Analysis of the Quality of Sudents’ Critical Reasoning Instruments Using Rasch Model

Authors

Yokhebed , Rexy Maulana Dwi Karmadi , Jayanti Silaen , Sherli Meidina , Nur Alfiatun Nikmah

DOI:

10.29303/jppipa.v11i3.9584

Published:

2025-03-25

Issue:

Vol. 11 No. 3 (2025): March

Keywords:

Critical reasoning, Instrument, Rasch model, Reliability, Validity

Research Articles

Downloads

How to Cite

Yokhebed, Karmadi, R. M. D., Silaen, J., Meidina, S., & Nikmah, N. A. (2025). Analysis of the Quality of Sudents’ Critical Reasoning Instruments Using Rasch Model. Jurnal Penelitian Pendidikan IPA, 11(3), 54–60. https://doi.org/10.29303/jppipa.v11i3.9584

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Abstract

This research aims to analyzes the validity and reliability of a students' critical reasoning questionnaire using the Rasch Model. This quantitative descriptive research employed a non-experimental survey design involving 122 fourth-semester undergraduate students from the College of Teacher Training and Education (STKIP) Melawi. Data were collected through an online questionnaire distributed via Google Forms. Validity was assessed through unidimensionality and item fit analysis, while reliability was measured using person reliability, item reliability, and Cronbach’s Alpha. The results indicate that the instrument has not yet achieved good item validity, with only 4 out of 20 items classified as valid. Reliability analysis showed Cronbach’s Alpha of 0.55 (bad category), item reliability of 0.98 (special category), and person reliability of 0.44 (weak category). Item difficulty analysis identified four difficulty levels: 2 items were very easy, 9 easy, 6 difficult, and 3 very difficult. The Rasch model’s item discrimination analysis produced eight item groups based on an item separation index of H = 9.97. These findings emphasize the need to refine questionnaire items and enhance respondent engagement. A validated instrument will support educational assessments by providing a reliable tool for measuring students’ critical reasoning skills in teacher training institutions.

References

Allen, E., & Seaman, C. (2007). Likert scales and data analysis. Quality Progress, 40(7), 64–65. Retrieved from https://www.bayviewanalytics.com/reports/asq/likert-scales-and-data-analyses.pdf

Alsoudi, S., & Shindi, Y. A. (2023). Rasch rating scale modelling of the Arabic version of the critical thinking disposition scale. Journal of Educational and Psychological Studies [JEPS], 17(4), 359–369. https://doi.org/10.53543/jeps.vol17iss4pp359-369

Anggraini, N., Nabillah, C., Lonan, H. D., & Sain, H. (2023). Application of the Rasch model to test tools in the analysis of survey design. Parameter: Journal of Statistics, 3(1), 19–27. https://doi.org/10.22487/27765660.2023.v3.i1.16412

Asmarita, D., & Prastowo, A. (2023). The learning process of Islamic education to foster students’ critical reasoning at Madrasah Ibtidaiyah Teluk Panji, North Sumatra. DAYAH: Journal of Islamic Education, 6(2), 233. https://doi.org/10.22373/jie.v6i2.18508

Azmi, I., & Prasetya, D. S. B. (2025). Profil berpikir kritis siswa SMP pada mata pelajaran IPA. Journal of Classroom Action Research, 7(1), 163–175. https://doi.org/10.21449/ijate.1210479

Balbuena, S. (2023). Application of the Rasch model in streamlining an instrument measuring depression among college students. International Journal of Assessment Tools in Education, 10(2), 257–278. https://doi.org/10.21449/ijate.1210479

Boone, W. J. (2016). Rasch Analysis for Instrument Development: Why, When, and How? CBE—Life Sciences Education, 15(4), rm4. https://doi.org/10.1187/cbe.16-04-0148

Darmana, A., Sutiani, A., Nasution, H. A., Ismanisa, I., & Nurhaswinda, N. (2021). Analysis of Rasch model for the validation of chemistry national exam instruments. Jurnal Pendidikan Sains Indonesia, 9(3), 329–345. https://doi.org/10.24815/jpsi.v9i3.19618

Denis, B. (2022). Сritical thinking technology as a tool for innovative activity in mathematics lessons. Baltic Journal of Legal and Social Sciences, 2(2), 23–30. https://doi.org/10.30525/2592-8813-2022-2-4

Erfan, M., Maulyda, M. A., Hidayati, V. R., Astria, F. P., & Ratu, T. (2020). Analisis kualitas soal kemampuan membedakan rangkaian seri dan paralel melalui teori tes klasik dan model Rasch. Indonesian Journal of Educational Research and Review, 3(1), 11–19. https://doi.org/10.23887/ijerr.v3i1.24080

Gustianingrum, R. A., Murni, A., & Maimunah. (2023). Analisis kemampuan berpikir kritis peserta didik dalam menunjang penguatan profil pelajar Pancasila. PRISMA, Prosiding Seminar Nasional Matematika, 6(2), 465–471. Retrieved from https://journal.unnes.ac.id/sju/prisma/article/view/66908

Jumadi, J., Sukarelawan, M. I., & Kuswanto, H. (2023). An investigation of item bias in the four-tier diagnostic test using Rasch model. International Journal of Evaluation and Research in Education (IJERE), 12(2), 622–629. https://doi.org/10.11591/ijere.v12i2.22845

Kaur, M. K. (2018). Critical thinking for global peace: A key for sustainable development. International Journal of Trend in Scientific Research and Development, 2(3), 1498–1501. https://doi.org/10.31142/ijtsrd11439

Lailiyah, L., Supriyati, Y., & Komarudin, K. (2018). Analysis of measures items in development of instruments self-assessment (Rasch modeling application). JISAE: Journal of Indonesian Student Assessment and Evaluation, 4(1), 1–9. https://doi.org/10.21009/jisae.v4i1.7819

Laliyo, L. A. R., Kilo, A. L., Paputungan, M., Kunusa, W. R., Dama, L., & Panigoro, C. (2022). Rasch modelling to evaluate reasoning difficulties, changes of responses, and item misconception pattern of hydrolysis. Journal of Baltic Science Education, 21(5), 817–835. https://doi.org/10.33225/jbse/22.21.817

Linacre, J. M. (2012). Winsteps Rasch tutorial 2. Retrieved from https://www.winsteps.com/a/winsteps-tutorial-2.pdf

Loh, S. L., Jamil, N. A., & Ng, K. T. (2022). Establishing the validity and reliability of a program evaluation questionnaire using Rasch measurement model. DINAMIKA: Jurnal Ilmiah Pendidikan Dasar, 14(2), 150–157. https://doi.org/10.30595/dinamika.v14i2.14786

Madzlan, N. A., Rus, R. C., Mustafa, M. C., & Yassin, S. M. (2022). Validity and reliability of survey items in employer perspective construct on the quality of ECCE: Rasch measurement model analysis. World Journal of English Language, 12(2), 288–293. https://doi.org/10.5430/wjel.v12n2p288

Mawardi. (2019). Rambu-rambu penyusunan skala sikap model Likert untuk mengukur sikap siswa. Scholaria: Jurnal Pendidikan Dan Kebudayaan, 9(3), 292–304. https://doi.org/10.24246/j.js.2019.v9.i3.p292-304

Mignogna, G., Carey, C. E., Wedow, R., Baya, N., Cordioli, M., Pirastu, N., Bellocco, R., Malerbi, K. F., Nivard, M. G., Neale, B. M., Walters, R. K., & Ganna, A. (2023). Patterns of item nonresponse behaviour to survey questionnaires are systematic and associated with genetic loci. Nature Human Behaviour, 7(8), 1371–1387. https://doi.org/10.1038/s41562-023-01632-7

Natsir, I., Mayasari, D., & Munfarikhatin, A. (2022). Rasch model: Quality of test instruments to measure students’ ability in algebra. SHS Web of Conferences, 149, 1–5. https://doi.org/10.1051/shsconf/202214901033

Packer, J., Croker, H., Goddings, A.-L., Boyland, E. J., Stansfield, C., Russell, S. J., & Viner, R. M. (2022). Advertising and young people’s critical reasoning abilities: Systematic review and meta-analysis. Pediatrics, 150(6). https://doi.org/10.1542/peds.2022-057780

Purwana, U., Rusdiana, D., & Liliawati, W. (2020). Pengujian kemampuan menginterpretasikan grafik kinematika calon guru fisika: The polytomous Rasch analysis. ORBITA: Jurnal Kajian, Inovasi Dan Aplikasi Pendidikan Fisika, 6(2), 259–266. https://doi.org/10.31764/orbita.v6i2.3264

Putra, M. D. K., & Retnawati, H. (2020). Analisis Rasch terhadap instrumen general self-efficacy scale-12 versi bahasa Indonesia: Studi komparasi rating scale model (RSM) dan partial credit model (PCM). Indonesian Journal of Educational Assesment, 3(1), 52–60. https://doi.org/10.26499/ijea.v3i1.55

Ramdani, A., Jufri, A. W., Jamaluddin, & Setiadi, D. (2020). Kemampuan berpikir kritis dan penguasaan konsep dasar IPA peserta didik. Jurnal Penelitian Pendidikan IPA, 6(1), 119–124. https://doi.org/10.29303/jppipa.v6i1.388

Ridzuan, M. F., Lim, L. H., Fozee, F. A. A., & Nasser, S. N. A. M. (2020). Rasch analysis model: Reliability and validity of superitem test instrument. International Journal of Academic Research in Progressive Education and Development, 9(4), 1–11. https://doi.org/10.6007/IJARPED/v9-i4/8166

Salsabila, F. (2023). Pengujian validitas dan reliabilitas instrumen manajemen diri remaja: Rasch model analysis. Jurnal Bimbingan Dan Konseling Terapan, 7(1), 15–25. https://doi.org/10.30598/jbkt.v7i1.1741

Sari, T. N. I., & Rakhmawati, A. (2024). Analysis the quality of critical thinking and creativity questions in high school biology subjects with the Rasch model. Research and Development in Education (RaDEn), 4(1), 602–616. https://doi.org/10.22219/raden.v4i1.32758

Sumintono, B., & Widhiarso, W. (2014). Aplikasi Model Rasch untuk penelitian ilmu-ilmu sosial. Bandung: Tim Komunika Publishing House

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan RASCH pada assessment pendidikan. Cimahi: Trim komunikata

Sumintono, B. (2016). Aplikasi Pemodelan Rasch pada asesmen pendidikan: Implementasi penilaian formatif (assessment for learning). Retrieved from https://shorturl.at/uKbbw

Tarigan, E. F., Nilmarito, S., Islamiyah, K., Darmana, A., & Suyanti, R. D. (2022). Analisis instrumen tes menggunakan Rasch model dan software SPSS 22.0. Jurnal Inovasi Pendidikan Kimia, 16(2), 92–96. https://doi.org/10.15294/jipk.v16i2.30530

Tennant, A., & Küçükdeveci, A. A. (2023). Application of the Rasch measurement model in rehabilitation research and practice: Early developments, current practice, and future challenges. Frontiers in Rehabilitation Sciences, 4, 1–17. https://doi.org/10.3389/fresc.2023.1208670

Wahyudi, A., Mahyuddin, M. J., Irawan, A. W., Silondae, D. P., Lestari, M., Basco, F. H., & Kurniawan, S. J. (2020). Model Rasch: Analisis skala resiliensi Connor-Davidson versi Bahasa Indonesia. Advice: Jurnal Bimbingan Dan Konseling, 2(1), 28–35. https://doi.org/10.32585/advice.v2i1.701

Wilson, M., & Dulhunty, M. (2019). Interpreting the relationship between item difficulty and DIF: Examples from educational testing. Journal of Physics: Conference Series, 1379(1). https://doi.org/10.1088/1742-6596/1379/1/012026

Yokhebed, Sutarno, Masykuri, M., Prayitno, B. A., & Sudirman. (2024). Validity and reliability of online socio-scientific issue-based critical reasoning instrument. Jurnal Pendidikan IPA Indonesia, 13(2), 229–238. https://doi.org/10.15294/2hnw4674

Yunus, F. A. N., Suki, N. A., Baser, J. A., Masran, S. H., Marian, M. F., & Rahim, M. B. (2017). Kesahan dan kebolehpercayaan instrumen kompetensi pengajar TVET terhadap pengajaran teknikal berdasarkan pendekatan model Rasch. Online Journal for TVET Practitioners, 2(2). Retrieved from https://publisher.uthm.edu.my/ojs/index.php/oj-tp/article/view/4780

Author Biographies

Yokhebed, Universitas Tanjungpura

Rexy Maulana Dwi Karmadi, Universitas Tanjungpura

Jayanti Silaen, Universitas Tanjungpura

Sherli Meidina, Universitas Tanjungpura

Nur Alfiatun Nikmah, Universitas Tanjungpura

License

Copyright (c) 2025 Yokhebed, Rexy Maulana Dwi Karmadi, Jayanti Silaen, Sherli Meidina, Nur Alfiatun Nikmah

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with Jurnal Penelitian Pendidikan IPA, agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC-BY License). This license allows authors to use all articles, data sets, graphics, and appendices in data mining applications, search engines, web sites, blogs, and other platforms by providing an appropriate reference. The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in Jurnal Penelitian Pendidikan IPA.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).