Integrating Technology and Psychometrics: Developing an App-Based Diagnostic Tool Using the Partial Credit Model to Uncover Student Misconceptions
DOI:
10.29303/jppipa.v11i12.12305Published:
2025-12-25Downloads
Abstract
Misconceptions in science learning remain a major barrier to students’ conceptual understanding, while traditional assessments often fail to detect them effectively. This study aimed to develop PhyTestApp, an app-based diagnostic tool that integrates the Partial Credit Model (PCM) under Item Response Theory (IRT) to uncover student misconceptions in science. A research and development design was employed, involving expert validation, limited trials, and psychometric testing. The two-tier items were designed to capture both factual knowledge and reasoning. Findings indicated that the instrument met psychometric requirements, with items demonstrating good fit and functioning across different levels of student understanding. Usability testing also showed positive responses from students and teachers regarding clarity, content relevance, and technical operation. Overall, PhyTestApp provides a reliable and practical diagnostic tool that facilitates immediate feedback and supports more targeted science instruction. These results highlight the potential of combining psychometric modeling with mobile technology to improve the quality of science education and more effectively address misconceptions in line with 21st-century learning goals.
Keywords:
Diagnostic assessment Educational technology Misconceptions Partial credit modelReferences
Abidin, Z. A. (2018). Pengembangan Computerized Adaptive Test (CAT) untuk Memetakan Keterampilan Berpikir Kritis Fisika Peserta Didik Kelas XI SMA. Universitas Negeri Yogyakarta.
Aiken, L. R. (1985). Three Coefficients for Analyzing the Reliability and Validity of Ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012 DOI: https://doi.org/10.1177/0013164485451012
Akbar. (2013). Instrumen Perangkat Pembelajaran. PT. Remaja Rosdakarya.
Andayani, A., & Ramalis, T. R. (2019). Kajian implementasi teori respon butir dalam menganalisis instrumen tes materi fisika. Prosiding Seminar Nasional Fisika 5.0, 37–42. Retrieved from https://fisika.upi.edu/wp-content/uploads/2020/02/Prosiding-Sinafi-2019.pdf
Çelikkanlı, N. Ö., & Kızılcık, H. Ş. (2022). A Review of Studies About Four-Tier Diagnostic Tests in Physics Education. Journal of Turkish Science Education, 19(4), 1291–1311. https://doi.org/10.36681/tused.2022.175 DOI: https://doi.org/10.36681/tused.2022.175
Chen, C. H. H. I. H., Lin, H. U. H., & Lin, M. I. N. G. I. (2003). Developing a Two-Tier Diagnostic Instrument to Assess High School Students ’ Understanding − The Formation of Images by a Plane Mirror. Proceedings of the National Science Council, 12(3), 106–121. Retrieved from https://shorturl.at/AiRkc
Dellantonio, S., & Pastore, L. (2021). Ignorance, misconceptions and critical thinking. Synthese, 198(8), 7473–7501. https://doi.org/10.1007/s11229-019-02529-7 DOI: https://doi.org/10.1007/s11229-019-02529-7
George, D., & Mallery, P. (2020). IBM SPSS Statistics 26 Step by Step: A Simple Guide and Reference (Sixteenth). Routledge. https://doi.org/10.4324/9780429056765 DOI: https://doi.org/10.4324/9780429056765
Greiff, S., Niepel, C., Scherer, R., & Martin, R. (2016). Understanding students’ performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files. Computers in Human Behavior, 61, 36–46. https://doi.org/10.1016/j.chb.2016.02.095 DOI: https://doi.org/10.1016/j.chb.2016.02.095
Gurel, D. K., Eryilmaz, A., & McDermott, L. C. (2015). A review and comparison of diagnostic instruments to identify students’ misconceptions in science. Eurasia Journal of Mathematics, Science and Technology Education, 11(5), 989–1008. https://doi.org/10.12973/eurasia.2015.1369a DOI: https://doi.org/10.12973/eurasia.2015.1369a
Istiyono, E. (2018). Pengembangan Instrumen Penilaian dan Analisis Hasil Belajar Fisika. UNY Press.
Juita, Z., Sundari, P. D., Sari, S. Y., & Rahim, F. R. (2023). Identification of Physics Misconceptions Using Five-tier Diagnostic Test: Newton’s Law of Gravitation Context. Jurnal Penelitian Pendidikan IPA, 9(8), 5954–5963. https://doi.org/10.29303/jppipa.v9i8.3147 DOI: https://doi.org/10.29303/jppipa.v9i8.3147
Kaltakci-Gurel, D., Eryilmaz, A., & McDermott, L. C. (2017). Development and application of a four-tier test to assess pre-service physics teachers’ misconceptions about geometrical optics. Research in Science and Technological Education, 35(2), 238–260. https://doi.org/10.1080/02635143.2017.1310094 DOI: https://doi.org/10.1080/02635143.2017.1310094
Laliyo, L. A. R., Botutihe, D. N., & Panigoro, C. (2019). The development of two-tier instrument based on distractor to assess conceptual understanding level and student misconceptions in explaining redox reactions. International Journal of Learning, Teaching and Educational Research, 18(9), 216–237. https://doi.org/10.26803/ijlter.18.9.12 DOI: https://doi.org/10.26803/ijlter.18.9.12
Nainggolan, J., Silaban, B., Sinaga, D., & Zendrato, F. (2023). Analysis of Physics Misconceptions of Students in Mechanic Materials Using the Tier Multiple Choice Diagnostic Test. AL-ISHLAH: Jurnal Pendidikan, 15(3), 3578–3586. https://doi.org/10.35445/alishlah.v15i3.3023 DOI: https://doi.org/10.35445/alishlah.v15i3.3023
Nugraeni, D., Jamzuri, J., & Sarwanto, S. (2013). Penyusunan tes diagnostik fisika materi listrik dinamis. Jurnal Pendidikan Fisika, 1(2), 12–16. Retrieved from http://jurnal.fkip.uns.ac.id/index.php/pfisika/article/view/2796
Nurhasanah, Hidayatullah, Z., Badrus, M., & Arif, S. (2024). Karakteristik Instrumen Tes Literasi Digital Ditinjau dari Validitas Isi dan Validitas Empiris (Kecocokan Butir dengan Model, Reliabilitas, serta Tingkat Kesukaran Butir). Journal of Classroom Action Research, 6(4), 917–923. https://doi.org/10.29303/jcar.v6i4.9650
Oriondo, & Dallo-Antonio. (1984). Evaluating Educational outcomes (test, measurement, and evalution). REX Printing Company, Inc.
Prihatni, Y., Kumaidi, K., & Mundilarto, M. (2016). Pengembangan instrumen diagnostik kognitif pada mata pelajaran IPA di SMP. Jurnal Penelitian dan Evaluasi Pendidikan, 20(1), 111-125. Retrieved from http://journal.uny.ac.id/index.php/jpep. Jurnal Penelitian Dan Evaluasi Pendidikan, 20(1), 111–125. DOI: https://doi.org/10.21831/pep.v20i1.7524
Pujayanto, P., Budiharti, R., Adhitama, E., Nuraini, N. R. A., & Putri, H. V. (2018). The development of a web-based assessment system to identify students’ misconception automatically on linear kinematics with a four-tier instrument test. Physics Education, 53(4). https://doi.org/10.1088/1361-6552/aac695 DOI: https://doi.org/10.1088/1361-6552/aac695
Rohmah, Z., Handhika, J., & Huriawati, F. (2018). E-diagnostic test untuk mengungkap miskonsepsi kinematika. SPEKTRA : Jurnal Kajian Pendidikan Sains, 4(2), 162. https://doi.org/10.32699/spektra.v4i2.57 DOI: https://doi.org/10.32699/spektra.v4i2.57
Sahidu, H., Gunawan, G., Indriaturrahmi, I., & Astutik, F. (2017). Desain Sistem E-Assessment Pada Pembelajaran Fisika Di Lptk. Jurnal Pendidikan Fisika Dan Teknologi, 3(2), 265. https://doi.org/10.29303/jpft.v3i2.422 DOI: https://doi.org/10.29303/jpft.v3i2.422
Salma, V. M. (2015). Pengembangan E-Diagnostic Test Untuk Mengidentifikasi Pemahaman Konsep Fisika Siswa Sma Pada Pokok Bahasan Fluida Statis Universitas Negeri Semarang. https://doi.org/10.15294/upej.v5i1.12701
Samsudin, A., Zulfikar, A., Saepuzaman, D., Suhandi, A., Aminudin, A. H., Supriyadi, S., & Coştu, B. (2024). Correcting grade 11 students’ misconceptions of the concept of force through the conceptual change model (CCM) with PDEODE*E tasks. Journal of Turkish Science Education, 21(2), 212–231. https://doi.org/10.36681/tused.2024.012 DOI: https://doi.org/10.36681/tused.2024.012
Sarasvati, A. (2016). Pengembangan Science Assesment Website (Sc-Wb) Tema Sistem Ekskresi Manusia untuk Kelas VIII SMP. Universitas Negeri Semarang. Retrieved from http://lib.unnes.ac.id/28833/
Soehato, S., & Csapo, B. (2021). Evaluating Item Difficulty Patterns for assessing student misconceptions in science across physics, chemistry, and biology concepts. Heliyon, 7. https://doi.org/10.1016/j.heliyon.2021.e08352 DOI: https://doi.org/10.1016/j.heliyon.2021.e08352
Suantari, N. M. D. P., Suma, K., & Pujani, N. M. (2018). Development of three tier static electricity diagnostic test to identify student conceptions about static electricity. International Research Journal of Engineering, IT & Scientific Research, 4(5). https://doi.org/10.21744/irjeis.v4n5.285 DOI: https://doi.org/10.21744/irjeis.v4n5.285
Suban, M. E., Hidayatullah, Z., & Nurhasanah. (2024). Identifikasi Miskonsepsi Menggunakan Three-Tier Diagnostic Test dan Representasi Gambar pada Konsep Gaya. Hamzanwadi Journal of Science Education, 1(2), 1–9. https://doi.org/10.29408/hijase.v1i2.26917 DOI: https://doi.org/10.29408/hijase.v1i2.26917
Sukatiman, Saputro, I. N., & Budiarto, M. K. (2024). Digital Classroom Innovations: Leveraging Smartphone-Based Application To Stimulate Students Creative Thinking Skills. Journal on Efficiency and Responsibility in Education and Science, 17(4), 349–360. https://doi.org/10.7160/eriesj.2024.170407 DOI: https://doi.org/10.7160/eriesj.2024.170407
Suparman, A. R., Rohaeti, E., & Wening, S. (2024). Development of Computer-Based Chemical Five-Tier Diagnostic Test Instruments: a Generalized Partial Credit Model. Journal on Efficiency and Responsibility in Education and Science, 17(1), 92–106. https://doi.org/10.7160/eriesj.2024.170108 DOI: https://doi.org/10.7160/eriesj.2024.170108
Susanti, M., Rusilowati, A., & Susanto, H. (2015). Pengembangan Bahan Ajar Ipa Berbasis Literasi Sains Bertema Listrik Dalam Kehidupan Untuk Kelas IX. Unnes Physics Education Journal, 4(3). https://doi.org/10.15294/upej.v4i3.9973
Syamsuddin, S. (2023). Implementasi Classic Test dan Item Respon Theory Pada Penilaian Tes Pembelajaran Matematika. EDUSCOPE: Jurnal Pendidikan, Pembelajaran, Dan Teknologi, 8(2), 28–43. https://doi.org/10.32764/eduscope.v8i2.3488 DOI: https://doi.org/10.32764/eduscope.v8i2.3488
Taslidere, E. (2016). Development and Use of a Three-tier Diagnostic Test to Assess High School Students’ Misconceptions about the Photoelectric Effect. Research in Science and Technological Education, 34(2), 164–186. https://doi.org/10.1080/02635143.2015.1124409 DOI: https://doi.org/10.1080/02635143.2015.1124409
Theobald, M., & Brod, G. (2021). Tackling Scientific Misconceptions: The Element of Surprise. Child Development, 92(5), 2128–2141. https://doi.org/10.1111/cdev.13582 DOI: https://doi.org/10.1111/cdev.13582
Thiagarajan, S. (1974). Instructional Development for Training Teachers of Exceptional Children. Sourcebook.
Tsui, C. Y., & Treagust, D. (2010). Evaluating secondary students’ scientific reasoning in genetics using a two-tier diagnostic instrument. International Journal of Science Education, 32(8), 1073–1098. https://doi.org/10.1080/09500690902951429 DOI: https://doi.org/10.1080/09500690902951429
Tumanggor, A. M. R., Supahar, S., Ringo, E. S., & Harliadi, M. D. (2020). Detecting Students’ Misconception in Simple Harmonic Motion Concepts Using Four-Tier Diagnostic Test Instruments. Jurnal Ilmiah Pendidikan Fisika Al-Biruni, 9(1), 21–31. https://doi.org/10.24042/jipfalbiruni.v9i1.4571 DOI: https://doi.org/10.24042/jipfalbiruni.v9i1.4571
Vosniadou, S. (2020). Students’ Misconceptions and Science Education. Oxford University Press. https://doi.org/10.1093/acrefore/9780190264093.013.965 DOI: https://doi.org/10.1093/acrefore/9780190264093.013.965
Wahidah, N., & Saptono, S. (2019). The Development of Three Tier Multiple Choice Test to Explore Junior High School Students’ Scientific Literacy Misconceptions. Journal of Innovative Science Education, 8(2), 190-198. Retrieved from https://journal.unnes.ac.id/sju/index.php/jise/article/view/27927
Wahyuningsih, T., Raharjo, T., & Fitriana Masithoh, D. (2013). Pembuatan Instrumen Tes Diagnostik Fisika SMA Kelas XI. Jurnal Pendidikan Fisika, 1(1), 111–117. Retrieved from https://jurnal.fkip.uns.ac.id/index.php/pfisika/article/view/1785
License
Copyright (c) 2025 Nurhasanah, Zul Hidayatullah, Hajra Yansa, Moh. Badrus Sholeh Arif, Muh Asriadi
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with Jurnal Penelitian Pendidikan IPA, agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC-BY License). This license allows authors to use all articles, data sets, graphics, and appendices in data mining applications, search engines, web sites, blogs, and other platforms by providing an appropriate reference. The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in Jurnal Penelitian Pendidikan IPA.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
