Beyond Content Validity: Comprehensive Validation of Scientific Literacy Assessment for Junior High School Teachers
DOI:
10.29303/jppipa.v11i12.13043Published:
2025-12-25Downloads
Abstract
Despite the central role of teachers in fostering students’ scientific literacy, assessment instruments targeting in-service science teachers remain limited. This study aimed to develop and examine the preliminary quality of a scientific literacy instrument for junior high school science teachers, informed by OECD documents outlining the direction of the PISA 2025. A design-based research approach was employed using Tessmer’s formative evaluation model, encompassing iterative stages of preliminary analysis to pilot field test. The instrument comprised nine context-rich stimuli and 25 items across multiple formats designed to elicit scientific reasoning and decision-making in real-world contexts. Expert validation produced a Mean Expert Score of 4.67, while teacher readability evaluation yielded a Mean Readability Score of 4.29, indicating strong content representation and clarity. Pilot psychometric analysis showed a balanced distribution of item difficulty (32% easy, 40% moderate, and 28% difficult), and item-level validity and discrimination indices provided diagnostic evidence for refinement. Pilot psychometric analysis indicated that the instrument’s reliability remains preliminary and highlights the need for further refinement through larger-scale field testing. Overall, this study contributes a forward-looking assessment instrument that supports formative evaluation of teachers’ scientific literacy and provides a robust foundation for subsequent large-scale validation aligned with emerging international assessment orientations.
Keywords:
Design-based research Junior high school Scientific literacy TeacherReferences
Aryani, S., Novia, H., & Setiawan, A. (2024). Development and Analysis Validation Instrument Critical Thinking Heat Concept (CTHT) Using Rash Model. Jurnal Penelitian Pendidikan IPA, 10(7), 3731–3738. https://doi.org/10.29303/jppipa.v10i7.7727 DOI: https://doi.org/10.29303/jppipa.v10i7.7727
Coppi, M., Fialho, I., & Cid, M. (2023). Developing a Scientific Literacy Assessment Instrument for Portuguese 3rd Cycle Students. Education Sciences, 13(9). https://doi.org/10.3390/educsci13090941 DOI: https://doi.org/10.3390/educsci13090941
Darling-Hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective Teacher Professional Development. Retrieved from https://learningpolicyinstitute.org/product/teacher-prof-dev. DOI: https://doi.org/10.54300/122.311
Darman, D. R., Suhandi, A., Kaniawati, I., Samsudin, A., & Wibowo, F. C. (2024). Development and Validation of Scientific Inquiry Literacy Instrument (SILI) Using Rasch Measurement Model. Education Sciences, 14(3). https://doi.org/10.3390/educsci14030322 DOI: https://doi.org/10.3390/educsci14030322
Eliyawati, E., Widodo, A., Kaniawati, I., & Fujii, H. (2022). Education For Sustainable Development (ESD) In Students’ Textbooks. In Journal of Engineering Science and Technology Special Issue on ICMScE2022. Retrieved from https://www.ipcc.ch/report/ar4/wg1/
Eliyawati, Widodo, A., Kaniawati, I., & Fujii, H. (2023). The Development and Validation of an Instrument for Assessing Science Teacher Competency to Teach ESD. Sustainability (Switzerland), 15(4). https://doi.org/10.3390/su15043276 DOI: https://doi.org/10.3390/su15043276
Faisal, S., Rusilowati, A., & Susilaningsih, E. (2023). Science Literacy Assessment Instrument for Additives and Addictive Substances: Development, Validation, and Rasch Model Analysis. Jurnal Penelitian Pendidikan IPA, 9(10), 7826–7836. https://doi.org/10.29303/jppipa.v9i10.4376 DOI: https://doi.org/10.29303/jppipa.v9i10.4376
Fe Bustamante, M. O., & EMercado, R. (2024). Professional Development And Scientific Literacy In Science Curriculum. Sci-Int. (Lahore), 36(6), 417–422. Retrieved from https://sci-int.com/pdf/638715227225281132.pdf
Habibi, H., & Suparman, S. (2020). Literasi Matematika dalam Menyambut PISA 2021 Berdasarkan Kecakapan Abad 21. JKPM (Jurnal Kajian Pendidikan Matematika), 6(1), 57-64. Retrieved from http://journal.lppmunindra.ac.id/index.php/jkpm/ DOI: https://doi.org/10.30998/jkpm.v6i1.8177
Hartina, L., Rosidin, U., & Suyatna, A. (2020). Pengaruh Penerapan Instrumen Performance Assessment pada Pembelajaran IPA Berbasis Laboratorium Real terhadap Hasil Belajar Siswa. Jurnal Penelitian Pendidikan IPA, 6(1), 25–31. https://doi.org/10.29303/jppipa.v6i1.299 DOI: https://doi.org/10.29303/jppipa.v6i1.299
Hasasiyah, S. H., Hutomo, B. A., Subali, B., & Marwoto, P. (2020). Analisis Kemampuan Literasi Sains Siswa SMP pada Materi Sirkulasi Darah. Jurnal Penelitian Pendidikan IPA, 6(1), 5–9. https://doi.org/10.29303/jppipa.v6i1.193 DOI: https://doi.org/10.29303/jppipa.v6i1.193
Hidayah, N., Rusilowati, A., & Masturi, M. (2019). Analisis profil kemampuan literasi sains siswa SMP/MTs di Kabupaten Pati. Phenomenon: Jurnal Pendidikan MIPA, 9(1), 36-47. https://doi.org/10.21580/phen.2019.9.1.3601 DOI: https://doi.org/10.21580/phen.2019.9.1.3601
Hung, C. S., & Wu, H. K. (2024). High school science teachers’ assessment literacy for inquiry-based science instruction. International Journal of Science Education, 46(7), 621–642. https://doi.org/10.1080/09500693.2023.2251657 DOI: https://doi.org/10.1080/09500693.2023.2251657
Hurian Kamela, R. S. A. (2022). Public Teacher Recruitment And Selection In Indonesia: A Legal Perspective. Scientia Business Law Review (SBLR), 1(1), 37–43. https://doi.org/10.56282/sblr.v1i1.51 DOI: https://doi.org/10.56282/sblr.v1i1.51
Jamaluddin, J., Jufri, A. W., Ramdani, A., & Azizah, A. (2019). Profil Literasi Sains Dan Keterampilan Berpikir Kritis Pendidik IPA SMP. Jurnal Penelitian Pendidikan IPA, 5(1). https://doi.org/10.29303/jppipa.v5i1.185 DOI: https://doi.org/10.29303/jppipa.v5i1.185
Kelp, N. C., McCartney, M., Sarvary, M. A., Shaffer, J. F., & Wolyniak, M. J. (2023). Developing Science Literacy in Students and Society: Theory, Research, and Practice. Journal of Microbiology & Biology Education, 24(2). https://doi.org/10.1128/jmbe.00058-23 DOI: https://doi.org/10.1128/jmbe.00058-23
Kioupi, V., & Voulvoulis, N. (2019). Education for sustainable development: A systemic framework for connecting the SDGs to educational outcomes. Sustainability (Switzerland), 11(21). https://doi.org/10.3390/su11216104 DOI: https://doi.org/10.3390/su11216104
Limiansih, K., Sulistyani, N., & Melissa, M. M. (2024). Persepsi Guru SMP terhadap Literasi Sains dan Implikasinya pada Pembelajaran Sains di Sekolah. Jurnal Pendidikan Mipa, 14(3), 786–796. https://doi.org/10.37630/jpm.v14i3.1858 DOI: https://doi.org/10.37630/jpm.v14i3.1858
Mckay, V. (2018). Literacy, lifelong learning and sustainable development. Australian Journal of Adult Learning, 58(3), 390–425. Retrieved from https://search.informit.org/doi/abs/10.3316/informit.069488054638414
McKenney, S., & Reeves, T. C. (2025). Educational design research for relevant & robust scholarship. Journal of Computing in Higher Education, 37(2), 614–638. https://doi.org/10.1007/s12528-025-09456-2 DOI: https://doi.org/10.1007/s12528-025-09456-2
Mulyana, V., & Desnita, D. (2023). Empirical Validity and Reliability of the Scientific Literacy Assessment Instrument Based on the Tornado Physics Enrichment Book. Jurnal Penelitian Pendidikan IPA, 9(5), 3961–3967. https://doi.org/10.29303/jppipa.v9i5.3290 DOI: https://doi.org/10.29303/jppipa.v9i5.3290
OECD. (2023a). Education at a Glance 2023 (Education at a Glance). OECD Publishing. https://doi.org/10.1787/e13bef63-en DOI: https://doi.org/10.1787/e13bef63-en
OECD. (2023b). PISA 2022 Results (Volume I): The State of Learning and Equity in Education. OECD. https://doi.org/10.1787/53f23881-en DOI: https://doi.org/10.1787/53f23881-en
OECD. (2023c). PISA 2025 Science Framework (Draft). Retrieved from https://pisa-framework.oecd.org/science-2025/
Osborne, J., & Allchin, D. (2024). Science literacy in the twenty-first century: informed trust and the competent outsider. International Journal of Science Education. https://doi.org/10.1080/09500693.2024.2331980 DOI: https://doi.org/10.1080/09500693.2024.2331980
Parenta, Y., Masykuri, M., & Saputro, S. (2022). Literature Study: Application of PBL-STEM on Simple Machine Topic to Improve Critical Thinking Skills. Jurnal Penelitian Pendidikan IPA, 8(2), 674–680. https://doi.org/10.29303/jppipa.v8i2.1181 DOI: https://doi.org/10.29303/jppipa.v8i2.1181
Rahmadani, Y., Fitakurahmah, N., Fungky, N., Prihatin, R., Majid, Q., & Prayitno, B. A. (2018). Profil Keterampilan Literasi Sains Siswa di Salah Satu Sekolah Swasta di Karanganyar. Jurnal Pendidikan Biologi, 7(3), 183–190. https://doi.org/10.24114/jpb.v7i3.10123 DOI: https://doi.org/10.24114/jpb.v7i3.10123
Rasimin, Semma, A. B., Zakiyuddin, Ali, M., & Helmy, M. I. (2024). Multi-dimensional challenges in the Indonesian social science information technology-based learning: A systematic literature review. Heliyon, 10(7). https://doi.org/10.1016/j.heliyon.2024.e28706 DOI: https://doi.org/10.1016/j.heliyon.2024.e28706
Roy, G., Sikder, S., & Danaia, L. (2025). Adopting scientific literacy in early years from empirical studies on formal education: a systematic review of the literature. In International Journal of STEM Education (Vol. 12, Issue 1). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1186/s40594-025-00547-1 DOI: https://doi.org/10.1186/s40594-025-00547-1
Sæleset, J., & Friedrichsen, P. (2021). Pre-service Science Teachers’ Pedagogical Content Knowledge Integration of Students’ Understanding in Science and Instructional Strategies. Eurasia Journal of Mathematics, Science and Technology Education, 17(5), 1–18. https://doi.org/10.29333/ejmste/10859 DOI: https://doi.org/10.29333/ejmste/10859
Schenk, L., Hamza, K., Arvanitis, L., Lundegård, I., Wojcik, A., & Haglund, K. (2021). Socioscientific Issues in Science Education: An opportunity to Incorporate Education about Risk and Risk Analysis? Risk Analysis, 41(12), 2209–2219. https://doi.org/10.1111/risa.13737 DOI: https://doi.org/10.1111/risa.13737
Sjöström, J. (2024). Vision III of scientific literacy and science education: an alternative vision for science education emphasising the ethico-socio-political and relational-existential. In Studies in Science Education. Routledge. https://doi.org/10.1080/03057267.2024.2405229 DOI: https://doi.org/10.1080/03057267.2024.2405229
Rezeki, S. K., & Lubis, F. (2022). Analisis Soal Buatan Guru Bahasa Indonesia Berdasarkan Taksonomi The Structure Of The Observed Learning Outcome (Solo) Siswa Kelas XI SMA Negeri 12 Medan Tahun Pembelajaran 2020/2021 (Vol. 11, Issue 1). Retrieved from http://ejournal.uika-bogor.ac.id/index.php/TEK DOI: https://doi.org/10.32832/tek.pend.v11i1.5997
Suwono, H., Maulidia, L., Saefi, M., Kusairi, S., & Yuenyong, C. (2022). The Development and Validation of an Instrument of Prospective Science Teachers’ Perceptions of Scientific Literacy. Eurasia Journal of Mathematics, Science and Technology Education, 18(1). https://doi.org/10.29333/EJMSTE/11505 DOI: https://doi.org/10.29333/ejmste/11505
Tasquier, G., Knain, E., & Jornet, A. (2022). Scientific Literacies for Change Making: Equipping the Young to Tackle Current Societal Challenges. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.689329 DOI: https://doi.org/10.3389/feduc.2022.689329
Tessmer, M. (2005). Planning and Conducting Formative Evaluations: Improving the Quality of Education and Training. Routledge.
Tonnetti, B., & Lentillon-Kaestner, V. (2023). Teaching interdisciplinarity in secondary school: A systematic review. In Cogent Education (Vol. 10, Issue 1). Taylor and Francis Ltd. https://doi.org/10.1080/2331186X.2023.2216038 DOI: https://doi.org/10.1080/2331186X.2023.2216038
Tripp, B., & Shortlidge, E. E. (2019). A framework to guide undergraduate education in interdisciplinary science. CBE Life Sciences Education, 18(2). https://doi.org/10.1187/cbe.18-11-0226 DOI: https://doi.org/10.1187/cbe.18-11-0226
United Nations. (2025). The Sustainable Development Goals Report. Retrieved from https://unstats.un.org/sdgs
Widiatmo, T., Jufri, A. W., & Jamaluddin, J. (2019). Analysis of Validation of Instruments to Measure Student’s Critical Thinking Ability and Science Literation. Jurnal Penelitian Pendidikan IPA, 5(2), 212–218. https://doi.org/10.29303/jppipa.v5i2.272 DOI: https://doi.org/10.29303/jppipa.v5i2.272
Wieman, C. (2007). Why Not Try a Scientific Approach to Science Education. The Magazine of Higher Learning, 9–15. https://doi.org/10.3200/CHNG.39.5.9-15 DOI: https://doi.org/10.3200/CHNG.39.5.9-15
Yusmar, F., & Fadilah, R. E. (2023). Analisis Rendahnya Literasi Sains Peserta Didik Indonesia: Hasil PISA Dan Faktor Penyebab. LENSA (Lentera Sains): Jurnal Pendidikan IPA, 13(1), 11–19. https://doi.org/10.24929/lensa.v13i1.283 DOI: https://doi.org/10.24929/lensa.v13i1.283
License
Copyright (c) 2025 Kintan Limiansih, Albertus Hariwangsa Panuluh, Niluh Sulistyani
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with Jurnal Penelitian Pendidikan IPA, agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC-BY License). This license allows authors to use all articles, data sets, graphics, and appendices in data mining applications, search engines, web sites, blogs, and other platforms by providing an appropriate reference. The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in Jurnal Penelitian Pendidikan IPA.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
