Effectiveness of Higher Order Thinking Skills-based Test Instruments in Science Learning in Indonesia: A Meta-analysis

Authors

Tomi Apra Santosa , Siska Angreni , Rona Taula Sari , Festiyed , Yerimadesi , Yuni Ahda , Heffi Alberida , Fitri Arsih

DOI:

10.29303/jppipa.v10i5.6654

Published:

2024-05-25

Issue:

Vol. 10 No. 5 (2024): May

Keywords:

Effect size, Higher order thinking skills, Science learning, Test instrument

Review

Downloads

How to Cite

Santosa, T. A., Angreni, S., Sari, R. T., Festiyed, Yerimadesi, Ahda, Y., … Arsih, F. (2024). Effectiveness of Higher Order Thinking Skills-based Test Instruments in Science Learning in Indonesia: A Meta-analysis. Jurnal Penelitian Pendidikan IPA, 10(5), 242–249. https://doi.org/10.29303/jppipa.v10i5.6654
Crossref
Scopus
Google Scholar
Europe PMC

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Abstract

This research is motivated by the many applications of Higher Order Thinking Skills-based test instruments in science learning. However, it has not been found how effective the instrument is in science learning. The purpose of this study was to determine the effectiveness of Higher Order Thinking Skills-based test instruments in science learning. This type of research is a meta-analysis. Data sources come from the databases of Google Scholar, Hindawi, ScienceDirect, Wiley, Taylor of Francis, Sage Journal, Springer, ProQuest and ERIC. Search keywords are test instruments, test instruments based on Higher Order Thinking Skills in science learning, and higher order thinking skills. This study analyzed 20 articles published in 2020-2023 used in the meta-analysis. Data analysis techniques with the help of JSAP software. The results concluded that the summary effect size value was 1.24 (very high effect size). This finding shows that the use of test instruments based on Higher Order Thinking Skills is effectively applied in science learning. Furthermore, the research provides higher order tinking skills-based test instruments in science learning in Indonesia and provides important information for the application of higher order thinking skills-based test instruments in the future

References

Ahmad, S., Prahmana, R. C. I., Kenedi, A. K., Helsa, Y., Arianil, Y., & Zainil, M. (2017). The instruments of higher order thinking skills. In Journal of Physics: Conference Series (Vol. 943, No. 1, p. 012053). IOP Publishing. https://doi.org/10.1088/1742-6596/943/1/012053

Alomery, H. (2022). The Effectiveness of Visual Mind Mapping Strategy for Improving English Language Learners’ Critical Thinking Skills and Reading Ability. European Journal of Educational Research, 11(3), 1245–1257. Retrieved from http://files.eric.ed.gov/fulltext/EJ1329573.pdf

Angganing, P., Budiningsih, C. A., & Haryanto. (2022). The Profile of Students’ Communication Skills on Science Learning in Elementary Schools. Pegem Egitim ve Ogretim Dergisi, 13(1), 117–124. https://doi.org/10.47750/pegegog.13.01.14

Apriani, S. P., Zuhro, M. V., Siregar, N., Ristanto, R. H., & Isfaeni, H. (2023). Diferensiasi Pembelajaran Virus Sebuah Pengembangan Instrumen Higher Order Thinking Skill (HOTS). Diklabio: Jurnal Pendidikan dan Pembelajaran Biologi, 7(2), 197-211. https://doi.org/10.33369/diklabio.7.2.197-211

Arafah, K., Amin, B. D., Sari, S. S., & Hakim, A. (2021, May). The development of higher order-thinking skills (HOTS) instrument assessment in physics study. In Journal of Physics: Conference Series (Vol. 1899, No. 1, p. 012140). IOP Publishing. https://doi.org/10.1088/1742-6596/1899/1/012140

Aybirdi, N. (2023). The Impact of Flipped Learning on L2 Learners’ Achievements : A Meta- Analysis. International Journal of Education, 11(1), 41–60. Retrieved from https://files.eric.ed.gov/fulltext/EJ1373405.pdf

Aydede, M. N. (2022). Examining the Primary School Teacher Candidates’ Science Learning Skills in Terms of Their Attitudes towards Science and Their Science Teaching Self-Efficacy Beliefs. International Journal of Educational Methodology, 8(4), 853–864. https://doi.org/10.12973/ijem.8.4.853

Azzahra, I., Nurhasanah, A., & Hermawati, E. (2023). Implementasi Kurikulum Merdeka pada Pembelajaran IPAS di SDN 4 Purwawinangun. Didaktik: Jurnal Ilmiah PGSD STKIP Subang, 9(2), 6230-6238. https://doi.org/10.36989/didaktik.v9i2.1270

Bantaokul, P., & Polyiem, T. (2022). The Use of Integrated 5Es of Inquiring-Based Learning and Gamification to Improve Grade 8 Student Science Learning Achievement. Journal of Educational Issues, 8(1), 459. https://doi.org/10.5296/jei.v8i1.19802

Baysal et al. (2023). The Effect Of Gender On Motivation Towards Science Learning: A Meta-Analysis Study. Research in Pedagogy, 13(1), 1–18. https://doi.org/10.5937/IstrPed2301001B

Bernard, R. M., Borokhovski, E., Schmid, R. F., & Tamim, R. (2014). A meta-analysis of blended learning and technology use in higher education : From the general to the applied. J Comput High Educ, 1–37. https://doi.org/10.1007/s12528-013-9077-3

Borenstein, M., & Hedges, L. V. (2009). Introduction to Meta-Analysis. John Wiley & Sons

Çevik, M., & Bakioğlu, B. (2022). The Effect of STEM Education Integrated into Teaching-Learning Approaches (SEITLA) on Learning Outcomes: A Meta-Analysis Study. International Journal of Progressive Education, 18(2), 119–135. https://doi.org/10.29329/ijpe.2022.431.8

Cohen, L., Manion, L., Lecturer, P., Morrison, K., & Lecturer, S. (2007). Research Methods in Education. Routledge.

Damaianti, V. S., Abidin, Y., & Rahma, R. (2020). Higher order thinking skills-based reading literacy assessment instrument : An Indonesian context. Indonesian Journal of Applied Linguistics, 10(2), 513–525. https://doi.org/10.17509/ijal.v10i2.28600

Divayana, D. G. H., Sudirtha, I. G., & Suartama, I. K. (2021). Digital Test Instruments Based on" Wondershare-Superitem" for Supporting Distance Learning Implementation of Assessment Course. International Journal of Instruction, 14(4), 945-964. https://doi.org/10.29333/iji.2021.14454a

Ernawati, M. D. W., Rusdi, M., Asrial, A., Muhaimin, M., Wulandari, M., & Maryani, S. (2022). Analysis of Problem Based Learning in the Scaffolding Design: Students' Creative-Thinking Skills. Cypriot Journal of Educational Sciences, 17(7), 2333-2348. Retrieved from http://files.eric.ed.gov/fulltext/EJ1350312.pdf

Fahrezi, I., Taufiq, M., Akhwani, A., & Nafia’ah, N. (2020). Meta-Analisis Pengaruh Model Pembelajaran Project Based Learning Terhadap Hasil Belajar Siswa Pada Mata Pelajaran IPA Sekolah Dasar. Jurnal Ilmiah Pendidikan Profesi Guru, 3(3), 408. https://doi.org/10.23887/jippg.v3i3.28081

Hariyadi, S., Santosa, T. A., & Sakti, B. P. (2023). Effectiveness of STEM-Based Mind Mapping Learning Model to Improve Students’ Science Literacy in the Era of Revolution. Jurnal Penelitian Pendidikan IPA, 9(10), 791–799. https://doi.org/10.29303/jppipa.v9i10.5125

He, X. X., Deng, Y. P., Liu, J. H., Sun, G. Y., Xiong, J. W., & Xiao, Y. (2023). The Chain Mediating Role of Perceived Family Support for Formal and Informal Science Learning in the Association between Family Socioeconomic Status and Informal Science Learning Experiences. Journal of Baltic Science Education, 22(2), 232-253. https://doi.org/10.33225/jbse/23.22.232

Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M. (2020). Computers & Education The potential of digital tools to enhance mathematics and science learning in secondary schools : A context-specific meta-analysis. Computers & Education, 153(April), 103897. https://doi.org/10.1016/j.compedu.2020.103897

Ichsan, Suharyat, Y., Santosa, T. A., & Satria, E. (2023). The Effectiveness of STEM-Based Learning in Teaching 21 st Century Skills in Generation Z Student in Science Learning : A. Jurnal Penelitian Pendidikan IPA, 9(1), 150–166. https://doi.org/10.29303/jppipa.v9i1.2517

Juandi, D., Kusumah, Y. S., & Tamur, M. (2022). A Meta-Analysis of the last two decades of realistic mathematics education approaches. International Journal of Instruction, 15(1), 381–400. https://doi.org/10.29333/iji.2022.15122a

Juandi, D., Kusumah, Y. S., Tamur, M., Perbowo, K. S., & Tanu, T. (2021). Heliyon A meta-analysis of Geogebra software decade of assisted mathematics learning : what to learn and where to go ? Heliyon, 7(October), e06953. https://doi.org/10.1016/j.heliyon.2021.e06953

Kim, S. Y., & Hamdan Alghamdi, A. K. (2023). Secondary School Students’ Perceptions of Science Learning Environment and Self-Efficacy in South Korea: Gender Differences. Journal of Baltic Science Education, 22(2), 269–281. https://doi.org/10.33225/jbse/23.22.269

Li, S., & Wang, W. (2023). Effect of blended learning on student performance in K-12 settings : A meta-analysis. J Comput Assist Learn, 1–19. https://doi.org/10.1111/jcal.12696

Martin, F., & Carolina, N. (2022). A Meta-Analysis on the Community of Inquiry Presences and Learning Outcomes in Online and Blended Learning Environments. Online Learning Journal, 26(1), 325–359. https://doi.org/10.24059/olj.v26i1.2604

Purnomo, H., Sa'dijah, C., Hidayanto, E., Permadi, H., & Anwar, L. (2022). Development of Instrument Numeracy Skills Test of Minimum Competency Assessment (MCA) in Indonesia. International Journal of Instruction, 15(3), 635-648. https://doi.org/10.29333/iji.2022.15335a

Puspita, S., & Irfandi, I. (2022). Meta-Analysis of Focusky Learning Media on Student Learning Outcomes. Jurnal Ilmu Pendidikan Muhammadiyah Kramat Jati, 3(2), 20-22. https://doi.org/10.55943/jipmukjt.v3i2.34

Puspita, A. D., Maryani, I., & Sukma, H. H. (2023). Problem-based science learning in elementary schools: A bibliometric analysis. Journal of Education and Learning, 17(2), 285–293. https://doi.org/10.11591/edulearn.v17i2.20856

Qisthi, C. U., Abidin, Z., & Sulistyono, S. (2023). Pengembangan Instrumen Literasi Numerasi Biologi Pada Pembelajaran IPA Dalam Materi Sistem Pernapasan Pada Manusia. Jurnal Ilmiah Wahana Pendidikan, 9(22), 191-202. https://doi.org/10.5281/zenodo.10096812

Rahayu, N. K. L. A., Widiartini, N. K., & Divayana, D. G. H. (2023). Pengembangan Instrumen Higher Order Thinking Skill (HOTS) Matematika Program Studi Penelitian dan Evaluasi Pendidikan. Jurnal Penelitian Dan Evaluasi Pendidikan Indonesia, 13(1), 17–26. https://doi.org/10.23887/jpepi.v13i1.1761

Rahman, A. A., Santosa, T. A., Nurtamam, M. E., & Widoyo, H. (2023). Meta-Analysis : The Effect of Ethnoscience-Based Project Based Learning Model on Students ’ Critical Thinking Skills. Jurnal Penelitian Pendidikan IPA, 9(9), 611–620. https://doi.org/10.29303/jppipa.v9i9.4871

Ramadhan, S., Mardapi, D., Prasetyo, Z. K., & Utomo, H. B. (2019). The Development of an Instrument to Measure the Higher Order Thinking Skill in Physics. European Journal of Educational Research, 8(3), 743–751. https://doi.org/10.12973/eu-jer.8.3.743

Ramadhan, S., & Sumiharsono, R. (2020). The Quality of Test Instruments Constructed by Teachers in Bima Regency, Indonesia : Document Analysis. International Journal of Instruction, 13(2), 507–518. https://doi.org/10.29333/iji.2020.13235a

Razak, A., Santosa, T. A., Lufri, & Zulyusri. (2021). Meta-Analisis: Pengaruh HOTS (Higher Order Thinking Skill) terhadap Kemampuan Literasi Sains dan Lesson Study Siswa pada Materi Ekologi dan Lingkungan pada Masa Pandemi Covid-19. Bioedusiana: Jurnal Pendidikan Biologi, 6(1), 79–87. https://doi.org/10.37058/bioed.v6i1.2930

Riopel, M., Nenciovici, L., Potvin, P., Chastenay, P., Charland, P., Sarrasin, J. B., & Masson, S. (2019). Impact of serious games on science learning achievement compared with more conventional instruction: an overview and a meta-analysis. Studies in Science Education, 55(2), 169-214. https://doi.org/10.1080/03057267.2019.1722420

Sa’dijah, C., Purnomo, H., Abdullah, A. H., Permadi, H., Anwar, L., Cahyowati, E. T. D., & Sa’diyah, M. (2023). Students’ numeracy skills in solving numeracy tasks: Analysis of students of junior high schools. In AIP Conference Proceedings (Vol. 2569, No. 1). AIP Publishing. https://doi.org/10.1063/5.0113664

Schmid, R. F., Borokhovski, E., Bernard, R. M., Pickup, D. I., & Abrami, P. C. (2023). A meta-analysis of online learning, blended learning, the flipped classroom and classroom instruction for pre-service and in-service teachers. Computers and Education Open, 5(January), 100142. https://doi.org/10.1016/j.caeo.2023.100142

Setiawan, A. A., Muhtadi, A., & Hukom, J. (2022). Blended Learning and Student Mathematics Ability in Indonesia: A Meta-Analysis Study. International Journal of Instruction, 15(2), 905-916. https://doi.org/10.29333/iji.2022.15249a

Shapiro, B. (2015). Studying lifeworlds of science learning: A longitudinal study of changing ideas, contexts, and personal orientations in science learning. Canadian Journal of Science, Mathematics and Technology Education, 4(1), 127–147. https://doi.org/10.1080/14926150409556601

Smith, L. A., Oakden-rayner, L., Bird, A., Zeng, M., To, M., Mukherjee, S., & Palmer, L. J. (2023). Articles Machine learning and deep learning predictive models for long-term prognosis in patients with chronic obstructive pulmonary disease : a systematic review and meta-analysis. The Lancet Digital Health, 5(12), e872–e881. https://doi.org/10.1016/S2589-7500(23)00177-2

Suendarti, M., & Virgana, V. (2022). Elevating natural science learning achievement: Cooperative learning and learning interest. Journal of Education and Learning (EduLearn), 16(1), 114–120. https://doi.org/10.11591/edulearn.v16i1.20419

Sugih, S. N., Maula, L. H., & Nurmeta, I. K. (2023). Implementasi kurikulum merdeka dalam pembelajaran IPAS di sekolah dasar. Jurnal Pendidikan Dasar Flobamorata, 4(2), 599-603. https://doi.org/10.51494/jpdf.v4i2.952

Sumiantari, N. L. E., Suardana, I. N., & Selamet, K. (2019). Pengaruh Model Problem Based Learning Terhadap Kemampuan Pemecahan Masalah Ipa Siswa Kelas VIII SMP Universitas Pendidikan Ganesha. JPPSI: Jurnal Pendidikan Dan Pembelajaran Sains Indonesia, 2(April), 12–22. https://doi.org/10.23887/jppsi.v2i1.17219

Suprapto, E., Sumiharsono, R., & Ramadhan, S. (2020). The Analysis of Instrument Quality to Measure the Students ’ Higher Order Thinking Skill in Physics Learning. Journal of Turkish Science Education, 17(4), 520–527. https://doi.org/10.36681/tused.2020.42

Suryanto, S. (2016). Developing An Assessment Instrument Of Junior High School Students’ Higher Order Thinking Skills In MathematicS. Research and Evaluation in Education, 2(1), 92–107. http://dx.doi.org/10.21831/reid.v2i1.8268

Suryono, W., Haryanto, B. B., Santosa, T. A., Suharyat, Y., & Sappaile, B. I. (2023). The Effect of The Blended Learning Model on Student Critical Thinking Skill : Meta-analysis. Edumaspul - Jurnal Pendidikan, 7(1), 1386–1397. https://doi.org/10.33487/edumaspul.v7i1.6087

Tamur, M., Juandi, D., & Kusumah, Y. S. (2020). The Effectiveness of the Application of Mathematical Software in. International Journal of Instruction, 13(4), 867–884. https://doi.org/10.29333/iji.2020.13453a

Tanujaya, B. (2016). Development of an Instrument to Measure Higher Order Thinking Skills in Senior High School Mathematics Instruction. Journal of Education and Practice, 7(21), 144–148. Retrieved from https://files.eric.ed.gov/fulltext/EJ1109390.pdf

Uluçinar, U. (2023). The Effect of Problem-Based Learning in Science Education on Academic Achievement: A Meta-Analytical Study. Science Education International, 34(2), 72–85. https://doi.org/10.33828/sei.v34.i2.1

Utama, C. (2020). The Instrument Development to Measure Higher-Order Thinking Skills for Pre-Service Biology Teacher. International Journal of Instruction, 13(4), 833–848.

Utomo, W., Suryono, W., Santosa, T. A., & Agustina, I. (2023). The Effect of STEAM-Based Hybrid Based Learning Model on Students’ Critical Thinking Skills. Jurnal Penelitian Pendidikan IPA, 9(9), 742–750. https://doi.org/10.29303/jppipa.v9i9.5147

Walid, A., Sajidan, S., Ramli, M., & Gamal, R. (2019). Construction of The Assessment Concept to Measure Students’ High Order Thinking Skills. Journal for the Education of Gifted Young, 7(June), 237–251. https://doi.org/10.17478/jegys.528180

Wang, Y. L., & Tsai, C. C. (2019). Exploring the structure of science learning self-efficacy: the role of science learning hardiness and perceived responses to capitalization attempts among Taiwanese junior high school students. Research in Science and Technological Education, 37(1), 54–70. https://doi.org/10.1080/02635143.2018.1480476

Wilson, D. M., & Narasuman, S. (2020). Investigating Teachers’ Implementation and Strategies on Higher Order Thinking Skills in School Based Assessment Instruments. Asian Journal of University Education (AJUE), 16(1), 1–15. https://doi.org/10.24191/ajue.v16i1.8991

Yakob, M., Hamdani, H., Sari, R. P., Haji, A. G., & Nahadi, N. (2021). Implementation of performance assessment in stem-based science learning to improve students’ habits of mind. International Journal of Evaluation and Research in Education, 10(2), 624–631. https://doi.org/10.11591/ijere.v10i2.21084

Yustiana, Mahadi, I., & Ariska, D. (2022). The Effect of E-Learning Based on the Problem-Based Learning Model on Students’ Creative Thinking Skills During the Covid-19 Pandemic. International Journal of Instruction, 15(2), 329–348. https://doi.org/10.29333/iji.2022.15219a

Zulfiana, S., Gunamantha, I. M., & Putrayasa, I. B. (2023). Pengembangan Instrumen Kemampuan Berpikir Tingkat Tinggi Dan Literasi Sains Pada Pembelajaran IPA Kelas V SD. PENDASI Jurnal Pendidikan Dasar Indonesia, 7(1), 13-24. https://doi.org/10.23887/jurnal_pendas.v7i1.1384

Zulyusri, Z., Santosa, T. A., Festiyed, F., Yerimadesi, Y., Yohandri, Y., Razak, A., & Sofianora, A. (2023). Effectiveness of STEM Learning Based on Design Thiking in Improving Critical Thinking Skills in Science Learning : A. Jurnal Penelitian Pendidikan IPA, 9(6), 112–119. https://doi.org/10.29303/jppipa.v9i6.3709

Author Biographies

Tomi Apra Santosa, Universitas Negeri Padang

Siska Angreni, Universitas Negeri Padang

Rona Taula Sari, Universitas Negeri Padang

Festiyed, Universitas Negeri Padang

Yerimadesi, Universitas Negeri Padang

Yuni Ahda, Universitas Negeri Padang

Heffi Alberida, Universitas Negeri Padang

Fitri Arsih, Universitas Negeri Padang

License

Copyright (c) 2024 Tomi Apra Santosa, Siska Angreni, Rona Taula Sari, Festiyed, Yerimadesi, Yuni Ahda, Heffi Alberida, Fitri Arsih

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with Jurnal Penelitian Pendidikan IPA, agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC-BY License). This license allows authors to use all articles, data sets, graphics, and appendices in data mining applications, search engines, web sites, blogs, and other platforms by providing an appropriate reference. The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in Jurnal Penelitian Pendidikan IPA.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).