Development and Implementation of a Custom GPT Model for Assessing Critical Thinking Skills in Higher Education
DOI:
10.29303/jppipa.v11i8.12067Published:
2025-08-25Downloads
Abstract
This study developed and evaluated a Custom GPT model designed specifically to assess the critical-thinking skills of higher-education students, focusing on synthesis (C4), evaluation (C5), and creation (C6) based on Bloom’s Taxonomy. Employing a design-based research (DBR) methodology, six iterative design cycles were systematically conducted, involving direct classroom observations, expert validations, and practical testing with fourteen mathematics education students. Findings demonstrated that the Custom GPT system is highly practical regarding ease of use (86.7%) and efficiency (85.1%), and practically engaging for learners (81.8%). Quantitative analyses showed significant improvements in students’ critical-thinking skills, particularly within synthesis and evaluation domains, although variations in creation domain performance indicated the need for incorporating more real-world, application-based scenarios. Qualitative feedback confirmed that real-time AI-driven feedback significantly enhanced students' self-reflection and learning strategies. This study contributes theoretically by integrating AI with Bloom’s Taxonomy-based assessment and offers practical implications by providing a scalable, objective, and innovative assessment approach for enhancing higher-order thinking in higher education contexts.
Keywords:
Artificial Intelligence Bloom’s taxonomy Critical thinking Custom GPTReferences
Akbar, S. (2013). Instrumen Perangkat Pembelajaran. Bandung: PT Remaja Rosdakarya.
Andiopenta. (2024). Use of Artificial Intelligence (AI) in Writing Scientific Works. Jurnal Penelitian Pendidikan IPA, 10(12), 10196–203. https://doi.org/10.29303/jppipa.v10i12.9414
Anisah, D. C., Nasrudin, H., & Sukarmin. (2025). Profile of Critical Thinking Skills of Phase F Learning in Chemistry Subjects Acid-Base Material. Jurnal Penelitian Pendidikan IPA, 11(1), 17–27. https://doi.org/10.29303/jppipa.v11i1.9842
Bitzenbauer, P. (2023). ChatGPT in Physics Education: A Pilot Study on Easy-to-Implement Activities. Contemporary Educational Technology, 15(3). https://doi.org/10.30935/cedtech/13176
Cahyani, C. P. N., & Setiawan, E. P. (2024). How Motivation Mediated Science Achievement of Indonesian Students: A Path Analysis on PISA 2018 Data. Jurnal Penelitian Pendidikan IPA, 10(12), 10493–501. https://doi.org/10.29303/jppipa.v10i12.10012
Elen, J., & Verburgh, A. (2023). Fostering Critical Thinking: Features of Powerful Learning Environments. European Journal of Education, 58(3), 434–46. https://doi.org/10.1111/ejed.12568
Essien, A., Bukoye, O. T., O’Dea, X., & Kremantzis, M. (2024). The Influence of AI Text Generators on Critical Thinking Skills in UK Business Schools. Studies in Higher Education, 49(5), 865–82. https://doi.org/10.1080/03075079.2024.2316881
Hoadley, C., & Campos, F. C. (2022). Design-Based Research: What It Is and Why It Matters to Studying Online Learning. Educational Psychologist, 57(3), 207–20. https://doi.org/10.1080/00461520.2022.2079128
Hopfenbeck, T. N., Zhang, Z., Sun, S. Z., Robertson, P., & McGrane, J. A. (2023). Challenges and Opportunities for Classroom-Based Formative Assessment and AI: A Perspective Article. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1270700
Indrašienė, V., Jegelevičienė, V., Merfeldaitė, O., Penkauskienė, D., Pivorienė, J., Railienė, A., & Sadauskas, J. (2023). Value of Critical Thinking in the Labour Market: Variations in Employers’ and Employees’ Views. Social Sciences, 12(4). https://doi.org/10.3390/socsci12040221
Jelica, S., Miroslav, M., & Uroš, Š. (2023). Possible applications of ChatGPT in higher education. International Conference on Technological Ecosystems for Enhancing Multiculturality, 135–144. https://doi.org/10.1007/978-981-97-1814-6_13
Kadrija, R., Shatri, Z. G., & Këndusi, V. V. (2022). Effects of Critical Thinking Implementation on Enhancing of Teaching Quality. Journal of Educational and Social Research, 12(6), 236–45. https://doi.org/10.36941/jesr-2022-0159
Kofahi, A., Mohammed, & Husain, A. (2025). ChatGPT for Operating Systems: Higher-Order Thinking in Focus. Journal of Information Technology Education: Research, 24(001). https://doi.org/10.28945/5382
Lee, H.-Y., Chen, P.-H., Wang, W.-S., Huang, Y.-M., & Wu, T.-T. (2024). Empowering ChatGPT with Guidance Mechanism in Blended Learning: Effect of Self-Regulated Learning, Higher-Order Thinking Skills, and Knowledge Construction. International Journal of Educational Technology in Higher Education, 21(1). https://doi.org/10.1186/s41239-024-00447-4
López, F., Contreras, M., Nussbaum, M., Paredes, R., Gelerstein, D., Alvares, D., & Chiuminatto, P. (2023). Developing Critical Thinking in Technical and Vocational Education and Training. Education Sciences, 13(6). https://doi.org/10.3390/educsci13060590
Naznin, K., Al Mahmud, A., Nguyen, M. T., & Chua, C. (2025). ChatGPT integration in higher education for personalized learning, academic writing, and coding tasks: a systematic review. Computers, 14(2), 53. https://doi.org/10.3390/computers14020053
OECD. (2024). PISA 2022 Results (Volume I): The State of Learning and Equity in Education. In Perfiles Educativos (Vol. 46, Issue 183). OECD Publishing. https://doi.org/10.22201/iisue.24486167e.2024.183.61714
Surin, S., & Damrongpanit, S. (2024). Developing Students’ Critical Thinking: Examining the Influence of Learning Management Approaches Through Meta-Analysis and Propensity Score Matching. European Journal of Educational Research, 13(3), 1391–1409. https://doi.org/10.12973/eu-jer.13.3.1391
Swindell, A., Greeley, L., Farag, A., & Verdone, B. (2024). Against Artificial Education: Towards an Ethical Framework for Generative Artificial Intelligence (AI) Use in Education. Online Learning, 28(2). https://doi.org/10.24059/olj.v28i2.4438
van den Berg, G., & du Plessis, E. (2023). ChatGPT and Generative AI: Possibilities for Its Contribution to Lesson Planning, Critical Thinking and Openness in Teacher Education. Education Sciences, 13(10). https://doi.org/10.3390/educsci13100998
Vygotsky, L. S., & Cole, M. (1978). Mind in Society: Development of Higher Psychological Processes. Harvard University Press.
Wang, W.-S., Lin, C.-J., Lee, H.-Y., Huang, Y.-M., & Wu, T.-T. (2025). Enhancing self-regulated learning and higher-order thinking skills in virtual reality: the impact of ChatGPT-integrated feedback aids. Education and Information Technologies, 1–27. https://doi.org/10.1007/s10639-025-13557-x
Wu, T.-T., Lee, H.-Y., Chen, P.-H., Lin, C.-J., & Huang, Y.-M. (2025). Integrating peer assessment cycle into ChatGPT for STEM education: A randomised controlled trial on knowledge, skills, and attitudes enhancement. Journal of Computer Assisted Learning, 41(1), 13085. https://doi.org/10.1111/jcal.13085
Xia, Q., Weng, X., Ouyang, F., Lin, T. J., & Chiu, T. K. F. (2024). A Scoping Review on How Generative Artificial Intelligence Transforms Assessment in Higher Education. International Journal of Educational Technology in Higher Education, 21(1). https://doi.org/10.1186/s41239-024-00468-z.
Xiaolei, S., & Teng, M. F. (2024). Three-Wave Cross-Lagged Model on the Correlations between Critical Thinking Skills, Self-Directed Learning Competency and AI-Assisted Writing. Thinking Skills and Creativity, 52. https://doi.org/10.1016/j.tsc.2024.101524
License
Copyright (c) 2025 Muhammad Fathurrahman, Faisal Eka Mahendra, Karim Paul Aronggear, Muhammad Syahrul Kahar, Teguh Hidayat Iskandar Alam
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with Jurnal Penelitian Pendidikan IPA, agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC-BY License). This license allows authors to use all articles, data sets, graphics, and appendices in data mining applications, search engines, web sites, blogs, and other platforms by providing an appropriate reference. The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in Jurnal Penelitian Pendidikan IPA.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
