Progress in International Reading Literacy Study-Based Reading Competency Assessment Instrument with Culturally Educative Texts for Primary School Students

: The objectives of this research and development are (1) to produce a product prototype of the PIRLS equivalent reading literacy competency assessment instrument with educative cultural content text for grade 6 elementary school students, (2) to test the PIRLS equivalent reading literacy competency assessment instrument with educative cultural content text for grade 6 elementary school students from the aspects of content validity, construct validity, Esay question reliability test, Objective question reliability test, difficulty level, and differentiability. This type of research and development uses the Borg & Gall model which consists of 10 stages. The data collected is qualitative and quantitative data. The data is related to content validity, construct validity, readability, and practicality of the assessment instrument as well as the practicality of the user manual. The collected data were analyzed with content validity (Gregory), Product Moment, and Cronbach's Alpha. Based on the results of the analysis and discussion, the results of the correlation person analysis for the overall r value of 30 items > r table for subject 28 the r table value is 0.361 with the acquisition of the smallest value of 0.383 and the largest of 0.724. In addition to the results of the limited subject test, the results of the analysis of the broad subject test using SPSS for windows version 26 software also showed the results that the person correlation value for the reading literacy competency assessment instrument from a total of 30 items obtained a calculated r value> r table for subjects 141 people the r table value of 0.361 with the acquisition of the smallest value of 0.439 and the largest value of 0.896. While the acquisition of the teacher response score is 0.882. So, based on these results it can be said that the product of the reading literacy competency assessment instrument equivalent to the Progress in International Reading Literacy Study with educational cultural content text is suitable for direct use for every grade 6 elementary school student.


Introduction
Literacy competencies play a central role in shaping the intellect and development of learners at the basic education level (Dorn & Soffos, 2023;Ng et al., 2021).Literacy competencies are very important in today's information and knowledge era because they are the main gateway to deeper knowledge and understanding (Tugtekin & Koc, 2020) and are an important foundation for providing skills and knowledge to the younger generation (Ismail et al., 2021).Literacy competence is an important indicator needed by the younger generation to improve and foster 21st century skills (Joynes et al., 2019), which require them to overcome the challenges of the times, improve the quality of life, become competitive, and develop national character (Indriyani et al., 2019).Therefore, literacy competencies must be applied and trained from an early age to build a smart, literate and cultured young generation (Flewitt et al., 2015;Lähdesmäki & Maunula, 2022).
Students' reading literacy competence has been proven that Indonesia has participated in the Progress in International Reading Literacy Study (PIRLS) in 2016 published by the International Association for the Evaluation of Educational Achievement (IEA) (Mulyani, 2021), which shows that Indonesia is 46th out of 50 countries that participated (Ismail et al., 2021).Not only that, Indonesia has also participated in the Program for International Student Assessment (PISA), the results of which show that students are still very weak in reading (Harsiati, 2018).According to the Tentor Anak Bangsa Team (2020), the results of the PISA study released by the OECD showed that Indonesian students ranked 72nd out of 77 countries, or the bottom 10 countries in terms of reading literacy (Koyuncu & Fırat, 2020).In addition, 30% of Indonesian students achieved reading proficiency at level 2 with an OECD average of 77%, namely in the ability to find one or two pieces of information needed to make inferences or to deal with several conditions, being able to find main ideas, understand relationships, construct meaning for inference (Van Ammel, 2021).
The analysis of low reading literacy competence is also supported by the situation observed (Hidayah et al., 2019) by researchers from several schools around Malang City.Observations show that students in elementary schools do not have adequate literacy skills or interest in reading.The implementation of reading activities in Indonesian language learning that has been carried out is not in accordance with the stages of reading such as pre-reading, reading process, and postreading.Before reading, there should be a pre-reading stage carried out.The purpose of this stage is to direct learners' attention to activate the schemata related to the topic or theme to be read.Next is the reading process phase, where learners are trained to understand the content or information of the text.At this stage of the reading process, the researcher found that many learners still faced difficulties in finding explicit and implied information, determining the message conveyed by the author, and interpreting values during this reading process.Furthermore, after reading, the post-reading stage is the assessment.The assessment should be measured using a standardized and practical reading literacy competency assessment instrument.
In addition, other information collected by researchers includes reading literacy assessment questions are still lacking in assessing students' reading skills.The questions still look monotonous; the amount of text given is still too long; it is not appropriate for the level or level of learners at the elementary school level, especially for grades 5 and 6 at level 3, and the questions rely more on the memory aspect.Given that grade 6 in primary school is a pivotal point where students receive more complex, significant and contextualized learning (Yohana et al., 2019).The challenges of accurately and relevantly measuring reading literacy skills should, therefore, be more complex, especially when inspiring local educational and cultural values are incorporated into the learning context (Mufidah et al., 2023).
Thus, as a teacher or educational practitioner, it must be responsible for developing PIRLS-equivalent reading literacy competency questions that are appropriate to the level of learners, taking into account critical thinking skills.By using the International Research on Reading Literacy and Comprehension (PIRLS) guidelines and standards, teachers can create more contextualized, appropriate and accurate reading literacy competency evaluation instruments to measure their learners' ability in reading at the primary school level.The assessment instrument should be linked to Indonesia's current learning conditions and culture.Learners in the Progress in International Reading Literacy Study survey were given tests of various genres.The results are aggregated into two reading categories, namely literary reading and informational reading.This analysis is done to differentiate learners who experience obstacles in reading literacy development and learners who are ready for conventional reading literacy learning (Winarni et al., 2020).
Thus, through the meticulous steps in the development of this instrument, it is hoped that this research will make a real contribution to efforts to improve more meaningful and contextualized reading literacy learning in grade 6 elementary schools.As such, this research has the potential to open new avenues in the development of assessment instruments that are more inclusive, diverse and relevant to the cultural and educational realities in Indonesia.In a broader context, this study also has the potential to provide valuable guidance for the development of similar assessment instruments at higher education levels.(Firman & Mirnawati, 2023;Gustiani, 2019).The selection of the Borg & Gall development model is because it is adjusted to the development objectives, namely to produce a prototype product in the form of a PIRLS equivalent reading literacy competency assessment instrument using educative culturally charged text that is feasible, valid, and effective, and can be used by grade 6 students in elementary schools.

Method
The first stage, conducting a preliminary study, conducted observations and interviews (Schoch, 2020) with Indonesian language teachers and students in grade 6 elementary school.The task given to teachers and students was to fill in a needs analysis questionnaire.The information data from the observations and interviews were used by researchers as the basis for developing the reading literacy competency assessment instrument.In addition, a literature study was conducted with the aim of reviewing various kinds of sources relevant to the research.The second stage is planning, namely analyzing the curriculum, then formulating indicators of reading literacy assessment assesment equivalent to PIRLS.After that, the researcher made an instrument framework design, and determined the content of the parts of the assessment instrument that would be developed according to the indicators.At this stage, the researcher determines the form of the product, the content of the product, and the form of the product.
The third stage is mapping the instrument product, mapping the educative cultural values used as a question stimulus, compiling the instrument lattice, then compiling the PIRLS equivalent reading literacy competency assessment instrument product with educative cultural content text.The activities of preparing the lattice include developing a matrix of reading literacy competency indicators, question indicators, cognitive levels, question context (educative cultural content), question numbers, and question forms.Furthermore, researchers developed reading literacy assessment instrument items equivalent to Progress in International Reading Literacy Study to measure reading literacy.Then compile the assessment rubric, and construct the assessment instrument and assessment rubric on digital applications, and books.The fourth stage is the expert team validation test, namely reading literacy assessment experts and language experts.The design of the assessment instrument that has passed the expert test stage will then be revised according to the direction of the expert team, then tested on students through a preliminary trial.
The fifth stage is revising the main product, namely revising the validated reading literacy competency assessment instrument based on suggestions for improvement and validation from a team of reading literacy assessment experts and linguists.This main product revision will result in new product improvements that are ready for a series of further tests.The sixth stage is the main trial with limited subjects, namely 28 students in elementary schools.The results of the trial were then analyzed, calculated the content validity, construct validation of the reliability of the description questions, the difficulty index, and the differentiating power which is a requirement that the test questions are feasible and standardized.The seventh stage is the revision of operational products after two trials.This is done to overcome new obstacles that have not been thought of.
The eighth stage was the Operational Trial.The product is ready to be tested on a wide range of subjects, namely 140 students in grade 6 elementary school, after going through 2 trials and two revisions.The ninth stage is the final product revision.At this stage, researchers made final revisions, after obtaining notes from the results of validation by reading literacy assessment experts, linguists, practitioners, and students.This final product revision is to improve things that are not good results, such as the weaknesses and shortcomings of the assessment instrument based on expert tests and field tests.After that the product is published to the target or users.The last stage is Dissemination and Implementation.In this last stage, researchers report the results in scientific forums through seminars and publish them in scientific journals, books, and Intellectual Property Rights (HKI).
Expert validation subjects involved one reading literacy assessment lecturer and one linguist lecturer.Field subjects involved two Indonesian language teachers in grades 5 and 6 of Sabilillah Islamic Elementary School of Malang, as well as 140 grade 6 students.The types of data generated are qualitative and quantitative.Qualitative data is in the form of statements containing suggestions or comments from material experts, language, and practitioners.Quantitative data is in the form of assessment results obtained from questionnaires and observation sheets.While the instruments used include validation questionnaires, assessment sheets, student and teacher response questionnaires.The validation questionnaire was given to reading literacy assessment experts and language experts.The aspects assessed were content validity, construct validity, language, teacher instructions, and readability and practicality of the product.The assessment sheets used were evaluation test, attitude questionnaire, skill questionnaire.Student response questionnaires were given during the field test.The aspect assessed is the attractiveness of the product.The teacher response questionnaire was given during the field test.The aspects assessed were the clarity of the student book and teacher's manual.
Data analysis techniques consisted of qualitative and quantitative descriptive analysis.Qualitative data includes three stages, namely data reduction; data presentation; and conclusion (Alam, 2021;Hennink et al., 2020).Data in the form of criticisms, suggestions, and comments from reading literacy and assessment experts, language experts, and practitioners collected were reduced by researchers by selecting suggestions, criticisms, and comments that were in accordance with the theoretical framework and adjusted to the development of assessment instruments.Then the data that has been reduced is presented in tabular form, after which it formulates conclusions, and is described to make it clearer and easier to understand.While quantitative data in this development research is obtained from score data from content validation test results, construct validity, empirical validity, reliability test of description questions, reliability test of objective questions, test level of difficulty, and differentiability of questions.

Product Description
The assessment instrument developed is a Level 3 reading literacy competency assessment book for grade 5 and 6 elementary school students, a teacher's guidebook, and a student's guidebook.This assessment instrument was developed as an effort to support the improvement of reading literacy at the primary level.This book is specifically designed to assist teachers at the primary level in measuring and understanding students' reading literacy levels so that they can identify areas for improvement and design more effective learning strategies.In addition, this book is also designed to measure students' reading literacy competencies at a level that is aligned with the Progress in International Reading Literacy Study, which is an international standard in reading literacy assessment.By having quality assessment instruments, it is expected to prepare students to face future literacy demands.
The systematics of the instrument product consists of a preface, table of contents, an introduction that briefly explains the Assessment of reading literacy competence and Progress in International Reading Literacy Study, the type of text consists of 50% literary text and 50% informational text.The text stimulus used is text that contains educative cultural values such as religious values, moral values, social values, and cultural values.The form of questions consists of 20% multiple choice questions, 40% complex multiplechoice questions, 10% matching questions, 5% short fill questions, and 25% description questions whose question weights are adjusted to the Progress in International Reading Literacy Study level, besides that there are successful strategies for working on assessment questions, equipped with practice questions and discussions, 2 packages of simulation questions, and a list of references.
This reading literacy competency assessment instrument book was developed based on the concept of reading taxonomy proposed by Barret (Amalya et al., 2020), with reference to the reading constructs measured by the Progress in International Reading Literacy Study.Barret's reading taxonomy can help in selecting various types of texts that reflect the level of difficulty and characteristics relevant to what is tested by the Progress in International Reading Literacy Study.From the two taxonomies, further derived into sub competencies and indicators by paying attention to the theory on relevant reading literacy competencies used by researchers in developing reading literacy competency assessments equivalent to Progress in International Reading Literacy Study.This reading literacy competency assessment instrument consists of three parts, namely the initial, core and final parts.The following is an image of the book cover of the Progress in International Reading Study (PIRLS) equivalent reading literacy competency assessment instrument using educative cultural texts packaged in the form of student books.

Figure 1. Student Book
In addition to the student book, the prototype is also in the form of a Progress in International Reading Literacy Study equivalent reading literacy competency assessment instrument guidebook for teachers and students (Lan & Yu, 2022).The content systematics of the test guidebook for teachers consists of a preface, table of contents, introduction to the reading literacy competency assessment instrument, instrument features, grids, assessment implementation, instrument features on E-Learning, instructions for opening the instrument on E-Learning, instructions for operating the instrument on E-Learning, instructions for opening 681 links, instructions for using the scoring rubric, and closing.This manual is designed to provide a clear and comprehensive guide for teachers to manage and implement this assessment instrument effectively and appropriately.The following guidebook for the Progress in International Reading Literacy Study equivalent reading literacy competency assessment instrument with educative cultural content text for teachers is presented in Figure 2.

Figure 2. Teacher's Handbook
While the systematic content of the test manual for students consists of a preface, table of contents, introduction to the reading literacy competency assessment instrument, instrument features, grids, assessment implementation, instrument features on E-Learning, instructions for opening the instrument on E-Learning, instructions for operating the instrument on E-Lerning, instructions for opening links, and closing.This student manual is designed to guide students through the PIRLS assessment instrument process, explaining what is being tested, how to answer the questions, and how students can best prepare.By understanding this instrument, students are expected to be more confident and ready to face the reading literacy competency test, so that it will bring proud results.The following section of the guidebook for the PIRLS equivalent reading literacy competency assessment instrument with educative cultural content text for students is presented in Figure 3.

Presentation of Data Content, construct, and language validity
After the product development of the reading literacy competency assessment instrument, the next step is the validation stage from assessment experts and language experts, as well as psychology experts.Aspects resulting from validation are content validity, construct validity, language and readability, teacher's guide, and student's guide.The purpose of the validation is to determine and measure whether the reading literacy competency assessment questions used are in accordance with the reading competencies and indicators on the Progress in International Reading Literacy Study construct and the cognitive level achieved.
The value of content validity, construct validity, linguistic and readability, teacher's guide, and student's guide or content validity (CV) of the instrument developed was then analyzed using the Gregory formula based on the assessment of 2 experts.The results of validation by two expert validators/assessment experts and language experts, as well as psychology experts can be seen in Table 1.
Based on the assessment results of the two validators, the reading literacy competency assessment instrument equivalent to Progress in International Reading Literacy Study is categorized as having a very high validity of 0.88.Gregory (2000) states that instrument products can be declared to have a level of "very high validity" if they obtain a calculation value of content validity > 0.79 or a range of values between 0.80 -1.Despite obtaining a high validity value, during the validation process, researchers received many suggestions and inputs on the preparation of the reading literacy competency assessment test questions, so that the test questions tested were good and very valid.The content validity, construct validity, linguistic and readability tests, teacher guides, and student guides are to determine the validity and feasibility (Hidayat et al., 2020;Sarmi et al., 2020) of the Progress in International Reading Literacy Study equivalent reading literacy competency assessment instrument products with educational cultural content texts for grade 6 elementary school students.The validation test conducted by assessment and language experts, as well as psychology experts, contained several suggestions that had to be revised so that they met the criteria for a valid assessment instrument.The suggestions given by the validators are changing the order of the texts, starting from literary texts, then informational texts, correcting the writing of wrong words and adjusting to the correct Indonesian language rules, changing the length of alternative answers to questions number 2, 14, 21, and 26 in package 1, and alternative answers to questions number 28 and 29 in package 2, changing the stimulus for questions number 1 to number 4 in package 2, and changing some alternative answers in the form of, for example, pictures with an attractive appearance.The five suggestions given need to be considered to develop a reading literacy competency assessment instrument equivalent to the Progress in International Reading Literacy Study, it is in accordance with the opinion of Damaianti et al. (2020) and Suma (2020) that the development of reading literacy competency assessment instruments needs to pay attention to the design in terms of letterforms, presentation of material, and completeness of teacher guides and student guides to clarify the expected information.

Empirical Validity
This empirical validity data analysis activity is divided into 2 parts, namely analyzing limited subject data totaling 28 people and analyzing broad subject data with a total of 141 people.The data from the correlation person analysis using SPSS for windows version 26 software can be seen in the following table.Based on the data in the Table 2, it can be seen that the results of the correlation person analysis for the overall r count value of 30 items > r table for subject 28 the r table value is 0.361 with the acquisition of the smallest value of 0.383 and the largest is 0.724.In addition to the results of the limited subject test, the results of the analysis of the broad subject test using SPSS for windows version 26 software also showed the results that the person correlation value for the minimum competency assessment instrument (AKM) reading literacy level 2 of the whole 30 items obtained a r count > r table value for 35 subjects, the r table value was 0.361 with the acquisition of the smallest value of 0.439 and the largest value of 0.896, so we can conclude that all questions of the Progress in International Reading Literacy Study equivalent reading literacy competency assessment instrument using educational cultural texts are valid in terms of limited and broad subject trials.
An instrument is said to be valid if the instrument is able to precisely measure what is to be measured.In other words, validity is related to "accuracy" with measuring instruments (Yati et al., 2023).A valid instrument will produce valid data as well.It can also be said that if the data generated from an instrument is valid, then the instrument is also valid (Pratiwiningtyas et al., 2017).This statement is in line with opinion which states that a valid instrument must have internal validity and external validity values.The internal validity of the instrument must meet construct validity and content validity, while for external validity is the empirical value of the instrument which when used everywhere will obtain valid data results.
Instruments that have been analyzed from the aspects of content and empirical validity then need to know how much the reliability value is.Calculations carried out using SPSS software version 26 obtained the results of the reliability value using the Cronbach Alpha formula shown in the table below.

Tabel 3. Reliability Calculation Results with Cronbach's Alpha
Cronbach's Alpha N of Items 0.725 30 Based on the results of the reliability test analysis assisted by SPSS software version 26, the reliability level of the reading literacy competency assessment instrument equivalent to Progress in International Reading Literacy Study with educative cultural content text is 0.725.The data value of the results of this calculation shows that the instrument is declared reliable because it provides a Cronbach Alpha value (α) > 0.60 and the data from the analysis is included in the classification of "Very high reliability".The acquisition of this high reliability value shows that the reading literacy competency assessment instrument equivalent to Progress in International Reading Literacy with educational cultural content text has an internal consistency coefficient for each item of the same type that has a value of confidence in its measurement is steady (Otaya et al., 2020).This is in accordance with the opinion of which states that a reliable instrument will provide stable and consistent measurement results.
In addition to the effectiveness test conducted by students, the reliability of effectiveness was also tested through a teacher response questionnaire.This was done to determine the clarity and ease of the product.The suggestions given by the teacher were to provide additional time, and reduce the number of words in the stimulus.The data from the practitioner validation results are presented in the following table.The teacher response score is 0.882 which shows very feasible results.The results of the teacher response score can be said that the product of the Progress in International Reading Literacy Study equivalent reading literacy competency assessment instrument with educational cultural content text is feasible to be used directly for every grade 6 elementary school student.

Conclusion
Based on the results of the analysis and discussion, it can be concluded that the reading literacy competency assessment instrument equivalent to the Progress in International Reading Literacy Study with educative cultural content text developed is an instrument that is suitable for use as an assessment of reading literacy competence and has good quality in terms of content validity, construct validity, readability and language.The instrument is a valid and reliable instrument.Based on the assessment results from both validators, the reading literacy competency assessment instrument equivalent to Progress in International Reading Literacy Study with educative cultural content text is categorized as having very high validity, namely 0.88.While the results of the correlation person analysis for the overall r count value of 30 items > r table for subject 28 the r table value is 0.361 with the acquisition of the smallest value of 0.383 and the largest of 0.724.In addition to the results of the limited subject test, the results of the analysis of the broad subject test using SPSS for windows version 26 software also showed the results that the person correlation value for the reading literacy competency assessment instrument from a total of 30 items obtained a value of r count > r table for 35 subjects, the r table value was 0.361 with the acquisition of the smallest value of 0.439 and the largest value of 0.896.The teacher response score was 0.882.Thus, the results of the teacher response score can be said that the product of the Progress in International Reading Literacy Study equivalent reading literacy competency assessment instrument with educational cultural content text is suitable for direct use for every grade 6 elementary school student.

Table 1 .
Expert Validation Results with Gregory Test using Likert scale

Table 2 .
Results of Empirical Validity Test Calculations using SPSS

Table 4 .
Practitioner Validation Results