Introduction
Reading comprehension is a complex process in which readers must make multiple inferences, add prior information to what is read, and integrate new information with prior knowledge [
1]. This skill is vital not only for understanding the text but also for academic and career success [
2].
Children with developmental dyslexia struggle with aspects of reading difficulties at the word level and decoding [
3, 4]. While poor comprehenders, even though they can read the text accurately, fluently, and according to their age, have difficulty in understanding what they have read. These children have specific reading comprehension deficits (SCD) [
2,
5].
Children who have had speech and language disorders in their preschool years may also experience difficulties with reading, especially comprehension, as they grow older. Studies have shown that at least 50% of these children have varying degrees of reading disorders when they reach school age. Recent research has found that between 22% and 83% of students with reading comprehension difficulties meet the diagnostic criteria for developmental language disorder, but their teachers may not have identified them as having oral language or reading disorders [
6].
Different statistics exist on the prevalence of reading disorders in different societies. In a study conducted on 25 000 sixth-grade students in France, according to the international classification of diseases (ICD), the prevalence of reading disability was 3.5% and according to the diagnostic and statistical manual in its fifth version American Psychiatric Association (APA), 6.6% is determined [
7]. On the other hand, it is estimated that between 10%-15% of 7-8-year-old children who have a normal performance in decoding have a deficit in reading comprehension [
8]. Large-scale identification studies have shown that the prevalence of specific reading comprehension disorders is likely around 8% for children between 9 and 14 years of age. A prevalence rate of 8% means that an average of two students in a classroom can meet the criteria for specific reading comprehension deficits [
9].
Although in Iran, the study of the prevalence of reading disorder has not been conducted at the national level, especially in the field of reading comprehension disorder, local studies are conducted on the prevalence of reading disorder. In a study, 1 200 male students in 6 grades from primary schools in Ahvaz City, Iran were investigated. The results of this research showed that the rate of reading disorder among students was 7%, the highest percentage was calculated in the second grade (17%), and the lowest percentage was calculated in the sixth grade (2%) [
10]
The assessment of reading skills is necessary to diagnose the problem and plan appropriate clinical interventions. The report of the results of the last period of the progress in international reading literacy study test, which was conducted in 2021 on the fourth-grade students of Iranian schools, showed that Iranian students with an average score of 413 performed poorly and below the world average level (have scored 500) and are at the bottom of the table among the 57 participating countries in this International evaluation [
11]. These results emphasize the need to strengthen reading assessment systems in the country to monitor student’s performance and plan for designing suitable educational and intervention protocols.
Currently, many tests and measurement scales are available in different languages to assess reading skills, especially reading comprehension. The Neale analysis of reading ability and Wechsler objective reading dimensions tests are among the most famous reading assessment scales in English, which have subtests to assess reading comprehension [
12, 13]. Also, tests in different languages have been designed to specifically assess reading comprehension. For example, The Suffolk reading scale is a group test to assess reading comprehension in English that uses a Clouse question format [
14]. Evaluación de la comprensión lectora (reading comprehension assessment test) (ECOMPLEC) is also a specific test for evaluating reading comprehension skills made in Spanish [
15].
In Persian language, tests are designed to measure different aspects of reading skills. One of the most famous reading assessment tests in Persian is the “NEMA” reading and dyslexia test, which exists to comprehensively evaluate reading skills in the first to fifth grades. This test has ten subtests, two of which are word comprehension and text comprehension subtests designed to evaluate reading comprehension dimensions. The text comprehension subtest of the NEMA test includes a common text for all grades and two specific texts for each grade, with 7 to 8 multiple-choice questions designed for each text. Cronbach’s α coefficient of this subtest is 0.48 [
16].
Even though the NEMA test is still used as a comprehensive reading assessment scale in the country, it is necessary to design a scale and task with appropriate psychometric characteristics for the specific assessment of reading comprehension skills. Therefore, we designed a set of items under the title of Persian reading comprehension assessment task (PRCAT) to evaluate reading comprehension skills in elementary students. In this regard, to check the validity of students’ grades in this assignment, the grades of this assignment were compared to the obtained grades of the text comprehension subtest of the NEMA test. Therefore, this research was conducted to investigate the correlation between the scores of the PRCAT and the scores of the text comprehension subtest of the reading and dyslexia test.
Materials and Methods
A total of 24 students (9 girls and 15 boys) of the fourth and fifth grades (15 fourth graders and 9 fifth graders) participated in this study. All students were at normal levels in terms of reading skills. All the students were right-handed and their first language was Persian. They were selected by cluster sampling method from schools in three education regions of Tehran City, Iran (regions 5, 7, and 9). These students were randomly selected from among 197 natural students of the fourth and fifth grades of elementary school and were included in the study.
Table 1 summarizes the demographic information of the study participants.
Evaluation criteria
Personal information questionnaire and teacher interview
First, an individual information registration form was completed to ensure the absence of specific vision, hearing, cognitive, and neurological problems for each student, and then, teachers’ reports were received regarding the presence or absence of reading problems in students.
Subtests of word reading, non-word reading, and text comprehension of the “NEMA” test
To detect decoding problems in reading, word and non-word reading subtests of the NEMA test were used. The word reading subtest includes three groups of high-frequency, medium-frequency, and low-frequency words. The subjects should read these words aloud and the examiner should write down their correct and incorrect answers. The non-word reading subtest also includes 40 non-words and is taken from the subject in the same way as word reading. With the difference that non-words, unlike words, are meaningless. Cronbach’s α coefficients reported for the high-frequency word reading subtest are 0.97, medium frequency 0.98, low frequency 0.98, and nonword reading subtest 0.97 [
16].
Also, to evaluate reading comprehension, the text comprehension subtest of the NEMA test was used. The text comprehension subtest includes a common text for all grades and two specific parallel texts for each grade, with 7 to 8 multiple choice questions designed for each text. In the current research, the common text and the specific text (a) have been used for the fourth and fifth grades. Cronbach’s α coefficient of this subtest is 0.48 [
16].
Persian reading comprehension assessment task (PRCAT)
The PRCAT includes 12 texts, each of which contains 3 to 4 four-choice questions. The student should read the texts and then answer the questions. This task has no time limit and the examiner can guide the student to solve decoding errors in reading. The content validity of the items of this task was done by using the opinion of 11 experts (including one psychologist, one linguist, three teachers, and six speech and language pathologists), and the content validity ratio (CVR) was calculated. Then, the items were reviewed based on the study conducted by Ayre and Scally (2014), who considered the minimum value of content validity ratio (CVR) to be 0.63 based on the opinions of 11 experts, and the items that obtained a lower score were removed [
17]. Finally, 12 texts and 40 questions were considered as final items.
Procedure
At first, after obtaining the consent form for the project and completing the information registration form, a short interview was conducted with the teachers about the students, and the teachers’ report was received regarding the presence or absence of reading problems in the students. Then, to detect decoding problems, word reading and non-word reading subtests of the NEMA were used. The students read all the words and non-words aloud and the results were recorded by the examiner. Then, the students who were considered to have “normal reading skills” after these steps, were asked to evaluate the PRCAT and the text comprehension subtest of the NEMA test. For both assessment criteria, students had to read texts and answer four-choice questions related to the text. The tests had no time limit and the students were allowed to refer to the text while answering the questions and read the text again silently. Also, if the students had a problem in decoding the words, they could ask the examiner questions. The examiner was not allowed to help the students with the meaning of the words and should only guide them in decoding the difficult words.
After data collection, the results were corrected. One point was awarded for each correct answer and no points for incorrect answers. The results were analyzed using SPSS software version 24, and Pearson’s correlation coefficient was used to check the correlation between the scores of the two tests.
Results
Table 2 presents the Mean±SD for the scores of the PRCAT and the text comprehension subtest of the NEMA test.
The results showed a significant correlation between the scores of the PRCAT and the text comprehension subtest of the NEMA test. The correlation between the scores of these two tests was calculated using the Pearson correlation coefficient (r=0.622, P<0.01).
Discussion
In this study, the scores obtained from students’ performance in the PRCAT and the text comprehension subtest of the NEMA test were investigated. The correlation coefficient between the results of these two assessments showed a good relationship between the scores obtained and the PRCAT can provide reliable scores about students’ reading skills.
Today, checking and comparing the scores of tests and evaluation scales with each other and the degree of correlation between them is not done just to obtain the criterion validity index; rather, the purpose is to check and compare the accuracy of the tests in diagnosing reading problems. Keenan et al. conducted four English oral reading tests, including the Gray oral reading test–3, the qualitative reading inventory–3, the Woodcock-Johnson passage comprehension, and the Peabody individual achievement test on students to evaluate reading comprehension problems. Although the students who had lower performance were expected to perform poorly on all tests, the researchers found that the average overlap between tests in identifying reading comprehension difficulties was only 43% [
18]. This outcome suggests that tests and assessment criteria that measure similar skills may provide different diagnostic results. This result emphasizes the necessity of reviewing and comparing tests to ensure accuracy and consistency in the evaluation process.
In addition to the present study, in some reading tests in Iran, the correlation between their scores and the scores of the NEMA test has been calculated. In Heidari et al.’s study, the correlation results between the comprehensive diagnostic test of dyslexia and the NEMA test were estimated to be 0.56 [
19]. This study was conducted to investigate the psychometric characteristics of the comprehensive diagnostic test of dyslexia.
This study has established the validity of the PRCAT based on its correlation with NEMA test scores. This is a crucial step toward creating comprehensive comparisons between diagnostic and assessment scales. The goal of future studies is a comprehensive and detailed analysis of the results of various reading tests in the Persian language, which can help to increase the accuracy of assessment and diagnosis, as well as to create more valid scales.
Conclusion
The results of the present study showed that an acceptable correlation is observed between the scores obtained in the Persian reading comprehension assessment task with the text comprehension subtest of the NEMA test. This result indicated that the PRCAT has good validity compared to the text comprehension subtest of the NEMA test. This preliminary research can help future studies with a larger sample size and more details.
Ethical Considerations
Compliance with ethical guidelines
This study was approved by the Ethics Committee of Iran University of Medical Sciences (Code: IR.IUMS.REC.1401.240).
Funding
The present article was extracted from the research project of Mohammad Hossein Hamdollahi approved by Department of Speech Therapy, School of Rehabilitation Science, Iran University of Medical Sciences.
Authors' contributions
All authors equally contributed to preparing this article.
Conflict of interest
The authors declared no conflict of interest.
Acknowledgments
The authors thank and appreciate all the teachers and administrators of the Tehran Education Department for their cooperation in issuing sampling permits. Also, the team extends its heartfelt gratitude to all the individuals who have supported us throughout their research. We express our particular appreciation to Zahra Lotfi, Sara Abbasi, Mahdiyar Nouri, Mohsen Pasha, and Mohammad Mahdi Hamdollahi for their valuable contributions.
References
- Ozuru Y, Dempsey K, McNamara D. Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learn Instruction. 2009; 19(3):228-42. [DOI:10.1016/j.learninstruc.2008.04.003]
- Spencer M, Wagner RK. The comprehension problems of children with poor reading comprehension despite adequate decoding: A meta-analysis. Rev Educ Res. 2018; 88(3):366-400. [DOI:10.3102/0034654317749187] [PMID] [PMCID]
- Bishop DV, Snowling MJ. Developmental dyslexia and specific language impairment: Same or different? Psychol Bull. 2004; 130(6):858. [DOI:10.1037/0033-2909.130.6.858] [PMID]
- Nation K, Cocksey J, Taylor JS, Bishop DV. A longitudinal investigation of early reading and language skills in children with poor reading comprehension. J Child Psychol Psychiatry. 2010; 51(9):1031-9. [DOI:10.1111/j.1469-7610.2010.02254.x] [PMID]
- Nation K. Children's reading comprehension difficulties. In: Snowling MJ, Hulme C, editors. The science of reading: A handbook. Oxford: Blackwell Publishing; 2005. [DOI:10.1002/9780470757642.ch14]
- Kelso K, Fletcher J, Lee P. Reading comprehension in children with specific language impairment: An examination of two subgroups. Int J Lang Commun Disord. 2007; 42(1):39-57. [DOI:10.1080/13682820600693013] [PMID]
- Di Folco C, Guez A, Peyre H, Ramus F. Epidemiology of reading disability: A comparison of DSM-5 and ICD-11 criteria. Sci Stud Read. 2022; 26(4):337-55. [DOI:10.1080/10888438.2021.1998067]
- Nation K, Snowling M. Assessing reading difficulties: The validity and utility of current measures of reading skill. Br J Educ Psychol. 1997; 67(Pt 3):359-70. [DOI:10.1111/j.2044-8279.1997.tb01250.x] [PMID]
- Keenan JM, Hua AN, Meenan CE, Pennington BF, Willcutt E, Olson RK. Issues in identifying poor comprehenders. Annee Psychol. 2014; 114(4):753-77. [DOI:10.3917/anpsy.144.0753] [PMID] [PMCID]
- Ghorbanibirgani A. [An epidemiologic study of dyslexia among male students in primary schools in Ahvaz-Iran 2013-14 (Persian)]. J Pediatr Nurs. 2014; 1(1):27-35. [Link]
- Mullis IV, von Davier M, Foy P, Fishbein B, Reynolds KA, Wry E. PIRLS 2021: International results in reading. Boston: Progress in International Literacy Study (PIRLS); 2021. [Link]
- Neale MD, McKAY MF, Childs G. The neale analysis of reading ability-revised. Br J Educ Psychol. 1986; 56(3):346-56. [DOI:10.1111/j.2044-8279.1986.tb03047.x]
- Rust J, Golombok S, Trickey G. Wechsler objective reading dimensions. London: Psychological Corporation; 1993. [Link]
- Hagley F. Suffolk reading scale: Teacher’s guide. Heslington: NFER-Nelson; 1987. [Link]
- León Cascón JA, Escudero I, Escudero Domínguez I, Olmos R, Olmos Albacete R. [ECOMPLEC: Evaluación de la comprensión lectora: Manual (Spanish)]. Milan: TEA; 2012. [Link]
- Moradi A, Hosaini M, Kormi Nouri R, Hassani J, Parhoon H. [Reliability and validity of reading and dyslexia test (NEMA) (Persian)]. J Adv Cogn Sci. 2016; 18(1):22-34. [Link]
- Ayre C, Scally AJ. Critical values for Lawshe’s content validity ratio: Revisiting the original methods of calculation. Meas Eval Couns Dev. 2014; 47(1):79-86. [DOI:10.1177/0748175613513808]
- Keenan JM, Meenan CE. Test differences in diagnosing reading comprehension deficits. J Learn Disabil. 2014; 47(2):125-35. [DOI:10.1177/0022219412439326] [PMID] [PMCID]
- Heidari K, Faramarzi S, Abedi A, Ghamarani A. [Designing the comprehensive diagnostic test of dyslexia (CDTD) and investigating its validity and reliability for primary school students of Isfahan in 2015-2016 (Persian)]. J Paramed Sci Rehabil. 2019; 8(2):17-28. [DOI:10.22038/jpsr.2019.30060.1771]