The Readability Level of the 11th Grade Geography Textbook and Students’ Reading Ability Scores Based on Gender and School Type

Document Type : Research Paper

Author

Assistant Professor, Department of Humanities and Social Sciences, Farhangian University, Tehran, Iran

Abstract

The purpose of this study is to evaluate the readability level of 11th grade geography textbook based on students’ gender and school type. The Gunning FOG, Flesch and Cloze test indexes were used to examine the difficulty level of the book. The findings of the study revealed that the book is appropriate for an undergraduate course, but it is difficult for 11th grade students. The readability scores of lessons showed that the content of the book is not organized from simple to difficult. The mean score of the 11th grade students was 34% in the cloze test. As a result, the content causes stress and frustration for learners due to its difficulty. The results showed that female students had a higher mean score of reading ability than male students, but this difference was not statistically significant. Also, students’ reading score in special public schools was significantly higher than that in conventional public schools. In addition, there was a significant positive correlation between students' reading ability score and geography lesson score.
Introduction                                                      
Curriculum planners consider several standards for developing school textbooks, one of which is text readability. The most important methods used to determine the readability level include the Flesch method, Cloze test, Gunning FOG method, Fry method and Mc Laughlin method. In recent years, many researchers have used these methods to determine the readability level of textbooks in Iran (Ghaderi Moghaddam and Sobhaninejad, 2016; Kondari, 2020; Seraj et al., 2020; Yousefi et al., 2019). The literature review shows that we need more research on the readability and appropriateness of geography textbooks in Iran. Given the lack of such a study on the 11th grade geography book in Iran, the present study aimed to use quantitative content analysis and Gunning Fog, Flesch, and Cloze indexes for this purpose. The researcher aimed to answer the following questions:

Are the reading texts used in the 11th grade geography book linguistically appropriate for 11th grade students?
Is the content of the 11th grade geography book organized from simple to difficult?
What is the mean score of 11th grade students’ reading ability?
Is there a significant difference between 11th grade male and female students in terms of the mean reading ability score?
Is there a significant difference between conventional public schools and special public schools in terms of students’ mean reading ability score?
Is there a significant relationship between the reading ability scores of the 11th grade students and the geography course score?

Method
Three readability indexes were used for the study: Gunning Fog, Flesch, and Cloze test. The readability of the 11th grade geography book was first determined by Gunning Fog and Flesch readability indexes. The researcher also used the cloze test to assess the reading ability of the students. Student's t-test and correlation were used to analyze the scores of cloze test.
 
 
Results
Gunning and Flesch indexes were used to answer the first research question. The mean score of the Gunning and Flesch test was 13.8 and 38.9, respectively, that is, the content of the 11th grade geography book is difficult to read and understand. To answer the second research question, the readability scores of the lessons were compared, and indicated no particular order in the scores of lessons. Overall, although the second half of the book is more difficult than the first half, the book is not organized from simple to difficult. A cloze test was then used to answer the third to the sixth research questions. The mean score of the cloze test was 17 for all students, that is, on average, students answered only 34% of the questions correctly. As a result, the content causes stress and frustration for learners due to its difficulty. The mean reading ability score of female and male students was 17.93 (36%) and 16.1 (32%), respectively. The results of the Student's t-test showed that female students had higher reading ability scores than male students, but this difference was not statistically significant. The mean reading ability score of students in special public schools and conventional public schools was 19.17 (38%) and 15.45 (31%), respectively. Based on Student's t-test, the reading ability score of students in special public schools was significantly higher than that in conventional public schools. The Pearson’s correlation coefficient for students' reading ability scores and geography lesson scores was 0.604, which is significant at 99% confidence level. Therefore, there is a significant positive correlation between students' reading ability scores and geography lesson scores.
Discussion and conclusion
The purpose of this study is to evaluate the readability level of the 11th grade geography textbook and compare the reading ability scores of the students by gender and school type. The findings of the study revealed that the content of the book is not appropriate for the 11th grade students in terms of readability and comprehension, and needs to be revised. Comparing the readability scores of lessons shows that the book is not organized from simple to difficult. The results of the cloze test showed: (1) the content of the book causes stress and frustration for learners due to its difficulty; (2) the mean reading ability score of female students was higher than that of male students, but this difference was not statistically significant; (3) the reading ability score of students in special public schools is significantly higher than that in conventional public schools, and (4) there is a significant positive correlation between students' reading ability score and geography lesson score.

Keywords


Albright, J., de Guzman, C., Acebo, P., Paiva, D., Faulkner, M., Swanson, J. (1996). Readability of patient education materials: implications for clinical practice. Appl Nurs Res. 9, 139–143. https://doi.org/10.1016/s0897-1897(96)80254-0
Ante, L. (2022). The relationship between readability and scientific impact: Evidence from emerging technology discourses, Journal of Informetrics: 16, 101252. https://doi.org/10.1016/j.joi.2022.101252
Badarudeen, S., Sabharwal, S. (2008). Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites. J Bone Joint Surg Am. 90, 199–204. https://doi.org/10.2106/jbjs.g.00347
Bednaraz, S. (2004). Us geography text book: their role in education reform. International Research in Geography and Environmental Education, 13 (3), 223-338. http://dx.doi.org/10.1080/10382040408668517
Chen, A. C. H. (2016). A critical evaluation of text difficulty development in ELT textbook series: A corpus-based approach using variability neighbor clustering. System, 58, 64–81. http://dx.doi.org/10.1016/j.system.2016.03.011
Cline, T. A. (1972). Readability of community college textbooks and the reading ability of the students who use them. Journal of Reading Behavior, 5 (2), 110–118.
Cooley, M. E., Moriarty, H., Berger, MS., Selm-Orr, D., Coyle, B., Short, T. (1995). Patient literacy and the readability of written cancer educational materials. Oncol Nurs Forum. 22, 1345–1351.
Dale, E., Chall, J. S. (1948). A formula for predicting readability. Educational Research Bulletin, 27, 1-20, 37-54.
Dayani, M.H. (2000). Reading Assessment of Persian Writings: Reading for children and adolescents. Ketabkhane Rayanei Press, Tehran. Iran. (Text in Persian)
Davids, V. (2002). Determining text difficulty. Basehor-Linwood, Virtual School, 27,1-3.
DuBay, W. H. (2004). The principles of readability. Costa Mesa, CA: Impact Information.
Fazlollahi, S., & Maleki Tavana, M. (2011). The evaluation and assessment of readability of the third-grade science book in elementary school according to 5 standard formulas: Fry, Gunning, Flesch, Laughlin and Close. Journal of Curriculum Studies, 6 (22), 141-162. (Text in Persian).
Friedman, D. B., Hoffman-Goetz, L. (2006).  A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 33, 352–373. https://doi.org/10.1177/1090198105277329
Flesch, R. (2016). How to Write Plain English. University of Canterbury. Archived from the original on July 12, 2016. Retrieved July 12, 2016.
Fry, E. B. (1989). Reading formulas: Maligned but valid. Journal of Reading, 32(4), 292–297.
Fulcher, G. (1997). Text difficulty and accessibility: Reading formulae and expert judgement. System, 25(4), 497–513. https://doi.org/10.1016/S0346-251X(97)00048-1
Ghaderi Moghaddam, M. E., Sobhaninejad, M. (2016). Validation methods to measure textbooks readability. Research in Curriculum Planning, 48, 44-55. (Text in Persian)
GEÇİT, Y. (2010). The Evaluation of High School Geography 9 and High School Geography 11 Text Books with Some Formulas of Readability. EDUCATIONAL SCIENCES: THEORY & PRACTICE, 2205- 2220.
Gordejeva1, J., Zowalla, R., Pobiruchin, M., Wiesner, M. (2022). Readability of English, German, and Russian Disease-Related Wikipedia Pages: Automated Computational Analysis, JOURNAL OF MEDICAL INTERNET RESEARCH, 24 (5): e36835). https://doi.org/10.2196/36835
Gysi, W. K. (2013). The Role of Readability in Science Education in Ghana: A Readability Index Analysis of Ghana Association of Science Teachers Textbooks for Senior High School. Journal of Research & Method in Education, 2 (1), 9-19.
Gunes, F. (2003). The importance of the teaching of reading-writing the sentence. Turkishness Science Research, 13, 39-48.
Hailat, S., Obeidat, H., Hailat, M. (2014). The readability of secondary school geography textbooks in Jordan. Journal of Institutional Research South East Asia, 12 (2), 74-89.
Hidayatillahi, N., Zainil, Y. (2020). The readability od student’ textbook used in semantic and pragmatic course in English language education program of UNP. Journal of English Language Teaching Volume, 9 (1), 144-159. https://doi.org/10.24036/jelt.v9i1.107848  
Ivan, A. (2010). Klariti.com. Fog Index and Readability Formulas. Available at http://www.klariti.com/business-writing/Fog-Index-Readability-formulas.shtm1#top. Accessed on March 12, 2013.
Jitenara, A., Nolet, V., Xin, Y., & Dacosta, J. (2001). An analysis of middle school geography textbooks: implication for students with learning problems. Reading & Writing Quarterly, 17, 151-173. http://dx.doi.org/10.1080/105735601300007606
Johns, A. M. (1997). Text, role and context: Developing academic literacies. Cambridge, UK: Cambridge University Press.
Izqi, U., Seker, B. S. (2012). Comparing different readability formulas on the examples of science-technology and social science textbooks. Procedia - Social and Behavioral Sciences, 46, 178 – 182. http://dx.doi.org/10.1016/j.sbspro.2012.05.089
Kaya, Z. (1998). İş eğitimi kitaplarının okunaklılığı. Eğitim ve Bilim, 108, 30-35.
Keshavarz, M., & Salimi, H. (2007). Collocational competence and cloze test performance: a study of Iranian EFL learners. International Journal of Applied Linguistics, 17(1), 81-92.
Klare, G. R. (1963). The measurement of readability. Ames, Iowa: Iowa State University Press. 
Kobayashi, M. (2002). Cloze test revisited: Exploring item characteristic with special attention to scoring methods. The Modern Language Journal, 86, 571-585.
Kondari, F. (2020). Assessing and comparing the readability level of Farsi reading books in the third grade of elementary school based on reading techniques, Pouyesh Journal in Teaching Educational Sciences and Counseling, 12, 22-41. (Text in Persian).
Krippendorff, k. (2011). Content analysis (Basics and methodology). Nayeb, H. Ney Publishing. (Text in Persian)
Miles, T. (1990). The fog index: a practical readability scale. West Virginia University. Retrieved April 2, 2006, from http://www.as.wvu.edu/~tmiles/fog.html
Nazari, N., Farhadpour, M. R., Soleymani, E. (2016). Measure the Readability of the Persian text of the ‘Lets know more’ Section of the Quran book for the Grades two, three, and four of Elementary School Based on the Flash-Diani and Galing-Diani Formulas. Knowledge and Information Management, 3, 85-92. (Text in Persian)
Oba, F. G. (2015). Readability of biology textbooks and student’ academic performance in senior secondary schools in Ektti State Nigeria. European Journal of Research and Reflection in Educational Sciences, 3 (3), 79-84.
Odo, D. M. (2018). A Comparison of Readability and Understandability in Second Language Acquisition Textbooks for Pre-service EFL Teachers. The Journal of Asia TEFL, 15 (3), 750-765. http://dx.doi.org/10.18823/asiatefl.2018.15.3.12.750
Plucinski, K., Olsavsky, J., & Hall, L. (2009). Readability of introductory financial and manage vial accounting text books. Academy of Educational Leadership Journal, 13(4), 119-127.
Sarmad, Z., Bazagan, A., and Hejazi, E. (2005). Research methods in behavioral sciences. Aghah Press, Tehran.
Seraj, F., Roshan, B., Najafivan, A., & Yousefi rad, F. (2020). Readability of Farsi Textbooks in Primary School: First to Sixth Grades, Journal of Curriculum Studies, 14 (55), 113-140. (Text in Persian)
Shekarey, A., Najareyan, Z. (2012). A study of the readability of Hedyehaye Asemani textbooks for grades four and five based on Gunning Scale. Research in Curriculum Planning, 33, 72-78. (Text in Persian)
Snyman, M. (2004). Using the printed medium to disseminate information about psychiatric disorders. African Journal of Psychiatry, 7(4), 15–20.
Smith, M. & Taffler, R. (1992). Readability and understandability: Different measures of the textual complexity of accounting narrative. Accounting, Auditing & Accountability Journal, 5(4), 84–98.
Tabatabaei, O., & Mirzaei, E. (2014). Correlational validation of cloze test and c-test against IELT. Journal of Educational and Social Research, 4(1), 345–356.
Taylor, W. (1956). Recent Developments in the use of Cloze Procedure. Journalism Quarterly, 33, (1), 42 -48, 99.
Wellington, J. (1994). Secondary science, contemporary issues and practical approaches. London: Mackays of Chatham, PLC.
Wissing, G. J., Blignaut, A. S., & Van den Berg, K. (2016). Using readability, comprehensibility and lexical coverage to evaluate the suitability of an introductory accountancy textbook to its readership. Stellenbosch Papers in Linguistics, 46 (1), 155–180.
Yarmohamadian, M. H. (2016). Principles and principles of curriculum planning, Yadvare book Press, Tehran, Iran. (Text in Persian)
Yousefi, M.J., Ahhah Karam Tajdi. P., Salehi, M. R. (2019). Assessing the readability and comprehensibility of the textbook of applied geography (3) of the twelfth grade of humanities using Flash, Fry and Gunning indices, Quarterly Journal of Education Studies, 20, 12-25. (Text in Persian)
Zamanian, M. & Heydari, P. (2012). Readability of texts: State of the art. Theory and Practice in Language Studies, 2(1), 43–53.