Interpreting evidence of differential item functioning (DIF) against learner responses: A South African example across three languages
DOI:
https://doi.org/10.56285/jltVol59iss2a6685Keywords:
differential item functioning, equivalence, item bias, large-scale assessments, PIRLS 2016, reading literacyAbstract
The aim of this study was to find evidence of differential item functioning (DIF) using an English, Afrikaans and isiZulu Grade 4 narrative text that was used during PIRLS 2016, followed by an investigation into learner responses to items that presented with DIF. One text that was used in PIRLS 2016, namely ‘The Pearl’ was used as an example here, where three out of 15 items presented with DIF across the three languages. Findings point to learners’ inability to respond to literal items, meaning those questions that do not require any inference making or evaluation at higher order levels of comprehension. Using a conceptual framework to establish linguistic, cultural, functional and metric equivalence, the current study provides evidence that there was no systematic bias. The current study presents results for a specific sub-set of qualitative analyses that was done during Phase II of a larger, mixed method study.
Downloads
References
Andrich, D., Sheridan, B. E., & Luo, G. (2012). RUMM2030: Rasch unidimensional models for measurement. RUMM Laboratory.
Bassnett, S. (2013). Translation. Routledge. https://doi.org/10.4324/9780203068892
Bermann, S., & Porter, C. (Eds.). (2014). A companion to translation studies. John Wiley & Sons.
Bergbauer, A., Van Staden, S., & Bosker, R. (2016). Differences in achievement between home language and language of learning in South Africa: Evidence from prePIRLS 2011. South African Journal of Childhood Education, 6(1), 1–10. https://doi.org/10.4102/sajce.v6i1.441
Chan, D. N. S., & So, W. K. W. (2017). Translation and validation of translation in cross- cultural research: strategies used in a study of cervical cancer screening among ethnic minorities. International Journal of Nursing Practice, 23(6), Article 12581. https://doi.org/10.1111/ijn.12581
Chesterman, A. (2016). Memes of translation: the spread of ideas in translation theory (revised edition). John Benjamins.
Combrinck, C., Roux, K., Tshele, M., Mokoena, G., & McLeod-Palane, N. (2017). Research design and methods in PIRLS 2016. In S. Howie, C. Combrinck, M. Tshele, K. Roux, N. McLeod-Palane & G. Mokoena (Eds.), PIRLS 2016 Progress in international reading literacy study 2016 Grade 5 benchmark participation: South African learners reading literacy achievement (pp. 29–47). Centre for Evaluation and Assessment, University of Pretoria.
Department of Education. (1997). Language in education policy 14 July 1997, in terms of The National Education Policy Act, 27 of 1996. Government Gazette 18546, Notice 1701, 19 December. Government Printer.
Du Plessis, G. A., Saccaggi, C. F., & De Bruin, G. P. (2015). Cross-cultural differences in the character strength of citizenship in South Africa. Psychology in Society, 48, 22–44. https://doi.org/10.17159/2309-8708/2015/n48a2
Duke, N., & Carlisle, J. (2011). The development of comprehension. In M. L. Kamil, P. D. Pearson, E. B. Moje, & P. Afflerbach (Eds.), Handbook of reading research (Vol. 4, pp. 199–228). Routledge.
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). SAGE Publications.
Graham, M. A., & Mtsweni, M. U. (2024). Parental involvement predicts Grade 4 learners’ reading literacy: An analysis of PIRLS data for students in Mpumalanga, South Africa. Educational Review, 1–23. https://doi.org/10.1080/00131911.2024.2379416
Howie, S. J., Combrinck, C., Roux, K., Tshele, M., Mokoena, G. M., & Mcleod Palane, N. (2017). PIRLS Literacy 2016: Progress in international reading literacy study 2016: South African children’s reading literacy achievement. Centre for Evaluation and Assessment, University of Pretroia.
International Test Commission. (2017). ITC guidelines for translating and adapting tests (2nd ed.). (2017). International Journal of Testing, 18(2), 101–134. https://doi.org/10.1080/15305058.2017.1398166
Keet, C. M., & Khumalo, L. (2017). Toward a knowledge-to-text controlled natural language of isiZulu. Lang Resources & Evaluation, 51, 131–157. http://dx.doi.org/10.1007/s10579-016-9340-0
La Roche, S., Joncas, M., & Foy, P. (2017). Sample design in PIRLS 2016. In M. O. Martin, I. V. S. Mullis, & M. Hooper (Eds.), Methods and Procedures in PIRLS 2016 (pp. 3.1–3.34). Retrieved August 17, 2025, from https://timssandpirls.bc.edu/publications/pirls/2016-methods/chapter-3.html
Land, S. (2015). Reading and the orthography of isiZulu. South African Journal of African Languages, 35(2), 163–175. https://doi.org/10.1080/02572117.2015.111300
Mahdavi, J. N., & Tensfeldt, L. (2013). Untangling reading comprehension strategy instruction: Assisting struggling readers in the primary grades. Preventing School Failure: Alternative Education for Children and Youth, 57(2), 77–92. https://doi.org/10.1080/1045988X.2012.668576
Millin, T. (2015). Reading to learn: A literature review within a South African context. Stellenbosch Papers in Linguistics, 44, 105–124. https://doi.org/10.5774/44-0-170
Mohammed, I., & Amponsah, O. (2018). Predominant factors contributing to low reading abilities of pupils at Elsie Lund Basic School in the Tamale metropolis, Ghana. African Educational Research Journal, 6(4), 273–278.
Morgan, D. L. (2014). Integrating qualitative and quantitative methods: A pragmatic approach. SAGE Publications. https://doi.org/10.4135/9781544304533
Mullis, I. V. S., & Martin, M. O. (2015). PIRLS 2016 assessment framework (2nd ed.). TIMSS & PIRLS International Study Center.
Mullis, I. V. S., & Prendergast, C. O. (2017). Developing the PIRLS 2016 achievement items. In M. O. Martin, I. V. S. Mullis, & M. Hooper (Eds.), Methods and procedures in PIRLS 2016 (pp. 1.1–1.29). Retrieved August 17, 2025, from https://timssandpirls.bc.edu/publications/pirls/2016-methods/chapter-1.html
Nida, E. A. (1964). Towards a science of translating. Brill.
Patterson, A., Roman, D., Friend, M., Osborne, J., & Donovan, B. (2018). Reading for meaning: The foundational knowledge every teacher of science should have. International Journal of Science Education, 40(3), 291–307. https://doi.org/10.1080/09500693.2017.1416205
Peña, E. D. (2007). Lost in translation: Methodological considerations in cross-cultural research. Child Development, 78(4), 1255–1264. https://www.jstor.org/stable/4620701
Pretorius, E. J. (2014). Supporting transition or playing catch-up in grade 4? Implications for standards in education and training. Perspectives in Education, 32(1), 51–76.
Pretorius, E., & Stoffelsma, L. (2017). How is their word knowledge growing? Exploring grade 3 vocabulary in South African township schools. South African Journal of Childhood Education, 7(1), 1–13. https://doi.org/10.4102/sajce.v7i1.553
Pretorius, E. J., & Klapwijk, N. M. (2016). Reading comprehension in South African schools: Are teachers getting it, and getting it right? Per Linguam, 32(1), 1–20. https://doi.org/10.5785/32-1-627
Roux, K. (2020). Examining the equivalence of the PIRLS 2016 released texts in South Africa across three languages [Doctoral thesis, University of Pretoria]. UPSpace Institutional Repository. http://hdl.handle.net/2263/80509
Roux, K., Van Staden, S., & Pretorius, E. J. (2022). Investigating the differential item functioning of a PIRLS Literacy 2016 text across three languages. Journal of Education, (87), 135–155. https://doi.org/10.17159/2520-9868/i87a07
Sandilands, D., Oliveri, M. E., Zumbo, B. D., & Ercikan, K. (2013). Investigating sources of differential item functioning in international large-scale assessments using a confirmatory approach. International Journal of Testing, 13(2), 152–174. https://doi.org/10.1080/15305058.2012.690140
Spaull, N., Pretorius, E, & Mohohlwane, N. (2020). Investigating the comprehension iceberg: Developing empirical benchmarks for early-grade reading in agglutinating African languages, South African Journal of Childhood Education, 10(1), Article 773. https://doi.org/10.4102/sajce.v10i1.773
Van Rooy, B., & Pretorius, E. J. (2013). Is reading in an agglutinating language different from an analytic language? An analysis of isiZulu and English reading based on eye movements. Southern African Linguistics & Applied Language Studies, 31(3), 281–297. https://doi.org/10.2989/16073614.2013.837603
Vottonen, E. (2016). Does theory meet practice? The realisation of global and local strategies in self-reflection analysis, In M. Dore (Ed.), Achieving consilience: Translation theories and practice (pp. 206–227). Cambridge Scholars Publishing.
Wilkes, A., & Nkosi, N. (2010). Complete Zulu beginner to intermediate book and audio course. John Murray Press.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Karen Roux, Surette van Staden
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
