Informative assessment: A supportive tool for systemic validity in language education
Abstract
Abstract
Accountability in language testing is normally supported through a validation process. In language teaching, however, the matter of accountability is sometimes overlooked, especially when the focus falls on curriculum completion. In view of the reciprocity that exists between language teaching and testing, core principles used for test validation are also useful for ensuring accountability in course design. This paper evaluates a language course offered to Education students at a tertiary institution by drawing on the notions of construct and content validity. Empirical data gleaned from a literacy survey and assessment artefacts are interpreted against course objectives and content. A need is identified for substantial revisions to the course in order to ensure closer alignment with students’ language learning needs. The study shows how accountability in language education coheres with the principle of systemic validity and the relevance of analysing and correlating different kinds of data.Downloads
References
References
Blommaert, J. 2005. Discourse: A critical introduction (Key topics in Sociolinguistics). Cambridge: Cambridge University Press.
Blommaert, J. 2010. The sociolinguistics of globalization. Cambridge: Cambridge University Press.
CDE (The Centre for Development and Enterprise). 2015. Teachers in South Africa: Supply and demand 2013-2025. Executive summary of a report. Accessed June 14, 2018. http://www.cde.org.za/wp-content/uploads/2015/03/Final-Revised-ES-TeacherSupplyandDemand2025.pdf.
Cohen, J.W. 1988. Statistical power analysis for the behavioural sciences. 2nd edition. Hillsdale, NJ: Lawrence Erlbaum Associates.
Council of Europe. 2018. Common European Framework of Reference for Languages: Learning, teaching, assessment (CEFR). Cambridge: Cambridge University Press. Available: http://www.coe.int/t/dg4/linguistic/Cadre1_en.asp. Accessed June 14, 2018.
Cumming, A. and R. Berwick, eds. 1996. Validation in language testing. Clevedon, UK: Multilingual Matters Ltd.
Cummins, J. 2001. Negotiating identities: Education for empowerment in a diverse society. 2nd ed. Los Angeles, CA: CABE.
Cummins, J. and C. Davison, eds. 2007. International handbook of English language teaching. New York: Springer Science+Business Media, LLC.
De Graaf, R. and A. Housen. 2011. Investigating the effects and effectiveness of L2 instruction. In The handbook of language teaching, eds. M.H. Long and C.J. Doughty, 726-755. Chichester, West Sussex: Wiley-Blackwell.
Author 2012. The design, refinement and reception of a test of academic literacy for postgraduate students. MA dissertation, University of the Free State, South Africa.
Author 2017. Developing a theoretical rationale for the attainment of greater equivalence of standard in the Grade 12 Home Language exit-level examinations. PhD thesis, University of the Free State, South Africa.
Ellis, R. 2008. The study of second language acquisition. 2nd ed. Oxford: Oxford University Press.
Fairclough, N. 1995. Critical discourse analysis: The critical study of language. London and New York: Longman.
Frederiksen, J.R. and A. Collins. 1989. A systems approach to educational testing. Educational researcher 18(9): 27-32.
Fulcher, G. 2010. Practical language testing. London: Hodder Education.
Fulcher, G. 2015. Re-examining language testing: A philosophical and social inquiry. Abingdon, Oxon: Routledge.
Green, A. 2014. Exploring language assessment and testing. New York: Routledge.
Grosser, M.M. and M. Nel. 2013. The relationship between the critical thinking skills and the academic language proficiency of prospective teachers. South African journal of education 33(2): 1-17.
Habermas, J. 1970. Toward a theory of communicative competence. In Recent sociology 2, ed. H.P. Dreitzel, 115-148. London: Collier-Macmillan.
Halliday, M.A.K. 1978. Language as social semiotic: The social interpretation of language and meaning. London: Edward Arnold.
Hogan, T.P. 2007. Psychological testing: A practical introduction. 2nd ed. Hoboken, New Jersey: John Wiley and Sons.
Hughes, A. 2003. Testing for language teachers. 2nd ed. Cambridge: Cambridge University Press.
Hymes, D. 1972. On communicative competence. In Sociolinguistics: Selected readings, eds. J.B. Pride and J. Holmes, 269-293. Harmondsworth: Penguin.
Janks, H. 2000. Domination, access, diversity and design: A synthesis for critical literacy education. Educational Review 52(2): 175–186.
Janks, H. 2001. Identity and conflict in the critical literacy classroom. In Negotiating critical literacies in classrooms, eds. B. Comber and A. Simpson, 137-150. Mahwah, NJ: Erlbaum.
Janks, H. 2010. Literacy and power. New York: Routledge.
Kerr, M.M. and K.M. Frese. 2017. Reading to learn or learning to read? Engaging college students in course readings. College teaching 65(1): 28-31. Available: https://www.tandfonline.com/doi/abs/10.1080/87567555.2016.1222577. Accessed June 14, 2018.
Kiely, R. 2009. Observing, noticing, and understanding: Two case studies in language awareness in the development of academic literacy. Language awareness 18: 3-4, 329-344. Available: https://dx.doi.org/10.1080/09658410903197298. Accessed June 14, 2018.
Messick, S. 1995. Validity of psychological assessment: Validation of inferences from person’s responses and performances as scientific inquiry into score meaning. American psychologist 50: 741-749.
Mogekwu, M. 2005. African Union: Xenophobia as poor intercultural communication. Ecquid Novi 26(1): 5-20.
Murphy, B., Conway, P.F., Murphy, R. and K. Hall. 2014. The emergence of reading literacy in post-primary teacher education: From the background to the foreground. European journal of teacher education 37(3): 331-347. Available: https://www.tandfonline.com/doi/abs/10.1080/02619768.2013.870995. Accessed June 14, 2018.
Neocosmos, M. 2008. The politics of fear and the fear of politics: Reflections on Xenophobic violence in South Africa. Journal of Asian and African studies 43(6): 586-594.
Nkosi, B. 2015. Most of our new teachers can’t. Mail&Guardian, 17 April 2015. Available: http://mg.co.za/article/2015-04-17-most-of-our-new-teachers-cant Accessed October 20, 2015.
Norris, J.M. and L. Ortega. 2000. Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning 50, 417-528.
Norris, J.M. 2011. Task-based teaching and testing. In The handbook of language teaching, eds. M.H. Long and C.J. Doughty, 578-594. Chichester, West Sussex: Wiley-Blackwell.
Orgeret, K.S.. 2010. Mediated culture and the well-informed global citizen: Images of Africa in the global North. Nordicom review 31(2): 47-61.
Orgeret, K.S. 2012. Intercultural educational practices: Opening paths for dialogue. Intercultural Communication Studies (XXI): 189-204. Available: http://hdl.handle.net/10642/1379. Accessed June14, 2018.
Rademeyer A. 2007. SA vaar swakste in toets: Leerlinge se leesvaardighede nie op peil. [SA performs the worst in test: Learners’ reading abilities not up to standard – own translation]. Beeld, 30 November.
Rambiritch, A. 2012. Challenging Messick: Proposing a theoretical framework for understanding fundamental concepts in language testing. Journal for language teaching 46(2): 108-121.
Robinson, P. 2011. Syllabus design. In The handbook of language teaching, eds. M.H. Long and C.J. Doughty, 294-310. Chichester, West Sussex: Wiley-Blackwell.
Shamim, A. 2011. Critical language awareness in pedagogic context. Language teaching 4(4): 28-35. Available: www.ccsenet.org/elt. Accessed June 14, 2018.
Shohamy, E. 2006. Language policy: Hidden agendas and new approaches. New York: Routledge.
Singh, R. J. 2013. Examining xenophobic practices amongst university students – a case study from Limpopo province. Alternation (special edition) 7: 88-108.
Singh, S. K. 2013. Zimbabwean teachers’ experiences of xenophobia in Limpopo schools. Alternation (special edition) 7: 51-66.
Svalberg, A.M.L. 2007. Language awareness and language learning. English language teaching journal 40: 287-308.
Svalberg, A.M.L. 2009. Engagement with language: Interrogating a construct. Language awareness 18(3-4): 242-258.
UFS (University of the Free State). 2015. English for professional purposes (ENGS1504) course guide. Bloemfontein: University of the Free State.
Van Dyk, T.J. 2010. Konstitutiewe voorwaardes vir die ontwerp van ’n toets van akademiese geletterdheid.[Constitutive conditions for the design of a test of academic literacy]. PhD thesis, University of the Free State, South Africa.
Weideman, A. 2009. Constitutive and regulative conditions for the assessment of academic literacy. Southern African linguistics and applied language studies 27(3): 235-251.
Weideman, A. 2017a. Does responsibility encompass ethicality and accountability in language assessment? Language & communication 57: 1-9. Available: http://dx.doi.org/10.1016/j.langcom.2016.12.004. Accessed August 3, 2017.
Weideman, A. 2017b. Responsible design in applied linguistics: Theory and practice. Cham, Switzerland: Springer.
Weir, C.J. 2005. Language testing and validation: An evidence-based approach. New York: Palgrave Macmillan.
Copyright (c) 2019 Colleen Lynne du Plessis, Carla Els
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
This journal is an open access journal, and the authors and journal should be properly acknowledged, when works are cited.
Authors, copyright holders, may use the publishers version for teaching purposes, in books, theses, dissertations, conferences and conference papers.
A copy of the authors' publishers version may also be hosted on the following websites:
- Non-commercial personal homepage or blog.
- Institutional webpage.
- Authors Institutional Repository.
The following notice should accompany such a posting on the website: This is an electronic version of an article published in SAJHE, Volume XXX, number XXX, pages XXX “XXX", DOI. Authors should also supply a hyperlink to the original paper or indicate where the original paper (http://www.journals.ac.za/index.php/SAJHE) may be found.
Authors publishers version, affiliated with the Stellenbosch University will be automatically deposited in the University Institutional Repository SUNScholar.
Articles as a whole, may not be re-published with another journal.
The following license applies:
Attribution CC BY-NC-ND 4.0