The Recurring Issue of Aviation English Test Validity: Echoes from Test-takers and Assessors of the English for Aviation Language Testing System in Algeria
DOI:
https://doi.org/10.33919/esnbu.23.2.6Keywords:
ESP, Assessment, Aviation English, test validity, test design, aviation phraseology, language proficiencyAbstract
The English for Aviation Language Testing System (EALTS) is one of the international tests for pilots wishing to operate in international airspace. This test presents a wide range of difficulties for Algerian pilots, but little research has been conducted to estimate its validity. This paper seeks to answer two questions: (1) What are test-takers perspectives on the EALTS test construct? and (2) What is the assessors' perception of the testing procedure and test validity? A descriptive study was conducted using qualitative data from a semi-structured questionnaire for ten pilots and a semi-structured interview for three certified assessors and four university researchers. The authors used 'first-hand' data from the targeted sample to cross-check results' validity through triangulation. The results show specific difficulties from an affective dimension, such as stress and anxiety caused by inefficient preparation and unfamiliarity with test tasks. A different interpretation of ICAO descriptions using the rating scale is another issue noted by assessors. Additionally, technical issues with the computer-based listening test and non-compliant features of the test contents with features of the target situation language use are among the main issues noted by both test-takers and assessors.
References
Assassi, T. (2017). The Effect of Formulaic Language on ESP Learners’ Conversation Competence: The Case of Aviation Professionals in Algeria. [Unpublished Doctoral Dissertation]. Tlemcen University, Algeria.
Assassi, T. (2020). Algerian Aeronautics Professionals’ Needs Taxonomy: A Target Situation Needs Analysis. International Journal of English Linguistics, 10, 90-103. https://doi.org/10.5539/ijel.v10n6p90 DOI: https://doi.org/10.5539/ijel.v10n6p90
Assassi, T., & Benyelles, R. (2016). Formulaic Language for Improving Communicative Competence. Arab World English Journal, 7, 163-176. https://doi.org/10.24093/awej/vol7no2.11 DOI: https://doi.org/10.24093/awej/vol7no2.11
Borowska, A. P. (2018). A Multilingual Speaker in Global Aviation Communication. Applied Linguistics Papers, 25(2), 11-19. https://doi.org/10.32612/uw.25449354.2018.2.pp.11-19 DOI: https://doi.org/10.32612/uw.25449354.2018.2.pp.11-19
Brindley, G. (1998). Assessing listening abilities. Annual Review of Applied Linguistics, 18(1), 171–191. https://doi.org/10.1017/S0267190500003536 DOI: https://doi.org/10.1017/S0267190500003536
Brown, A. (1993). The role of test-taker feedback in the test development process: Test-takers’ reactions to a tape-mediated test of proficiency in spoken Japanese. Language Testing, 10(3), 277-301. https://doi.org/10.1177/026553229301000305 DOI: https://doi.org/10.1177/026553229301000305
Civil Aviation Authority (CAA International). (2012). English for Aviation Examinations. Handbook 2012. https://sciarium.com/file/283262
Civil Aviation Authority. (2016). CAP 413: Radiotelephony Manual. Intelligence, Strategy and Policy, Safety and Airspace Regulation Group. Civil Aviation Authority.
Cookson, S. (2009). Zagreb and Tenerife: Airline accidents involving linguistic factors. Australian Review of Applied Linguistics, 32(3), 22.1-22.14. https://doi.org/10.2104/aral0922 DOI: https://doi.org/10.1075/aral.32.3.02coo
Cox. A., & Karimi. M. (2022). Creating a rubric for placement tests for Aviation English programs. In P. Tosqui-Lucks & J.C. Santana (Eds.), Aviation English a Global Perspective: Analysis, teaching, assessment (pp. 182-199). Bookerfield. https://doi.org/10.53268/BKF22080510 DOI: https://doi.org/10.53268/BKF22080510
Cushing, S. (1994). Fatal words: Communication clashes and aircraft crashes. University of Chicago Press.
Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing. Cambridge University Press.
Denscombe, M. (2014). The Good Research Guide: for small-scale social research projects. McGraw-Hill Education.
Dornyei, Z. (2003). Questionnaires in second language research: Construction, Administration, and Processing. Lawrence Erlbaum Associates.
Douglas, D. (2000). Assessing language for specific purposes. Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511732911
Douglas, D. (2005). Testing languages for specific purposes. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp. 857-868). Lawrence Erlbaum.
Douglas, D., & Myers, R. (2000). Assessing the communication skills of veterinary students: Whose criteria? In A. J. Kunnan (Ed.), Fairness and validation in language assessment: Selected papers from the 19th Language Testing Research Colloquium, Orlando, Florida. Cambridge University Press.
Duff, P. (2008). Case Study Research in Applied Linguistics. Taylor & Francis.
Elder, C. (1993). How do subject specialists construe classroom language proficiency? Language Testing, 10(3), 235-254. https://doi.org/10.1177/026553229301000303 DOI: https://doi.org/10.1177/026553229301000303
Federal Aviation Administration’s Aeronautical Center (FAA). (2021, December 31). U.S. Civil Airmen statistics. https://www.faa.gov/data_research/aviation_data_statistics/civil_airmen_statistics
Field, J. (2019). Rethinking the Second Language Listening Test - From Theory to Practice. Equinox.
Field, J. (20022, July 6). Idle chatter: What really goes on in tests of interactive communication? CRELLA Symposium, 2020. [Video]. YouTube. https://www.youtube.com/watch?v=ONM7xcJTPD0
Fowler, R., Matthews, E., Lynch, J., & Roberts, J., (2021). Aviation English Assessment and Training. Collegiate Aviation Review International, 39(2), 26-42. https://doi.org/10.22488/okstate.22.100231 DOI: https://doi.org/10.22488/okstate.22.100231
Hymes, D. (1972). On Communicative Competence. In J.B. Pride and J. Holmes (Eds.), Sociolinguistics. Selected Readings (pp. 269-293). Penguin.
International Civil Aviation Organization (ICAO). (2004). Manual on the Implementation of ICAO Language Proficiency Requirements (Doc. 9835). International Civil Aviation Organization.
International Civil Aviation Organization (ICAO). (2006). ICAO Manual of Radiotelephony. Doc 9432-AN/925. Annex 10, Volume II, Chapter 5. International Civil Aviation Organization.
International Civil Aviation Organization (ICAO). (2010a). GBAS CAT II/III development baseline SARPs. International Civil Aviation Organization.
International Civil Aviation Organization (ICAO). (2010b). Manual on the Implementation of ICAO Language Proficiency Requirements (2nd ed.) (Doc. 9835). International Civil Aviation Organization.
International Civil Aviation Organization (ICAO). (2013). Language Proficiency Requirements: Critical to aviation safety. ICAO Journal, 5, 64-64.
Jacoby, S., & McNamara, T. (1999). Locating competence. English for Specific Purposes, 18(3), 213-241. https://doi.org/10.1016/S0889-4906(97)00053-7 DOI: https://doi.org/10.1016/S0889-4906(97)00053-7
Katsarska, V. (2021). Formulaic Language in Aeronautical English-Friend or Foe. Journal of Teaching English for Specific and Academic Purposes, 9(2), 239-245. https://doi.org/10.22190/JTESAP2102239K DOI: https://doi.org/10.22190/JTESAP2102239K
Lumley, T., & Brown, A. (1996). Specific-purpose language performance tests: Task and interaction. Australian Review of Applied Linguistics, Series S, 13, 105-136. https://doi.org/10.1075/aralss.13.06lum DOI: https://doi.org/10.1075/aralss.13.06lum
Lynch, T., Mendelshon, D. (2010). Listening. In N. Schmitt (Ed.), An introduction to applied linguistics (2nd ed) (pp. 180-196). Hodder Arnold.
Mekkaoui, G. (2013). English for Aviation in the Algerian context: The case of Pilots and Air Traffic Controllers in Zenata-Messalli El Hadj Airport. [Unpublished master’s thesis]. Tlemcen University, Algeria.
Mekkaoui, G., & Mouhadjer, N. (2019). Addressing air traffic controllers’ English language proficiency needs: Case of Zenata Airport. Global Journal of Foreign Language Teaching, 9(3), 167-183. https://doi.org/10.18844/gjflt.v9i3.4245 DOI: https://doi.org/10.18844/gjflt.v9i3.4245
Moere, A.V., Suzuki, M., Downey, R., & Cheng, J. (2009). Implementing ICAO Language Proficiency Requirements in the Versant Aviation English Test. Australian Review of Applied Linguistics, 32, 27.1-27.17. https://doi.org/10.1075/aral.32.3.07van DOI: https://doi.org/10.1075/aral.32.3.07van
Monteiro, A.L.T. (2022). The Assessment of English in Aeronautical Radiotelephony Communications: A mixed methods study. In P. Tosqui-Lucks & J.C. Santana (Eds.), Aviation English a Global Perspective: Analysis, teaching, assessment (pp. 224-245). https://doi.org/10.53268/BKF22080512 DOI: https://doi.org/10.53268/BKF22080512
Netherlands Aviation Safety Board (1978). Final Report and Comments of the Netherlands Aviation Safety Board of the Investigation into the Accident with the Collision of KLM Flight 4805 and Pan American Flight 1736. https://www.faasafety.gov/files/gslac/courses/content/232/1081/finaldutchreport.pdf
Taylor, L., Geranpayeh, A. (2011). Assessing listening for academic purposes: Defining and operationalising the test construct. Journal of English for Academic Purposes, 10(2), 89-101. https://doi.org/10.1016/j.jeap.2011.03.002 DOI: https://doi.org/10.1016/j.jeap.2011.03.002
Wagner, E. (2014). Assessing listening. In A. Kunnan (Ed.), Companion to language assessment Volume 1 (pp. 47-63). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbcla094 DOI: https://doi.org/10.1002/9781118411360.wbcla094
Wu, Y. (1998). What do tests of listening comprehension test? - A retrospection study of EFL test-takers performing a multiple-choice task. Language Testing, 15, 21-44. https://doi.org/10.1177/026553229801500102 DOI: https://doi.org/10.1177/026553229801500102
Zhou, Y., Yoshitomi, A. (2019). Test-taker perception of and test performance on computer-delivered speaking tests: the mediational role of test-taking motivation. Lang Test Asia 9(10), 1-19. https://doi.org/10.1186/s40468-019-0086-7 DOI: https://doi.org/10.1186/s40468-019-0086-7
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Tarek Assassi , Tarek Ghodbane
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
All published articles in the ESNBU are licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). This license lets others remix, tweak, and build upon your work non-commercially, and although their new works must also acknowledge you and be non-commercial, they don't have to license their derivative works on the same terms.
In other words, under the CC BY-NC 4.0 license users are free to:
Share - copy and redistribute the material in any medium or format
Adapt - remix, transform, and build upon the material
Under the following terms:
Attribution (by) - All CC licenses require that others who use your work in any way must give you credit the way you request, but not in a way that suggests you endorse them or their use. If they want to use your work without giving you credit or for endorsement purposes, they must get your permission first.
NonCommercial (nc) - You let others copy, distribute, display, perform, and modify and use your work for any purpose other than commercially unless they get your permission first.
If the article is to be used for commercial purposes, we suggest authors be contacted by email.
If the law requires that the article be published in the public domain, authors will notify ESNBU at the time of submission, and in such cases the article shall be released under the Creative Commons 1 Public Domain Dedication waiver CC0 1.0 Universal.
Copyright
Copyright for articles published in ESNBU are retained by the authors, with first publication rights granted to the journal. Authors retain full publishing rights and are encouraged to upload their work to institutional repositories, social academic networking sites, etc. ESNBU is not responsible for subsequent uses of the work. It is the author's responsibility to bring an infringement action if so desired by the author.
Exceptions to copyright policy
Occasionally ESNBU may co-publish articles jointly with other publishers, and different licensing conditions may then apply.
