Potential Impact of Automated Writing Evaluation (AWE) on EFL Learners’ Writing Accuracy: Learners’ Attitudes in Focus

Mahshid Asfa; Ehsan Rezvani

doi:10.22159/ijoe.2023v11i2.47035

Authors

Mahshid Asfa English Department, Isfahan (Khorasgan) Branch, Islamic Azad University, Isfahan, Iran
Ehsan Rezvani English Department, Isfahan (Khorasgan) Branch, Islamic Azad University, Isfahan, Iran https://orcid.org/0000-0001-8996-0312

DOI:

https://doi.org/10.22159/ijoe.2023v11i2.47035

Keywords:

automated writing evaluation , l 2 writing, syntactic accuracy, attitude

Abstract

The present study aimed to examine the potential impacts of Automated Writing Evaluation (AWE) on Iranian English as Foreign Language (EFL) learners’ writing accuracy. Another study objective was to identify the learners’ attitudes toward using AWE in English classes. As such, 50 female EFL students at the intermediate level of language proficiency were chosen via convenience sampling to participate in the present study. They were then randomly divided into two groups of Control Group (CG) and the Experimental Group (EG), each consisting of 25 learners. Before the treatment, a researcher-made writing composition pre-test was administered to both groups. Next, the learners in the EG received the treatment in which an AWE system was employed to check the learner’s written productions. Next, learners in the CG were exposed to the conventional pen-and-paper methods of English writing classes. Then, a writing composition post-test was administered. The performances of learners on pre and post-tests were scored based on accuracy in terms of syntactic errors. After administering the post-test, EG learners were asked to fill in a questionnaire on AWE.

Moreover, they were asked two questions in a semi-structured interview on their attitudes toward applying to the AWE program. The researcher recorded and later transcribed the learners’ answers for further analysis. The independent sample t-test revealed that Iranian EFL learners’ writing accuracy significantly improved after receiving AWE. Furthermore, the Chi-square test results, in conjunction with the interview results, showed that learners had positive attitudes toward AWE in writing classes. The study will have implications for the second language (L2) writing practices of EFL teachers and students.

Downloads

Download data is not yet available.

References

Attali, Y. (2004). Exploring the feedback and revision features of the Criterion service. http://www.ets.org/Media/Research/pdf/erater_NCME_2004_Attali_B.pdf

Barkaoui, K. (2007). Teaching writing to second language learners: Insights from theory and research. TESL Reporter, 40(1), 35–48. http://ojsdev.byuh.edu/index.php/Issue1/article/view/1112

Bayraktar, S. (2001). A Meta-analysis of the effectiveness of computer-assisted instruction in science education. Journal of Research on Technology in Education, 34(2), 173–188. https://doi.org/10.1080/15391523.2001.10782344

Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17(2), 102–118. https://doi.org/10.1016/j.jslw.2007.11.004

Bitchener, J., & Knoch, U. (2008). The value of written corrective feedback for migrant and international students. Language Teaching Research, 12(3), 409–431. https://doi.org/10.1177/1362168808089924

Blake, R. (2000). computer mediated communication: A window on L2 Spanish interlanguage. Language Learning and Technology, 41(1), 120–136. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/973bed3f-8657-45ef-8717-bbd758105b45/content

Braine, G. (1997). Beyond word processing: Networked computers in ESL writing classes. Computers and Composition, 14(1), 45–58. https://doi.org/10.1016/S8755-4615(97)90037-2

Chappelle, C. A. (2001). Innovative language learning: Achieving the vision. ReCALL, 13(1), 3–14. https://doi.org/10.1017/S0958344001000210

Chen, C. F. E., & Cheng, W. Y. E. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning and Technology, 12(3), 94–112. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/77d263c9-dfcf-4382-b179-7002b43cd5ef/content

Chou, H. N., & Chung, C. M. (2013). EFL college students’ writing performance and perceptions of the use of an automated writing evaluation system. English Education, 2(1), 93–126.

Elliot, N., & Klobucar, A. (2013). Automated essay evaluation and the teaching of writing. In M. D. Shermis & J. Burstein (Eds.), Handbook of automated essay scoring (pp. 16–35). Routledge.

Ellis, R. (1999). Learning a second language through interaction. John Benjamins Publishing Company.

Ferris, D. R., & Roberts, B. (2001). Error feedback in L2 writing classes? Journal of Second Language Writing, 10(3), 161–184. https://doi.org/10.1016/S1060-3743(01)00039-X

Grimes, D. (2008). Assessing automated assessment: Essay evaluation software in the classroom.

Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. Journal of Technology, Learning, and Assessment, 8(6), 14–30.

Howrey, J. D., & Tanner, P. (2009). Writing portfolios: Empowering students, teachers, and the curriculum. in A. M. stoke (Ed.), JALT2008 Conference Proceedings. tokyo: JAlt.

Klobucar, A., Elliot, N., Deess, P., Rudniy, O., & Joshi, K. (2013). Automated scoring in context: Rapid assessment for placed students. Assessing Writing, 18(1), 62–84. https://doi.org/10.1016/j.asw.2012.10.001

Lavolette, E., Polio, C., & Kahng, J. (2015). The accuracy of computer-assisted feedback and students’ responses to it. Language Learning and Technology, 19(2), 50–68. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/53b08da7-fa29-4ad8-8594-9ea68bf9ef3d/content

Leki, I. (1992). Understanding ESL writers: A guide for teachers. Boynton/Cook Publishers.

Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18. https://doi.org/10.1016/j.jslw.2014.10.004

Meng, Y. (2013). Written corrective feedback: A review of studies since Truscott (Working Paper). TESOL & Applied Linguistics. https://doi.org/10.7916/salt.v13i2.1339

Moseley, M. H. (2006). Creating recursive writers in middle school: The effect of a writing program on student revision practices. Capella University.

Noroozi, A., Rezvani, E., & Ameri, A. (2021). Students’ perceptions of the incorporation of flipped learning into L2 grammar lessons. Teaching English with Technology, 21(1), 112–130. https://files.eric.ed.gov/fulltext/EJ1283678.pdf

Noroozi, A., Rezvani, E., & Ameri-golestan, A. (2020). The effect of flipped classrooms on L2 learners’ development and retention of grammatical knowledge. Turkish Online Journal of Distance Education, 21(4), 14–30. http://doi.org/10.17718/tojde.803344

Palermo, C., & Wilson, J. (2020). Implementing automated writing evaluation in different instructional contexts: A mixed-methods study. Journal of Writing Research, 12(vol. 12 issue 1), 63–108. https://doi.org/10.17239/jowr-2020.12.01.04

Phillips, S. M. (2007). Automated essay scoring: A literature review. TASA Institute, Society for the Advancement of Excellent in Education.

Salteh, M. A., & Sadeghi, K. (2012). Teachers’ corrective feedback in L2 writing revisited: Concerns against and suggestions for its employment. World Applied Sciences Journal, 17(3), 375–383. https://www.idosi.org/wasj/wasj17(3)12/16.pdf

Soltanpour, F., & Valizadeh, M. (2017). The effect of the collaboration of reflective notes with CALL on EFL learners’ writing accuracy. European Journal of English Language Teaching, 2(2), 22–52. https://oapub.org/edu/index.php/ejel/article/view/644

Thuy, N. N. (2012). The effects of semantic mapping on vocabulary memorizing.

Vygotsky, L. S. (1981). The genesis of higher mental function. In J. V. Wertsch (Ed.), The concept of activity in Soviet psychology. Sharp.

Wang, P. (2015). Effects of an automated writing evaluation program: Student experiences and perceptions. Electronic Journal of Foreign Language Teaching, 12(1), 79–100. https://e-flt.nus.edu.sg/wp-content/uploads/2020/09/v12n12015/wang.pdf

Wang, Y. J., Shang, H. F., & Briody, P. (2013). Exploring the impact of using automated writing evaluation in English as a foreign language university students’ writing. Computer Assisted Language Learning, 26(3), 234–257. https://doi.org/10.1080/09588221.2012.655300

Warschauer, M. (2005). Sociocultural perspectives on CALL. In J. L. Egbert & G. M. Petrie (Eds.), CALL research perspectives. Lawrence Erlbaum Associates, Inc.

Warschauer, M., Turbee, L., & Roberts, B. (1996). Computer learning networks and student empowerment. System, 24(1), 1–14. https://doi.org/10.1016/0346-251X(95)00049-P

Williams, J. (2005). Teaching writing in second and foreign language classrooms. McGraw-Hill.

Wilson, J., Myers, M. C., & Potter, A. (2022). Investigating the promise of automated writing evaluation for supporting formative writing assessment at scale. Assessment in Education: Principles, Policy and Practice, 29(2), 183–199. https://doi.org/10.1080/0969594X.2022.2025762

Yu, Y. T., & Yeh, Y. L. (2003). Computerized feedback and bilingual concordance for EFL college students’ writing. Taipei, Taiwan: Crane.