Exploring syntactic complexity and its relationship with writing quality in EFL argumentative essays

Over the last two decades, L2 writing scholars in the field of instructed second-language acquisition (ISLA) have taken up investigating the elaborated degree of linguistic features in L2 learners' written output to better understand learners writing competence and L2 growth from a linguistic and psycholinguist perspective ^{9, 29}. Syntactic complexity refers to the variety and sophistication of syntactic features exhibited in L2 outputs, including the length of production units i.e., the number of words per clause, the amount of subordination, or the frequency of distinct clauses, or phrase structures used ³⁵. Per se, analyzing L2 output in terms of indices measuring these syntactic features is a common means to gauge L2 growth or writing quality. The default hypothesis has been that as L2 learners' interlanguage develops, they begin to produce less frequent and diverse syntactic features ^{5, 27}. That said, L2 learners’ language-using ability or writing quality is demonstrated through their control over complex and sophisticated levels of syntactic knowledge (Li & Yang, 2023) ³⁴.

Presuming this, investigating the relationship between L2 writing quality human ratings writing scores and syntactic complexity indices has been the focus of much L2 writing research ^{2, 23}. This focus has been motivated by at least three reasons conceptually, researchers are interested in identifying features that both adequately reflect the multidimensional nature of syntactic complexity and strongly correlate with quality ratings of L2 writing ⁵. From the assessment perspective, this line of research has informed the empirical development of writing proficiency scales and facilitated automated scoring ¹⁵. Pedagogically, a detailed understanding of this relationship can offer useful insights into the aspects of syntactic complexity to focus on for different learner groups, genres, and writing tasks (Bulté & Housen, 2019). Additionally, this focus has been motivated by the advent of computational tools such as L2 Syntactic Complexity Analyzer (L2-SCA) ²¹ has made it possible to examine this relationship on a larger scale quickly and accurately ^{4, 24, 30}.

Despite recent advances, existing L2 research on syntactic complexity has several theoretical, conceptual, and methodological flaws ³ (Housen, De Clercq, Kuiken, & Vedder, 2019). As an illustration, syntactic complexity has traditionally been conceived as the basic internal formal structure of the text ⁶ and measured using only large-grained complexity indices such as clausal subordination and T-unit (consisting of one main clause plus extra, embedded clauses, Hunt, 1965) which have high predictive power ³⁵. However, in recent years, the use of these absolute complexity, large-grained, indices has come under criticism for failing to account for syntactic variation, the specific subtypes of clausal and phrase level elaboration that emerge as L2 learners advance (Housen et al., 2019) and for the difficulty in interpretation ⁵. Recent studies ³⁰ have indicated that fine-grained indices that capture specific subtypes of phrasal or clausal structures (e.g., clause complements per clause) are better indicators of holistic scores of TOFEL independent essay quality ¹⁹.

Another criticism is that the traditional absolute complexities do not account for the relative frequency of syntactic forms and the entangled link between syntactic forms and lexical items ¹⁷. That said, the absolute complexity indices do not strongly coincide with some aspects of L2 acquisition theories, such as usage-based approaches, which posit that linguistic constructions emerge from language use, such that, features related to the frequency, saliency, and contingency of syntactic constructions, not absolute complexity, is the main indicator of L2 acquisition ^{11, 13}. Aligned with the theory of usage-based approach, some scholars most notably Kyle (2016) proposed features related to the use of Verb Argument Constructions (VACs, which consist of a verb slot and its related arguments) as a reliable method of measuring relative complexity or syntactic sophistication ²⁴ (Li & Yang, 2023) ³⁰. These studies have confirmed that the newly proposed VACs sophistication indices are better indicators of L2 writing quality. However, the majority of L2 writing studies in this line of inquiry have primarily focused on argumentative, and narrative essays written by advanced or college-level L2 learners in large-scale standardized tests (e.g., TOEFL) in the English-dominant contexts ^{19, 36}. Thus, it is unclear as to what absolute and fine-grained along with VACs sophistication features capture the writing quality and syntactic developmental trajectories of Ethiopian EFL. Moreover, little is known about the link between syntactic features and EFL course instructors’ holistic ratings of genres commonly assessed in EAP classrooms context. Therefore, the present study then tapped into what specific syntactic features predict EFL teachers' holistic rating score of argumentative essays produced by undergraduate EFL learners in their classroom context using computational tools namely TAASSC ¹⁷. As such, exploring the relationship between L2 writing quality and syntactic features of argumentative essays produced by this group of learners enables us to obtain useful information regarding their linguistic competence and repertoires, helping to create a more comprehensive picture of the writing performance of EFL learners at different stages of L2 learning.

The overall structure of this study consists of five sections, including the introductory part. In the following, section two presents a brief overview of syntactic complexity as a multi-dimensional construct by laying out the theoretical dimensions and looking at how syntactic complexity is evaluated in L2 acquisition. The third section is concerned with the methodology used for this study. The fourth section presents the analysis and results of the study. Finally, the conclusion gives a summary as well as the implication of the findings to pertinent future studies.

Recognizing the multidimensional nature of syntactic complexity, Ortega (2015) defined it as the "expansion of the capacity to use the additional language in ever more mature and skillful ways, tapping the full range of linguistic resources offered by the given grammar to successfully fulfill various communicative goals" ²⁹. Her definition highlights that the term "syntactic complexity" refers to the ability to generate longer and more complex structures (i.e., absolute complexity) as well as knowledge of how to skillfully leverage the complexity and subtleties of language ^{6, 17}. This implies that these two dimensions are critical in comprehending and analyzing syntactic complexity as a multidimensional construct ⁹, and this viewpoint is taken into account for the current study.

The first construct, "syntactic complexity," as defined in this study, pertains to the hierarchical organization and nesting of the basic formal structures of language units (e.g., phrases and clauses) within a sentence (Housen et al., 2019). Measuring this dimension helps to demonstrate the extent to which EFL/ESL learners can manipulate and combine diverse grammatical patterns to build complex linguistic systems as their interlanguages develop. Given this, in ISLA literature, various indices have been proposed to quantify syntactic complexity as a multidimensional entity, ²², which can be summarized into two approaches: absolute complexity indices ²¹ and fine-grained indices ¹⁷. The former assesses the internal complexity of the text in a largely holistic way, without differentiating specific subtypes of the structures concerned ⁵. Several L2 writing studies ^{8, 21, 37} assess this dimension of complexity using L2 Syntactic Complexity Analyzer. Whereas the latter, fine-grained indices, take into account specific subtypes of phrasal (e.g., prepositions per direct object) or clausal structures (e.g., clause complements per clause, which taps into the use of a particular type of dependent clause). Several fine-grained indices and computational tools that measure these indices have been proposed and developed by different scholars ^{3, 17} and employed in L2 writing studies. Of these fine-grained syntactic complexity indices operationalized by Kyle (2016) and computed in TAASSC which have had an impact on recent discussions of clausal and phrasal complexity are then considered in the current study ¹⁹ (Li & Yang, 2023) ³⁰.

The second construct, "syntactic sophistication," relates to the acquisitional sequence of specific sets of linguistic construction including syntactic constructions and the mental processes required for producing, and breaking them down ^{7, 17}. The basic premise is that syntactic construction (e.g., VACs) learned later are more sophisticated ¹⁸. It takes its theoretical cues from the usage-based approach, which maintains that L2 acquisition is a process of contingency learning in which L2 learners create form-meaning pairings through repeated exposure to the target language constructions (e.g., words) using cognitive processes like entrenchment, categorization, association, and generalization ¹². This suggests that grammatical constructions that emerge from the interaction of language input and cognitive processes carry meaning independent of specific lexical items within the sentence. For instance, the ditransitive construction consistently forms (i.e., subject–verb–indirect object–direct object) structure carries the meaning of transferring something from one entity to another as in the sentence “she got/give/kicked/ me a ball,” regardless of the verbs (Park & Sung, 2022). These sentence or clause-level parings between form and meaning are referred to as Verb Argument Constructions (VACs) and are analyzed as independent syntactic constructions in English and other languages ^{14, 17}.

According to the usage-based view, the acquisition of VACs as a type of construction is subject to multiple psycholinguistic factors ¹², among which input frequency is the most important driving factor. Specifically, VACs with relatively high frequency in input have a better chance to entrench in the learner's memory and are more likely to be acquired earlier and deemed as less complex, while VACs with low input frequency are considered to be acquired in later stages and thus more complex ^{13, 33}.

Another key factor predicting VAC acquisition is the contingency of mapping between a main verb and the VAC in which it occurs ¹². It refers to the degree of reliability to which a given cue (a particular verb) can predict a certain outcome (VAC) or in reverse, how likely a verb will occur given a certain VAC ¹⁸. Previous research has shown that L2 learners employ verbs that are less strongly associated with a target construction as their proficiency increases ^{18, 9}. The findings of these studies lent support for the claim that closely associated verb-VAC acquired early, and are thus regarded as less complex, while the more unusual combinations are deemed more difficult to acquire and thus more complex. For example, Kyle and Crossley (2017) found that higher-scoring L2 argumentative essays contained less frequent and more strongly associated verb-VAC combinations (Li & Yang, 2023). In keeping with the taxonomy in recent studies ^{18, 19, 22}, we measured syntactic complexity using traditional absolute complexity indices and fine-grained complexity indices that differentiate structural subtypes of clauses and phrases, and syntactic sophistication using VAC-related features, i.e., frequency and contingency features, ( see Table 1, Table 2, Table 3, Table 4 in the methodology section). In what follows, we provide a systematic review of previous studies that have examined the relationship of L2 writing quality to these different dimensions of syntactic complexity features.

A number of cross-sectional L2 studies have investigated the relationship between writing quality and different measures of syntactic complexity ^{1, 8, 36}. Most of these studies have relied heavily on various large-grained indices that quantify the average length of a certain linguistic unit (e.g., mean length sentences (MLS), mean length T-unit (MLT), mean length clauses (MLC), etc.) and the amount of subordination such as dependent clauses per clause (DC/C) and clauses per T-unit (C/T) ⁹. Based on the notion that academic writing derives its complexity from the elaborate use of clausal constructions or clausally complex, with many subordinate clauses (Hyland, 2002).

In their synthesis and meta-analyses, Ortega (2003) and Wolfe-Quintero et al. (1998) noted that the majority of these investigations, as they (1988) reported 23 studies out of 40 studies, discovered a largely consistent positive link between global complexity features and overall assessment of composition quality. To illustrate, among the earliest studies, Homburg (1984) investigated the relationship between a holistic evaluation of ESL writing quality based on the Michigan Test of English Language Proficiency grading scheme (MTELP) and 10 measures of syntactic complexity and found a significant relationship between MLS, MLC, finite clausal subordination (DC/C), and writing score. More recently, to identify key linguistic features on the writing performance of EFL learners, Chuenchaichon, (2022) analyzed a collection of opinion essays written by Thai EFL university students from CU-TEP (The Chulalongkorn University Test of English Proficiency) corpus and found significant correlations between MLT, MLC, MLS and the quality of argumentative essays score. That said, essays with longer T-units and higher amounts of subordination received higher scores and were considered quality text. This finding is also corroborated by a few longitudinal research. For instance, Bulté and Housen (2014) reported longitudinal growth in MLT scores throughout a semester-long English for Academic Purposes (EAP) L2 writing course ²⁰.

In addition, in search of more global indices, Lu (2011) included 14 measures of syntactic complexity recommended by Ortega (2003) and Wolfe-Quintero et al. (1988) and developed an automated analyzing tool, the L2-SCA, based on large-scale data gathered from the Written English Corpus of Chinese Learners (WECCL). He found that seven out of 14 measures progressed linearly across levels of L2 writing quality these were the mean length of MLT, MLC, MLS, and coordinate phrases per clause (CP/C), and T-Unit (CP/T), complex nominals per clause (CN/C) and per T-unit (CN/T). Moreover, Lu found that contextual factors like institution (i.e., the universities the learners attended), timing condition (timed versus untimed), and genre (argumentative versus narrative) have an impact on the linguistic characteristics of written products in addition to the proficiency level of L2 learners. According to his findings (2011), argumentative writing has lengthier units and is more complicated at the phrasal level than narrative essays ³⁷ or application letter writings ³⁸. Subsequent research has tested the reliability of the 14 measures of absolute complexity, large-grained, indices provided by L2-SCA, with a wide range of variables such as topics ³⁶, instructional setting ²⁸ that affect the production of complexity.

Yang et al. (2015) used seven indices from L2-SCA to analyze a corpus of argumentative essays on two topics written by ESL graduate students, scored using the TOEFL iBT independent writing rating rubric, and found that MLT significantly predicted essay scores on various topics. In another study, using 10 indices from L2-SCA, Ai and Lu (2013) examined the differences in syntactic complexity in the writing of native speakers (NS) and non-native speakers (NNS) of English, showed that NNS produced shorter clauses, sentences, and T-units, less subordination, and fewer noun phrases than do NS. This shows that EFL writers are characterized by distinct syntactic structures from L1 or L2 learners ²³. Taken together the aforementioned L2 writing studies have indicated a positive relationship between large-grained indices of syntactic complexity and writing quality.

Despite the ubiquity and usefulness of such structure-based measures (MLT, and DC/C) recently, however, a number of scholars have questioned their use to investigate syntactic complexity in L2 writing ^{5, 18}. The first, limitation is that absolute complexity measures mostly capture the degree of elaboration, but give less attention to the degree of variation, in other words, syntactic diversity ²⁷. Second, ³ argue, with corpus-based evidence, that clausal elaboration assessed by T-unit indices is more characteristic of conversation whereas academic writing is characterized syntactically by the use of noun complex phrase constituents, particularly noun phrase complexity, thus they recommend phrase-level structures as more reliable measures for L2 writing in an academic setting.

Subsequent cross-sectional and longitudinal research (e.g., Biber et al., 2016) ^{34, 38} has indicated that phrasal complexity is a better predictor of writing quality scores than clausal subordination. For instance, Kyle and Crossley (2018) reported that fine-grained phrasal complexity indices (related to nominal subject, direct object, and prepositional object modifiers) accounted for a larger proportion of variance in essay scores than both holistic syntactic complexity indices and fine-grained clausal complexity indices. Similar results were reported by Zhang and Lu (2022) in their analysis of a collection of application letters and argumentative essays produced by college-level Chinese EFL learners. Similarly, Taguchi et al. (2013) analyzed a collection of argumentative essays written by a number of non-native speakers of English in a university in the United States using ten syntactic complexity indices provided by Biber tagger and found that noun phrase modification features such as attributive adjectives, and post modifying prepositional phrases, contributed to essay quality. Demonstrating that as the academic level increased, so did the use of phrasal complexity features in writing. This finding is also corroborated by some longitudinal research, in which Crossley and McNamara (2014) discovered that high-quality university L2 learners' descriptive texts included more complex noun phrases and fewer embedded clauses.

Although the above reviewed, studies have revealed a positive relationship between different dimensions of syntactic complexity and L2 writing quality, this model is based on considering syntactic structure only and is difficult to link to the frequency of occurrence of form-meaning mappings found in usage-based perspectives on language learning. In complementing this gap, Kyle (2016) by developing and validating TAASSC proposed a range of usage-based VAC indices related to frequency and contingency as a new way of representing syntactic sophistication (M. Abdi Tabari et al., 2023). This proposal inspired a new wave of research on comparing absolute versus relative complexity measures in terms of their predictive power of L2 writing quality ¹⁸ (Li & Yang, 2023) ³⁰. This line of research has generated evidence for greater predictive power of VACs-based complexity measures over absolute ones. As an instance, using a corpus of TOEFL independent and integrated essays as well as descriptive essays selected from the Michigan State University corpus, Mostafa and Crossley (2020), found the stronger predictive power of usage-based VAC indices, as these measures explained more variance in L2 writing particularly in the argumentative task than in descriptive and integrated tasks (sources based writing).

Similarly, M. Abdi Tabari et al., (2023) examined the extent to which a number of VAC-based syntactic sophistication indices could predict ESL learners’ argumentative writing quality in different writing task conditions and found that only verb-VAC combinations (contingency-related features) were the most significant predictor of syntactic sophistication for all groups. In another study, Li and Yang (2023) compared indices of absolute complexity and VAC sophistication in terms of their ability to predict the writing quality of the overall as well as major sections of English research articles (RA) written by Chinese Ph.D. students. The finding indicated that VAC-based measures are more useful for indexing the writing quality of RAs at the whole text level while absolute measures have a stronger predictive power at the part-genre level, pointing out that both holistic and fine-grained indices usefully complement each other in capturing the syntactic complexity of L2 production.

Three important observations can be made in the body of research reviewed above. First, the majority of studies of the relationship of syntactic complexity to L2 writing quality have primarily focused on advanced and college-level L2 learners’ writing for large-scale standardized test samples (e.g., TOEFL) in the English-dominant context, without taking into account relevant contextual factors including in particular texts written by different EFL learner groups in different socio-cultural contexts. Additionally, few studies on L2 writing have systematically examined the writing produced by these students within the context of their writing classes, though with a few exceptions ³¹. Furthermore, the recent conceptualization of syntactic complexity as a multidimensional construct ²⁷, calls for the use of multiple measures that tap into multiple dimensions of complexity, yet as the forgoing review highlighted that there are no studies that explore the syntactic complexity employed a large set of multiple measures in a single study ³⁰. Specifically, in the Ethiopian context to the best of our knowledge, no single study has systematically investigated the link between writing quality and syntactic complexity features using a large set of measurements including global, fine-grained clausal and phrasal measures and syntactic sophistication thus far.

Despite the paucity of research comparing the relationship of different types of syntactic indices to the L2 writing quality of university-level students in the Ethiopian context, there are at least three reasons for studying it regarding the classroom context. First, Ethiopian students seem to suffer from several difficulties in L2 writing in that expressing complex ideas in words and sentences to form coherent writing is a challenging task (Dawit, 2013; Eskinder, 2018). In particular, at the university level, the researchers, and instructors at university, have observed the texts by university students are generally shorter, not discipline-specific, and less dependent on the use of reliable evidence or sources to defend one’s arguments; these are evidenced by their tests, examinations, class works, assignments and senior essay papers. Second, it is clear that a writing sample (i.e., an argumentative essay) produced in a classroom context may produce different syntactical features than writing samples produced in high-stakes assessment settings. Third, the assessment of students' writing samples is generally perceived to be very demanding and time-consuming for writing teachers. It is, therefore, necessary to investigate the relationship between argumentative writing quality and certain syntactic features to inform teachers’ manual rating, as well as to lay a foundation for the development of automated programs to supplement manual assessment.

Motivated by the afforded mentioned research insight and discussed gaps in the literature the present study aims to investigate the extent to which a number of syntactic complexity and VAC-based syntactic sophistication indices could predict EFL learners’ argumentative writing quality. To this end, we focus on two specific questions:

What is the relationship between syntactic complexity (with its different dimensions) and the quality of EFL students' writing?

What is the predictive power of syntactic complexity (with its different dimensions) on EFL students’ writing quality?

Thirty (13 female and 17 male) first-year undergraduate students, studying at Mekedela Amba University (MAU) in Ethiopia, served as the participants of this study. The participants were recruited through stratified random sampling their ages ranged from 19 to 25 years old (M =21.1; SD=1.4) and they had different first language (L1) backgrounds, including Amharic, Awi, and Oromia meanwhile, they had received formal English language education for 12 years in school. The participants were in a context where English is taught as a foreign language and thus have almost no access to producing something outside the classroom. At the time of this study, the participants were studying an Academic English course namely Communicative English II. This course aids first-year students in enhancing their overall communicative competence in particular the writing section striving to develop the higher-level academic writing abilities required to flourish in their disciplinary courses so they have background knowledge and prior experience in writing an argumentative essay. Additionally, the participants reported that they were familiar with writing an argumentative genre and found it easy to do so. The participants' mean rating for their level of familiarity with writing in an argumentative genre was 5.2 on a 9-point Likert scale (1 = least familiar, 9 = extremely familiar). On another 9-point Likert scale (1 = least difficult, 9 = most difficult), the mean rating for perceived difficulty in writing in an argumentative genre was 5.2.

Data for this study were collected through an argumentative writing task adapted from A. Fazilatfar et al. (2020) study for several reasons. Firstly, argumentative writing is a common mode of academic discourse in many L2 writing courses. Secondly, previous studies have reported that argumentative writing tasks elicit more complex syntactic structures than other writing tasks such as narrative. The task prompt required the participants to justify their opinion with the following statement:

Some people believe that the Internet causes many problems although it has a lot of advantages. To what extent do you agree or disagree? In your writing, use your own ideas and experience, and support them with examples and relevant evidence

This task was selected to minimize the variability that could have resulted from the topical knowledge; the researchers assumed that the topic of the 'Internet' enabled them to tap into the participants' perspectives regarding a broadly discussed social issue, rather than other personal issues, was probably more related to university level students. The participants were given 30 minutes for the writing activity. In this way, both time and topic constraints were controlled to make results comparable ³⁵.

After receiving approval from the Mekedla Amba University community services and research directorate office, we contacted the instructors who had agreed to participate in the study. We explained the study's objectives and procedures to them, and they agreed to conduct the writing experiments as in-class activities and award bonus credits to students who submitted the essays through the course site. They all participated in the research voluntarily, signing an informed consent form. Participants completed the writing task with paper and pencil in a regular writing classroom, and their English teachers administered the procedures. First, the students completed a self-assessment questionnaire adopted from (M. Abdi Tabari et al., 2023). Then, the teacher distributed the writing prompt and the essay sheet in total, 30 argumentative essays were collected. Two-course instructors, who had master's degrees in English, graded the essays. Intra-class correlation coefficients (ICCs=.870, 95% CI showed 0.671˗˗0.957, p=.000; which was high) were calculated to check for the inter-rater reliability, we used a two-way mixed effects model with absolute agreement definition, which assumes that the essay was rated by two or more raters and that these raters were the only raters participating in the study. For computational analyses of syntactic complexity measures, all the essays were transferred from handwriting to typed texts and saved in a .txt file format which is the format required by the TAASSC program.

The current study employed computational tools namely TAASSC to measure the syntactic features of the writing sample. This is mainly; because TAASSC is a freely available and fully automated text analysis tool (cf Biber Tagger) ³. Furthermore, previous studies have reported that the fine-grained clausal and phrasal as well as syntactic sophistication indices provided in TAASSC are more detailed than the indices proposed by other tools ^{18, 30}. TAASSC calculates approximately 370 indices of syntactic variation. Each index is calculated about four sub-corpora written registers in the Corpus of Contemporary American English (COCA) namely, academic, fiction, magazine, and newspaper (Davies, 2015).Since our samples are academic writings, we only used the indices generated from the academic sub-corpus ofCOCA.After the target, syntactic feature indices were obtained for each writing sample, or the results of TAASSC calculation were analyzed in SPSS (Ver. 21) for significance.

A pilot study was conducted to test the reliability of the POS tagging and TAASSC parsing in learner data. We chose two random essays 5 and 28 and the automated analysis was manually checked; we found high accuracy, with POS tagging at 98% and TAASSC parsing at 93% with minor difficulties encountered in unknown or misspelled words. The precision of the tagging and parsing may be due to the written, punctuated style of the text.

To assess the internal complexity of the text such as syntactic embedding in a largely holistic way, we used the 14 indices reported by Lu’s (2011) L2SCA analyzer embedded in TAASSC. These indices calculate syntactic complexity into five dimensions with different measures in each dimension. Table 1 summarizes the complexity measures included in L2SCA. It should be noted that in the L2SCA a clause is defined only as a structure with a subject plus a finite verb. Thus for the subsequent more detailed analysis, we included a broad range of fine-grained clausal and phrasal measures made use of TAASSC ¹⁷.

We specifically selected 31 fine-grained clausal complexity indices tap into the amount of usage of specific types of dependent clauses structure from TAASSC, which are seen in Table 2. As seen in the table below twenty-nine indices calculate the average number of particular structures per clause. As well, TAASSC calculates two additional general indices of clausal complexity that take into account the total number of dependents per clause and provide a measure of syntactic variation.

Given the importance of phrase-level syntactic complexity in academic writing, we used 66 phrase complexity indices from TAASSC. The rationale for this is unlike to L2SCA; TAASSC phrasal complexity measures focuses on noun modifiers (dependent types) and the place that noun modifiers occur (phrase types). The terminology and abbreviation of dependents and phrase types are presented in Appendix A for clarity purposes. In addition, TAASSC considers standard deviation in order to measure syntactic variation. Table 3

To measure this dimensions of complexity, we also selected indices that calculated verb-argument construction complexity, such as the frequency of VAC, TTR of verb argument constructions, and the strength of association.

Foremost, we calculated VAC frequency indices using TAASSC which include average frequency scores for main verb lemmas (e.g. to make), VACs (e.g. subject-verb-direct object), and verb-VAC combinations (e.g. subject-to make- direct object) in a target text (in this case the writing samples) based on counts derived from COCA. These measures were calculated based on both the tokens (all instances of a particular structure) and types (unique instances of a particular structure) of the structures extracted from the text and standard deviations for these scores. To account for the potential Zipfian nature, distributional skewness, of the data, we also included the logarithm-transformed versions of these frequency measures. Besides, we also counted type-token ratios (TTR) for main verb lemmas, VACs, and verb-VAC combinations to provide measures of the diversity of the VAC structures in the text. These indices comprise a rough measure of frequency. In the current analysis, in total 18 frequency indices based on the frequency norms derived from the academic section of COCA are used (as seen in Table 4).

We also measured the association strength of the VAC structures, i.e. the conditional probability that the main verb lemma and a VAC will occur together. TAASSC utilizes three types of association strength measures to calculate mean association strength scores for both the types and tokens of each VAC structure in a target text: faith, delta P, and collostructional analysis. TAASSC also calculates intra-text standard deviation scores for tokens. Faith calculates the probability of whether the main verb lemma (the outcome) will occur given a particular VAC structure (the cue), and vice versa. Delta P is similar to faith but is a more conservative method. It measures the probability of an outcome given a cue minus the probability of the outcome without the cue. In addition to, Faith and Delta P that measure association strength directionally, we used the average, the ratio and standard deviations for collostructional strength analysis computes the joint probability that two items (the main verb lemma and the VAC structure) will co-occur without the concern for directionality (for a total of 17 indices).

We pruned the 146 syntactic complexity indices before statistical analysis. First, the normal distribution of these indices was verified by assessing their skewness and kurtosis levels, syntactic features that were normally distributed were included in further consideration. After that, Pearson correlations were performed on the remaining syntactic features to see if there was any real association between them and the holistic essay scores. Any syntactic features that did not achieve an absolute correlation value of r.100 with the holistic essay score (which Cohen, 1988 defines as the threshold for a "small" influence) and a significance level of p.001 were eliminated from further consideration. For those variables, we controlled for multicollinearity to ensure we did not include indices that measured similar language features. We did so using VIF (variance inflation factors) test score such that any variables showing a VIF below ten were retained (Levshina, 2015).

Based on the correlational results, (for RQ2) we built five multiple regression models in a stepwise fashion independently for each index type (traditional, fine-grained clausal, fine-grained phrasal, VACs indices and combined all these measures) to investigate the predictive power of the various syntactic features measures (as predictor variables) on human ratings of essay quality (as an outcome variable). Since multiple regression shows combined and independent contributions of predictors, this analytical strategy was used to determine the extent to which the selected syntactic features predict the human-rated holistic writing score. To determine goodness of fit of the multiple regression model we report, Adjusted R-squared (Adj.R²) this is because unlike the R-squared (R²) value assumes all the independent variables considered affect the result of the model, adjusted R-squared only adds new predictors to its model if it improves the model's predicting power.

First, the potential for the 14 indices in Lu's (2011) SCA to explain the variance in holistic scores of essay quality was investigated. To achieve this, a number of preliminary analyses were performed to confirm that the data was appropriate for correlation and stepwise multiple regression analysis. Per se, eight of the 14 indices violated the assumptions of normality and were excluded from further investigation. Even though the remaining six indices demonstrated normal distributions, five of these indices did not reach the minimum correlation thresholds of r> = 0.100 and p < .001 and were removed from further consideration. One variable namely the number of complex nominals per T-unit (CN_T), which measured phrase level elaboration, demonstrates a strong and nearly significant correlation with holistic writing score (r=.342, P=.065), but was not included in the predictor model. Overall, the result revealed that there was no significant relationship between the traditional syntactic complexity indices and writing quality, implying that these indices had little explanatory power on L2 writing quality ^{19, 30, 38}

As for clausal level complexity measures, twenty of the 31 clausal complexity indices violated the assumption of normality (e.g., passive constructions), due to rare occurrences in participants’ essays and were excluded from further examination. Seven of the remaining 11 variables did not meet the minimal correlation requirements of r > = 0.100 and p < .001 and thus were eliminated from further analysis (e.g., direct object). As seen in Table 5 only four variables, namely the frequent usage of adverbial modifiers, modal auxiliaries, nominal complement, and nonfinite clausal complement per clause, which demonstrated meaningful relationship and had a large effect size with writing score, were entered into a stepwise regression analysis to see if the retained indices may explain the variation in L2 argumentative writing scores.

The resulting model, as seen in Table 6, comprised two predictors, namely the use of adverb modifier per clause and modal auxiliary per clause was significant (r= .696, adj. R²=.446, F_{2, 15.83} = 12.64, p= .000) and explained 44.6% of the variation in writing scores. To look at the unique contribution of each of the predictors the coefficient model was assessed, the coefficient of adverb modifier per clause was (β= -.529, t= -3.782, P < .001) and the coefficient of modal auxiliary per clause was (β=.376, t=2.687, P<.005).

Simply put, the model showed a statistically strong negative relationship between adverb modifiers per phrase and a moderately significant positive relationship between the use of modal auxiliary per clause and writing scores implying that higher-quality essays typically contain fewer adverbial modifiers and more modal verbs per clause, the result is in line with ^{4, 30}.

As previously mentioned, the current study takes into account 66 phrasal complexity indices of which 16 showed normal distributions. After pruning these 16 variables, three indices (see Table 7) specifically, determiner per nominals, determiner per direct object, and possessive per direct object demonstrated a meaningful relationship with writing score and were entered in the stepwise regression model.

The resulting model (Table 8) consisted of one index namely the use of possessives per direct object was significant and explained 24.3% of the variance in holistic essay scores (r= .519, adj. R²=.243, F_22.40 = 10.334, P <.005). The coefficient of dependents possessives per direct object was (β= -.519, t= -3.215, P=.003). The model demonstrated a strong significant negative correlation between the number of possessive phrases per direct object and writing scores, implying that higher-scoring essays contain fewer dependents, such as possessive and more determiners, as modifiers indirect objects.

Overall, the model showed that indicators of phrasal elaboration were indicators of the quality of the essay.

Penultimately, to examine the relationship between syntactic sophistication and writing quality 35 VAC indices (18 frequency related and 17 related to association strength) were employed, of these ten indices violated the premise of normalcy and were excluded from further consideration. Twenty-one indices out of the 25 remaining variables did not meet the minimum correlation standards of r > = 0.100 and p <.001 and were therefore eliminated from additional analysis. The remaining four variables (presented in Table 9) were included in a stepwise regression analysis.

The final model, as seen in Table 10, comprised a single syntactic diversity measure namely verb lemma frequency type was significant (r=.503, adj. R² =.227, F _22.9 = 9.51, p =.005) and explained 22.7% of the variation in writing score. The coefficient of this index was (β= 6.5E-006, t= 3.08, p=.005) revealing that essays with a wider range of frequent VAC structures received a better writing score.

Finally, the ability of the 11 syntactic complexity and sophistication variables included in each preceding regression model to explain variance in essay holistic scores was explored. As all of these variables satisfied normality and minimum correlation with writing score, and none of the indices exhibited collinearity, a stepwise regression was performed on all of them. The resulting model, see Table 11, included two fine-grained clausal complexity and one syntactic sophistication index was significant (r=.762, Adj.R² =.532, F ₃, 12.87 = 11.99, p=.000) and explained 53.2% of the variation in writing scores, suggesting that the best predictive results were achieved by the combination of syntactic complexity and sophistication features. The results indicated that clausal elaboration and lemma type-token ratio contributed to a model of development in a complementary manner.

The study provides predictors of writing quality at different dimensions of syntactic complexity, allowing EFL instructors to design classroom activities to promote writing quality. Put simply, as our study indicates that an increased repertoire of certain complex structures along with the flexible use of frequent verb-VAC patterns is likely to improve writing quality, we believe that incorporating and explicit teaching some of the clausal and phrasal complexity structures and VAC-based elements into writing programs will be extremely beneficial. We contend that this kind of instruction could make use of genre-based pedagogy and explicit instruction, which helps students create complex structures that are in line with academic writing conventions and content is particularly promising. Forinstance, Casal and Lu (2021) drawn on, Concept-Based Instruction (CBI) approach, propose an instructional activities such as complexity-focused activities targeting essential complexity features (e.g., clausal complement or prototypical VACs) via reflection-oriented group discussions or individual sample text-analysis activities targeting the frequency and functional affordances enable advanced students internalize such features.

Furthermore, Ethiopian teachers should compile a corpus of argumentative essay, identify key verb-VAC expressions, and analyze their distribution across different sections. This can be used for independent analysis and reflection-oriented group discussions on the functions of key verb-VAC patterns. As previously stated, EFL teachers face challenges in manual evaluation of academic genres in classrooms, as they have few criteria or rubrics to rely on. The findings of the study provides a pool of linguistic features predictive of argumentative quality, allowing for the development of a writing rubric especially the coefficients can be used to set weights for different descriptive factors, enhancing construct coverage and assessment accuracy as well the efficiency of EFL teachers ¹⁵.

Finally, this study provide valuable insights using an automated assessment system to help EFL teachers provide detailed writing feedback to students at different dimensions of syntactic complexity because we believe that a finer-grained analysis can be rendered possible by developing an automated assessment system that can predict the writing quality of different genres based on carefully pre-selected features.