Development of Recognition of Pragmatic Routines
At the beginning of the semester, pretests of the VKS were conducted. Looking at the descriptive statistics, the AH group (M = 17.42; SD = 4.74; Min = 5; Max = 24) performed better than the SA group (M = 17.23; SD = 4.38; Min = 7; Max = 24), which is contrary to the anticipation. While results from independent-samples T test (t = -0.189; df = 84; p = .850 > 0.05) indicated that the difference between the pretests of the two groups was not significant. The effect size of the T test was very small (d = 0.04), which suggests that although the AH group performed better, its advantage was very slim. Unitary linear regression analyses showed that in the SA group (β = 0.823; p = .000 < 0.05; R² = 0.678), intensity of interaction significantly influenced recognition of pragmatic routines. While in the AH group (β = -0.042; p = .787 > 0.05; R² = 0.002), there was no such significant relationship. This may be due to the nature of the two learning contexts. The SA group participants can contact with NSs frequently, however, the majority of the AH group participants do not have opportunities for exposure.
In the end of the semester, post-tests were conducted. From the descriptive statistics of the SA group (M = 21.47; SD = 4.35; Min = 10; Max = 25) and the AH group (M = 19.16; SD = 3.87; Min = 10; Max = 25), it can be found that the former outperformed the latter. Independent-samples T test (t = 2.592; df = 84; p = .011 < 0.05) demonstrated that the difference between the two groups in routine recognition was significant. Effect size of the T test was medium (d = 0.56). This means learning contexts differentiate the two groups after a semester. According to the unitary linear regression analyses, intensity of interaction can to some extent predict recognition of routines in the SA group (β = 0.701; p = .000 < 0.05; R² = 0.491), while for the AH group (β = 0.063; p = .688 > 0.05; R² = 0.004), the relationship has not been demonstrated.
To find out gains of the two groups in pragmatic routines recognition over a semester, paired-samples T tests were conducted. Results suggested SA group gained significantly in recognition of pragmatic routines (M = -4.23; SD = 6.75; t = -4.112; p = .000 < 0.05), the effect size was medium (d = 0.63). While the AH group did not (M = -1.744; SD = 5.80; t = -1.971; p = .055 > 0.05), the effect size was small (d = 0.30). From the data analysis, it can be assumed that although students in both groups made progress after a semester, the amount was definitely not the same, which verified the assumption that SA is more advantageous in pragmatic development.
Nevertheless, the degree of difficulty of 13 pragmatic routines involved in the VKS is different to both SA and AH group. Gain ratios of the 13 pragmatic routines among SA group participants are demonstrated in Table 4 − 1, and statistics of the AH group are in Table 4 − 2.
Table 4-1
Recognition of pragmatic routines (SA group).
| | | Gains | |
Pragmatic routine | Pretest score | Post-test score | score | % |
That works for me | 1.14 | 1.74 | 0.60 | 15.12 |
Help yourself | 1.16 | 1.67 | 0.51 | 12.79 |
Thanks for your time | 1.26 | 1.72 | 0.47 | 11.63 |
Do you want to come to my place? | 1.00 | 1.44 | 0.44 | 11.05 |
Would you mind? | 1.40 | 1.84 | 0.44 | 11.05 |
Do you think you could make it? | 1.40 | 1.81 | 0.42 | 10.47 |
I gotta go | 1.26 | 1.58 | 0.33 | 8.14 |
Can I get you anything else? | 1.23 | 1.53 | 0.30 | 7.56 |
I was wondering | 1.58 | 1.84 | 0.26 | 6.40 |
Thanks for coming | 1.47 | 1.72 | 0.26 | 6.40 |
My bad | 1.56 | 1.77 | 0.21 | 5.23 |
Could you do me a favor? | 1.74 | 1.79 | 0.05 | 1.16 |
Do you have the time? | 1.05 | 1.00 | -0.05 | -1.16 |
As can be found in Table 4 − 1, gain scores varied from 0.60 (15.12%) in "That works for me" to -0.05 (-1.16%) in "Do you have the time?", the average was 0.33 (8.14%). On one hand, students experienced the highest gains in "That works for me" (0.60, 15.12%) and "Help yourself" (0.51; 12.79%). On the other hand, they showed the lowest gains in "Could you do me a favor?" (0.05; 1.16%) and "My bad" (0.21; 5.23%). While "Do you have the time?" was the only one where students' scores decreased. The results indicated that during SA, students may come across certain circumstances frequently so they were able to acquire the prototypical routines. However, some circumstances did not occur frequently, so they were still unfamiliar with those prototypical expressions. In "Could you do me a favor?", it can be noticed that the pretest score was the highest among 13 situations (1.74), which indicated that the SA group were already familiar with it, so it was reasonable that they did not improve a lot. This explanation is also applicable to others. While, "Do you have the time?" which decreased and got the lowest score in the post-test (1.00) was the one that worth attention. Maybe to the NSs, it is a common time asking expression, while to the L2 learners, it is easily-confused since it is very similar to "Do you have time?" as in asking somebody if he or she is available or inviting someone to do something. It can be hypothesized that most participants gave wrong answers not because they could not distinguish, but due to carelessness.
Table 4-2
Recognition of pragmatic routines (AH group).
| | | Gains | |
Pragmatic routine | Pretest score | Post-test score | score | % |
Can I get you anything else? | 1.00 | 1.42 | 0.42 | 10.47 |
My bad | 1.19 | 1.51 | 0.33 | 8.14 |
I was wondering | 1.51 | 1.81 | 0.30 | 7.56 |
I gotta go | 1.16 | 1.47 | 0.30 | 7.56 |
Thanks for your time | 1.33 | 1.60 | 0.28 | 6.98 |
Do you have the time? | 1.00 | 1.21 | 0.21 | 5.23 |
That works for me | 1.23 | 1.40 | 0.16 | 4.07 |
Do you want to come to my place? | 1.07 | 1.12 | 0.05 | 1.16 |
Could you do me a favor? | 1.56 | 1.56 | 0.00 | 0.00 |
Thanks for coming | 1.53 | 1.51 | -0.02 | -0.58 |
Would you mind? | 1.63 | 1.56 | -0.07 | -1.74 |
Do you think you could make it? | 1.56 | 1.47 | -0.09 | -2.33 |
Help yourself | 1.65 | 1.53 | -0.12 | -2.91 |
As for the AH group, it can be observed in Table 4 − 2, gains ranged from 0.42 (10.47%) in "Can I get you anything else?" to -0.12 (-2.91%) in "Help yourself", with an average of 0.13 (3.35%). On one hand, students experienced comparatively high gains in "Can I get you anything else?" (0.42, 10.47%) and "My bad" (0.33; 8.14%). On the other hand, their scores decreased in "Help yourself" (-0.12; -2.91%) and three other routines. In the AH context, although the overall progress was not significant, the change of each routine varied. Students may encounter certain situations frequently, so they could get the appropriate routines. SA was not 100% advantageous than AH context as it can be found in the very routine "Do you have the time?", AH group (0.21; 5.23%) gained significantly greater than their SA counterparts (-0.05; -1.16%). It can be imagined that although the AH context was short of authentic exposure, students still could get valuable information from the Internet and classrooms, which was beneficial to their pragmatic development.
To explore the changing patterns of routine recognition, participants' answers to the VKS were analyzed, in terms of the number of responses. Pretest, post-test, and differences of SA group were displayed in Table 4 − 3, and AH group in Table <link rid="tb12">4</link>–4 (both were expressed in the number of participants and percentages).
Table 4-3
Descriptive statistics on answers to the VKS (SA group).
Answers in the VKS | Pretest | Post-test | Difference |
| N | % | N | % | N | % |
c) Recognition + plausible meaning | 222 | 39.71 | 370 | 66.19 | 148 | 26.48 |
c) Recognition + non-plausible meaning | 119 | 21.29 | 128 | 22.90 | 9 | 1.61 |
b) Partial recognition | 178 | 31.84 | 55 | 9.84 | -123 | -22.00 |
a) No recognition | 40 | 7.16 | 6 | 1.07 | -34 | -6.08 |
As can be detected in Table 4 − 3, on one hand, SA group participants improved their recognition of routines, submitting more plausible meanings and recognizing more routines, even if the meanings were not plausible enough. On the other hand, answers reporting that they have seen but didn't recognize the routine or completely haven't seen the expression decreased largely. The routine elicited most non-plausible meanings was "Do you have the time?". Most participants mistook it for "Do you have time?" and gave answers like "Are you available?" or "Can I ask you out?".
Table 4-4
Descriptive statistics on answers to the VKS (AH group).
Answers in the VKS | Pretest | Post-test | Difference |
| N | % | N | % | N | % |
c) Recognition + plausible meaning | 245 | 43.83 | 273 | 48.84 | 28 | 5.01 |
c) Recognition + non-plausible meaning | 103 | 18.43 | 195 | 34.88 | 92 | 16.46 |
b) Partial recognition | 156 | 27.91 | 83 | 14.85 | -73 | -13.06 |
a) No recognition | 55 | 9.84 | 8 | 1.43 | -47 | -8.41 |
Progress of the AH group is demonstrated in Table <link rid="tb12">4</link>–4. Although the AH group did not demonstrate significant development in routines recognition, their answer pattern is changing in a positive direction. Answers on "a) I don't remember seeing or hearing this expression before." and "b) I have seen or heard this expression before but I don't know what it means." decreased a lot, while answers on "c) I know this expression." increased notably although explanations were not always plausible.
Development of Production of Pragmatic Routines
Similar to recognition, most SA group participants made significant gains in the production of pragmatic routines. In general, SA students were approximating the NSs' norm.
At the beginning of the semester, DCT pretests were conducted. Looking at descriptive statistics, the SA group (M = 13.67; SD = 3.30; Min = 4; Max = 20) performed better than the AH group (M = 11.93; SD = 3.36; Min = 4; Max = 19). Independent-samples T test (t = 2.430; df = 84; p = .017 < 0.05) indicated that the difference between the pretests of the SA and the AH group was significant. Effect size of the T test was medium (d = 0.52). Despite students got alike scores in the pretests of recognition, they differed in production from the beginning, this maybe due to the fact that SA group have already lived in English speaking countries for a period. Unitary linear regression analyses showed that in the SA group (β = 0.667; p = .000 < 0.05; R² = 0.445), intensity of interaction significantly influenced production of pragmatic routines. While in the AH group (β = 0.091; p = .561 > 0.05; R² = 0.008), there was no such significant relationship, which is concordant with recognition results.
In the end of the semester, post-tests were conducted. From descriptive statistics, it can be found that the SA group (M = 15.67; SD = 3.16; Min = 8; Max = 21) exceeded the AH group (M = 12.95; SD = 3.75; Min = 5; Max = 21). Independent-samples T test (t = 3.636; df = 84; p = .000 < 0.05) demonstrated that there was significant difference between the two groups in routines production. Effect size of the T test was medium (d = 0.78). According to the unitary linear regression analyses, intensity of interaction can to some extent predict production of routines in the SA group (β = 0.715; p = .000 < 0.05; R² = 0.511), while for the AH group (β = 0.273; p = .077 > 0.05; R² = 0.074), such relationship has not been demonstrated.
To find out gains of the two groups in pragmatic routines production over a semester, paired-samples T tests were conducted. Results suggested the SA group students gained significantly in production of pragmatic routines (M = -2.00; SD = 4.97; t = -2.641; p = .012 < 0.05), the effect size was small (d = 0.40). While the AH group did not (M = -1.023; SD = 4.40; t = -1.525; p = .135 > 0.05), the effect size was small (d = 0.23). It is confirmed that although both groups made progress, the SA group is significant while the AH group is not. SA is more effective than AH context in pragmatic development, both recognition and production.
To shed more light on the details of pragmatic routines production, descriptive analysis was carried out on the 13 settings in the DCT. The expressions were classified into three types: high-prototypical pragmatic routines, low-prototypical pragmatic routines, and freely-generated utterances. The average number of L2 learners and NSs that produced each type of expression was demonstrated in Table 4–5 (SA group) and Table 4–6 (AH group).
Table 4-5
Production of high-prototypical routines, low-prototypical routines and freely-generated utterances. (SA group)
| Pre-test (N = 43) | Post-test (N = 43) | Difference | NSs (N = 92) |
| (n) | % | (n) | % | (n) | % | (n) | % |
High-prototypical routines | 15 | 34.70 | 20 | 45.80 | 5* | 11.09 | 66 | 71.31 |
Low-prototypical routines | 15 | 35.78 | 12 | 28.98 | -3* | -6.80 | 19 | 20.75 |
Freely-generated utterances | 13 | 29.52 | 11 | 25.22 | -2* | -4.29 | 7 | 7.24 |
Note: the values for the difference column are changes from the pretest to post-test. *p < .05 (paired-samples t-test). |
Table 4–5 introduces the changes of the SA group. Learners' L2 pragmatic routines production increased a lot in highly-prototypical routines (11.09%), decreased notably in low-prototypical routines (-6.80%) and freely-generated utterances (-4.29%). After a semester abroad, the SA learners approximated the native way of pragmatic routines production.
Table 4-6
Production of high-prototypical routines, low-prototypical routines and freely-generated utterances. (AH group)
| Pretest (N = 43) | Post-test (N = 43) | Difference | NSs (N = 92) |
| (n) | % | (n) | % | (n) | % | (n) | % |
High-prototypical routines | 14 | 31.48 | 15 | 34.70 | 1* | 3.22 | 66 | 71.31 |
Low-prototypical routines | 12 | 28.80 | 13 | 30.23 | 1* | 1.43 | 19 | 20.75 |
Freely-generated utterances | 17 | 39.71 | 15 | 35.06 | -2* | -4.65 | 7 | 7.24 |
Note: the values for the difference column are changes from the pretest to post-test. p > .05 (paired-samples t-test). |
Table 4–6 showed the changes of the AH group. The use of both high (3.22%) and low (1.43%) prototypical routines raised, and the use of freely-generated utterances (-4.65%) lowered. Overall, the development of learners' pragmatic routines production is not symbolic enough.
To probe deeper into the development pattern, the 13 situations were analyzed individually. Since the development of routines production is significant in the SA group, not in the AH group, the close analysis is conducted on the former group (N = 43) only, as demonstrated in Table 4–7.
Table 4-7
Production gains in each pragmatic routine. (SA group)
Situation | Pragmatic routines elicited | Pretest (N = 43) | Post-test (N = 43) | Difference |
| | | | (n) | % |
1. Dinner table | No thanks, I'm full | 3 | 5 | 2 | 4.65 |
No, thank you* | 12 | 15 | 3 | 6.98 |
I'm stuffed* | 10 | 8 | -2 | -4.65 |
2. Introduction | Nice {to meet/meeting} you | 22 | 23 | 1 | 2.33 |
3. Restaurant | For here or to go? | 3 | 6 | 3 | 6.98 |
How can I help you?* | 22 | 25 | 3 | 6.98 |
4. Puddle | Watch out | 21 | 28 | 7 | 16.28 |
5. Farewell | {Thanks/thank you/-} You too | 28 | 35 | 7 | 16.28 |
6. Late | Sorry I am late | 12 | 16 | 4 | 9.30 |
7. Phone | Hello? | 33 | 39 | 6 | 13.95 |
8. Borrow pen | {Could/Can/May} I borrow a pen? | 11 | 20 | 9 | 20.93 |
Do you have (a/an extra) pen I [could/can] borrow?* | 10 | 4 | -6 | -13.95 |
9. Store | No thanks, I'm just looking | 2 | 10 | 8 | 18.60 |
(No, thanks) I'm just browsing* | 19 | 15 | -4 | -9.30 |
10. Decease | I am sorry for your loss | 2 | 2 | 0 | 0.00 |
I am (so) sorry* | 14 | 12 | -2 | -4.65 |
Sorry to hear that* | 21 | 24 | 3 | 6.98 |
11.Messy house | Sorry for the mess | 9 | 15 | 6 | 13.95 |
Sorry my {place/house} is a mess* | 9 | 12 | 3 | 6.98 |
12. Piece of paper | Here you go | 31 | 32 | 1 | 2.33 |
13. Careful driving | Be careful | 16 | 27 | 11 | 25.58 |
First of all, among the 13 situations, learners' use of highly-prototypical routines increased in 12 in different degrees. And the only one that showed no progress was "Decease" which should elicit "I am sorry for your loss" (0.00%). In terms of the gains, the top three situations are: Careful driving eliciting "Be careful" (25.58%), Borrow pen eliciting "{Could/Can/May} I borrow a pen?" (20.93%) and Store eliciting "No thanks, I'm just looking" (18.60%). Progress in those pragmatic routines may benefit from the frequent appearance of corresponding situations during the learners' stay in the TL country. By comparison, the following three situations gained the least in highly-prototypical routines production: Introduction eliciting "Nice {to meet/meeting} you" (2.33%), Piece of paper eliciting "Here you go" (2.33%) and Dinner table eliciting "No thanks, I'm full" (4.65%). There are probably three reasons: First, learners may have encountered those situations before, they have already given the right answers in the pretest so that they did not show much progress. Second, learners have not encountered the situations frequently enough to acquire the corresponding appropriate routines. Third, learners were able to encounter certain situations frequently, but without appropriate routines input.
The second trend is an increase in the use of high-prototypical routines and a decrease of the low-prototypical ones. This can be observed in 3 out of the 6 situations that elicit both high- and low-prototypical routines. The three situations are: "Dinner table", "Borrow pen" and "Store". As the high- and low-prototypical standards were defined according to the NSs' agreement, to some extent, this trend can indicate that the SA group has approximated the NSs in routines production.
The last trend is an increase in the use of both high- and low- prototypical routines in one situation. This can be found in the other 2 out of the 6 situations that elicit both high- and low-prototypical routines. The two situations are "Restaurant" and "Messy house". Take "Messy house" for example, the use of high-prototypical routine "Sorry for the mess" and low-prototypical routine "Sorry my {place/house} is a mess" respectively increased by 13.95% and 6.98%.
After summarizing the three patterns in Table 4–7, it is reasonable to say that the SA group have approximated the NSs in pragmatic routines production. SA context and language exposure have benefited L2 learners a lot in pragmatic development.