- K. Koedinger, T. Mitamura, Ruth Wylie. 2009. Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor. Abstract: Is Self-Explanation Always Better? The Effects of Adding Self-Explanation Prompts to an English Grammar Tutor Ruth Wylie (rwylie@cs.cmu.edu) Human-Computer Interaction Institute, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA 15217 USA Kenneth R. Koedinger (koedinger@cmu.edu) Human-Computer Interaction Institute, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA 15217 USA Teruko Mitamura (teruko@cs.cmu.edu) Language Technologies Institute, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA 15217 USA Abstract Several studies have demonstrated the benefits of self- explanation on learning well-defined domains like math, biology, and physics. However, these findings have yet to be replicated in probabilistic domains like second language acquisition. Working with adult English as a Second Language students (n=61) within the domain of the English article system (i.e. teaching students the difference between a dog vs. the dog) we conduct the first experimental study of the effects of prompting self-explanation on second language grammar acquisition. We compare two different modes of self-explanations (free-response and menu-based), each implemented in an intelligent tutoring system, to a control tutor with no explicit self-explanation prompts. Students in all conditions show significant learning gains but contrary to theoretical predictions, the self-explanation tutors did not lead to better learning over the no self-explanation condition. We discuss why and under what specific conditions target- specific practice without self-explanation may be a more effective instructional strategy. Keywords: Self-Explanation Effect; Computer Assisted Language Learning; ESL Grammar Learning Introduction Self-explanation has been shown to be a successful learning strategy for multiple domains, contexts, and learners. One limitation of the existing work is the domains in which it has been tested have all been math and science domains like biology (Chi, et al., 1994), physics (Chi, 1989; Conati & VanLehn, 2000), and geometry (Aleven & Koedinger, 2002), and, to the best of our knowledge, there have never been any experimental studies on the effects of self- explanation on second language grammar acquisition. Thus, an open question exists: is self-explanation truly domain independent (Roy & Chi, 2005) or are there constraints to its applicability? In the original self-explanation studies, Chi et al. (1989) examined students’ spontaneous self-explanations of a physics text. This work revealed a positive correlation between the number and type of self-explanations and student learning. In subsequent experimental studies, Chi et al. (1994) showed that students who were prompted to self- explain demonstrated greater learning gains than those who were not. Furthermore, Aleven and Koedinger (2002) demonstrated that prompting self-explanations can be an effective learning strategy even when students only select a general problem-solving principle. Within the second language acquisition community, there is a large body of research that looks at implicit versus explicit instruction. A meta-analysis of the relative effectiveness of different types of second language instruction revealed that treatments involving explicit focus on rules were more effective than those that did not (Norris & Ortega, 2000). Thus, self- explanations, which highlight explicit rules, may be beneficial for the second language learner. Our goal was to see if the success of self-explanation could be replicated within second language acquisition. To this end, we developed two tutoring systems with different types of self-explanation prompts and compared student learning gains and learning efficiency scores to a control tutor that had no explicit self-explanation prompts. Results show that while students in all three conditions demonstrate significant pre-post learning gains, students in the self- explanation conditions did no better than those in the control group. In fact, a significant learning efficiency by tutor condition interaction reveals that there may be limits to the benefits of self-explanation. Adding Self-Explanation to an Existing Tutor Self-explanation prompts were added to an existing tutoring system designed to teach the English article system (teaching students the difference between “a dog” and “the dog”). In the existing system (Figure 1), developed using the Cognitive Tutoring Authoring Tools (Koedinger, et al., 2004), students select an article (a, an, the, or no article) from a drop-down menu to complete the sentence. They receive immediate feedback on their selections (the answer
