- A. Graesser, G. T. Jackson, Pamela W. Jordan, A. Olney, C. Rosé, K. VanLehn. 2005. When is Reading Just as Effective as One-on-One Interactive Human Tutoring?. Abstract: When is Reading Just as Effective as One-on-One Interactive Human Tutoring? Kurt VanLehn (vanlehn@cs.pitt.edu), Art Graesser (agraesser@memphis.edu), G. Tanner Jackson (gtjacksn@memphis.edu), Pamela Jordan (pjordan@pitt.edu), Andrew Olney (aolney@memphis.edu), Carolyn P. Rose (cprose@cs.cmu.edu) Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA 15260 USA and Institute for Intelligent Systems, University of Memphis, Memphis, TN USA Abstract Many human tutorial dialogues have a similar format (Graesser, Person & Magliano, 1995) wherein the tutor poses a question, the student answers it, and they work collaboratively to improve the student’s answer. We hypothesized that the well-known effectiveness of human tutoring was due to the last phase, and in particular, to the interactive nature of the tutorial dialogue there. To test the hypothesis, we compared tutoring which held constant the first two steps, the question and the student’s initial answer, and varied whether the student engaged in tutorial dialogue or simply read an explanation of how to derive a correct answer. To our surprise, the initial experiment showed equal learning gains. Two further experiments painted a confusing picture, but it is safe to say that interactive tutoring is not always superior to reading. Keywords: Natural language tutoring, dialogue, tutoring, learning, qualitative physics Introduction In the scientific literature, the educational enterprise, and the general public, expert human tutors are believed to provide better instruction than computer tutors or classroom teaching. Many studies (reviewed below) have tried to isolate the source of human tutors’ effectiveness. The present paper adds to the accumulating evidence. First, let us describe the type of tutoring under study. Graesser, Person and Magliano (1995) found that much human tutoring followed a 5-step pattern: 1. The tutor poses a question or problem. 2. The student attempts to answer it. 3. The tutor provides brief evaluative feedback. 4. The tutor and student collaboratively improve the answer or solution. This can involve a moderately long dialogue. 5. The tutor ends the discussion, often by asking the student if they understand, and almost always getting a positive response. Here is an example: 1. Tutor: What does a t-test tell you? 2. Student: It tests whether a mean is significant. 3. Tutor: Sorta. 4. Tutor: Can it be applied to experiments with just one group, or do you need two or more groups? Student: More than one. Tutor: Right. Because the t-test compares the means of the two groups. What does it tell you about the two means? Student: Whether they are significant. Tutor: Almost. What you care about is whether one mean is really and truly higher than the other, or whether the difference between them is just an accident of sampling. Does the t-test tell you anything about that? Student: Yes. 5. Tutor: So do you understand the t-test now? Student: Yes. Tutor: Good. Let’s go on. Graesser et al. (1995) hypothesized that the effectiveness of tutoring lies in the tutorial dialogue of step 4, and in particular, that it is the interactive nature of that dialogue which accelerates learning. For instance, learning would probably be harmed if students simply listened or read an explanation at step 4 instead of participating in a dialogue. This hypothesis is consistent with several earlier studies of human tutoring. Wood, Wood and Middleton (1978) had a human tutor implement 4 different strategies for teaching preschool children how to assemble a complicated block structure. One strategy implemented the following rule: “If the child succeeds, when next intervening offer less help. If the child fails, when next intervening take over more control.” (Wood et al, 1978, pg 133). The other strategies were less interactive. For instance, the least interactive strategy had the tutor just demonstrate the to-be-learned procedure. As predicted by the interaction hypothesis, the most interactive tutoring strategy produced the best performance on a post-test. Swanson (1992) compared the highly interactive tutoring strategy of Wood et al. (1978) to simply lecturing. As in the Wood study, the same tutor implemented both forms of instruction, but Swanson’s students were college students learning how lens work. As predicted by the interaction hypothesis, the more interactive tutoring produced more gains. Swanson also found that a second tutor could not learn to be interactive, and tended to lecture in both conditions. Chi et al. (2001) took advantage of the propensity of untrained tutors to lecture, and first had a group of tutors work with tutees naturally. These tutors were then trained to
