Perceptual Learning and Perceptual Recognition II: Workshop Report By Kevin Connolly, John Donaldson, David M. Gray, Emily McWilliams, Sofia Ortiz-Hinojosa, and David Suarez This report highlights and explores five questions which arose from the workshop on perceptual learning and perceptual recognition at the University of Toronto, Mississauga on May 10th and 11th, 2012. 1. How Should We Demarcate Perceptual Learning from Perceptual Development? Eleanor Gibson defines perceptual learning as, "any relatively permanent and consistent change in the perception of a stimulus array, following practice or experience with this array" (1963, p. 29). Consider two such examples from William James. James writes, "One man will distinguish by taste between the upper and lower half of a bottle of old Madeira. Another will recognize, by feeling the flour in a barrel, whether the wheat was grown in Iowa or Tennessee" (1890, p. 509). These are examples of perceptual expertise. However, ordinary perceptual development-such as the natural improvement of visual acuity in children-is a relatively permanent change in the perception of a stimulus array, following practice or experience. Does ordinary perceptual development then count as perceptual learning? During the workshop, Daphne Maurer proposed a way to demarcate perceptual learning from perceptual development. There is, on the one hand, developmental tuning-a kind of change which is primarily due to pruning and is the consequence of maturation. For example, children get worse at discriminating non-native speech sounds as they grow older. Their perceptual systems become tuned to native speech sounds. On the other hand, when we speak of perceptual learning, we are generally talking about adults, or about certain kinds of perceptual training with children. In perceptual learning, one intervenes in the system in order to get better 2 at a certain task. For example, Rob Goldstone's talk outlined how people who have learned enough mathematics attend to mathematical equations in a way that follows the order of operations. So, for instance, if they are given the equation 5 + 3 x 7, they attend to the "x" before the "+" since that's the order in which the problem gets solved. This led him to suggest that one can train perception and action systems "to do the right mathematical thing." Maurer admitted that her way of demarcating perceptual learning from perceptual development presupposes that during development a child's brain is plastic and easily changeable, while during adulthood things are less plastic and much harder to change. Michael Rescorla challenged this view, and Maurer granted that if we drop this assumption, perhaps the kinds of perceptual learning we have distinguished begin to blur. References: Gibson, E. J. (1963) "Perceptual learning." Annu. Rev. Psychol. 14, 29–56. James, William (1890). The Principles of Psychology. New York: Henry Holt and Company. 2. What Are the Origins of Multimodal Associations? What are the origins of multimodal associations, that is, associations between features detected by distinct sense modalities? There are two main possibilities: (1) multimodal associations are the result of neural hardwiring, or (2) multimodal associations are learned responses to environmental correlations. In her talk, Cecilia Heyes discussed a specific kind of multimodal association, namely, the association between when a person performs an action and when that person observes that same type of action. Mirror neurons are neurons that fire in both cases. Heyes argues that these neurons are a byproduct of learned visual/tactile associations. That is, she thinks that correlated 3 sensorimotor experience forges mirror neurons, such that without such experience, we would not have such neurons. On Heyes' view, mirror neurons arise from associative learning (see Heyes, 2009). She rejects the nativist hypothesis that mirror neurons are present at or shortly after birth such that the role for experience in the development of these neurons is limited to 'tuning'. If Heyes is right, then this may be taken as evidence in favor of the hypotheses that at least some multimodal associations are learned responses to environmental conditions. This associative hypothesis may or may not extend more broadly, to sensory integration in general. Daphne Maurer argued in her talk that in neurological development, infants begin life with lots of connections between areas of the brain that seem later to handle discrete modalities (see Maurer, Gibson, and Spector, 2012). For instance, the visual cortex initially receives input from many different sense modalities. Connections from the visual input end up getting reinforced, while the other connections are pruned away. Experience plays a role in the pruning of connections, strengthening those that correspond to the environment, and doing away with most of the rest. If this is right, then it is evidence that some multimodal associations may be the result of neural hardwiring. Though these connections are reinforced and strengthened by the environment, they are not learned. That is, the connections do not arise anew as the result of a learning process. Louise Richardson pointed out in her comments on Maurer's talk that although it is tempting to conclude from the evidence that infant perception is multimodal, in infants, the sensory cortices are not yet specialized. So, evidence of (what in adults is) an auditory area responding to visual input is not evidence that the infant's experience of that input is partly auditory. Nonetheless, if these early connections persist once the sensory cortices become specialized, then it seems reasonable to conclude that some multimodal associations are the 4 result of neural hardwiring. From all this, we might conclude that both (1) and (2) have a role to play. Some multimodal associations are the result of neural hardwiring that is present at birth, and reinforced/strengthened by exposure to environmental correlations. And some such associations are the result of associative learning. In particular, the mirror neurons that associate visual with tactile experience come about as a result of associative learning. References: Heyes, C. M. (2010). "Where do mirror neurons come from?" Neuroscience and Biobehavioural Reviews, 34, 575-583. Maurer, D., Gibson, L. C., and Spector, F. (2012). "New insights into the development of multisensory perception." In Bremner, Lewkowicz, and Spence (Eds.) Multisensory Development. Oxford: Oxford University Press. 3. Does Our Representation of Time Provide an Amodal Framework for Multi-Sensory Integration? A previous workshop discussed the idea that amodal representations of space and time could provide a framework for integrating information from different sensory modalities (see http://networksensoryresearch.utoronto.ca/Brown-Q4.html). The current workshop discussed evidence from experimental psychology suggesting that temporal features are used to coordinate information from proprioception and vision. This evidence lends some support to a temporal framework hypothesis, which claims that our representation of time serves as an amodal framework for multi-sensory integration. There are some obstacles to drawing such a conclusion, however. Even if temporal features are used for multi-sensory integration, these features might not be represented amodally, since each modality might encode temporal features differently. In her talk, Cecilia Heyes discussed how we are better at visually recognizing our own 5 bodily movements than at recognizing the movements of friends, even when those movements are reduced to point-lights and shown from a third-person point of view (Prasad and Shiffrar, 2009). Visual experience seems to be insufficient to explain this advantage because we typically have far more experience viewing others than we do viewing ourselves. What then explains the visual self-recognition advantage? In a series of experiments testing visual recognition of recorded facial movements (displayed by means of anonymized computer avatars), Cook, Johnston, and Heyes found that although manipulating the spatial orientation of facial movements impeded the recognition of friends' movements, it had little effect on selfrecognition (2012). By contrast, manipulating the temporal features of recorded movements impeded self-recognition far more than manipulation of spatial features. For instance, changes in timing and rhythm affected visual self-recognition far more than changes in orientation and topographical configuration. Since by itself visual experience seems insufficient to account for the visual self-recognition advantage, it is plausible to suppose that the advantage depends instead on a transfer of information across different sensory modalities. One reasonable hypothesis is that information derived from first-person proprioception is deployed in the visual self-recognition of bodily movements. Cook, Johnston, and Heyes' results suggest that it may be temporal information in particular which enables the requisite coordination of representations across different modalities. While it would be overly hasty to conclude that these results confirm the temporal framework hypothesis, nevertheless, if temporally-based integration turned out to be very prevalent, the hypothesis might be more plausible since the existence of such a framework would help to explain a great many cases of sensory integration. Most troubling for supporters of the temporal framework hypothesis, however, is the possibility that there is a 'Molyneux problem' for temporal features. That is, it might be the case 6 that each modality encodes temporal features differently (as suggested by Barry Smith), and that the correspondence of temporal features across different modalities must be learned through the association of modality-specific representations. If the functioning of unimodal perceptual mechanisms turned out to underlie temporally-based sensory integration, this would make the temporal framework hypothesis less plausible, since it would reduce the need for an amodal representational framework. References: Cook, R., Johnston, A., and Heyes, C. (2012). "Self-Recognition of Avatar Motion: How do I know it's me?" Proceedings of the Royal Society B: Biological Sciences 279 (1729) (February 22): 669–674. doi:10.1098/rspb.2011.1264. Prasad, S., and Shiffrar, M. (2009). "Viewpoint and the recognition of people from their movements." Journal of Experimental Psychology: Human Perception and Performance 35 (1) (February 1): 39–49. 4. What Counts as Cognitive Penetration? In his talk, Rob Goldstone presented a putative case of cognitive penetration: people internalize mathematical rules in a way that modifies their perception. In particular, people proficient in mathematics attend to equations in a way that follows the order of operations. So, for instance, if they are given the equation 14 4 ÷ 2, they will attend to the "÷" before the "-" in conformity with the order of operations. In his commentary on Goldstone's talk, Michael Rescorla gave a brief history of research into cognitive penetration. The New Look movement in psychology, which arose in the middle of the 20th century, held that cognitive penetration was ubiquitous, citing, for example, studies purporting to show that hungry perceivers would see more items as edible, and impoverished perceivers would see coins as larger. This brought into doubt the prima facie plausible claim that 7 there can be a tribunal of experience-that perception can be made to answer to beliefs. In the 1980s, however, Jerry Fodor and Zenon Pylyshyn argued that perception was in fact modular and segregated from cognition, and that if there was any cognitive penetration then it was of a sort so trivial as not to be worthy of the name: shifts in attentional focus, choosing to wear glasses, and so on (Fodor & Pylyshyn 1981; Fodor 1983; Pylyshyn 1999). Rescorla suggested that if take on board the lessons of the historical debate over cognitive penetration, we might then conceive of Goldstone's case, not as a case of cognitive penetration, but as a largely attentional phenomenon. In the Q and A, Kevin Connolly pushed Rescorla's point. Suppose we train someone with no knowledge of the meaning of the mathematical symbols to attend to equations in a way that follows the order of operations. What this shows in the mathematics case is that knowledge of the order of operations is not constitutive of the perception that the person knowledgeable of math has. Rather, there is something more basic, the attentional habit common to both the math expert and the person with no knowledge of mathematics who is trained to attend in the way that the expert does. If one has that habit, then one might have the same type-experience that the math expert has, even if one has no knowledge of the order of operations. Goldstone's talk also offered a second, perhaps more convincing case of cognitive penetration. and may look similar, at least at first glace. However, to someone proficient in mathematics, in the second case the 4s look like they can be canceled, but not in the first case. Fiona Macpherson suggested that the grouping of the 4s in the second case might actually manifests itself in one's perceptual phenomenology. The idea is that unlike in the first case, in the second case we see the 4s as grouped together. It is a putative case of cognitive penetration in which one's knowledge of mathematics affects one's perceptual phenomenology 8 through the grouping of particular numbers. References: Fodor, J. (1983) Modularity of Mind, Cambridge MA: MIT Press. Fodor, J. & Pylyshyn, Z. W. (1981) 'How Direct is Visual Perception?' Cognition, 9: 139-196. MacPherson, F. (2012), 'Cognitive penetration of colour experience: rethinking the issue in light of an indirect mechanism,' Philosophy and Phenomenological Research, 84 (1): 24-62. Pylyshyn, Z. W. (1999) 'Is Vision Continuous with Cognition? The Case for Cognitive Impenetrability of Visual Perception,' Behavioral and Brain Sciences, 22: 341–423. 5. How Can Philosophers and Psychologists Most Fruitfully Collaborate? Much of the final panel discussion focused on what the goals of interdisciplinary work between philosophers and psychologists should be, and how those goals might be best achieved. The panelists and other workshop participants identified three potential goals: (1) to investigate traditional philosophical questions empirically; (2) to construct and test psychological theories; and (3) to develop and clarify the concepts employed in both disciplines. Several suggestions were made regarding how philosophers and psychologists might most effectively collaborate in the pursuit of these goals, and extract maximum benefit from each other's research and expertise. First, philosophers might benefit from psychologists by simply becoming more familiar with their work. Diana Raffman observed that philosophy of mind is or ought to be constrained, and even directly informed, by psychologists' data and theories. Mohan Matthen and Michael Rescorla mentioned Tyler Burge's allegations that the theories of contemporary naïve realists are inconsistent with empirical findings in psychology (Burge, 2005), highlighting the potential disadvantages of conducting philosophical inquiry into the nature of the mind without sensitivity to recent psychological research. 9 Second, beyond simply being informed by psychological research, philosophers might also play a role in setting the agenda for that research. Cecilia Heyes noted that the institutional incentives in psychology discourage theoretical work in favor of experimentation (though Rescorla noted that some theoretical work is being actively done by psychologists, for instance Susan Carey's 2009 work on concepts, leaving a vacuum for others to potentially fill. Fiona Macpherson proposed that philosophers might use their skills in conceptual analysis and theory building to help psychologists construct theories and devise experiments to test those theories. This could make it much easier for philosophers to probe traditional philosophical questions empirically. However, two problems were raised for this second suggestion. Heyes argued that psychologists are not always interested in the questions philosophers are asking and may not want philosophers to help set their research agendas. She cited questions about phenomenology as an example of questions that would not be of interest to most psychologists. She warned that philosophers sometimes want psychologists to pursue an agenda that "is not in the nature of their discipline to be pursuing." Additionally, Andreas Keller pointed out that philosophers tend to want scientific theories to be stated partly in terms of the concepts that figure in traditional philosophical questions, whereas psychologists often prefer theories that elide those distinctions in favor of a simpler but still powerful explanatory framework. Thus the philosopher's preference for psychological theories that have obvious relevance to philosophical questions and the psychologist's imperative to construct theories with the theoretical virtue of simplicity may put them at cross-purposes. Heyes agreed, expressing skepticism that the best scientific theories of 10 the mind that we can devise will be of clear relevance to the kinds of questions that philosophers want to ask. Third, workshop participants from both fields suggested that psychologists might benefit from philosophers through the kind "conceptual therapy" philosophers can provide. Rescorla and Matthen emphasized, for example, the potential benefit to psychology of an appreciation of recent work on the nature of representation. Heyes suggested that psychologists would be more welcoming of assistance from philosophers if it were targeted at the kinds of questions that psychologists are already interested in. Fourth, and finally, perhaps philosophers and psychologists can aid one another by simply providing a fresh pair of eyes, problematizing assumptions that from a different theoretical perspective may seem obvious. Heyes mentioned an example from the workshop itself: Louise Richardson (philosophy, Oxford) asked a question during the Q&A following her talk that led her to question an assumption that she had previously taken to be obvious. References: Burge, Tyler (2005). "Disjunctivism and Perceptual Psychology", Philosophical Topics, 33: 1– 78. Carey, S. (2009). The Origin of Concepts. Oxford University Press.