Philosophy Study, ISSN 2159-5313 February 2012, Vol. 2, No. 2, 73-80 Synesthesia, Experiential Parts, and Conscious Unity Rocco J. Gennaro University of Southern Indiana Synesthesia is the "union of the senses" whereby two or more of the five senses that are normally experienced separately are involuntarily and automatically joined together in experience. For example, some synesthetes experience a color when they hear a sound or see a letter. In this paper, I examine two cases of synesthesia in light of the notions of "experiential parts" and "conscious unity." I first provide some background on the unity of consciousness and the question of experiential parts. I then describe two very different cases of synesthesia. Finally, I critically examine the cases in light of two central notions of "unity." I argue that there is good reason to think that the neural "vehicles" of conscious states are distributed widely and can include multiple modalities. I also argue that some synesthetic experiences do not really enjoy the same "object unity" associated with normal vision. Keywords: synesthesia, experiential parts, consciousness, unity, visual perception, auditory perception Synesthesia is literally a "union of the senses" whereby two or more of the five senses that are normally experienced separately are involuntarily and automatically joined together in experience (Ramachandran and Hubbard 2001; Cytowic 2003; Ramachandran 2004, Ch. 4; Robertson and Sagiv 2005).1 For example, some synesthetes experience a color when they hear a sound or see a letter. In this paper, I examine two cases of synesthesia in light of the notions of "experiential parts" and "conscious unity." In section one, I provide some background on the unity of consciousness and the question of experiential parts. In section two, I describe the two very different cases of synesthesia. In section three, I critically examine these cases in light of two central notions of "unity." I argue that there is good reason to think that the neural "vehicles" of conscious states are distributed widely and can include multiple modalities. I also argue that some synesthetic experiences do not really enjoy the same "object unity" associated with normal vision. 1. Conscious Unity and Experiential Parts An important aspect of conscious experience is that it seems to be "unified" in some important sense. What is meant by the "unity of consciousness" and explaining just how the brain achieves such unity has become a central topic in consciousness studies. There are many different senses of "unity" (Bayne and Chalmers 2003; Tye 2003; Bayne 2010; Brook and Raymont 2010), but perhaps most common is the notion that, from the first-person point of view, we experience the world and its objects in an integrated way and as a single phenomenal field of experience (Cleeremans 2003). This conjoint experiential character is sometimes referred to as "phenomenal unity" (Bayne 2010, 10-11). However, when one looks on the neural level at how the brain processes information, one only sees complex discrete regions of the cortex processing separate aspects of perceptual objects. Even different aspects of the same object, such as its color, shape, and motion, are Rocco Joseph Gennaro, Ph.D., Professor of Philosophy and Department Chair, University of Southern Indiana, USA; main research fields: Philosophy of Mind/Consciousness. DAVID PUBLISHING D SYNESTHESIA, EXPERIENTIAL PARTS, AND CONSCIOUS UNITY 74 processed in different parts of the brain. When a blue ball is thrown, we visually experience the motion, color, shape, and object all at the same time. It is not as if these properties come apart in our visual experience. So these properties also become unified in an object.2 Given that there is no single place (or "Cartesian theater") in the brain where all this information comes together (Dennett 1991), the problem arises as to how the resulting conscious experience is unified. What binds together such disparate neural activity to produce the kind of unity we experience from the first-person point of view? As Cleeremans puts it: "The problem of integrating the information processed by different regions of the brain is known as the binding problem" (2003, 1). Bayne and Chalmers (2003) attempt to clarify what is meant by the "unity of consciousness" and a number of important interconnected theses emerge (cf. Bayne 2010, 9-18). For example, they call the "unity thesis" the view that "necessarily, any set of conscious states of a subject at a time is unified" (24). However, the main type of unity discussed above has to do with what Bayne and Chalmers call "objectual unity": "two states of consciousness are objectually unified when they are directed at the same object" (24). The earlier example of seeing a single moving (round) blue ball fits this description. It would seem that such binding takes place unconsciously, involuntarily, and is simply presupposed in the resulting experience of an object. For my purposes, another important sense of "unity" arises when we consider whether or not a conjunction of conscious states yields a further conscious state. This is closer to what Bayne and Chalmers call "subsumptive unity": "two conscious states are subsumptively unified when they are both subsumed by a single state of consciousness" (27). For example, we might suppose that auditory and visual conscious states can combine into an overall conscious perceptual state. As Brook and Raymont (2010, Sec. 6) nicely explain, one way to frame the disagreement is in terms of those who favor the EP view ("experiential parts") in contrast to the NEP view ("no experiential parts"). The EP view says that unified conscious experience includes simpler experiences as parts or something like parts. The NEP view is basically that the conscious state through which diverse contents are presented does not have other conscious states, experiences, as parts. On the EP theory, I am conscious of many experiences when I have a unified conscious experience (Dainton 2000; Bayne and Chalmers 2003). On the NEP view, I am conscious of just "one experience" with many different contents (Tye 2003). The unified composite experience replaces or supersedes any included conscious states. I am inclined to favor the NEP theory for two main reasons: First, with respect to objectual unity, it does not normally seem to be the case that we can "separate out" the shape, color, and movement experiences of the ball. Phenomenologically, we experience the object and its properties all at once. It is, of course, logically true that if I consciously experience an object O with properties A, B, and C, then I am also experiencing A, B, and C. But this logical entailment does not mean that I have three further distinct conscious experiences over and above my experience of A, B, and C together. Second, with respect to subsumptive unity, and perhaps more important, the EP view seems to undermine the main idea behind the very binding problem itself. After all, isn't the binding problem generated mainly because we do experience the object's properties all together in a single visual experience?3 2. Synesthesia: Two Cases Synesthesia is certainly a fascinating phenomenon. Some synesthetes experience color when they hear sounds or read words. Others experience tastes, smells, shapes, or touches in almost any combination. These sensations are automatic and cannot be turned on or off. Unlike many other abnormal psychological phenomena, synesthesia is not a disease or illness and is not harmful. In fact, the vast majority of synesthetes prefers to have SYNESTHESIA, EXPERIENTIAL PARTS, AND CONSCIOUS UNITY 75 synesthesia and could not imagine life without it. Synesthesia can, for example, aid one's memory of names and phone numbers and be an asset for creative art. Although there are numerous forms of synesthesia, I will focus on the following two here: Case 1 (grapheme → color): "grapheme-color synesthesia" involves experiencing (black) letters or numbers as inherently colored. For example, one might always experience the letter "R" or the number "2" as red, or the letter "N" and the number "8" as purple. All letters and numbers are experienced as having clearly distinct and regular colors. A more specific example of grapheme-color synesthesia might have "A" experienced as blue, "B" as green, "C" as yellow, and so on. This is perhaps the most common form of synesthesia. Case 2 (visual motion → sound): "motion-sound synesthesia" involves hearing sounds in response to visual motion and flickers (Saenz and Koch 2008). Saenz and Koch report evidence that, for at least four synesthetes, seeing visual motion or non-moving visual flashes automatically causes the perception of sound. These synesthetes outperformed control subjects on a difficult visual task involving rhythmic temporal patterns, for example, judging whether two successive sequences (either both auditory or both visual) were the same or different. This is presumably because these synesthetes not only see but also hear the patterns. 3. Conscious States, Unity, and Synesthesia Let us then systematically examine the above two cases in light of both objectual and subsumptive unity. It may very well be that other forms of synesthesia require different treatment and thus would result in different analyses. It is also useful to compare and contrast our two cases to "normal" instances of objectual and subsumptive unity. It is necessary first to present an important fourfold distinction. First, we have the conscious state, that is, the vehicle which is identical with a mental representation and is presumably a brain state of some kind. This is particularly relevant when we examine the neural basis of synesthesia. Second, there is the representational content of the state in question, that is, what the state is about or directed at. Third, there is the mental attitude (or "mode") of the state, that is, what type of mental state it is, e.g., a doubt, a thought, a perception, and so on. Fourth, there are different modalities, each of which corresponds to one of the five senses: visual, auditory, somatosensory, and so on. So, for example, one might have a visual perception of a red house. The visual perception itself is presumably a brain state, the vehicle, distributed over some area of the brain. Vision is the modality in this case with perception as its attitude and "red" and "house" as the content. Notice also that, unlike Case 1, Case 2 is an instance of cross-modal synesthesia. We thus need to examine four combinations. 3.1. Case 1 and Objectual Unity Many grapheme-color synesthetes experience both the color and the letter or number as properties of the same object. The focus with objectual unity is obviously on the object, not the vehicle or mental representation. So in many grapheme-color cases there is clearly objectual unity much like the way the rest of us normally perceive a letter or number written in color ink or crayon. Both the shape and color are experienced as properties of an object, namely, the letter or number (although the color is not actually present). So, for example, we could have a case where there is a conscious state or vehicle that is a visual perception (i.e., the modality and mental attitude) directed at an "F"shape and a "red" color (the content). We might represent this state as SYNESTHESIA, EXPERIENTIAL PARTS, AND CONSCIOUS UNITY 76 follows: VEHICLE [visual, perception, red colored F shape] We can thus schematically represent the conscious state as a VEHICLE, which is the mental state or representation, with its modality, attitude, and content in square brackets. However, many (perhaps, most) synesthetes do not really even experience the color as bound to the letter or number itself. Dixon, Smilek, and Merikle (2004) show that many synesthetes experience the color "in the mind's eye" or "in the head" whereas others experience the color in external space on a page. They call the former "associator" synesthetes and the latter "projector" synesthetes. So it is really only projector synethetes who enjoy normal objectual unity in Case 1. There is thus clearly a different experience for the associator synesthete in the sense that there is no objectual unity. There is no true binding of the color and shape features for associator synesthetes, unlike the experience we would have, say, when looking at an "F" written in red ink. It would thus seem that, in these cases, we instead have the following: VEHICLE [visual, perception, F shape, red color] It is also interesting that "when a typical projector synesthete ... is presented with a digit in the 'wrong' color, she reports that even though she sees both colors, the [color] lies above the colored grapheme" (Dixon, Smilek, and Merikle 2004, 336). So there is also not the usual binding of shape and color, even for the projector synesthete, when the grapheme is actually colored with a different color from the one that is normally bound with that grapheme. 3.2. Case 1 and Subsumptive Unity Recall the disagreement between those who favor the experiential parts view (EP) and supporters of the no experiential parts view (NEP). The EP view says that unified conscious experience includes simpler experiences as parts or something like parts. On the NEP view, I am conscious of just "one experience" which can have many different contents and involve multiple modalities. What follows can be seen as further support for the NEP view. It is true that we often individuate conscious mental states by their content or mental attitude, such as a thought about a house as opposed to a perception of a house, or a perception of a blue ball as opposed to a perception of a yellow glass. Of course, it is widely held that a given mental state or vehicle can have multiple contents, meaning that the mental state is directed at more than one property or object. Indeed, mental states can clearly have very complex content. Moreover, some have even argued that a single conscious mental state can have multiple attitudes.4 With regard to the projector version of Case 1, we thus seem to have another example of dual content, albeit with a combination that is very unusual such as perceiving both the shape of a letter or number and a specific color not typically experienced together by most other human beings in similar conditions. One explanation for this kind of synesthesia is that there is "cross-activation" or "cross-wiring" of adjacent brain regions (Ramachandran and Hubbard 2001). The idea is that the brain representation (i.e., the vehicle) involves more neural integration than is typical between the brain areas responsible for the two contents. The fusiform gyrus [in the temporal lobes] contains the color area V4 ... which processes color information, but ... the number area of the brain, which represents visual numbers ..., is right next to it ... [and] imaging experiments on people with synesthesia suggest that showing black and white numbers to a synesthete produces activation in the color area ...." (Ramachandran 2004, 65) SYNESTHESIA, EXPERIENTIAL PARTS, AND CONSCIOUS UNITY 77 There is thus a kind of brain "hyperconnectivity" in synesthetes not found in other people. Other related neural explanations appeal to "disinhibited cortical feedback" between brain areas such that information is processed in a bottom-up fashion but later stage brain activation feeds back to activate earlier stages. It is this abnormal feedback that causes the unusual synesthetic experiences (Grossenbacher and Lovelace 2001).5 The kind of unity described above seems closest to what Tye calls "neurophysiological unity" such that "conscious states may be said to be neurophysiologically unified if and only if they are realized in a single neural region or via a single neurological mechanism" (2003, 12). Of course, one difficult question is just how to individuate neural states, which Tye never really addresses at length. However, the discussion above suggests that we ought to individuate the vehicles of synesthete experiences rather widely. Indeed, I have elsewhere argued at length that feedback loops and top-down integration of brain activity are necessary for having any kind of conscious state (Gennaro 2006; 2012). For example, the brain structures involved in feedback loops seem to resemble the structure of at least some form of higher-order theory of consciousness whereby lower-order and higher-order states combine to produce conscious states. On my view, there is essential and mutual interaction between the relevant neuronal levels. Edelman and Tononi (2000), for example, also emphasize the more global nature of conscious states and it is reasonable to interpret this as the view that conscious states are composed of both the higher and lower order states. My own view is that concepts in higher-order thoughts are directed at a first-order perception in order for the perception to become conscious in the first place. In cases of synesthesia, we also seem forced to include multiple overlapping brain areas as the vehicle for, say, the conscious experience described in Case 1. So, on the NEP view, we might again represent the single overall conscious experience from the previous subsection as follows: VEHICLE [visual, perception, F shape, red color] In contrast to the EP view, however, there is little reason to suppose that there are also the following two conscious states: VEHICLE1 [visual, perception, F shape], and VEHICLE2 [visual, perception, red]6 3.3. Case 2 and Objectual Unity Case 2 is quite different than Case 1 in some interesting ways. For example, it involves two different sensory modalities, namely, visual and auditory. Motion-sound synesthesia involves hearing sounds in response to visual motion and flickers (Saenz and Koch 2008). Of course, what makes this case unusual is that only the motion-sound synesthete hears sounds during the motion or flickers. So, unlike Case 1, an additional sensory modality is activated in the synesthete whereas the rest of us only have experience via the one visual modality. One issue, however, is whether or not these synesthetes experience the motion and sound as properties of the same object. Is there objectual unity in these cases? I am, somewhat tentatively, inclined to think so mainly because there is nothing inherently unusual about the idea that one can experience the same object as both moving and making sounds. This occurs when we see (and hear) a car driving near us or when we see (and hear) a band playing music at a concert. If one's visual and auditory perceptions are "directed at the same object," then there is objectual unity as defined by Bayne and Chalmers. These cases differ from those where, say, one sees the ocean waves and simultaneously hears a flock of seagulls overhead. However, a complication arises in that there is arguably an important difference between seeing colors on SYNESTHESIA, EXPERIENTIAL PARTS, AND CONSCIOUS UNITY 78 an object and hearing sounds from that same object. One might suppose that sounds are not really experienced as properties of outer objects in the same way that we experience colors of objects. That is, in Case 2, perhaps we should hold that the synesthete visually experiences the object's motion and that the motion merely causes the sounds heard. There is perhaps something to this especially since one might think of sounds as being caused by events, not as "attached" to the objects per se, which stands in contrast to the visual experience of color and shape. It is not the guitar itself, one might suppose, that sounds a certain way; rather, it is the strumming or picking that makes the sound. Similarly, it is not the object itself which has the sound but rather the motion which causes the sound. This raises further interesting and difficult questions at the heart of recent work on auditory perception and the nature of sounds but also has its roots in the traditional dichotomies between direct and indirect realism as well as between the primary and secondary qualities of objects (for a nice review, see O'Callaghan 2009). I will not attempt to settle the issue here, but it is interesting to note that if one does not take sounds to be properties of objects (or events), then objectual unity may not be possible in Case 2 or at least it is quite different from single modality cases like Case 1. In addition, much more research needs to be done to determine just how such synesthetes experience sounds. Are sounds mere "add-ons" in one's mind more like associator grapheme-color synesthesia? How does the color experience differ from the auditory perception in cases of motion-sound synesthesia? Nonetheless, to the extent that one's visual and auditory perceptions are both "directed at the same object," we might tentatively conclude that there is objectual unity in such cases. Thus, we can think of Case 2 as follows: VEHICLE [visual/auditory, perception, motion/sound] 3.4. Case 2 and Subsumptive Unity Let us again suppose that the NEP view is correct and that, in Case 2, there is only one all-encompassing conscious state. Now much like Case 1, we can then suppose that there is an unusually integrated or overlapping brain state which serves as the vehicle. However, in Case 2, we have both the auditory and the visual modalities involved. So again, on the NEP view, we might represent the single overall conscious state as follows: VEHICLE [visual/auditory, perception, motion/sound] If one insists on treating each modality as involving separate conscious parts or experiences, however, then there would also be the following: VEHICLE1 [visual, perception, motion], and VEHICLE2 [auditory, perception, sound] In contrast to the EP view, however, I think there is little reason to suppose that there would also have to be the above two conscious states. There is good reason to suppose that the neural "vehicles" of conscious states are distributed widely and can include multiple modalities, especially given that multiple contents and attitudes can be anchored in a single vehicle. Due to the evidence of neural overlap and cross-activation described earlier with respect to Case 1, it seems reasonable to extend the same logic to multiple modalities such that a widely individuated neural vehicle can include multiple modalities. Moreover, such neural overlap helps to explain the kind of first-person phenomenal unity mentioned in Section 1. That is, neural overlaps may not be necessary for phenomenal unity in more typical cases of phenomenal unity but may be sufficient in at least some atypical cases of synesthesia. In any case, the idea here is that the brain representation (i.e., the vehicle) involves much more neural SYNESTHESIA, EXPERIENTIAL PARTS, AND CONSCIOUS UNITY 79 integration than is typical between the two brain areas responsible for two modalities, at least in cases of motion-sound synesthesia. There is presumably some kind of additional cross activation or overlapping neural activity between the visual and auditory cortices. What is interestingly different about Case 2, then, is not only that there are vehicles with multiple attitudes and contents, but also that there are vehicles of some conscious states involving multiple modalities as well. This may also be so for other crossor multi-modal cases of synesthesia and seems to favor the NEP view. 4. Conclusion In closing, then, we can conclude from Cases 1 and 2 that there is good reason to suppose that vehicles of at least some conscious states include multiple modalities in addition to multiple contents and attitudes. There is nothing empirically implausible about the view that conscious states can be distributed fairly widely in the brain. This also seems to lend support to the NEP view in the sense that once we accept that there are such all-encompassing and widely individuated conscious states, there is little reason to hold that there are also numerous experiential parts of such states. With respect to Case 1, I also argued that many synesthetic experiences do not really enjoy the usual objectual unity associated with normal vision. With respect to Case 2, it was argued that objectual unity may indeed be present, but only to the extent that the synesthete's conscious visual and auditory experiences are directed at the same object. What complicates the matter here is just how to understand the relationship between visually experiencing a moving object and hearing sounds caused by the object. Works Cited Bayne, Tim. The Unity of Consciousness. New York: Oxford UP, 2010. Bayne, Tim, and David Chalmers. "What Is the Unity of Consciousness?" Ed. Axel Cleeremans. The Unity of Consciousness: Binding, Integration and Dissociation. Oxford: Oxford UP, 2003. 23-58. Brook, Andrew, and Paul Raymont. "The Unity of Consciousness." The Stanford Encyclopedia of Philosophy. Ed. Edward N. Zalta. Fall 2010 Edition. <http://plato.stanford.edu/archives/fall2010/entries/consciousness-unity/>. Carruthers, Peter. Consciousness: Essays from a Higher-Order Perspective. New York: Oxford UP, 2005. Cleeremans, Axel, ed. The Unity of Consciousness: Binding, Integration and Dissociation. Oxford: Oxford UP, 2003. Cytowic, Richard. The Man Who Tasted Shapes. Cambridge, MA: MIT, 2003. Dainton, Barry. Stream of Consciousness. New York and London: Routledge, 2000. Dennett, Daniel. Consciousness Explained. Boston: Little, Brown, and Co., 1991. Dixon, Mike, Daniel Smilek, and Phil Merikle. "Not All Synaesthetes Are Created Equal: Projector versus Associator Synaesthetes." Cognitive, Affective, & Behavioral Neuroscience 4.3 (2004): 335-43. Edelman, Gerald, and Giulio Tononi. A Universe of Consciousness. New York: Basic Books, 2000. Gennaro, Rocco. "Between Pure Self-Referentialism and the (Extrinsic) HOT Theory of Consciousness." Ed. Uriah Kriegel and Ken Williford. Self-Representational Approaches to Consciousness. Cambridge, MA: MIT, 2006. 221-248. ---. The Consciousness Paradox: Consciousness, Concepts, and Higher-Order Thoughts. Cambridge, MA: MIT, 2012. Grossenbacher, Peter, and Christopher Lovelace. "Mechanisms of Synesthesia: Cognitive and Physiological Constraints." Trends in Cognitive Science 5.1 (2001): 36-41. Hubbard, Edward. "Neurophysiology of Synesthesia." Current Psychiatry Reports 9.3 (2007): 193-99. Hubbard Edward, and Vilayanur S. Ramachandran. "Neurocognitive Mechanisms of Synesthesia." Neuron 48.3 (2005): 509-20. Kriegel, Uriah. "Consciousness, Higher-Order Content, and the Individuation of Vehicles." Synthese 134.3 (2003): 477-504. O'Callaghan, Casey. "Auditory Perception." The Stanford Encyclopedia of Philosophy. Ed. Edward N. Zalta. Summer 2009 Edition. <http://plato.stanford.edu/archives/sum2009/entries/perception-auditory/>. Ramachandran, Vilayanur S. A Brief Tour of Human Consciousness. New York: Pearson, 2004. SYNESTHESIA, EXPERIENTIAL PARTS, AND CONSCIOUS UNITY 80 Ramachandran, Vilayanur S., and Edward M. Hubbard. "Synaesthesia: A Window into Perception, Thought and Language." Journal of Consciousness Studies 8.12 (2001): 3-34. Robertson, Lynn, and Noam Sagiv, eds. Synesthesia: Perspectives from Cognitive Neuroscience. Oxford: Oxford UP, 2005. Saenz, Melissa, and Cristof Koch. "The Sound of Change: Visually-Induced Auditory Synesthesia." Current Biology 18.15 (2008): R650-51. Tye, Michael. Consciousness and Persons. Cambridge, MA: MIT, 2003. Notes 1. This etymologically driven "definition" is not meant to cover all kinds of synesthesia. As we will see, one common form of synesthesia occurs within the visual modality, namely, perceiving black numbers or letters as colored. 2. When unity breaks down in various ways we see various abnormal conditions, such as schizophrenia and agnosia. 3. I will later consider a multi-modal example along these lines. But for much more on the disagreement between the EP and NEP views, see Tye 2003, Ch. 1 and Bayne 2010, Ch. 2. I do not mean to imply that, on the NEP view, all aspects of the one total conscious state are experienced equally in terms of detail or attention. On the contrary, much of our conscious experience at any given time is likely lacking in detail and at the periphery of consciousness, such as the case with peripheral vision or when one is consciously aware of some background music while performing an attention occupying task. 4. Such a view is particularly common among those who advocate some form of higher-order or self-monitoring theory of consciousness such that what makes a mental state conscious is some kind of higher-order or self-referential thought directed at the mental state (see Kriegel 2003; Carruthers 2005; Gennaro 2006; 2012). These views typically develop from the highly intuitive claim that has come to be known as the Transitivity Principle which says that a conscious state is a state whose subject is, in some way, aware of being in it. One motivation for these theories is the desire to use this principle to explain what differentiates conscious and unconscious mental states. 5. See Hubbard and Ramachandran (2005) and Hubbard (2007) for a review of several related, but distinct, neural models of synesthesia. It is also interesting to note that, for some projector synesthetes, they experience the same color even when the same letter is presented in very different fonts. This suggests that the "meaning" or "concept" of the grapheme determines the experienced color as opposed to the mere shape. Such meanings presumably involve higher levels in the brain which leads to the distinction between higher (those who experience a color due to the concept of a number) and lower (those who experience a color solely due to a grapheme shape) synesthetes (Ramachandran and Hubbard 2001). Since different brain areas are responsible, this may be some evidence for the (abberant) reentrant feedback model. Indeed, it may very well be that a combination of these neural models is necessary to provide a fully adequate neurological account of different types of synesthesia. 6. For more discussion of neurophysiology of something more like Case 1 and objectual unity, see Dixon , Smilek, and Merikle, 2004, 340-42.