Synesthesia and Cross-Modal Perception

        For years, the medical and scientific communities have been fascinated by the neurological phenomenon known as synesthesia.  This condition, which results in one type of sensory input leading to “automatic, involuntary experiences in a second sensory or cognitive pathway” (1), allows some people to have unique sensory experiences such as “seeing” sounds or “tasting” colors.  In recent years, synesthesia has become more well-known among the general public, due to the publication of books such as “A Mango-Shaped Space” by Wendy Mass and “The Man Who Tasted Shapes” by Richard Cytowic, and also due to an increase in the amount of research being done in an attempt to better understand the neurological processes of synesthesia.  It is difficult for non-synesthetes to imagine what it must be like to perceive the world in such an exceptional way, for example, seeing music in colors and shapes.  However, what if such combined perceptions were not so unusual after all?  What if synesthetes simply experience a more intensified, more conscious version of the cross-modal sensory integration that takes place in all of our brains?

        There are many different types of synesthesia, the most common of which are grapheme-color (in which numbers and letters are perceived as having signature colors) and sound-color (in which musical notes and other sounds are accompanied by the perception of colors or shapes) (1).  Some less common forms connect sound with taste, or associate numbers with particular spatial orientations (4).  Synesthesia has a genetic component (it is believed to be an X-linked trait) and so it is common for multiple individuals in the same family to be synesthetes.  However, symptoms vary greatly between all individuals with synesthesia, and no two people (not even parent-child pairs who experience the same general type of perceptions) will have the exact same experiences (i.e. no two grapheme-color synesthetes will associate the same colors with the same letters) (1).  This non-concordance is a major characteristic of synesthesia, and it would seem to refute the idea that there are uniform, underlying connections between areas of sensory integration in all individuals.  However, there are a few very general trends among synesthetic perceptions; for example, the majority of individuals with grapheme-color synesthesia tend to perceive the letter “O” as being white, and there is a tendency for sound-color synesthetes to associate higher-pitched sounds with lighter, brighter colors (1).  So, it is possible that some inter-sensory associations exist that are common to all synesthetes, if not to all people.

        One very interesting test that demonstrates a visual-sound association common among non-synesthetes is the “Bouba-Kiki test.” In this test, which was designed by Wolfgang Köhler, participants are presented with two shapes, one being bulbous and rounded like a clover, and the other being sharp and jagged like a lightning bolt.  The participants are then asked to decide which of the shapes would be identified as a “bouba” and which would be called a “kiki.”  An impressive 98% of participants chose the rounded shape for “bouba” and the sharp shape for “kiki.” (6). These results suggest that people are making some type of fundamental connections between sounds (auditory input) and shapes (visual input).  One might argue that these results were culturally influenced (ex: many “round” objects in the English language—ball, bowl, bulb—are associated with the letter B).  However, the test was also administered to native Tamil speakers, as well as to very young English children who could not yet read, with similar results.  Additionally, autistic children (who often exhibit abnormalities in vocal behavior (7)) tend to perform poorly on the Bouba-Kiki test, suggesting some neurological basis for the assignment of words to objects (5).  These observations are very interesting when we consider the possibility that the creation of language may have been influenced by unconscious visual-grapheme associations.

        The phenomenon of shared sound symbolism across many different languages provides further evidence in favor of such associations.  Across the globe, and among groups of people that speak unrelated languages, certain sounds have continuously been assigned to particular objects.  For example, the word for “crab” begins with a hard “c” or “k” sound in at least 14 languages, including Korean, Swahili, English, Latin, and Malayan, while the word for “name” has a prominent “m” or “n” sound in at least 20 languages, including German, Japanese, Turkish, Finnish, and Amerindian (9).  Perhaps these sounds and concepts simply “match” in the human brain, just as some synesthetes would argue that the color blue “matches” the letter A.

  
        We also have cross-modal associations of which we are entirely conscious. All children, as they grow, develop certain connections between different sensory inputs (i.e., learning to associate the sound of a bird chirping with the image of a bird).  These connections, learned by the individual over time through experience, are considered to be normal.  Furthermore, if a person with his or her eyes closed were to examine two objects using touch alone, it is expected that he or she would later be able to identify the objects simply by looking at them, due to connections made in the mind between how something feels and what it is expected to look like (3).  This ability is not due to any type of synesthesia, because it is developed over time, and the associations are non-spontaneous.  However, there is evidence to suggest that cross-modal transfers can occur in the brains of newborns, seemingly before they have the chance to “learn.”

  
        In the 1970’s, Meltzoff and Borton conducted an experiment on infants, in which they allowed the infants to suck on pacifiers that they had never seen before.  Each infant received either a smooth pacifier, or a pacifier with nubs.  After 90 seconds, the infants were shown both a smooth pacifier and a nubbed pacifier side by side.  Results show that the infants spent a significantly greater amount of time looking at the type of pacifier that had just been in their mouths, as if they recognized it based on what it had felt like on the tongue (3).  In a second experiment, Lewkowicz and Turkewitz showed that infants experienced little increase in heart rate when exposed to white light followed by a burst of “white noise” (noise that was considered by adults to “match” the white light in intensity), but that their heart rates did change significantly when the white light was followed by a burst of either very high or very low intensity noise (2).  This reaction may be analogous to the discomfort that some synesthetes experience when they are presented with “mismatching” stimuli (ex: if music is accompanied by colors that do not match the perceived colors of the notes) (8).

        Some researchers, such as D. Maurer, go so far as to propose that all human infants are synesthetes, but that most people quickly lose many inter-sensory connections, so that each sense is experienced separately (2).  There is some evidence to support this theory: When infants are exposed to auditory and visual stimuli, activity is recorded over several areas of the cortex (corresponding to several different senses) whereas in adults, each stimulus elicits a response only in a single area of the cortex.  This indicates that, at the very least, the primary sensory cortex is much less specialized in infants than it is in adults, and could mean that infants are experiencing sights, sounds, tastes, etc…in response to a single stimulus (2).

        Synesthesia has proven to be useful to some synesthetes for purposes such as memorizing names and telephone numbers, or learning how to spell words.  For example, if one remembers that a word, when spelled correctly, is supposed to look red, one will know to use a “red” letter (10).  If synesthetes can use these unique, conscious perceptions to aid them in certain behaviors, is it possible that subconscious cross-modal perceptions are influencing our decisions on a regular basis?  Some synesthetes perceive emotions or personality traits for numbers, letters, or days of the week (1).  This emotional component to the condition leads researchers like Richard Cytowic to believe that the limbic system, the part of the brain associated with emotional response, plays an important part in synesthetic experiences (11).

  
        If the limbic system does play a role in perception, or in sensory integration, then perhaps “gut reactions” or feelings of “intuition” are in fact carefully thought-out responses, based on the unconscious integration of all available sensory input.  Whatever the case may be, there is ample evidence suggesting that all people experience cross-modal sensory integration to some degree, and that connections, both learned and innate, exist between the areas of sensory processing in the human brain.  Synesthetes are unique in that they spontaneously and consciously experience more than one type of perception in response to certain inputs.  However, I think that we must remember that we are all “perceiving” on several different levels, both consciously and subconsciously, and that our behaviors are much more than meets the eye.

Resources:

1) http://en.wikipedia.org/wiki/Synesthesia
2) http://psyche.csse.monash.edu.au/v2/psyche-2-27-baron_cohen.html, “Is There a Normal Phase of Synaesthesia in Development?”, Simon Baron-Cohen, PSYCHE, 2(27), June 1996
3) http://pegasus.cc.ucf.edu/~gallaghr/molyneux.html, “Neurons and Neonates: Reflections on the Molyneux Problem”, Shaun Gallagher, presented at  the Philosophy Colloquium, Trinity College, Dublin, November 2000
4) http://www.answers.com/topic/synaesthesia?cat=health
5) http://www.autismvox.com/boubakiki/, “Bouba/Kiki”, Kristina Chew, PhD, January 9, 2007
6) http://en.wikipedia.org/wiki/Bouba/kiki_effect
7) http://en.wikipedia.org/wiki/Autism#Communication
8) http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6SYR-4MBT27P-6&_user=400777&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000018819&_version=1&_urlVersion=0&_userid=400777&md5=99c3c63d7f201ab9886ee94f9923ab5f, “Green love is ugly: Emotions elicited by synesthetic grapheme-color perceptions”, Alicia Callejas et al, Brain Research, Volume 1127, 5 January 2007, Pages 99-107
9) http://www.percepp.com/soundsmb.htm, “SOUND SYMBOLISM”, Robin Allott, 1995, in Language in the Würm Glaciation, ed. by Udo L. Figge, 15-38. Bochum: Brockmeyer
10) http://www.bbc.co.uk/health/conditions/synaesthesia1.shtml
11) http://www.tqnyc.org/NYC074509/causes.htm, “What Causes Synesthesia?”