Communication Restoration Through the Use of Modalities

I. Communication and Modalities: It was Daniel Webster who said, "...if all my possessions were taken from me with one exception, I would choose to keep the power of communication, for by it I would soon regain all the rest. This succinct and yet profound statement encapsulates the purpose and the story of AAC (Augmentative and Alternative Communication) – giving back the power of communication to one who has lost it. Communication is defined in the dictionary as a conveyance of information. We can add that it is accomplished by the transmission via some modality of an organized and recognizable pattern of energy. A Modality is a path or method, and among the domain of the living there are two major modalities used for communication: signs and symbols. Semiology is the study of these modalities, both individually and combined into systems. Hence, Semiology is the study of the “life blood” of the communication process. Symbols and Signs are transmitted by humans via a number of sub modalities, some of which are Linguistic and some which are not. Examples of Linguistic modalities would be speech, Sign Language, and Morse Code. These are symbolically based communication pathways that typically convey very precise information. Non linguistic symbols would include Art, Music and Rituals. Signs, all of which are Non Linguistic in nature, would include facial grimaces, intensity of movement, proximity and many others variables. Signs typically provide a more general background for meaning. Frequently, communicators take advantage of more than one modality at a time. Multi modalities not only provide a richness to the communication process, but also an element of redundancy so that if one modality fails, there are others available to fill in the gap. These will be discussed in more detail in the next Section. But for now, the question might be, “How do Signs as a modality differ from Symbols?” II. Signs Typically Transmit Information Necessary for Survival: What is the difference between a symbol and a sign? If we were driving along the road and saw the following image, what would we do? Hopefully we would stop. This is a stimulus called a Sign and it is a call to action -to stop! Hence, it is typically a communication that is necessary for our survival. Disregard it and the results can sometimes be fatal! Signs can be events, motions or marks that have meaning. They may be generated voluntarily or involuntarily in the realm of the living. But even inanimate objects produce signs, although in this case they are not always calls to action. Rocks, for example, omit signs that indicate their age. Stars transmit signs which reveal how hot they are, how fast they are rotating, how fast they are moving and in what direction, and what gases are present in their atmosphere to mention a few. For animate objects, however, there are three important features of Signs: 1. They are typically calls to action. 2. They are important for our immediate survival; and 3. They are typically connected to their (referent) meaning through an iconic relationship or a cause-effect relationship that can easily be recognized and learned. A foot print in the sand, for example, has both iconic and cause & effect significance. It both looks like a foot, and was caused by some one stepping there. What is the call to action here? If you were Robinson Crusoe ostensibly alone on an Island in a part of the world where there were cannibals, and that was not your footprint, you would probably take great heed! (A Sign that Someone was here) It must be noted that the Signs we are referring to here are not the same as the Signs used as the basis for Sign Language, but more the signs used in the Pavlovian sense (i.e., a bell can be learned to be a sign for the imminent arrival of food; or black clouds are a sign of the possible arrival of rain). In the first case, the call to action is to salivate; and in the second, to grab a rain jacket. The so calla Signs of Sign Language on the other hand are not signs at all but in reality they are visual symbols. Many communicatively disabled persons with no speech are still capable of using some natural and learned Signs. In a familiar environment, these signs may be the most efficient form of communication being easy to produce and being associated mostly with the communication of important needs. Hence, an AAC user should not be discouraged from using these signs, even when they have a very sophisticated and expensive speech communication device at their disposal. In other situations, however, the speech modality is preferable because the symbols covey messages with considerably greater preciseness. Einstein, for example, would have had trouble explaining his Theory of Relativity by jumping up and down, waiving his arms and yelling “aarrgh. In fact I have seen the same dialogue applied by some of my students who were commenting on the some un-relevant facet of class final exam. III. Symbols Typically Transmit Information Necessary for Thought: If we see this, what do we do? Probably we would do nothing. That is because it was NOT a call-to-action. And it is certainly was NOT necessary to our survival. It was a call for a bit of organized information, however, to be brought into consciousness from our memory banks. For example, it may have brought to mind an idea -- about a man who is faster than a speeding bullet; more powerful than a locomotive; and who loves to leap tall buildings in a single bound. He often wears a blue leotard with a pink cape and boots; and has a quirky habit of frequently changing his clothes in the nearest telephone booth. He is employed by a newspaper company, and he has a girl friend of questionable intelligence (she never recognizes him in his leotard without his glasses.) All these threads of interrelated information, which we will call a concept, were brought to mind by the stimulus (in this case the word) "Superman." That is what symbols in communication do. They bring into our minds information (concepts) that we can then modify, enjoy, discard or use to plan our day. But typically, they are not calls to action. There are three important features of symbols: 1. They are calls for information to be brought into consciousness; 2. They can have a relatively arbitrary relationship to their referent (that which they signify); and 3. They can act as surrogates in the thought process for the very concepts they signify. For example we can and do think in words rather than images. Symbols, then, not only transmit information but they also become a major part of the thought processes. One might easily think that the arbitrary connection of symbols to their referents would create a major disadvantage, in comparison to signs, as a currency for communication, especially for children. Signs with their iconicity or apparent cause and effect relationship should be much easier to learn. But this is where nature has provided humans with an awesome edge--the prodigious ability to create and recall great numbers of arbitrary Concept--Symbol (Word) associations. This is the "backbone" of language. And just as it is no problem for a giraffe to reach nutritious leaves high up in a tree, or birds to fly because of their special structures which are a product of inheritance, it is no problem for humans to make these word—concept associations. In fact many persons by the time they finish college have often acquired over 100,000 concept-word associations in their lexicon! This capacity is our genetic inheritance. Indeed, for human communication, the arbitrary nature of the concept--symbol association, has one major advantage: boundless POWER for communication! Because there is no limit to the concepts that can be associated with symbols, there is nothing in the Universe that cannot be communicated symbolically! Many concepts moreover, cannot be seen, heard or touched, and can only be accessed mentally through their symbols. The words “electron or democracy” would be two examples. Thus symbols open up to the mind phenomenon and ideas in the Universe hitherto out of reach of the human senses. In addition, these symbols with their surrogate function carry the power of communication to the very mind itself. The potential and advantage for self communication (which we call verbal thought or language) is awesome. Hence, for severely sensory and/or motor impaired children, a goal for Assistive Technology in general (AT) and Alternative and Augmentative Communication (AAC) in particular is to open up the modalitis of communication so that they can have access to this powerful instrument of thought and communication. IV. Words, Art, Dance, Music, Ritual are all Symbolic Modalities: Humans display many kinds of symbolic behaviors. Probably, the earliest in history were Rituals, Music and Dance and Pictures. They are still as important today as they were in primitive times, nevertheless, because they have the capacity to communicate visceral human thoughts and feelings that cannot always be fully expressed in words. Although limited, AAC and AT (Assistive Technology) devices can provide “virtual” opportunities to communicate through rituals, music and dance. Words as symbols, on the other hand, have five features in particular which set them apart from the other symbol forms, and make them particularly suitable for precise communication. These include: 1. There is a one-to-one relationship between a symbol (word) and its referent (a thing or event in the world). For example, the word "table" can be identified with a specific object. The body of words which individuals store in their brain is referred to as a lexicon (vocabulary). This provides the capacity for precision in communication. 2. The Symbol (word)--Referent relationship is arbitrary, and provides the capacity to communicate almost anything. 3. Many symbols can exist for the same object. This, of course, provides the bases for many languages. For example, “table” (in English), “mesa” (in Spanish), “stol” (in Russian {pardon my script}) all mean the same thing. 4. Symbols can serve as surrogates for concepts in the thought process! This gives humans the ability to think in words about the past, the present and the future, or to visit a world and a time that never was, through verbal make believe, to mention a just few advantages. In fact, most adults typically think in this modality, that is, verbally. It is this feature, moreover, that makes this Modality so valuable. It is literally a passageway between the mind and the universe of things beyond our immediate sight, sound and touch!!! 5. The symbols of language can be used to discuss language. This is referred to as meta-language. V. There are Two Kinds of Symbol Sub Modalities: Presentational and Discursive: 1. Presentational Symbols are perceived as complete units. This typically involves the right side of the brain. Pictures and maps are some excellent examples. Many of the AAC devices or systems, particularly for non literate and/or low cognitive users rely on some form of pictures for encoding messages. These may be photographs, sketches or two dimensional drawings. One non computer communication system called the Picture Exchange Communication System (PECS) relies totally on picture symbols. This system can stand alone for communication, or serve as a pre training process for picture based AAC devices. 2. Discursive Symbols occur over a period of time. This typically involves the left side of the brain. Hence, it is not possible to perceive the whole unit except through the cognitive process of short term memory. Many of the AAC devices provide discursive stimuli, not only in the form of words but also in combinations of words and/or pictures to form phrases and sentences. A major problem, as we shall see, is the amount of time it takes to produce these strings of symbols on an AAC device when the user is severely motor impaired. This, of course, taxes the patience, if not the short term memory capacity of the communication partner (the listener). Hence, the “Holy Grail” of the AAC world is always to find techniques and strategies to speed up the communication process. VI. Signs and Symbols can be Communicated via Two Modalities. These are Graded and Combinative signals. 1. Graded signals are stimuli that vary along a continuum, for example, from more to less. If you change the degree of the stimulus, you change the message. Imagine yelling at the top of your voice versus mumbling softly as I often do in class. Students react differently to each. To the latter they smile complacently as they finger their Ipods. They know the lecture will continue benignly for hours. To the former they frown complacently as they finger their Ipods. They know the lecture is about to end in a furor of declarations. Pitch, tension, timing, space and color are other examples of signals that can convey meaning when they are changed from more to less along a graded continuum. These can be important modalities for communication for a motor impaired individual. For example, an increase in vocalization or body movement (albeit uncoordinated) may signify distress, anticipation, frustration or happiness etc. Because of the lack of precision of signs, the context of the situation may be necessary to decode the message. 2. Combinative signals are stimuli that occur in a sequence. If you change the order of the sequence, you change the meaning of the message. Certainly, speech is a prime example of a combinative stimulus. VII. Three Basic Types of Messages: Nominal, Expressive and Predicative, provide a total of SIX Modalities for Communication: Using the two types of signals (Graded and Combinative) humans can transmit three types of messages for a total of SIX possible modalities. These messages types are: 1. Nominal, 2. Expressive (sometimes called Emotional) and 3. Predicative, (sometimes called Propositional). 1. Nominal Graded Modality: This includes Messages that are transmitted by Signs. If a change in the degree of the signal creates a change in the meaning it is a graded signal as we mentioned above. They are Nominal if they are relatively instinctive and typically generated by internal body processes. Nevertheless, they can provide an important modality of communication for severely cognitively and/or motor impaired individuals. Two examples of Nominal Graded Communication are my rumbling stomach, and the early crying of an infant. a. My rumbling stomach, for example, is one form of communication that generates a sign that may signify to a listener that I am hungry, even if I say I’m not. A stomach rumble is a Nominal sign because it is an instinctive response arising from internal body processes. I have no control over it, despite what my wife says! If you are not familiar with stomach rumblings, check out the old classic movie, “The African Queen” with Humphrey Bogart. He (via special sound effects, of course) gives an excellent demonstration early in the movie. b. The Crying of an infant is perhaps a more relevant example of Nominal Graded communication. No one teaches a baby how to communicate so effectively. It is an instinctive reflex generated automatically by internal body processes. This vocalization is a form that generates a Sign, which is certainly a compelling call for action. The degree of intensity (e.g., cooing versus screaming) determines the meaning of the vocalization. Interestingly, humans appear to be neurologically wired to respond to an infants cry. Hence, the baby's cry is more compelling and is more difficult to ignore than one might expect, even for other babies. Put one crying baby in a nursery of ten, and soon you will have eleven crying babies. Besides crying, there are other body signs that are significant of personal needs. These can include an increase in vocalization, or body movements (agitation). These may be important signs and perhaps the only modality of communication for a disabled individual, young or old. 2. Nominal Combinative Modality: These messages are also transmitted by Signs but their structure is sequential. Change the sequence and the meaning changes. Two examples of Nominal Communication are the “Talk” of Parrots, and the Babbling of a baby. a. Parrots, like some Professors, talk but they don’t say anything. Their speech calls are clearly Combinative, but unlike the baby's cry, they are not instinctive but are learned from other birds. The propensity to learn the calls, however, is instinctive and hence, they are still Nominal Signs. Because they are learned, baby parrots exposed to human speech instead of other parrots’ calls, will pick up sequences of speech sounds that they frequently hear. These sequences can get meaning in a Pavlovian sense in that if every time I come home, I say to the Parrot, “Ed is Home,” the parrot may learn to say this sequence when Ed comes into the room. It could be a call to action to be fed. This is not language, however, because the phrase “Ed is Home” is in essence a “bird song” uttered in its totality with no individual words or grammatical rules distinguished or intended by the Parrot. The parrot will not say, “Ed is lazy,” unless of course Ed’s wife has been frequently nearby. The peculiar aspect of Parrot talk is that meaning is inferred on the sounds by humans that is not necessarily shared by the bird. Patrick Parrot may look adoringly at another cute Polly Parrot and sing, "Patrick wants a cracker." But food is probably not what is on his mind. But these phrases can be useful. For the pre linguistic and/or low cognitive adult AAC user, canned phrases that can be made by one stroke of a key are very valuable. Like, “My stomach hurts” or “How are you today,” or “Please call my doctor!” As with the Parrot, it is not necessary to know the individual words or the grammatical rules involved. c. Babies from 3 to 9 months do not have language or speech, of course, but still communicate vocally using a form of Nominal Combinative messages called Babbling. It is Nominal because it is instinctive. All humans, regardless of race, culture or location begin to babble at around the same age of development. (Even deaf children begin to babble but quickly give it up and attend to a more rewarding, though analogous, activity using the hands). Initially, Babbling may be an indication that the child is content. In time, it becomes social and meaningful depending on the sequence. If dad comes in the room, the baby may say dadada. If the child sees water, he may say wawawa. 3. Expressive (or emotional) Graded Modality: Communications through the Expressive Modality are more complex than Nominal, being mediated by higher centers of the brain dealing with emotion. Hence, there is more room for decision making on the part of the communicator. A very common example of an expressive graded signal is the whimpering versus the snarling behaviors of animals, notably a dog. A snarl from a dog is a warning to stay away. A less intense vocalization, such as a whimper, is a sign of submission. The dog may choose his message based upon past experiences and learning rather than instinct. Humans too, engage in a lot of expressive graded communication. I whimper and my wife snarls. But the best examples are provided in a popular book by Edward T. Hall on Body Language. I, of course, am Edward P. Hall and missed the “boat of fame” perhaps by just one letter! However, I would argue with Edward P’s use of the word “Language.” Body Language, as important as it is, is a communication system based on Expressive Graded Signs and consequently is NOT language. It is Graded in the sense that messages are conveyed by changes in the degree of some variable, like a change of voice pitch, intensity or modulation, the speed of movement, the degree of body distance, or tension, etc. This is not to imply that body “language” is not important! To the contrary it is usually very important. If on my tenth wedding anniversary, for example, when I took my wife out to see the Mike Tyson heavy weight slugging match (instead of the Bolshoi Ballet as I had promised), I should have deduced from a number of the communication modalities she employed (viz., her clenched jaw, her monotone voice, and the length of time (30 minutes) it took her to walk the 20 feet from the front door to the car,) that she was not happy about my choice of activities. These Signs are typically a prelude to my being locked out of the house in favor of the chicken coup for the night. Where does the decision part come in? If I resembled and had the demeanor of Robert Redford or Tom Cruise, I suspect she might have decided to alter her body language in spite of these same circumstances and react more like Marilyn Monroe than Koko the Gorilla! For the communicatively handicapped individual, body language can be a rich modality to communicate many needs and feelings. These forms should be attended to and encouraged even after the acquisition of an AAC device. The rubric below summarizes were we are at this point in our discussion of Communication. ( In the interest of brevity, and suspecting some of our female students hate bugs, I deleted the discussion of the insects here.) 4. Expressive (or emotional) Combinative Communications don’t exist according to many credible psycholinguists. But I believe I can point to one example. You can judge for yourselves the voracity of this idea. The example I would give is swearing. Although it is a speech form it is NOT language. It is an Expressive communication. For example, if you show me someone who is swearing, I will show you someone (excluding anyone with Tourettes Syndrome) who is typically mad, surprised, scared, happy, in love, or in pain, -all emotional states. Hence, Swearing is a Sign of an emotional state of mind. It is also a Combinative communication. For example, if I hit my thumb with a hammer, it doesn't feel like I have communicated my feelings if I say, "Oh green peas!" Hence, changing the combination of phonemes from what I typically might say (viz., “G@%$*^it!!!”) does change the meaning. Incidentally, it is typically the Right Cerebral Hemisphere that swears. This is because this hemisphere is for most folks a gestalt processor, which organizes information in whole units. This includes such things as memories of pictures, maps and even songs as well as swearing. There is also in the right hemisphere lots of words and phases as they appear in over-learned clichés and expressive epithets. Speech Formulas, which are over used familiar phrases (like, “How are you today”), Idioms (like, “The Fat is in the Fire”), and Proverbs (like, “Two’s company and Three’s a crowd”) are all stored in and emanate from the right cerebral hemisphere. More like songs than language, these expressions are pretty much units, fairly rigid in their order and tone. They come as a whole or they don’t come at all. This is the property that places them in the Right Cerebral Hemisphere, which is, as I have said, a Gestalt processor. The Left Cerebral Hemisphere on the other hand is a Segmental processor. Hence, it carries the burden of language acquisition and use. So, for example, if I should receive a severe injury to my left cerebral hemisphere, like from a hard blow to the left side of the head with a heavy purse on my way to the Mike Tyson Fight, I may find upon regaining consciousness that I am unable to use words to describe my headache or anything else. I may have what is called Expressive Aphasia. But in my attempt to speak, in the absence of expressive language, clear and fluent swear words may yet stream out. In other words, when I have lost my language ability, I may still be able to say many of those phrases that are not language, such as swear words. Ironically, this body of words that exists in the right hemisphere can be used beneficially as a springboard to both speech and language rehabilitation. And, with some reservations, for the teenage AAC user, some swear words or phrases might often be included in an AAC device so he can fit in with his peer group, and/or he may have some opportunities to learn the pragmatics of when to use and not to use such phrases. 5. Predicative Graded Modality: There are no examples available in the literature that I could find for this modality. It is conceivable, however, that Sign Language, one form of linguistic communication, includes some of these propositional and graded qualities. For example the Signs (really Visual Symbols) for “Far” and “Very Far” appear to be distinguishable by the degree of motion of one hand relative to the other. 6. Predicative Combinative Modality: This Modality includes all sub modalities of Language based communications which use Symbols. These include speaking, writing, Signing and Morse Code etc. This is in contrast to all the previous modalities of communication mentioned so far, which are based on the use of Signs. Language is Combinative because it relies on the order of phonemes, morphemes and words to determine the meaning. It is Predicative because the symbols bring to consciousness ideas or concepts which can be communicated. It is this very process of symbolization that in some cases breaks down and causes the communication disability. This is referred to as Aphasia. Although this is not basically a motor disorder, there are nevertheless AAC devices (e.g., Lingraphica) which are helpful in both rehabilitating language and restoring the communication processes. The Language modality for communication does exist in a variety of macro modalities in the brain. It is important to the SLP to recognize these so that when one modality (or more appropriately, a segment of a modality) becomes disabled, the Speech Pathologist will know to determine where the break occurs and to seek other combinations of modalities to reestablish or improve communication competency. For this reason a full discussion of these macro Modalities and their accessories will be the focus of the next Section.