How Early do Children Understand Iconic Co-Speech

How Early do Children Understand Iconic Co-Speech Gestures Conveying Action? 1. Introduction Speakers, including children at the early stages of language learning, rely on both gesture and speech in unpacking communications addressed to them—with gesture influencing the processing of the accompanying speech (GoldinMeadow, 2003). We know from previous work that children understand gesturespeech combinations with deictic gestures between ages 1 to 2 years (‘open’+point at box; Morford & Goldin-Meadow, 1992) and combinations with iconic gestures between ages 3 to 4 (‘open’+book gesture; Stanfield, Williamson & Özçalışkan, 2013). However, most of these earlier studies focused on gesturespeech combinations in which the gesture conveyed information about an object (e.g., its size, shape), leaving combinations with gestures conveying action information relatively unexplored. Previous work on early gesture production shows that iconic gestures become part of children’s repertoire later than deictic gestures (Özçalışkan & Goldin-Meadow, 2005a, 2011; Iverson & Goldin-Meadow, 2005; Iverson, Capirci & Caselli, 1994). Children begin to use deictic gestures frequently to indicate objects in their immediate environment by 12 months of age, but the initial increase we observe in iconic gesture production does not become evident until 26 months of age—a whole year later than the onset of deictic gestures. More importantly, these early iconic gestures predominantly (74%) convey action information (e.g., eating, throwing) and rarely convey object information (e.g., big ball, round circle, Özçalışkan & Goldin-Meadow, 2005a, 2011; Özçalışkan, Gentner & Goldin-Meadow, in press). Given their predominance in children’s early gesture repertoire, it is possible that iconic gestures conveying action information are understood earlier in development than are the ones conveying object information. On the other hand, understanding iconicity in gestures is a relatively late emerging ability, largely due the less transparent nature of the symbol-referent relationship in iconic gestures versus in deictic gestures. In fact, there is wealth of evidence suggesting that young children find it difficult to make use of the iconicity of a gesture to map it to its intended referent until 2-3 years of age (Namy & Waxman, 1998; Namy, 2001; Namy, Campbell, & Tomasello, 2004; Stanfield et al., 2013). As such, understanding gesture-speech combinations with iconic gestures conveying action information might be as difficult as understanding ones conveying object information. We test these possibilities by studying children’s comprehension of gesture-speech combinations in which the iconic gesture conveys action information. More specifically, we focus on the domain of motion (e.g., running into a house) as a prototypical action type, the expression of which elicits high rates of gesture production in both children and adults (e.g., Kita & Özyürek, 2003; Özçalışkan, 2012). Previous work (Talmy, 2000) identified two key components of motion in gesture production, namely manner (i.e., how one moves, such as hopping, crawling) and path of motion (i.e., the direction with which one moves, such as up, down, across). Speakers typically produce one of three gesture types when describing motion events: manner+path gestures which synthesize manner and path components into a single gesture (e.g., wiggle fingers left to right to convey running left to right), manner-only gestures that express only the single motion component of manner (e.g., wiggle fingers in the same location to convey running) and path-only gestures that express only the single motion component of path (e.g., move finger left to right to convey left to right trajectory)—with a preference for manner+path gestures among English speakers (Kita & Özyurek, 2003; Özçalışkan, 2012). In this study, we present English-speaking adults and children with language-typical gesture-speech combinations in which the iconic gesture conveys manner+path information. We ask whether children glean action information (i.e., manner+path, manner-only and path-only) conveyed in these co-speech gestures as early as they extract object information from iconic cospeech gestures. 2. Methods Participants The study included 108 participants; 36 3-year-olds (M = 3.1 years, range = 2.75-3.7), 36 4-year-olds (M = 4.2 years, range = 4.0-4.9), and 36 adults (M = 20.8, range = 18-40). The children came from middle- to upper-middle- class families and were predominantly Caucasian (69%) or African American (11%). The adults were college students at an urban research university and were predominantly African American (39%), Caucasian (31%), or Asian (23%). All adults were native English speakers; all children were learning English as their native language. Procedure for Data Collection The participants were tested individually in a laboratory by an experimenter. Each participant completed two training trials followed by eight test trials. Training Trials To ensure that participants (particularly the children) understood the task, two training trials were conducted at the beginning of the session. Participants watched a video demonstration of a gesture+speech combination that labeled an animal (e.g., ‘Look it is a bunny’ + bounce index and middle fingers in V-shape as if a bunny hopping). Following this demonstration, pictures of two animals were displayed side-by-side (e.g., a rabbit and a duck), only one of which was the correct match for the demonstrated referent. Participants were asked to choose which picture matched the demonstration (“Which one was that?”). Only one participant, a 3-year-old, made incorrect choices on both training trials and was excluded from the study. We collected data from an additional three-year-old to have equal sample sizes in each age group. Test Trials At the beginning of the study, each participant was assigned to one of three test conditions (manner+path, path-only, manner-only), resulting in 12 participants per age group and condition. Each participant was presented with video demonstrations of eight gesture+speech combinations involving a neutral verbal description, accompanied by an iconic gesture conveying both manner and path of motion (e.g., ‘he is moving’ + rapidly move bent fingers right to left to convey running left to right, see Fig.1, Panel A). The video demonstrations conveyed motion events based on eight different manners and four different paths, resulting in 16 manner+path variations (see Table 1 for a listing of all video demonstrations used in the study). Each participant was tested with only half of the videos. After each video demonstration, the participant was presented with two animations and asked to choose the one that matched the initial demonstration (e.g., ‘Which one is he?’; see Fig.1, Panels B1-B2). One of the animations (the correct choice) showed a figure moving with the same manner and path conveyed through the actor’s gesture. The other animation (the incorrect choice) showed a figure moving using (1) a different manner (manner-only condition), (2) a different path (path-only condition) or (3) both a different manner and path (manner+path condition) of the motion conveyed through the gesture. Within each condition, we counterbalanced both the order in which the video demonstrations were presented across participants and the side (left vs. right of child) on which the correct animation was presented across trials. A. Video B1. Animation for gesture+speech correct match B2. Animation for gesture+speech mismatch Figure 1. Sample stills from video demonstration and test animations for manner+path condition Scoring The adult participants recorded their own responses into the computer with a mouse click. For the children, the experimenter identified the animation the child pointed to first and inputted this choice into the computer. Each response was scored as correct (1) or incorrect (0), yielding a score range of 0-2 for the training trials and 0-8 for the test trials. An independent coder (who was blind to the age group, experimental group, and hypothesis of the study) then re-scored from video the selections of a randomly-chosen 25% of the children. Agreement between the experimenter at the time of the study and the later coder was 100%. Preliminary analyses showed no effect of child’s sex on the choice score; therefore we collapsed across gender for subsequent analyses. Item Manner Path Gestural description 1. Run Right 2. Run Left 3. Dance Right 4. Dance Left 5. Fly Up Wiggle index and middle fingers rapidly left to right Wiggle index and middle fingers rapidly right to left Sway V-shaped index and middle fingers left to right Sway V-shaped index and middle fingers right to left Flap open palm upward 6. Fly Down Flap open palm downward 7. Tumble Up 8. Tumble Down 9. Crawl Right 10. Crawl Left 11. Flip Right 12. Flip Left 13. Roll Up Move open palm in circles away from body upward Move open palm in circles away from body downward Extend and close fingers bent at knuckles left to right Extend and close fingers bent at knuckles right to left Rotate V-shaped index and middle fingers left to right Rotate V-shaped index and middle fingers right to left Circle index finger upward 14. Roll Down Circle index finger downward 15. Climb Up Alternate index and middle fingers upward 16. Climb Down Alternate index and middle fingers downward Table 1. Descriptions of the gestures used in the video demonstrations 3. Results A 3 (age: 3-year-olds, 4-year-olds, adults) x 3 (condition: path+manner, path only, manner only) ANOVA showed a significant main effect of age, F(3, 105) = 74.68, p < .001, but no effect of condition, F(3, 105) = .34, p = .71, or interaction F(4, 105) = .9, p = .47. As can be seen in Figure 2, across groups adults performed significantly better than the four-year-olds, who, in turn, performed significantly better than the three-year-olds (Bonferroni, ps = <.01). More importantly, both the adults and the four-year-olds correctly chose the animation that matched the gesture demonstrated in the video at levels significantly above chance, t(35) = 29.38, p < .001; t(34) = 4.1, p < .001, respectively. Three-year-olds, however, showed chance performance, t(35) = .00, p = 1.0. M Correct Responses 8 6 3-year-olds 4-year-olds Adults 4 2 0 Path+Manner Path-only Manner-only Figure 2. Mean number of correct choices as a function of age group and condition (max possible score=8). 4. Discussion We examined the ability of 3-year-olds, 4-year-olds, and adults to understand gesture-speech combinations with iconic gestures conveying action information. We found that by age 4, children were able to distinguish the motion of an iconic gesture conveying manner and path information simultaneously from other motion events that differed only in path or in manner. However, 3-year-olds failed to perform significantly above chance levels, even when the incorrect motion event conflicted with the demonstrated gesture on both path and manner cues. Although early iconic gesture conveys predominantly action (not object) information (Özçalışkan & Goldin-Meadow, 2011; Özçalışkan, Gentner & Goldin-Meadow, in press), these results suggest that children’s comprehension of action information conveyed through gesture develops at a similar age as their comprehension of object information conveyed through iconic gestures. Together, these findings support reports of a general development in understanding iconicity within the preschool years (Namy, 2001; Namy, Campbell, & Tomasello, 2004). Earlier work has shown that children on average produce their first action words (i.e., verbs) in speech at 18 months and their first action gestures at 25 months, showing a lag of seven months in the production of iconic gestures conveying action information (Özçalişkan , Gentner, & Goldin-Meadow, 2013). Interestingly, our results show that it takes children another year and a half to understand similar action gestures—a pattern that suggests the opposite trajectory to speech production, namely comprehension of iconic gestures not preceding but following the production of iconic gestures. The question remains as to why we observe later comprehension of iconic gestures conveying action information (compared to their production). One possible reason for the later comprehension of iconic co-speech gestures conveying action information might be the elicitation task we have used. Previous reports of increased iconic gesture production at age 26 months relied largely on naturalistic data collection methods (see for example Özçalışkan, Gentner & Goldin-Meadow, in press). However, in this study, the paradigm was quasi-experimental; as such, it might have imposed greater cognitive demands on the children in grasping the meaning of such co-speech gestures. Another reason could be the relatively late-emerging language-specific patterns in children’s speech about motion, which begin around age three (Özçalışkan & Slobin, 1999). It is possible that children first need to learn the language-specific patterns in speech (and possibly in gesture) about motion before beginning to glean the relevant information in the co-speech gestures conveying similar action meanings. Future studies exploring children’s comprehension of a broader range of co-speech iconic gestures conveying action information, possibly across structurally different languages, will shed further light on the trajectory with which children learn to comprehend such iconic gestures. In summary, our study shows that children can comprehend iconic cospeech gestures conveying action information around 4 years of age and that they can easily dissect these gestures into their semantic components (i.e., manner vs. path). These findings extend existing studies by showing early emerging abilities in children’s comprehension of co-speech gestures, and also mark gesture as an essential component of the receptive communicative abilities in early development.