Introduction

Sign phonetics is often discussed and analyzed with reference to sign phonology, both by sign language researchers and by others. As is the case with speech research, phonetics and phonology are related and interact with each other, but sign phonetics and sign phonology have different areas of emphasis. Sign phonetics is related to the physical forms of signs and their accompanying non-manual movements, whereas sign phonology focuses on the features of signs that can create phonemic contrasts. Like spoken languages, signed languages are made up of sublexical elements that can be combined in different ways to form lexical items. Stokoe identified three phonological parameters that can differentiate signs in American Sign Language (ASL): handshape, movement, and location (Stokoe, 1960). Handshape describes the configuration of the hands as a sign is produced. Movement describes how the hands and arms move during a sign. Location describes where the hands are located during production of a sign. Battison (1978) later added hand orientation—the direction that the palm faces during a sign—to this list. Liddell and Johnson (1989) analyzed the structure of continuous signing and developed a movement-hold model to parse sign boundaries. The physical form and perception of all of these features have been analyzed by sign phoneticians.

In the sign modality as well as the speech modality, phonetics deals with research questions along the lines of:

A defining feature of signed language is that it uses the hands and arms, rather than the vocal tract, as its primary articulators. While the implications of this structural difference between sign and speech have been discussed at length (Meier, 2002; Sandler, 1993), there is more work to do in identifying the full range of differences between sign and speech and in clarifying which of those are primarily the effect of the set of articulators that the two language modalities use, rather than the effect of perception modality or of abstract linguistic structure.

While the hands and arms are the primary articulators for signed language, movements of the head, mouth, torso, eyebrows, eyes, and other secondary articulators show rule-governed patterns during signing, which relate to syntax and prosody. These secondary articulators and their movements are referred to as nonmanuals (Herrmann & Steinbach, 2013; Sutton-Spence & Boyes Braem, 2001). The nearest analogue to sign nonmanuals in the speech modality is probably co-speech gesture, which is increasingly being analyzed as it relates to prosody (Esteve-Gibert & Prieto, 2013; Krivokapic, Tiede & Tyrone, 2017; Loehr, 2007). Only a small number of studies have looked at the phonetics of nonmanuals (Udoff, 2014; Weast, 2008), or at the compensatory movements of the head and torso which facilitate contact with the hand during signing (Mauk & Tyrone, 2012; Tyrone & Mauk, 2016).

Another structural characteristic that is distinct to sign languages is the existence of fingerspelling systems. Fingerspelling is a mechanism that allows sign languages to borrow vocabulary from the written form of a spoken language. During fingerspelling, a signer produces a sequence of hand configurations or hand configurations plus locations which represent a sequence of written characters. Some sign languages use fingerspelling more than others, and not all fingerspelling systems take the same physical form. For example, even though ASL and British Sign Language (BSL) use fingerspelling to borrow words from written English, ASL fingerspelling is one-handed, while BSL fingerspelling is two-handed. Fingerspelling is interesting from the standpoint of phonetics, because fingerspelling movements are smaller and more rapid than the movements in lexical signs, and thus more prone to coarticulation.

Methods for Analyzing Sign Phonetics

The earliest sign phonetics research relied on citation forms of signs or on production data recorded with standard video. Since the 1990s, many studies have used instrumented measures of sign production, such as motion capture or glove-based systems. These allow considerably more measurement precision and experimental reliability. At the same time, the equipment is usually expensive and requires substantial technical support. In addition, motion capture systems are generally not portable, so they can only be used in a laboratory setting. This is likely one reason that only a few sign languages have been studied closely at a phonetic level. Also, when sign production data is collected in a lab context, there may be limitations to the naturalness of the signing represented. There has been a recent return to video-based analyses of sign phonetics, because video can be used in a broader range of settings, and because there has been an increase in video sampling rate and an improvement in measures of movement speed and displacement, based on quantitative voxel analyses (Karppa, Jantunen, Koskela, Laaksonen & Viitaniemi, 2011; Tkachman, Hall, Fuhrman & Aonuki, 2019).

A related area of research that has included instrumented measures of sign production is the interface between sign language and human-computer interaction, with a focus on either automated sign recognition and synthesis or on the improvement of communication technologies for the deaf (Lu & Heuerfauth, 2009; Muir & Richardson, 2005; Vogler & Metaxas, 2004). While these studies have collected very detailed phonetic data, their broader objective is to identify the potential for meaningful contrasts between signs, rather than to examine phonetic variation.

Another methodological approach is a qualitative analysis of the distribution of phonetic forms in the lexicon as an effect of articulatory or perceptual constraints. For example, Siple (1978) performed qualitative analyses of the distribution of different hand configurations in ASL in relation to signs’ location. She observed that during signing, interlocutors fixate on the signer’s face rather than tracking the movements of the signer’s hands. Siple suggested that this was the reason why hand configurations are more varied when a sign is produced on or near the face, given that those signs have greater visual salience for the perceiver. Ann (2005) and Eccarius (2008) carried out similar analyses of Taiwan Sign Language and American Sign Language, respectively. Both studies concluded that the frequency of specific handshapes in those languages was largely determined by their ease of articulation.

More recently, Sanders and Napoli (2016) carried out a cross-linguistic study of the distribution of certain sign movements in Italian Sign Language (LIS), Al-Sayyid Bedouin Sign Language (ABSL), and Sri Lankan Sign Language (SLSL). They examined the citation forms of signs with simultaneous or alternating bimanual movements. Their findings suggest that the movements that most often occur in these languages are the ones that do not biomechanically destabilize the torso’s position. This adds to the literature considerably, because most sign phonetics research has investigated either hand location or handshape, rather than hand movement.

In summary, the field of sign phonetics has benefitted from a range of different methodological approaches; however, these methods have not been applied consistently across different sign languages. Moreover, the different methods have their own advantages and disadvantages, in terms of measurement precision, portability, and ease of use. These factors must be considered in comparing one sign phonetics study to another.

Relationship Between Sign Structure and Anatomy/Physiology

Important early research in sign phonetics examined the anatomy of the human hand and forearm in order to determine the inherent constraints on the formational structure of signs. Mandel (1979, 1981) analyzed the handshapes used in ASL in light of the configuration and function of the tendons in the forearms. His analysis suggests that the selected finger in a sign’s hand configuration is unlikely to be only the middle finger or the ring finger, because it is difficult to control the movements of these fingers given that they share an extensor tendon with other fingers.

Similarly, Ann (1996) analyzed the anatomy of the hands, outlining the configuration of the bones, muscles, tendons, and ligaments that allow or prevent the fingers’ movement. Based on this analysis, she developed a scoring system for which sign language handshape should be easiest or most difficult to produce. She then compared her calculated scores to the distribution of those handshapes in ASL and in Taiwanese Sign Language. In both languages, the handshapes that she identified as more difficult to produce occurred less frequently in the lexicon.

Later research pursued the question of which joints are flexed/extended to produce a sign. In one study of articulatory differences related to language experience, Mirus, Rathmann, and Meier (2001) examined non-signers’ imitations of signs produced in isolation by a native signer, which were viewed on a video monitor. The researchers coded the data descriptively from video and found that non-signers tended to initiate movement from more proximal joints than the native signers who had served as models. Similarly, Brentari, Poizner, and Kegl (1995) found that signers with Parkinson’s disease used more distal joints to produce ASL signs, compared to the citation forms of those signs. More recently, Eccarius, Bour, and Scheidt (2012) developed a detailed procedure for calibrating data gloves in order to convert raw sensor data into useful joint angle information that could be interpreted and analyzed across productions and across signers. This method has enormous potential in allowing researchers to assess the reliability and generalizability of their findings related to the phonetics of handshape in ASL and in other sign languages.

Many of the earliest studies to use instrumented data collection to analyze sign production investigated differences between typical signing and signing that was disrupted by neurological disorders such as stroke or Parkinson’s disease. Poizner, Klima, and Bellugi (1987) were among the first researchers to use motion capture to analyze production of ASL. They carried out a series of studies comparing the productions of ASL signers who had aphasia, apraxia, or right hemisphere damage as a result of stroke. The goal of their research was to determine whether aphasia would take a different articulatory form from deficits in the production of meaningful gestures (apraxia) or from visuospatial deficits caused by right hemisphere damage. Poizner, Bellugi, and Klima (1990) extended this line of research to include signers with Parkinson’s disease, a disorder that is primarily motoric rather than linguistic in nature. Similarly, Brentari et al. (1995) carried out an Optotrak study to compare an ASL signer with Parkinson’s disease to an ASL signer with aphasia. Using motion capture data, they were able to show that the signer with Parkinson’s disease preserved linguistic contrasts in production but showed a deficit in the coordination of handshape and movement. By contrast, the errors produced by the signer with aphasia were linguistic rather than motoric in nature. These findings illustrate that like speech, sign language can break down at the level of motor control or sensory feedback, as well as at the level of linguistic function. Moreover, the nature of the breakdown is consistent with what would be predicted from analogous speech disorders (cf. Tyrone, 2014).

Some of the early sign language research on ASL, such as Stokoe (1960) and Klima and Bellugi (1979), strongly emphasized the similarities between sign and speech. This was an important first step in establishing sign language as a legitimate research topic in the field of linguistics. After that initial parallel was drawn between sign and speech, sign phoneticians were then able to explore the relationships between sign structure, limb biomechanics, motor control, and cognitive function.

Prominence, Coarticulation, and Undershoot/Reduction

Several studies in sign phonetics have examined the effects of linguistic prominence, phrase position, and phonetic environment on the realization of signs. Wilbur and Schick (1987) collected ASL data from video and analyzed the form of contrastive and emphatic stress in semi-spontaneous signing. Their results suggested that stress was realized primarily through the lengthening of sign duration and the raising of signs in the signing space. Wilbur (1990) used a WATSMART system to examine the realization of stress in ASL. (WATSMART was one of the earliest motion capture systems—its name is an acronym for Waterloo Spatial Motion Analysis and Recording Technique.) Using this technique, she showed that native signers modified the duration of the movement transition prior to a stressed sign, whereas non-native signers increased the hand’s displacement during the stressed sign. This distinction between native and non-native stress patterns would likely not be identifiable from descriptive analyses of video.

Multiple studies have investigated coarticulation and other effects of phonetic context in signing and fingerspelling. Some of these studies have directly compared sign production to speech or to sequences of non-linguistic limb movements, to tease apart the effects of linguistic structure and of articulator size and physiology. Cheek (2001) examined coarticulation of handshape in the production of ASL signs with the index finger extended (1-handshapes) or with all the fingers extended (5-handshapes). Target signs with each of those handshapes were embedded in carrier phrases so that they were preceded and followed by the other handshape. She found variation in handshape that was rate-dependent and consistent with models of coarticulation. In addition, she found both articipatory and perseverative coarticulatory effects.

Grosvald and Corina (2012) used a motion capture system to examine linguistic and non-linguistic coarticulation in ASL sign production, which they compared to coarticulation in acoustic speech data in English. They examined not only the effects of adjacent signs on the realization of location, but also the effects of signs that precede or follow the target sign at a distance of up to three intervening signs. The researchers collected productions of schwa vowels in English, which were embedded in carrier phrases, such that the target vowel was at varying distances from the vowel /i/ or /a/. From these data, they measured F1 and F2 for the target vowels to look for coarticulatory effects. Similarly, for the signing data, they used a motion capture system to collect productions of ASL signs located in the neutral space in front of the body. Those target signs were embedded in carrier phrases which contained a sign located at the forehead or at the waist. The target neutral space sign was placed at varying distances from the high or low sign, and the hand’s vertical position was measured. To assess non-linguistic coarticulation, the researchers cued signing participants to flip a switch that was either above or below the middle of the signing space during a signing task.

Grosvald and Corina found that coarticulatory effects were weaker in the sign modality than in speech—in particular, distant speech segments had a stronger influence on vowel formants than distant signs did on sign location. In addition, they found that in terms of coarticulation, linguistic coarticulation patterned more like non-linguistic coarticulation than coarticulation in speech.

Ormel, Crasborn, and van der Kooij (2013) used a data glove in conjunction with motion capture to investigate coarticulatory effects of hand height in Sign Language of the Netherlands (NGT). They found that hand height was influenced by the location of the preceding and following signs, but that certain sign locations and certain forms of contact were more subject to coarticulation than others. This research extends the earlier findings by Grosvald and Corina (2012) and by Tyrone and Mauk (2010) by considering sign locations that required contact with another articulator (such as the non-dominant hand).

Wilcox (1992) examined the production of ASL fingerspelling, using motion capture data from the WATSMART system. His findings demonstrated that there was a large amount of coarticulation in fingerspelling and that features from an individual letter in a fingerspelling sequence would carry over into subsequent letters in the sequence. Moreover, he proposed that the transitions between letters were important for comprehension of fingerspelling in ASL. More recently, Keane (2014) carried out a study on handshape coarticulation in ASL fingerspelling. He extended the earlier research by Wilcox and others by collecting a large data sample using video and motion capture, and also by analyzing coarticulation for what it reveals about the phonetics-phonology interface. Keane’s findings are consistent with the early research on anatomy and physiology, which suggested that extension of the pinky finger is easily realized because of the configuration of the bones and tendons of the hand. He built upon the initial studies by Mandel and Ann, given that he was analyzing production data rather than citation forms of signs.

Mauk (2003) used a Vicon motion capture system to examine articulatory undershoot of handshape and location in ASL (that is, when the articulators do not move far enough to achieve handshape or location targets). He found that undershoot occurred in both of these parameters as an effect of signing rate and phonetic environment. Similarly, Tyrone and Mauk (2012) collected a larger data sample in order to investigate phonetic reduction in the realization of location in ASL. Like the earlier study by Mauk (2003), they found that phonetic reduction in ASL occurred as an effect of factors that would be predicted from speech research. Their main result was that ASL signs with locations that are high in the signing space tended to be lowered at faster signing rates and when they were preceded or followed by a sign that was low in the signing space.

While Tyrone and Mauk (2012) examined the lowering of forehead-located signs in ASL in lab settings, sociolinguists of ASL examined the same phenomenon in more naturalistic contexts (Lucas, Bayley, Rose, and Wulf, 2002). Perhaps because of the difference in methodology, Lucas et al. (2002) found limited effects of phonetic environment, while Tyrone and Mauk found strong effects of phonetic environment. Russell, Wilkinson, and Janzen (2011) set out to investigate the seeming contradiction between findings from sociolinguistic field work and experimental phonetic research on the lowering of signs that are high in the signing space. In order to do so, they examined a set of videotaped corpus data and looked at productions of signs located at the forehead, head, or neck in order to determine whether these signs were lowered and to what extent. In addition, they investigated whether lowering occurred in a manner that was gradient or categorical.

Russell et al. used a more controlled data collection procedure than earlier sociolinguistic studies, but unlike the phonetic studies, their data still consisted of relatively naturalistic conversation. They used standard video to record the conversations but worked to maximize the consistency and precision of the measurements. For instance, they implemented a procedure for correcting the data for the position of the head and a procedure for normalizing the data to allow for comparisons across signers. Like the earlier field studies, Russell et al. found that the extent of lowering differed according to the grammatical category of the sign that was lowered. In addition, they found that the signs that occurred more often in the language, by their estimation, were lowered more often and to a greater extent.

This body of research indicates that sign languages exhibit phonetic effects of context and prominence, similarly to how spoken languages do. Indeed, a few of these studies made direct comparisons between context effects in one modality as opposed to the other. Future research could probe this parallel further to try to explain the subtle differences in how context effects are manifested across modalities.

Sign Perception

Multiple studies of sign perception have investigated signers’ and non-signers’ perception of the phonological parameters identified by Stokoe (1960). Most of these studies investigated the possibility of categorical perception. Categorical perception is a phenomenon that was documented for some speech sounds, and it refers to the idea that individuals can only discriminate stimuli as well as they can identify or label them (Liberman, Harris, Hoffman, & Griffith, 1957). This was first identified for perception of F2 transitions and place of articulation in stop consonants. Listeners were incapable of discriminating different tokens that were within the same identification category, as determined by participants’ performance on an identification task. Moreover, listeners showed their best discrimination accuracy when two stimuli were on opposite sides of the identification boundary. Not all phonemic contrasts are perceived categorically in spoken language, but the possibility has been examined for multiple speech sound contrasts as well as multiple languages.

Emmorey, McCullough, and Brentari (2003) examined categorical perception for location and handshape in deaf ASL signers and hearing non-signers. The two groups performed similarly on the identification task, but only deaf signers showed categorical perception for handshape, based on the position of the discrimination peak. Neither group showed categorical perception for sign location. The stimuli were computer generated still images of ASL signs. Baker, Idsardi, Golinkoff, and Petitto (2005) carried out a similar study, examining only handshape, and they found categorical perception for signers but not non-signers, and greater within-category discrimination in non-signers.

Along similar lines, Best, Mathur, Miranda, and Lillo-Martin (2010) compared early and late deaf ASL signers, late hearing signers, and hearing non-signers on identification and discrimination of a handshape contrast. The stimuli were dynamic pseudosigns presented on video. The researchers did not find categorical perception in any group, but language experience affected participants’ ability to discriminate different handshapes.

Morford, Grieve-Smith, MacFarlane, Staley, and Waters (2008) looked at computer-generated dynamic sign stimuli and used handshape contrasts as well as location contrasts. Their participants were deaf native signers, deaf non-native signers, and hearing non-native signers. They found that all groups performed similarly on identification, but only the deaf participants showed statistically significant categorical perception, and the effect was stronger in the native than in the non-native deaf signers.

A different type of perception study by Dye and Shih (2006) examined phonological priming in native and non-native deaf signers of BSL. Participants were asked to carry out a lexical decision task after viewing an image of a sign that was related to the target. The researchers found that phonological priming occurred for native signers when the prime included the same location and movement as the target sign. Handshape had no effect, and neither of the other two parameters had an effect in isolation. Non-native signers completed the lexical decision task quickly and accurately, but they showed no effect of priming. These findings are consistent with earlier research by Hildebrandt and Corina (2002), which suggested that signers and non-signers view signs or pseudosigns as more "similar" when they share a location and a movement than when they share other combinations of structural features.

To date, there has been very little research on the interaction between perception and production in the sign modality. The research thus far suggests that signers do not rely heavily (if at all) on visual feedback of their own signing. Emmorey, Gertsberg, Korpics, and Wright (2009) carried out a study in which they modified visual feedback during signing, so that signers’ view of the signing space was reduced or completely blocked. The researchers found that signers did not alter their signing as an effect of either of these conditions. They concluded that there is no sign equivalent to the Lombard effect, in which hearing people speak louder when they have reduced auditory feedback. In a related study, Emmorey, Bosworth, and Kraljic (2009) found that signers perform poorly at recognizing signs viewed as if the signer were producing them. In addition, they showed that signers were skilled at learning new signs from other sign languages, even when visual feedback was limited or blurred. These findings are perhaps not surprising, given that many signs are produced outside the central visual field of the signer, but spoken words are not usually produced outside the auditory perception of a hearing speaker. In addition, there may be a different role for kinesthetic feedback in the two modalities.

Sign Phonetics, Sign Phonology, and Units of Production/Perception

While spoken language phoneticians have debated whether it is the articulatory gestures or the acoustic correlates of speech that are the units of speech perception and production, no such debate has arisen in sign language research. No one has suggested that the objects of sign perception are inherently hidden or difficult to access. It has been taken as a given that signers directly perceive the articulatory gestures of signed language. Therein lies the challenge for sign phonetics and phonology: what are the underlying units in the sign modality and how do they differ (if at all) from what a human researcher can observe and annotate? The visual nature of the sign modality can make it difficult for us to appreciate the importance of non-contrastive variation in sign production, because it seems that sign gestures should be transparent to any observer. As a result, there has been only limited quantitative research in sign phonetics, and not much consideration given to the interface (or distinction) between sign phonetics and sign phonology. (For a recent discussion of these questions, see Brentari, 2019). The field of sign linguistics would benefit tremendously if sign phonetics and phonology were clearly delineated and could inform each other more explicitly.