1. Overview

Cochlear implants (CIs) provide partial restoration of the sensation of hearing to people who would otherwise experience severe to profound deafness. Since the mid-1980s, thousands of adults with severe to profound hearing loss have received CIs, and since 2000 they have been approved for children as young as 1 year of age (NIDCD, 2020) and are now available for children as young as 9 months of age (Cochlear Corp, 2020).

For both children and adults who are deaf, CIs have been the most successful sensory aid to develop or restore spoken communication abilities. However, the nature of that hearing sensation is so dramatically different than typical hearing that many of the basic principles of auditory perception cannot be generalized to those who use CIs. Here we introduce the essential terminology, patterns of performance, and assumptions related to CIs, which will lay the groundwork for informed discussion and interpretation of the literature. The role of CIs in a larger scientific and cultural context has evolved in the past three decades, with meaningful lessons not only about the science of hearing and language but also about how communication is a part of a person’s identity.

2. What a Cochlear Implant Is

Although CIs might look similar to hearing aids, they are much different. Specifically, CIs do not amplify sound and send sound into the ear canal the way a hearing aid does. Instead, CIs directly stimulate the auditory nerve electrically, in a way that is designed to crudely represent the neural activation that might have resulted from typical hearing. The CI is not a new physical cochlea (like a limb prothesis) but a device that mimics the function of the cochlea and exploits its existing structure. It is a significantly more invasive and risky treatment for hearing loss compared to hearing aids, and thus CI candidates try hearing aids as a first approach before committing to the long and effortful process of using a CI.

A CI consists of a microphone and signal-processing hardware that rest on the outside of the head (usually on the ear) and a small array of physical electrodes that are inserted into the cochlea. The microphone and processor capture the external sound and divide it up into bands of different frequencies. The amplitude of each frequency component is tracked and converted into instructions for the electrodes that simulate the place in the cochlea that is appropriate for the frequency of the original sound (see Figure 1). In the top panel, the full waveform of a syllable is shown. For simplified illustration, it is filtered into six frequency bands between 100 and 8000 Hz, shown in the second panel. The envelope of each band’s waveform is extracted (shown in the third panel). Finally, a sequence of electrical pulses is generated whose amplitude follows the extracted envelope in each band (lower panel). These pulse trains are delivered through the six corresponding implant electrodes.

Instructions to active electrical pulse trains on each electrode are sent by radio communication from the external device to the internal device. There is usually some recovery time after the surgical implantation of the internal device, and there is almost always a period of “relearning” how to interpret sound after the device is initially activated. CI activations in infants have been popularized for their emotional appeal, but they represent a small minority of the reality of CIs. Initial activation can be very surprising and disorienting, and it is not at all like the moment of immediate clarity from the first time putting on a pair of well-fit eyeglasses.

Open in new tab

Figure 1. Illustration shows the original (top panel) waveform of the word “pinch” (/pɪntʃ/) followed by the filtered frequency bands (second panel), the continuous extracted envelopes of those frequency bands (third panel), and the discrete electrical pulses that follow the envelopes (bottom panel).

3. Brief Synopsis of Speech Perception With a Cochlear Implant

4. The Structure and Sound of a Cochlear Implant

5. Patterns of Perception: What Is Heard, What Is Not Heard, What Is Mistaken

The most important observation to make about speech recognition performance with CIs is that it is extremely variable (Gifford & Dorman, 2018). Perception of individual words can be just 5% in one person but 95% in another person, even if those people appear to have a similar audiological history. Performance for whole sentences is typically better, because the syntactic and semantic coherence of a sentence can support inferences about what was spoken. In fact, semantic context appears to be used heavily by CI listeners, underscoring the reliance on top-down cognitive processing when speech input is degraded (e.g., O’Neill et al., 2019), underscoring the reliance on top-down cognitive processing.

Perception of individual consonants is relatively less affected by linguistic constraints, and thus is more straightforwardly predictable based on what is known about how the sound is coded. Consonant place of articulation is contrasted mainly by frequency differences (such as formant transitions or spectral peaks in a fricative). Since a CI distorts frequency information so severely, place of articulation is typically the consonant feature that is perceived with the least accuracy. Conversely, the voicing feature is perceived much more reliably, owing to the fact that segment duration plays a large role in voicing contrasts (Winn et al., 2012); consider the duration of the vowel in word pairs like rope/robe, or the duration of consonant cognates such as s [long] and z [short]). It is possible that the robustness of voicing is constrained to English, where duration-based voicing cues tend to be exaggerated relative to other languages. Perception of manner of articulation (the degree of airflow constriction during the consonant) is more complex (Rødvik et al., 2018), as it suffers from the constraints of limited dynamic range. The disparities between slightly different degrees of vocal tract constriction could result in sound amplitude envelopes that are too small to be discriminable by a CI listener, yet there are typically other probabilistic constraints on manner of articulation that render it somewhere in between voicing and place of articulation in terms of difficulty.

Vowel perception with a cochlear implant is notoriously poor (Harnsberger et al., 2001). As vowels typically are contrastive mainly by their spectral properties (e.g., formants) without very informative cues in the amplitude envelope, the listener must rely on the exact auditory domain that is most heavily distorted by the CI. In that way, vowel perception is somewhat akin to perception of consonant place of articulation. However, as the spectral composition of vowels is neatly described even within the constrained frequency-channel analysis in a CI, there are some CI recipients who perceive vowels successfully.

For CI listeners, pitch perception is extremely poor and almost completely absent (Moore & Carlyon, 2005; Zeng, 2002). Pitch perception in people with typical hearing involves a combination of at least three mechanisms: (a) harmonic pitch, driven by the encoding of specific sound components at equally spaced interval frequencies; (b) rate pitch, or the rate of repetition of waveform peaks; and (c) spectral pitch, or the placement of activation in the cochlea. Cochlear implant users’ pitch perception has revealed some complexities of pitch perception that were not fully understood from studies of acoustic hearing alone. Specifically, although harmonic pitch might be intuitively understood as an example of the place code, the poor pitch perception in CI users who have a place code but not harmonicity demonstrates that these are separate concepts. Harmonic pitch is completely absent from a CI because there are no harmonics represented in the cochlear stimulation; the sites of electrical stimulation are simply too broad to form the compact and precise peaks needed for harmonics. As resolved harmonics lie at the heart of a human’s ability to finely distinguish F0 differences on the order of 0.2%, this is a significant loss (Mehta & Oxenham, 2017). Perception of pitch based on rate is much less sensitive, requiring roughly 6% change to be noticeable. Although pitch information can theoretically be encoded by the rate of stimulation in a CI, the real implementation is severely limited in many ways. First, the rate of stimulation in a CI is almost always fixed at a constant rate, and therefore unrelated to the F0 of the sound being heard. F0 can therefore only be encoded by sampling the amplitude modulations that correspond to rate pitch. However, the range of amplitude modulation resulting from voice pitch can be rather shallow and also might be undersampled by the device, such that it might not be detectable. Finally, even when experimenters override the internal workings of the device to specifically represent perfectly maximal amplitude modulation at the exact F0 rate of the input speech, there are psychophysical limitations on how high that rate can go before it is simply perceived as an undifferentiated continuous buzz. Unfortunately, that limiting rate—estimated to be around 300 Hz—hovers near the middle-upper end of the F0 range for adult women, potentially jeopardizing the ability of a CI listener to detect pitch of everyday conversation partners.

As a consequence of poor pitch perception, CI listeners generally are thought to struggle with talker identification (Fuller et al., 2014), prosody and lexical tone (Chatterjee & Peng, 2008), and perceiving speech in noise (since F0 is a very powerful cue to separate one talker from background noise). Perceiving speech in noise is also weakened by the fact that the ability to localize sound is extremely poor in CI listeners (Jones et al., 2014), owing to the lack of synchrony across devices (Kan & Litovsky, 2014). There is thus little to no representation of the binaural cues that are essential for detecting the position of a sound in space. Pitch perception and speech in noise are commonly thought to be the most prominent aspirational goals of the next generation of CI technology.

Because the key element to successful CI signal processing is the representation of changes in the amplitude envelope, many realistic listening scenarios such as background noise and reverberation are known to severely disrupt speech understanding through a cochlear implant (e.g., Nelson & Jin, 2004; Oxenham & Kreft, 2014). The problem is that background noise fills in and masks the gaps and variations in intensity that would normally signal linguistically meaningful changes in the signal. Even what might be presumed to be a very favorable listening condition for persons with typical hearing (such as a +10 dB speech-to-noise ratio) can be very problematic for users of cochlear implants, and reverberation levels that are even as minor as a half-second or less (e.g., Zheng et al., 2011) can cause significant problems. Because of the implant’s poor spectral resolution and the absence of fine temporal and spectral cues, segregating speech from any kind of noise is a very challenging task.

After the initial activation of a CI, the patient typically goes through a prolonged period of gradual improvement while acclimating to the dramatically new sound quality. The patient essentially has to remap the world’s sounds to new perceived sound qualities. During the first 3 to 12 months, performance on basic speech recognition tasks typically improves steadily and finally reaches a plateau. That plateau can be highly variable across individuals and is the consequence of dozens of factors (Blamey et al., 2013; Lazard et al., 2012). Some reach a stage where their auditory abilities meet their basic communication needs, but many must make compromises, such as avoiding telephones, requiring face-to-face visual cues with good lighting, avoiding crowded and noisy situations, and so forth.

6. Replicating the Sound of a Cochlear Implant

Simulation of cochlear implant processing is done using a vocoder, which breaks down the speech sound into a set of small number of discrete spectral bands and represents a simplified form of those bands that preserves the changes in intensity but discards the frequency details. Vocoders can vary by their frequency range, number of spectral channels, type of carrier, or degree of temporal precision. Despite these meaningful differences that should always be specified, the term “vocoder” is sometimes used as shorthand for processing that analyzes a frequency range between ~100 Hz to an upper edge of 5000 to 8000 Hz, from which frequency eight channels are extracted and represented by noise. The reason this style of vocoder has prevailed is that performance by typical-hearing listeners on speech processed by an eight-channel noise vocoder usually a very good match to the performance by better-performing CI listeners in the same task. However, the exact number of spectral channels needed depends on the type of speech that is being heard and what is being asked of the listener.

7. Outcome Measures

New research is looking beyond spoken language recognition as primary measures of cochlear implant success. Measures of listening effort (Winn & Moore, 2018) and functional use of hearing (e.g., Kronenberger et al., 2020) may be able to shed light on the realistic benefits and shortcomings of cochlear implants for interpersonal communication and other functional abilities. Laboratory measures of speech recognition can be of little use when individuals withdraw from spoken communication. Measures of social isolation and interaction, and the impact of cochlear implants on quality of life are emerging as important indicators of cochlear implant success.

O’Neill (2020) showed that intelligibility scores reported in percent correct (keywords) might not indicate the conversational difficulty experienced by a cochlear implant user. Listeners repeated sentences and were scored for their percent words correct; at the same time, they were asked to rate how difficult that listening situation had felt. Listeners with normal hearing sensitivity rated listening as “somewhat difficult” or worse when their scores feel below about 90% correct. For CI users, though, some who scored quite poorly (18% correct) classified the task as only “somewhat difficult” while others who did well (75%) rated the task as “difficult.” Perceived self-efficacy and effort might therefore reflect success relative to expectations, where “18% is better than nothing” and “75% is not as good as I want” are both realistic and common feelings. The relationship between intelligibility and perceived difficulty was very weak, indicating that there is more to realistic communication than simply getting the words correct. Structured interviews with numerous CI recipients by Hughes et al. (2018) showed in vivid detail how individuals gauge the effort, anxiety, and investment needed to socially communicate, and make decisions based on effortfulness. Notably, there was little to no mention of word-recognition accuracy in these patient reports. The CI recipients in that study further went on to describe the feelings of isolation and invisibility involved in spoken communication, underscoring how psychological factors can completely overpower any contribution of the technology itself.

8. Cochlear Implants and Language Development

The primary goal of a cochlear implant for a child who is born deaf is to facilitate communication and language development. But of course not everyone who is deaf chooses to communicate using spoken language. Disagreements still exist between those who advocate early full-time use of sign language to ensure language development during a critical period (e.g., Humphries et al., 2012) and those who emphasize auditory development through the use of cochlear implants and spoken language. Some of those disagreements go far past the scope of this report because they deal with the child’s agency to choose participation in a particular culture or to choose their own path. As most children with congenital deafness are born to parents who have typical hearing, there is no simple and obvious path for socialization and cultural inclusion.

Cochlear implants have had a very positive effect on the language and academic development of children who are deaf (e.g., Geers et al., 2017). Nonetheless, they have not provided full access to spoken language. Evidence from numerous studies is converging on the general finding that for most language measures children with CIs achieve an ability level of about one standard deviation below their peers with normal hearing sensitivity (see, for example, van Wieringen & Wouters, 2015). As might be predicted by the sparse acoustic representation of speech through a CI, language skills requiring access to complex phonological structure is more challenging for children using a CI than are other language skills such as lexical or syntactic abilities (e.g., Nittrouer et al., 2018). Numerous studies have shown that literacy skills for children with severe to profound hearing loss still lag those of their peers, even if they do get a CI (Harris et al., 2017). In general, cochlear implants have not resulted in dramatic widespread academic performance increases for children with profound hearing loss (Marschark et al., 2015).

The debate about whether to use sign language with children who are deaf remains lively. Several retrospective studies of children with and without early sign language exposure suggest that those children without exposure to sign language tend to score better on measures of spoken language than those who allowed access to sign (e.g., Geers et al., 2017). However, standardized measures of spoken language should not be considered the only measure of communication and interaction. For a child who is deaf, the fundamental need is language access—that is, the child must be able to perceptually receive and cognitively process the auditory and visual signals in the communication setting (e.g., Hall, 2020). Children also use language to express and receive emotions and to establish trust and other social connections—abilities that are not necessarily measured by standardized language tests but which are potentially aided by nonauditory forms of communication. Longitudinal studies of language abilities in school-age children support the idea that earlier implantation leads to better outcomes on average, but there is a very wide range of variability, such that outcomes cannot be predicted solely based on the time of implantation (or any other singular factor). Despite the uncertainty of prognosis, some practices have become standard, such as the tendency to evaluate a child’s abilities based on hearing age rather than on chronological age.

9. Social Impacts and Global Outcome Measures

To understand the social impact of cochlear implants, one must appreciate the difference between deaf—someone who does not have typical hearing ability—and Deaf—being a member of a fully-realized community of people whose communication style is central to their identity. In the early years of cochlear implants, the response of the Deaf community was to reject them in favor of sign language (see National Association of the Deaf [NAD] position statement from 1999). Cochlear implants and the emphasis on spoken language was perceived as a threat to Deaf culture and to American Sign Language. Additionally, CIs could be described as a “treatment” for something that was not believed to be a problem. In the decades that have passed since the NAD position statement, there still remains some antagonism toward CIs by the Deaf community, but their use is substantially less controversial. In recent years, the NAD has recognized that “deafness is diverse, especially in the choices that deaf adults and parents of deaf children continue to make about the range of communication and assistive technology options.” The amount and quality of the language input matters greatly, and variation in language input may be one of the important factors predicting communication success (e.g., Holt et al., 2020). It would be fair to say that CIs are generally not controversial among people who grew up using spoken language, because the implant would restore an ability that was lost. But the Deaf community primarily includes people who never used spoken language, and for whom a CI may fundamentally change rather than restore a part of their identity.

At the Laurent Clerc National Deaf Education Center at Gallaudet University, cochlear implants are now a part of the conversation, where in the past they may have been rejected:

There is no single language and communication approach appropriate for the diverse children who are using cochlear implant technology. Effective language and communication strategies following implantation should reflect the complex interaction of characteristics specific to each child and family. (Gallaudet Clerc Center FAQ)

10. Conclusions and Future Directions

Cochlear implants can substantially improve quality of life among adults who have lost their hearing, and they can also support near-normal language development in children with congenital hearing loss. The perception of speech is difficulty because of various distortions (especially to the spectral peaks and pitch-related cues). Any speech sounds that contrast mainly by frequency (e.g., place of articulation contrasts) will be especially difficult. However, perception of speech with a CI is not simply a less accurate version of speech perception with typical hearing. CI recipients rely heavily on contextual cues, visual cues, and other compensation strategies in situations where they opt in to spoken communication. On many occasions, they will also be economical with their effort, choosing instead to abandon spoken interaction because it is too draining. CIs were once considered highly controversial but are now much more widely accepted.

There are a number of recent and ongoing developments in CI technology to anticipate in the coming years. There are efforts to address the major shortcomings of current devices by partially restoring access to cues for voice pitch, and introducing better specificity of the electrical stimulation. Additionally, the candidacy criteria for CIs has gradually become less stringent over the years, so that one needs not have complete deafness to be considered for implantation. Many individuals have substantial residual hearing, inviting the use of a hearing aid in addition to the CI. Fortunately the hearing aid is complementary to the CI in that it transmits low-frequency voice pitch information that complements the high-frequency phonetic information delivered by the implant. In recent years, there have been a large number of implanted patients who can even use acoustic hearing in the same ear that is implanted, and who can take advantage of so-called hybrid devices. Bilateral implantation has become the standard of care in many places.

There are more invasive treatment approaches that share a lineage with the cochlear implant. The auditory brainstem implant bypasses the cochlea altogether and directly stimulates the cochlear nucleus; this option is suitable for patients who lack a cochlear structure or lack auditory nerve integrity.

The current frontier of CI innovation is the expansion of candidacy, opening up the doors for adults and children to receive a CI in cases where they would have been denied in past years. Those who retain usable hearing in one or both ears are now routinely considered to have potential benefit from a CI. This even includes individuals who have normal hearing in the ear opposite to the implanted ear, giving rise to a small population of so-called single-sided deaf implant recipients who integrate an acoustic signal in one ear with an electrical signal in the other ear. CIs are also increasingly becoming recognized for their use as a potential treatment of severe tinnitus (ringing in the ears) and auditory nerve dyssynchrony.

In the span of less than 50 years, the idea to electrically stimulate the auditory nerve has evolved from a radial proposal to a routine medical treatment. This has had a transformational effect on quality of life for those with hearing loss, for children acquiring spoken language, and for scientists who study the auditory system. The patterns of speech and language development—especially speech perception—have also bestowed valuable lessons to linguists who can marvel at the robustness and flexibility of human language.