Facial Motion Caption Used to Identify Apraxia and Other Speech Disorders

child wearing motion capture technology on head
Facial motion capture – the same technology used to develop realistic computer graphics in video games and movies – has been used to identify differences between children with childhood apraxia of speech and those with other types of speech disorders.

Facial motion capture – the same technology used to develop realistic computer graphics in video games and movies – has been used to identify differences between children with childhood apraxia of speech and those with other types of speech disorders, finds a new study by NYU’s Steinhardt School of Culture, Education, and Human Development.

“In our study, we see evidence of a movement deficit in children with apraxia of speech, but more importantly, aspects of their speech movements look different from children with other speech disorders,” said study author Maria Grigos, associate professor in the Department of Communicative Sciences and Disorders at NYU Steinhardt. The study, coauthored by Aviva Moss and Ying Lu of NYU Steinhardt, is published in the August issue of the Journal of Speech, Language, and Hearing Research.

Childhood apraxia of speech is a complex speech impairment in which children have difficulty planning and making accurate movements to create speech sounds. Children with apraxia of speech often are delayed in developing speech, have atypical speech patterns, and make slow progress in speech therapy.

Movement tracking technology has emerged as a useful tool in studying motor speech disorders, including apraxia. Tiny reflective markers are placed on the face, and using the motion capture technology, researchers can quantify facial movements by measuring how the lips and jaw move. Beyond simply listening to speech sounds, measuring motor deficits with facial movement tracking adds a layer of understanding to measuring speech sounds.

“This research enables us to look at the movement patterns used to produce a word in relation to the way that word is perceived. Including the perceptual component is key because as clinicians, we rely heavily on the judgments we make when listening to children speak. One of our aims was to determine if we could identify differences in how the lips and jaw move even when speech is perceived to be accurate by the listener,” Grigos said.

Grigos and her colleagues sought to understand if by measuring facial movements, children with apraxia of speech can be distinguished from children with other types of speech impairment. The researchers examined the lip and jaw movement of 33 children, ages three to seven, during speech tasks. Three groups were studied: 11 children with childhood apraxia of speech, 11 children with other speech impairments, and 11 children without speech impairments.

The children were asked to repeat one, two, and three syllable words while the motion capture technology tracked jaw, lower lip, and upper lip movements. The researchers looked at metrics including the timing, speed, and variability of the movement, as well as how far the lips and jaw moved during speech. They only analyzed words that they perceived to be pronounced accurately.

Using the movement tracking technology, the researchers were able to pick up subtle differences that the ear couldn’t hear. The most notable finding was that children with childhood apraxia of speech produced lip and jaw movements that varied more than the other two groups of children.

“Variability can be viewed in two ways: it can indicate that there is flexibility to achieve the speech goal, or it might reflect a lack of control,” Grigos said. “We’re still trying to identify the source of such variability and whether speech movement variability would decrease over the course of intervention involving intense practice.’”

The researchers also found that the timing of the movement was longer in both speech impaired groups, meaning that the two groups took longer to produce words than typically developing children.

Interestingly, when the children were asked to repeat three syllable words, the most difficult of the speech tasks, the two groups with speech impairments handled the words differently in terms of movement duration and variability, with more deficits seen in the apraxia group.

“Children with apraxia don’t improve quickly with treatment. Our findings suggest that the motor deficits seen in children with apraxia may contribute to their slow progress in treatment and difficulty generalizing newly acquired speech skills to untrained tasks,” Grigos said.

The study provides evidence that movement variability – as measured by facial motion capture – distinguishes children with childhood apraxia of speech from children with other speech disorders, and children respond differently to linguistic challenges depending on their speech impairment.

The research was supported by a grant from the National Institute on Deafness and Other Communication Disorders (R03DC009079).

About the Steinhardt School of Culture, Education, and Human Development (@nyusteinhardt)
Located in the heart of Greenwich Village, NYU’s Steinhardt School of Culture, Education, and Human Development prepares students for careers in the arts, education, health, media, and psychology. Since its founding in 1890, the Steinhardt School's mission has been to expand human capacity through public service, global collaboration, research, scholarship, and practice. To learn more about NYU Steinhardt, visit steinhardt.nyu.edu.

Press Contact

Rachel Harrison
Rachel Harrison
(212) 998-6797