Artificial Intelligence to Evaluate Surgical Outcomes of Dynamic Facial Reanimation
Kevin Zuo, MD, MASc1, Tomas J. Saun, MD, MASc1, Christopher Forrest, MD, MSc2 and Gregory Borschel, MD3, (1)University of Toronto, Toronto, ON, Canada, (2)The Hospital for Sick Children, Toronto, ON, Canada, (3)Division of Plastic and Reconstructive Surgery, The Hospital for Sick Children, Toronto, ON, Canada
PURPOSE
Facial reanimation outcomes are measured with patient- or provider-rated scales and cephalometric landmarks. These measures are quantitative but may not objectively assess emotional expression. Novel artificial intelligence and deep neural network algorithms are able to recognize common facial emotional states in humans. The study objectives were to evaluate emotional expression in patients following facial reanimation and to assess the use of a pretrained deep neural network to classify facial emotions.
METHOD
A literature review was performed to obtain previously published pre- and post-operative photographs of patients who underwent facial reanimation surgery. Surgical procedures were categorized as facial nerve innervation, non-facial nerve innervation, and regional muscle transfer. Face photos were analysed using a pretrained deep neural network-based facial analysis algorithm (Microsoft Azure's Face and Emotion Cognitive Services). Facial expressions were evaluated for eight predefined emotional states and assigned classification confidence scores between 0.00 and 1.00. The primary outcome measure was change in emotional classification confidence scores pre- and post-surgery.
RESULTS
This study included 135 patients (83 adults, 25M:58F; 52 children, 26M:26F) with facial paralysis (unilateral n = 120, bilateral n = 15). Comparison of pre- and post-operative photos revealed significant increases in classification confidence scores for happiness (mean 0.60±0.39, p<0.001) and significant decreases for contempt and neutrality (p<0.05) in adults and children. There were no significant changes for anger, disgust, fear, sadness, or surprise. Comparing facial nerve versus non-facial nerve innervated procedures, there were no significant differences in classification confidence scores for post-operative happiness.
CONCLUSIONS
Facial analysis algorithms based on deep neural networks can detect and quantify changes in emotional expression following facial reanimation surgery. These artificial intelligence assessment methods may be a useful adjunct to objectively evaluate patient outcomes.
LEARNING OBJECTIVES
- To evaluate assessment of emotional expression in facial reanimation
- To describe changes in emotional expression following facial reanimation surgery
Back to 2020 Abstracts