Prerpints.org logo
Preprint
Article

Dynamic Emotion Recognition and Social Inference Ability in Traumatic Brain Injury: An Eye-Tracking Comparison Study

Altmetrics

Downloads

75

Views

29

Comments

0

A peer-reviewed article of this preprint also exists.

Submitted:

31 August 2023

Posted:

31 August 2023

You are already at the latest version

Alerts
Abstract
Emotion recognition and social inference impairments are well-documented post-traumatic brain injury (TBI) yet the mechanisms underpinning these are not fully understood. We examined dynamic emotion recognition, social inference abilities, and eye fixation patterns between adults with and without TBI. Eighteen individuals with TBI and 18 matched non-TBI participants were recruited and underwent all three components of The Assessment of Social Inference Test (TASIT). The TBI group were less accurate in identifying emotions compared to the non-TBI group. Individuals with TBI also scored lower when distinguishing sincere and sarcastic conversations but scored similarly to those without TBI during lie vignettes. Finally, those with TBI also had difficulty understanding the actor’s intentions, feelings, and beliefs compared to participants without TBI. No group differences were found for eye fixation patterns and there were no associations between fixations and behavioural accuracy scores. This conflicts with previous studies and might be related to an important distinction between static and dynamic stimuli. Visual strategies appeared goal- and stimulus-driven, with attention being distributed to the most diagnostic area of the face for each emotion. These findings suggest that low-level visual deficits may not be modulating emotion recognition and social inference disturbances post-TBI.
Keywords: 
Subject: Social Sciences  -   Psychology

1. Introduction

Traumatic brain injury (TBI) is a global public health concern with approximately 69 million new cases each year [1]. The long-term sequelae of TBI include physical (i.e. headaches), cognitive (i.e. executive function, concentration), affective (i.e. mood disorders), and social cognition (i.e. diminished theory of mind) impairments [2]. Importantly, social cognition impairments are documented across all severities of TBI [3,4], and diminished abilities are associated with poor patient outcomes [5-7] and negatively impact long-term family functioning and wellbeing [8]. Despite the likelihood that social cognition impairments after TBI are underreported [9,10], the evidence base for their prevalence is strong [11-14]. One prominent area of disruption post-TBI is facial emotion recognition [11,14-16], with estimates that up to 39% of individuals with moderate-severe TBI may exhibit this impairment [17]. This is highly problematic as facial expressions are one of the richest tools for social inference and communication [18]. Additionally, individuals with TBI display difficulties in interpreting social cues such as tone of voice and gestures which impede the understanding of other people’s feelings, thoughts, and intentions [14,19,20].
Although emotion recognition impairments are well documented post-TBI, the mechanisms underpinning these changes are not fully understood. Specifically, it is unknown whether they stem from disruption to early visual processing (i.e. aberrant eye fixation patterns) or social cognition, or a combination of both. An association between eye movement dysfunction and poor emotion recognition has been documented for neurological disease and injury [21-27]. Although research on TBI populations is scant, it is unsurprising that eye fixation patterns are disrupted as, anatomically, several frontal brain areas modulating eye movements are susceptible to pathology due to their position within the skull and the trajectory of the brain on impact [28,29]. Furthermore, emotion and social inference recognition are complex processes, requiring several synchronised visual strategies (e.g. saccades, fixations) to the body and face. Complexity is increased as different emotional expressions elicit distinct fixation scan path patterns, for example, angry and sad faces produce earlier and longer fixations to the eye region compared to happy faces where the mouth receives more attention [30-33]. Unfortunately, the emotion recognition field has been hampered by an experimental bias towards behavioural data with a paucity of studies exploring visual strategies of real-time physiological data. Furthermore, there has been a growing discussion regarding the poor ecological validity of frequently implemented static facial expression stimuli and a call for increased use of dynamic stimuli [11,34]. The current study had three aims:
(1)
To explore dynamic emotion recognition and social inference abilities between adults with and without TBI
(2)
To determine if adults with TBI exhibited different fixation patterns compared to adults without TBI in response to dynamic social interactions
(3)
To investigate relationships between fixation duration and fixation counts to AOI and emotion recognition and social inference accuracy scores

2. Materials and Methods

2.1. Participants

The study included 18 TBI participants and 18 matched non-TBI participants (N = 36). Due to a dataset error, there was one less non-TBI participant for the social inference minimal (SI-M) and social inference enhanced (SI-E) components (N = 35). Ethical approval was obtained through Leeds East NHS Research Ethics Committee and Sheffield Hallam University Faculty Research Ethics. TBI participants were recruited through two brain injury rehabilitation services within the UK National Health Service. The non-TBI participants were recruited via stratified opportunity and snowballing sampling and matched to the TBI participants for sex, age, and education. All participants provided informed consent and completed all three components of The Assessment of Social Inference Test (TASIT)[14] whilst undergoing eye tracking.
TBI information was acquired from medical records, including scan data and hospital admission notes. Participant pathology was heterogeneous, including brain haemorrhage, skull fracture, and contusion, which is typical for this patient cohort [22]. Mechanisms of injury included assault, road traffic accidents, falls, and pedestrian collisions. Thirteen participants had frontal lobe pathology, five had pathology outside of frontal cortices to other cortical and/or subcortical regions (see Appendix A). As anticipated, we recruited more males than females which is in line with existing evidence that males are more likely to sustain a brain injury compared to females [23,24]. Years of education did not differ between groups but the group with TBI fell significantly below the non-TBI group for all three components of the IQ test. This is a pattern frequently reported in the brain injury literature [18,19]. The non-TBI group all reported as neurotypical with no history of brain injury or concussion. See Table 1 for the demographic and clinical characteristics of TBI participants and non-TBI groups.

2.1.1. Inclusion/exclusion criteria

TBI inclusion criteria
TBI sustained in adulthood is typically determined by assessment at hospital admission or brain pathology based on imaging scans.
  • Participants were required to be at least one-year post injury to ensure that chronic rather than acute effects of brain injury were measured
  • Aged between 18 and 65 to account for any effects of natural aging
Exclusion criteria for all participants
  • History of psychiatric illness
  • Severe recent drug and alcohol abuse assessed by Michigan Alcohol Screening Test (MAST) [25] and Drug Abuse Screening Test (DAST) [26]
  • Significant depression and anxiety using the Hospital Anxiety and Depression Scale (HADS) [27]
  • Visual deficits using the Visual Object and Space Perception Battery (VOSPB) [28] Warrington & James, 1991), and Cortical Vision Screening Test (CORVIST) [29]
None of the participants had visual defects measured by screening tasks and none met the criteria for severe anxiety or depression. However, the TBI group did have a higher overall HADS score (anxiety M = 8.11, SD = 4.61, depression M = 6.00, SD = 3.31, overall HADS M = 14.11, SD = 6.28); matched non-TBI-group (anxiety M = 4.94, SD = 2.04, depression M = 2.67, SD = 1.94, overall HADS M = 7.61, SD = 3.48), but this pattern is not unusual in post-TBI populations, even if the injury is mild [30,31].

2.1.2. Injury severity

Currently, there are two primary brain injury severity measurements, post-traumatic amnesia (PTA) and loss of consciousness (LOC) based on the Glasgow Coma Scale (GCS) [32]. PTA can be defined as the period between a TBI occurring and the patient regaining full awareness and standard memory function [33]. PTA less than 24 hours is indicative of mild TBI, greater than 24 hours but 7 days or less moderate and PTA longer than seven days indicates severe injury. This study cohort included 16 severe, one moderate, and one mild TBI. The GCS assesses motor, verbal, and eye responses providing a score between 3 and 15. Scores between 3 and 8 indicate severe injury, between 9 and 12 moderate and between 13 and 15 mild injuries. Collaborating referring clinicians determined severity of injury rating based on standard procedures – LOC and PTA.

2.2. Design

A quasi-experimental design was utilised to compare social cognition abilities of participants with and without TBI. A Tobii T120 Eye Tracker [34] recorded participants eye movement metrics including (1) time to first fixation, (2) first fixation duration, (3) total fixation duration, and (4) total fixation count. Three areas of interest (AOI) were chosen for the study, the eyes, nose, and mouth of the actors in the TASIT task.

2.2.1. Stimuli and procedure

The TASIT [14] comprises video vignettes of professional actors engaged in social interactions. The actors were employed to use a ‘method style’ of acting where they were required to demonstrate an emotion relevant to the situation. This method provided a realistic, spontaneous, and natural test context. The TASIT includes three sub-component measures: the emotion evaluation test (EET), the social inference-minimal test (SI-M), and the social inference-enriched (SI-E) test. All three sub-components of the TASIT were administered during the study. In brief, the EET assessed the understanding of communication components such as facial expressions, tone of voice, and gestures. The SI-M provided a measure of the participant’s ability to read paralinguistic features, facial expressions, and tone of voice in either sincere or sarcastic situations. Finally, the SI-E assessed the participant’s ability to use contextual cues (i.e. tone of voice and facial expressions) to determine if everyday conversations were deceptive or sarcastic. The primary behavioural score for each component was accuracy scores (i.e. identifying the correct emotion or expression). A more detailed description of the TASIT sub-components can be found in Appendix B. Only correct responses were included in all analyses. There was one practice trial at the beginning of each sub-component and participants verbally indicated their response which the researcher documented.
The TASIT is a superior social inference task as it includes naturally occurring visual cues during social situations (demeanour of the speaker and reaction of the listener) which many other experimental inference tasks do not. For example, while the Movie for the Assessment of Social Cognition (MASC) [36] includes dynamic social cues, it does not explore the processing of sincere and sarcastic exchanges in as much depth as the TASIT. It has been reported that the TASIT has good reliability and high test-retest reliability levels (r = 0.74-0.88) as well as good construct validity, low practice effects, and high ecological validity [14,37]. The TASIT is gaining traction in brain injury research [38-40].

2.2.2. Apparatus Eye-Tracker

A Tobii T120 eye-tracker and Tobii Studio Eye Tracking Analysis software [41] was used to collect and analyse eye-tracking data. The eye-tracker has a 17-inch thin-film-transistor screen (1280 x 1024 pixels) with an embedded infrared camera. The technology does not have a head frame ensuring participants are free to exhibit natural head and eye movements. Eye tracking data was sampled at 120 Hz with an accuracy of 0.5°. The default Tobii fixation filter algorithm was used for all three components of the TASIT (fixation threshold at 35 pixels for velocity and 35 pixels for distance per sample). An eye movement velocity above the 0.5° per second threshold was classified as a saccade sample and below it was classed as a fixation. The standard five-point calibration of each eye was undertaken for each component of the TASIT for each participant. Participants were seated approximately 62 cm away from the screen in a stationary chair.

2.2.3. Eye tracking metrics

Eye tracking metrics were calculated by the Tobii software [41]. Those analysed were ‘total fixation duration (seconds)’ which was the sum of all fixations within an active area of interest (AOI) e.g. how long the participant spent looking at the eyes, the nose and the mouth; and ‘fixation count (count)’ which indicates the number of times the participant fixates on an active AOI. Fixation counts are amalgamated throughout the experiment as participants redirect their fixations back and forth between AOIs.

3. Results

Parametric assumptions were checked for raw data. The behavioural data was relatively normally distributed but the eye-tracking data had moderate violations of normality regarding skewness and kurtosis, homogeneity of variance, and outliers. As violations were moderate and not severe, and because the study had approximately equal sample sizes, the effects of violation were thought minimal [42]. Furthermore, to guard against the correction of accurate but ‘non-normal’ data and to avoid the challenges associated with transforming skewed data parametric analyses were conducted on untransformed data [43]. Although the present study recruited similar participant numbers to previous dynamic social inference research in different populations [44,45], the sample size is still modest and, as such, the α for main inferential analyses was set at .05 with and adjusted α = .01 for all post-hoc follow-up analyses (α = .01).

3.1. Emotion Evaluation Test

For the EET, a 2 (TBI vs. no TBI group)*(7 - emotion) ANOVA was conducted to explore potential differences in emotion perception. A further 2*(7) *(3 – eyes, nose, mouth AOI) repeated measures ANOVA was conducted to investigate potential differences in fixation duration and count between the TBI and non-TBI groups.

3.1.1. Behavioural data

As demonstrated in Table 2, the descriptive statistics indicated that the group with TBI had lower overall accuracy scores on the EET compared to the group without TBI.
The analysis showed a significant main effect of emotion, (F (6, 204) = 6.85, p ≤ .001, ηp2 = 0.17), but the interaction between emotion and group was non-significant, (F (6, 204) = 1.08, p = .375, ηp2 = 0.03). The test of between-subjects effects was also significant, (F (1, 34) = 20.28, p ≤ .001, ηp2 = 0.37) with the group means indicating that the TBI participants had significantly fewer correct responses across the EET compared to the non-TBI group. When comparing the descriptive statistics, it appeared that the TBI group scored lower on negative emotions compared to positive, particularly interpreting sad, anxious, and revolted displays of emotion, compared to the non-TBI group, although these differences were not statistically significant.
Eye-tracking data
Fixation duration across the EET in seconds
The group with TBI had shorter fixation durations to the eyes and nose compared to participants without TBI, while both groups displayed similar fixation durations to the mouth (Table 3).
The analysis showed a significant main effect of emotion, (F (1.83, 62.17) = 38.99, p ≤ .001, ηp2 = 0.53), and a significant interaction between emotion and AOI, (F (4.19, 142.35) = 12.29, p ≤ .001, ηp2 = 0.27). The main effect of AOI, (F (1.70, 57.87) = 0.09, p = .884, ηp2 = 0.003), and the interactions between emotion and group, (F (1.83, 62.17) = 1.67, p = .198, ηp2 = 0.05), AOI and group, (F (1.70, 57.87) = 2.17, p = .130, ηp2 = 0.06), and emotion, AOI and group, (F (4.19, 142.35) = 1.26, p = .289, ηp2 = 0.04) were all non-significant. The tests of between-subjects effects was also not significant, (F (1, 34) = 2.84, p = .101, ηp2 = 0.08).
The significant interaction between emotion and AOI was explored through three one-way ANOVAs, with emotion as the independent variable (IV) and fixation duration as the dependent variable (DV). There was a significant effect of eyes, (F (1.40, 48.90) = 36.35, p ≤ .001, ηp2 = 0.51), nose, (F (2.53, 88.41) = 5.99, p = .002, ηp2 = 0.15) and mouth, (F (1.40, 48.90) = 36.35, p ≤ .001, ηp2 = 0.51) on fixation duration in response to different facial expressions, with follow-up paired-samples t tests in Appendix C. In brief, the post-hoc tests indicated that participants made longer fixations to the eyes when viewing sad faces compared to all other emotions. Participants also spent significantly longer looking at the eyes of neutral faces compared to the other emotions (bar sad). The group elicited longer fixations to the nose when viewing angry, sad, and neutral faces and more fixations to the mouth when looking at happy and sad faces.
Fixation count across the EET
The group affected by TBI had fewer fixations compared to those without TBI across all of the emotions and across most of the AOIs within the emotions (Table 4).
The analysis showed a significant main effect of emotion, (F (3.40, 115.65) = 41.78, p ≤ .001, ηp2 = 0.55) and a significant interaction between emotion and AOI, (F (5.85, 198.88) = 10.77, p ≤ .001, ηp2 = 0.24). The main effect of AOI, (F (1.60, 54.41) = 0.44, p = .602, ηp2 = 0.01) and the interactions between emotion and group, (F (3.40, 115.65) = 1.79, p = .146, ηp2 = 0.05), AOI and group, (F (1.60, 54.41) = 2.26, p = .124, ηp2 = 0.06), and emotion, AOI and group, (F (5.85, 198.88) = 1.88, p = .088, ηp2 = 0.05) were all non-significant. The tests of between-subjects effects was also not significant, (F (1, 34) = 2.96, p = .094, ηp2 = 0.08).
The significant interaction between emotion and AOI was explored through three one-way ANOVAs, with emotion as the IV and fixation count as the DV. There was a significant effect of eyes, (F (1.40, 48.90) = 36.35, p ≤ .001, ηp2 = 0.44), nose, (F (4.17, 146.02) = 5.87, p ≤ .001, ηp2 = 0.14) and mouth, (F (3.94, 138.06) = 30.97, p ≤ .001, ηp2 = 0.47) on fixation count in response to different facial expressions, with follow-up paired-samples t tests presented in Appendix D. In summary, the group made more fixations to the eyes and nose when viewing neutral and sad faces. The mouth was looked at more when participants viewed happy and sad faces.
Correlations
One-tailed Spearman’s Rho correlations were run to explore possible relationships between the behavioural data (accuracy for emotion recognition) and eye-tracking data (fixation duration and fixation count). When the groups were combined and the α-level had been adjusted to account for multiple comparisons (α= .01) there were no significant correlations (all, r's ≤ 0.29, all p’s ≥ .043). When the groups were separated there were also no significant correlations for the group with TBI (all r’s ≤ -0.23, p ≥ .352) or non-TBI group (all r’s ≤ 0.58, p ≥ .013).
Social Inference-Minimal
A 2 (TBI vs. no TBI group)*(3 – simple sarcasm, paradoxical sarcasm, sincere) ANOVA was conducted to investigate group differences during the understanding of conversational meanings. A further 2*(4 – intentions, meaning, beliefs, feelings) ANOVA was conducted to investigate the difference between the groups in understanding different facets of social interactions. With eye-tracking data, a 2*(3) *(3) mixed-design ANOVA was conducted to investigate potential differences in fixation duration and count between the TBI and control groups.
Behavioural data
Accuracy for conversation style
As demonstrated in Table 5, the descriptive statistics indicated that the TBI group had lower overall accuracy scores across the conversational styles (simple sarcasm, paradoxical sarcasm and sincere) compared to the control group.
There was no significant effect of conversation style, (F (1.59, 52.47) = 1.37, p = .260, ηp2 = 0.04) and no interaction between conversation style and group, (F (1.59, 52.47) = 1.63, p= .203, ηp2 = 0.05). The tests of between-subjects effects was significant, (F (1, 33) = 21.03, p ≤ .001, ηp2 = 0.39) and referring to the estimated marginal means, the group with TBI scored significantly lower across the three conversation styles of the SI-M (M= 15.54, SD= 3.65) compared to the control group (M= 18.55, SD= 1.94).
Accuracy
for comprehension probes
The descriptive statistics for the four different comprehension probes (beliefs, meanings, intentions, feelings) suggested that the group affected by TBI were less accurate at understanding what the actor was trying to do, what they were trying to say, what they were thinking, and what they were feeling (Table 6). See Appendix B for a comprehensive description of the TASIT behavioural metrics.
There was a significant effect of comprehension probe, (F (2.38, 78.36) = 3.19, p = .038, ηp2 = 0.09) but there was no interaction between comprehension probe and group, (F (2.38, 78.36) = 1.37, p= .261, ηp2 = 0.04). To further explore the significant effect of comprehension probe, independent t tests were conducted. When post-hoc correction was applied, the intentions, meaning, beliefs, and feelings probes were significant (p ≤ .001). The tests of between-subjects effects was significant, (F (1, 33) = 17.67, p ≤ .001, ηp2 = 0.35) and referring to the estimated marginal means, the group with TBI scored significantly lower across the four comprehension probes of the SI-M (M= 11.67, SD= 2.29) compared to the control group (M= 13.84, SD= 1.00).
Eye-tracking data
Fixation duration across the SI-M in seconds
The mean scores indicated that participants spent more time fixated on the mouth than the nose and eyes during simple sarcasm and sincere videos (Table 7). The sincere videos appeared to elicit the longest fixations, followed by the sarcastic and then paradoxical videos.
The analysis showed a significant main effect of conversational style, (F (1.34, 44.16) = 23.59, p ≤ .001, ηp2 = 0.42) and AOI, (F (2, 66) = 6.29, p = .003, ηp2 = 0.16). The interactions between conversation style and group, (F (1.34, 44.16) = 0.25, p = .78, ηp2 = 0.01), AOI and group, (F (1.78, 58.82) = 0.39, p =0.657, ηp2 = 0.01), conversation style and AOI, (F (2.39, 78.98) = 2.62, p = .070, ηp2 = 0.07), and conversation style, AOI and group, (F (2.39, 78.98) = 0.15, p =.894, ηp2 = 0.01) were all non-significant. The tests of between-subjects effects was also not significant, (F (1, 33) = 0.52, p = .476, ηp2 = 0.02).
To explore the significant main effect of conversation style, two-tailed paired samples t tests were conducted and the post-hoc correction was applied. Participants generated significantly longer fixation durations during the sarcastic videos compared to paradoxical videos, (t (34) = 5.67, p ≤.001, d = 0.96, CI= 0.61-1.30), sincere videos compared to sarcastic videos, (t (34) = -3.00, p =.005, d = 0.51, CI= -1.82- -0.35), and sincere videos compared to paradoxical videos, (t (34) = -6.50, p ≤.001, d = 1.09, CI= -2.68- -1.41).
Additional two-tailed paired-samples t tests were conducted to investigate the significant main effect of AOI. When the post-hoc correction was applied, participants exhibited significantly longer fixations to the mouth (M =2.30, SD= 1.69) compared to the eyes (M =1.13, SD= 1.07), (t (34) = -3.24, p = .003, d = 0.55, CI= -1.90- -0.44).
Fixation count across the SI-M
From the mean scores presented in Table 8, there appeared to be a similar pattern for both the TBI and control participants, with both groups having higher fixation counts for the sarcastic and sincere compared to paradoxical videos.
The analysis showed a significant main effect of conversational style, (F (1.39, 45.91) = 22.40, p ≤ .001, ηp2 = 0.40). The other main effect of AOI, (F (1.69, 55.77) = 2.96, p = .068, ηp2 = 0.08), and the interactions between conversation style and group, (F (1.39, 45.91) = 0.37, p = .617, ηp2 = 0.01), AOI and group, (F (1.69, 55.77) = 0.38, p = .648, ηp2 = 0.01), conversation style and AOI, (F (2.26, 74.71) = 2.40, p = .091, ηp2 = 0.07), and conversation style, AOI and group, (F (2.26, 74.71) = 0.44, p =.671, ηp2 = 0.01) were all non-significant. The tests of between-subjects effects was also not significant, (F (1, 33) = 0.002, p = .965, ηp2 ≤ 0.001).
To explore the significant main effect of conversation style, two-tailed paired-samples t tests were conducted with post-hoc corrections applied. Participants generated significantly more fixations during the sarcastic videos compared to paradoxical videos, (t (34) = 7.24, p ≤.001, d = 1.22, CI= 1.83-3.26), and to sincere (M = 5.73, SD = 3.41) videos compared to paradoxical videos, (t (34) = -6.30, p ≤.001, d = 1.06, CI= --4.97- -2.54).
Correlations
One-tailed Spearman’s Rho correlations were used to investigate potential relationships between the behavioural data (accuracy for conversation style and comprehension probe) and eye-tracking data (fixation duration and fixation count). When the groups were combined and once the α-level had been adjusted to account for multiple comparisons (α= .01), there were no significant correlations (all, r's ≤ 0.001, all p’s ≥ .110). When the groups were separated, there were still no correlations for the group with TBI (all r’s ≤ 0.51, p ≥ .032) but there were significant correlations for the control group between fixation duration to the eyes and the intentions comprehension probes (r= 0.74, p= .001), fixation count to the eyes and the intentions comprehension probes (r= 0.73, p= .001) and between fixation duration to the eyes and simple sarcasm comprehension probes (r= 0.69, p= .002).
SI-E
A 2 (TBI vs. no TBI group)*(2 – sarcastic and lie) ANOVA was conducted to investigate group differences during the understanding of conversational meanings. A further 2*(4 – intentions, meanings, beliefs and feelings) ANOVA was conducted. Again, this analysis investigated the difference between the groups in understanding different facets of social interactions. A 2*(2) *(3 – eyes, nose and mouth AOI) repeated measures ANOVA was conducted to investigate potential differences in fixation duration and count between the TBI and control groups.
Behavioural data
Accuracy for conversational style
As demonstrated in Table 9, the mean scores indicated that the group affected by TBI had lower overall accuracy scores across the two conversational styles (sarcastic and lie) compared to the control group.
The effect of conversation style was non-significant, (F (1.00, 33.00) = 0.64, p = .428, ηp2 = 0.02) but there was a significant interaction between conversation style and group, (F (1.00, 33.00) = 9.86, p= .004, ηp2 = 0.23). Simple effects analysis revealed that the group with TBI scored significantly lower on the sarcastic vignettes (M = 22.56, SD = 4.77) compared to controls (M = 29.41, SD = 3.37) (p ≤ .001). The tests of between-subjects effects was also significant, (F (1, 33) = 16.31, p ≤ .001, ηp2 = 0.33). Referring to the estimated marginal means, the group affected by TBI had significantly lower accuracy scores across the two conversation styles of the SI-E (M= 23.94, SD= 4.78) compared to the control group (M= 28.58, SD= 3.24).
Accuracy for comprehension probes
The descriptive statistics for the four different comprehension probes (intentions, meanings, beliefs, and feelings) suggested that the group living with TBI were less accurate at understanding what the actor was trying to do, what they were trying to say, what they were thinking, and what they were feeling (Table 10).
There was a significant effect of comprehension probe, (F (2.46, 81.19) = 3.25, p = .034, ηp2 = 0.09) and a significant interaction between comprehension probe and group, (F (2.46, 81.19) = 4.69, p= .008, ηp2 = 0.12). Simple effects analysis revealed that the group with TBI scored significantly lower on the ‘intention’ probes (M = 11.28, SD = 2.49) compared to controls (M = 14.82, SD = 1.63) (p ≤ .001) and the same for the ‘meaning’ probes (TBI M = 11.17, SD = 2.60; control group M = 14.47, SD = 1.70) (p ≤ .001). The tests of between-subjects effects was also significant, (F (1, 33) = 21.41, p ≤ .001, ηp2 = 0.39). Referring to the estimated marginal means, the group affected by TBI scored significantly lower across the four comprehension probes of the SI-E (M= 11.97, SD= 2.34) compared to the control group (M= 14.47, SD= 1.63).
Eye-tracking data
Fixationduration across the SI-E in seconds
Table 11 shows that individuals with TBI had shorter fixation durations to the eyes, nose and mouth compared to controls (Table 43).
The analysis showed a significant main effect of conversational style, (F (1, 33) = 44.18, p ≤ .001, ηp2 = 0.57) with participants exhibiting longer fixations for sarcastic videos compared to lie videos. The main effect of AOI, (F (1.22, 40.16) = 0.92, p = .405, ηp2 = 0.03) and the interactions between conversation style and group, (F (1.00, 33.00) = 0.002, p = .969, ηp2 ≤0.01), AOI and group, (F (1.22, 40.16) = 0.01, p = .991, ηp2 ≤ 0.01), conversation style and AOI, (F (1.61, 53.26) = 2.54, p = .099, ηp2 = 0.07), and conversation style, AOI and group, (F (1.61, 53.26) = 1.42, p =.249, ηp2 = 0.04) were all non-significant. The tests of between-subjects effects was also not significant, (F (1, 33) = 1.13, p = .296, ηp2 = 0.03).
Fixation count across the SI-E
From the descriptive statistics presented in Table 12, the TBI cohort appeared to produce fewer fixation counts to all AOI across the sarcastic and lie conditions compared to control participants. Both the TBI and control participants generated fewer fixation counts during the videos depicting lies compared to videos where the actors were being sarcastic.
The analysis showed a significant main effect of conversational style, (F (1, 33) = 75.77, p ≤ .001, ηp2 = 0.70) with participants displaying more fixations during the sarcastic vignettes compared to vignettes depicting lies. The main effect of AOI, (F (1.36, 44.95) = 0.28, p = .673, ηp2 = 0.01) and the interactions between conversation style and group, (F (1, 33) = 0.08, p = .780, ηp2= 0.002), AOI and group, (F (1.36, 44.95) = 0.20, p = .732, ηp2 = 0.01), conversation style and AOI, (F (1.58, 52.23) = 1.29, p = .279, ηp2 = 0.04), and conversation style, AOI and group, (F (1.58, 52.23) = 1.49, p =.236, ηp2 = 0.04) were all non-significant. The tests of between-subjects effects was also not significant, (F (1, 33) = 0.76, p = .390, ηp2 = 0.02).
Correlations
One-tailed Spearman’s Rho correlations explored the relationships between the behavioural data (accuracy for conversation style and comprehension probe) and eye-tracking data (fixation duration and fixation count). When the groups were combined and once the α-level had been adjusted to account for multiple comparisons (α= .01), there were no significant correlations (r ≤ 0.39, p ≥ .022). When the groups were separated, there were still no significant correlations for the group with TBI (all, r's ≤ 0.56, all p’s ≥ .016) or controls (all, r's ≤ 0.56, all p’s ≥ .019). Although the analysis did not reach the .01 set for post-hoc significance, the results are nearing significance and are significant at the typically accepted α level of .05.

4. Discussion

We compared the emotion recognition and social inference abilities of participants with TBI to those without TBI. As expected, the TBI group were significantly less accurate in identifying emotions on the EET than those without TBI. In line with our previous work[20], there was a distinct deficit for negative compared to positive emotions, particularly interpreting sad, anxious, and revolted displays of emotion. Individuals with TBI also scored significantly lower across all three of the conversation styles of the SI-M, in comparison to the control group, with simple-sarcasm vignettes scoring lowest followed by sincere, and then paradoxical-sarcastic exchanges. This effect was mirrored in the SI-E where the group with TBI were significantly poorer at recognising sarcastic conversations compared to those without TBI. Interestingly, there was no significant difference between the two groups during the lie vignettes. These findings are consistent with McDonald and colleagues who reported that participants with TBI displayed impairments in deciphering sarcastic exchanges but scored in a typical range when interpreting lies [14]. These results imply that while TBI diminishes the ability to comprehend sarcasm, the aptitude to recognise lies remains intact. This model has previously been proposed by McDonald [46] and is based on the conceptual differences between lies and sarcasm. For example, it has been proposed that understanding sarcasm requires more cognitive effort compared to understanding sincere and untruthful exchanges, mainly because understanding sarcasm necessitates first-order theory of mind (ToM; understanding what someone is thinking or feeling) and second-order ToM (predicting what one person thinks or feels about what another person is thinking or feeling) [47]. Analysis of the comprehension probes on both the SI-M and SI-E also demonstrated that the group with TBI had difficulty understanding the actor’s intentions, feelings, beliefs, and the meaning of the conversation compared to the group without TBI. In summary, these results align with prior research that highlights disruptions in emotion recognition and social inference following a traumatic brain injury (TBI) [11,14-16,40,48]. However, our study contributes to the field by pioneering an investigation into the potential involvement of visual impairments in this issue, utilising ecologically valid assessments.
There were no significant differences between the groups in terms of eye fixation patterns and no associations between fixation patterns and behavioural accuracy scores. This differs from previous studies that have reported TBI groups exhibiting abnormal fixation patterns compared to non-TBI groups [24-26]. Interestingly, these studies utilised static stimuli which have been criticised for potential low ecological validity, for instance, lack of motion, depth cues, and contextual cues [23,27,49-52]. Evidence has demonstrated that static and dynamic stimuli activate different brain networks [53,54] as well as different levels of attention. For example, Kujawa and colleagues reported that individuals with impaired consciousness resulting from brain injury exhibited higher visual attention to dynamic compared to static stimuli [55]. Others have also shown that eye fixation patterns differ in normative populations in response to dynamic and static stimuli [56,57]. It appears that when more real-world dynamic social stimuli are implemented, there are no group dissimilarities in terms of eye fixation patterns between those with and without TBI, a finding we replicated in a previous study [23]. These results might suggest that the challenges in everyday emotion recognition and social inference following a TBI aren't necessarily linked to visual alterations. Nevertheless, in the present study, we did observe distinctions between our TBI and non-TBI groups at a descriptive level. Additionally, considering previous reports on abnormal eye fixation patterns post-TBI in response to static stimuli, it may be rash to entirely dismiss the possibility that visual changes might exert some influence on social cognition. It seems that there is a multifaceted relationship between vision and social cognition after TBI, necessitating further investigation, potentially with larger sample sizes and increased statistical power to detect subtle group differences. Currently, it remains unclear what is driving these social impairments but, speculatively, they may be attributed to a combination of low-level visual functions and higher-order functions such as working memory load [48], education and/or executive functioning [40].
There was a significant interaction between AOI and emotion during the EET with visual strategies appearing to reflect attention to the most diagnostic area of the face for each emotion (e.g. mouth for identifying happiness and nose for recognising disgusted faces). These findings indicate a goal- and stimulus-driven influence on eye gaze patterns, as distinct patterns of fixation count and duration were observed for both emotional and neutral faces and for conversational styles. This finding is in line with previous research [31-33]. During the SI-M, all participants generated longer fixation durations and counts during the sarcastic videos compared to paradoxical and sincere compared to paradoxical videos. For the group without TBI, there was a correlation between fixation duration to the eyes and intentions, fixation count to the eyes and intentions, and between fixation duration to the eyes and simple sarcasm. Recognising sincere, sarcastic, and deceptive communication requires the integration of verbal, non-verbal, and paralinguistic cues and is paramount to everyday interactions. Despite this, previous research exploring this complex area has heavily relied on written and static stimuli. Our findings conflict with existing research which reported that individuals had longer fixation durations to nonliteral compared with literal interactions [58]. Gaining a deeper understanding of typical literal and non-literal language perception can inform existing theoretical frameworks as well as aid clinical rehabilitation areas, such as TBI, where these skills are frequently diminished [59].
The primary limitation of the present research was the relatively small sample size. Future work might include larger samples, with more female participants, or more conservative alpha levels during significance testing. Demographically, TBI and non-TBI groups were matched for age, sex, and years of education, but those with TBI fell below the non-TBI group on IQ measures. This is typical for TBI populations [60] but if social impairments post-TBI are related to working memory deficits then future studies could control for this.

5. Conclusions

Our results demonstrate two things. First, eye fixation patterns for dynamic social stimuli do not statistically differ between individuals with and without TBI but do differ on a descriptive level, and second, dynamic emotion recognition and social inference abilities are impaired after TBI. These combined findings seem to indicate that impairments are not related to abhorrent eye fixation patterns and must be driven by a disruption outside of the low-level visual system. Our research also highlights the important distinction between static and dynamic stimuli when trying to map structure to function in eye-tracking research and we urge other research teams to implement dynamic stimuli in the future.

Author Contributions

Conceptualization, LG, LAB, and JR.; methodology, LG, LAB, and JR; formal analysis, LG, LAB, and JR; investigation, LG; writing—original draft preparation, LG and LAB.; writing—review and editing, LG, LAB, JR, NM, AA.; supervision, LAB, JR; project administration, LG, LAB, JR; funding acquisition, LAB; patient selection, referral and risk screening, NM, AA. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by PhD bursary.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by (1) Leeds East NHS Research Ethics Committee and (2) Sheffield Hallam University Faculty Research Ethics.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

Thank you to all the participants who took part in this study.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Demographic and clinical characteristics of TBI participants (n=18).
Preprints 83825 i001

Appendix B

Description of the three subcomponents of the TASIT
The Emotion Evaluation Test (EET)
The EET subtest of the TASIT assesses social communication components that include facial expressions, tone of voice, and gestures. As part of this subtest, participants are required to identify the emotion expressed by the actor. Participants are provided seven emotions to choose from; anger, anxiety, surprise, sadness, happy, revulsion, and neutral. These emotions are then grouped into positive (surprise & happy) and negative (anger, anxiety, sadness & revulsion) emotions for analysis. The EET consists of 28 short videos (15-60 seconds) displaying actors interacting with each other in everyday situations. There are four videos for each of the seven emotions which are presented in a quasi-randomised fashion. For some videos, there is one actor, either on the telephone or directly talking to the camera. For the remaining videos, two actors are interacting with each other.
The Social Inference-Minimal (SI-M)
The SI-M assesses the ability to read paralinguistic features, facial expressions, and tone of voice in either sincere or sarcastic situations. Participants are required to pay close attention to the tone of voice, facial expressions, gestures, and body language to correctly identify whether the speech is sincere or sarcastic. There are 15 short videos (20-60 seconds) in the SI-M displaying conversational exchanges between actors. Five of the videos have sincere exchanges, another five have sarcastic exchanges, and the remaining five videos have paradoxical scripts, that is, the scripts are nonsensical unless it is understood that one actor is being sarcastic.
After each video, the participant is required to answer four questions about the target actor. The first question indexes what the speaker was trying to ‘do’ (their intentions, for example, to insult or to reassure), what they were trying to say (what they meant), what they were thinking (what the participant thinks the target actor knows), and lastly, what the target actor was feeling in a given context.
The Social Inference-Enriched test (SI-E)
The SI-E part of the TASIT included 16 short (15-60 seconds) videos of everyday conversations. The SI-E assesses the participants’ ability to use contextual cues (tone of voice and facial expressions) to determine if the conversation in the video is deceptive or if the meaning is opposite to what the actor is saying (i.e. the actor is being sarcastic).
The sarcasm and deception videos have identical scripts but during the sarcastic videos, actors do not try to obscure the truth but instead seek to emphasise it. The deception exchanges include white lies and sympathetic lies. White lies include situations such as a mother lying to a father about their child finishing dinner, with sympathetic lies including situations where a friend tells another friend that they do not look fat in a new outfit. Again, participants must use paralinguistic cues to make the correct answer. Furthermore, participants are provided with additional, enriched contextual cues during this subtest. Half of the video clips have camera edits to help participants establish the correct category of the video clip, for example, in the deceptive mother scenario, participants are provided with a camera shot of the child’s unfinished dinner.
The additional camera shots providing supplementary information can only be seen by the main actor during the deception videos (mother); the other actor (father) cannot see the extra visual cue. During the sarcastic video clips, both actors can see the visual cues highlighting the correct subtest category. The remaining halves of the videos use a prologue or an epilogue scene where the main actor discusses their thoughts with a third actor. For example, during the new outfit scenario, Ruth tells Gary that she does not think he has put on weight. Nevertheless, a prologue scene features the sympathetic friend (Ruth) telling another friend (Keith) that she does think Gary has put on weight. These additional cues should aid participants in understanding the video. The SI-E has a similar answer format to the SI-M in that it asks the participant’s four questions about what the actor was doing, saying, thinking, and feeling.

Appendix C.

Post-hoc tests for the TASIT EET (fixation duration).
(1)
Post-hoc paired samples t tests exploring the effect of emotion on fixation duration to the eyes following the significant ANOVA interaction between AOI and emotion.
Preprints 83825 i002
(2)
Post-hoc paired samples t tests exploring the effect of emotion on fixation duration to the nose following the significant one-way ANOVA interaction between AOI and emotion.
Preprints 83825 i003
(3)
Post-hoc paired samples t tests exploring the effect of emotion on fixation duration to the mouth following the significant one-way ANOVA interaction between AOI and emotion.
Preprints 83825 i004

Appendix D.

Post-hoc tests for the TASIT EET (fixation count)
(1)
Post-hoc paired samples t tests exploring the effect of emotion on fixation count to the eyes following the significant one-way ANOVA interaction between AOI and emotion.
Preprints 83825 i005
(2)
Post-hoc paired samples t tests exploring the effect of emotion on fixation count to the nose following the significant one-way ANOVA interaction between AOI and emotion.
Preprints 83825 i006
(3)
Post-hoc paired samples t tests exploring the effect of emotion on fixation count to the mouth following the significant one-way ANOVA interaction between AOI and emotion.
Preprints 83825 i007

References

  1. Dewan, M.C.; Rattani, A.; Gupta, S.; Baticulon, R.E.; Hung, Y.C.; Punchak, M.; Agrawal, A.; Adeleye, A.O.; Shrime, M.G.; Rubiano, A.M.; et al. Estimating the global incidence of traumatic brain injury. J Neurosurg 2018, 1–18. [Google Scholar] [CrossRef] [PubMed]
  2. Pavlovic, D.; Pekic, S.; Stojanovic, M.; Popovic, V. Traumatic brain injury: neuropathological, neurocognitive and neurobehavioral sequelae. Pituitary 2019, 22, 270–282. [Google Scholar] [CrossRef] [PubMed]
  3. Theadom, A.; McDonald, S.; Starkey, N.; Barker-Collo, S.; Jones, K.M.; Ameratunga, S.; Wilson, E.; Feigin, V.L. Social cognition four years after mild-TBI: An age-matched prospective longitudinal cohort study. Neuropsychology 2019, 33, 560–567. [Google Scholar] [CrossRef] [PubMed]
  4. Venkatesan, U.M.; Lancaster, K.; Lengenfelder, J.; Genova, H.M. Independent contributions of social cognition and depression to functional status after moderate or severe traumatic brain injury. Neuropsychol Rehabil 2021, 31, 954–970. [Google Scholar] [CrossRef]
  5. Wearne, T.; Kelly, M.; McDonald, S. Disorders of social cognition in adults with acquired brain injury. In Clinical Disorders of Social Cognition; Routledge: 2021; pp. 145–177.
  6. Milders, M. Relationship between social cognition and social behaviour following traumatic brain injury. Brain Inj 2019, 33, 62–68. [Google Scholar] [CrossRef]
  7. Rigon, A.; Turkstra, L.S.; Mutlu, B.; Duff, M.C. Facial-affect recognition deficit as a predictor of different aspects of social-communication impairment in traumatic brain injury. Neuropsychology 2018, 32, 476–483. [Google Scholar] [CrossRef]
  8. Grayson, L.; Brady, M.C.; Togher, L.; Ali, M. The impact of cognitive-communication difficulties following traumatic brain injury on the family; a qualitative, focus group study. Brain Inj 2021, 35, 15–25. [Google Scholar] [CrossRef]
  9. Maggio, M.G.; Maresca, G.; Stagnitti, M.C.; Anchesi, S.; Casella, C.; Pajno, V.; De Luca, R.; Manuli, A.; Calabrò, R.S. Social cognition in patients with acquired brain lesions: An overview on an under-reported problem. Applied Neuropsychology: Adult 2022, 29, 419–431. [Google Scholar] [CrossRef]
  10. Kelly, M.; McDonald, S.; Frith, M.H.J. A Survey of Clinicians Working in Brain Injury Rehabilitation: Are Social Cognition Impairments on the Radar? Journal of Head Trauma Rehabilitation 2017, 32, E55–E65. [Google Scholar] [CrossRef]
  11. Murphy, J.M.; Bennett, J.M.; de la Piedad Garcia, X.; Willis, M.L. Emotion Recognition and Traumatic Brain Injury: a Systematic Review and Meta-Analysis. Neuropsychology Review 2022, 32, 520–536. [Google Scholar] [CrossRef]
  12. Lin, X.; Zhang, X.; Liu, Q.; Zhao, P.; Zhang, H.; Wang, H.; Yi, Z. Theory of mind in adults with traumatic brain injury: A meta-analysis. Neuroscience & Biobehavioral Reviews 2021, 121, 106–118. [Google Scholar] [CrossRef]
  13. McDonald, S. Impairments in Social Cognition Following Severe Traumatic Brain Injury. Journal of the International Neuropsychological Society 2013, 19, 231–246. [Google Scholar] [CrossRef] [PubMed]
  14. McDonald, S.; Flanagan, S.; Rollins, J.; Kinch, J. TASIT: A new clinical tool for assessing social perception after traumatic brain injury. J Head Trauma Rehabil 2003, 18, 219–238. [Google Scholar] [CrossRef] [PubMed]
  15. Rosenberg, H.; McDonald, S.; Dethier, M.; Kessels, R.P.; Westbrook, R.F. Facial emotion recognition deficits following moderate-severe Traumatic Brain Injury (TBI): re-examining the valence effect and the role of emotion intensity. J Int Neuropsychol Soc 2014, 20, 994–1003. [Google Scholar] [CrossRef]
  16. Babbage, D.R.; Yim, J.; Zupan, B.; Neumann, D.; Tomita, M.R.; Willer, B. Meta-analysis of facial affect recognition difficulties after traumatic brain injury. Neuropsychology 2011, 25, 277–285. [Google Scholar] [CrossRef]
  17. Biszak, A.M.; Babbage, D.R. Facial affect recognition difficulties in traumatic brain injury rehabilitation services. Brain Inj 2014, 28, 97–104. [Google Scholar] [CrossRef]
  18. Jack, R.E.; Schyns, P.G. The Human Face as a Dynamic Tool for Social Communication. Curr Biol 2015, 25, R621–R634. [Google Scholar] [CrossRef]
  19. Turkstra, L.S. Conversation-based assessment of social cognition in adults with traumatic brain injury. Brain Injury 2008, 22, 397–409. [Google Scholar] [CrossRef]
  20. Greene, L.; Barker, L.A.; Reidy, J.; Morton, N.; Atherton, A. Emotion recognition and eye tracking of static and dynamic facial affect: A comparison of individuals with and without traumatic brain injury. J Clin Exp Neuropsychol 2022, 44, 461–477. [Google Scholar] [CrossRef]
  21. Olivetti Belardinelli, M.; Hünefeldt, T.; Meloni, R.; Squitieri, F.; Maffi, S.; Migliore, S. Abnormal visual scanning and impaired mental state recognition in pre-manifest Huntington disease. Experimental Brain Research 2021, 239, 141–150. [Google Scholar] [CrossRef]
  22. Black, M.H.; Chen, N.T.M.; Lipp, O.V.; Bölte, S.; Girdler, S. Complex facial emotion recognition and atypical gaze patterns in autistic adults. Autism 2019, 24, 258–262. [Google Scholar] [CrossRef]
  23. Greene, L.; Barker, L.A.; Reidy, J.; Morton, N.; Atherton, A. Emotion recognition and eye tracking of static and dynamic facial affect: A comparison of individuals with and without traumatic brain injury. Journal of Clinical and Experimental Neuropsychology 2022, 44, 461–477. [Google Scholar] [CrossRef]
  24. Douglas, J.; Vassallo, S.; White, E. Interpreting facial expression after traumatic brain injury: The role of visual scanning. In Proceedings of the International Brain Injury Association’s Eighth World Congress on Brain Injury; 2010. [Google Scholar]
  25. Vassallo, S.; Douglas, J.; White, E. Visual scanning in the recognition of facial affect in traumatic brain injury. i-Perception 2011, 2, 250–250. [Google Scholar] [CrossRef]
  26. Vassallo, S.; White, E.; Douglas, J. Visual Scanning to Emotional Facial Expressions is Impaired After Severe Traumatic Brain Injury. Investigative Ophthalmology & Visual Science 2010, 51, 2547–2547. [Google Scholar]
  27. Oatley, A.E.A.; Torsein, A.; Sadeghi, M.; Green, R.E.A. Reading Facial Emotions After Traumatic Brain Injury (TBI): Implications for Social Functioning and Treatment Development. Archives of Physical Medicine and Rehabilitation 2014, 95, e48. [Google Scholar] [CrossRef]
  28. Barker, L.A.; Andrade, J.; Morton, N.; Romanowski, C.A.J.; Bowles, D.P. Investigating the ‘latent’ deficit hypothesis: Age at time of head injury, implicit and executive functions and behavioral insight. Neuropsychologia 2010, 48, 2550–2563. [Google Scholar] [CrossRef]
  29. Barker, L.A.; Morton, N. Editorial: Executive Function(s): Conductor, Orchestra or Symphony? Towards a Trans-Disciplinary Unification of Theory and Practice Across Development, in Normal and Atypical Groups. Frontiers in Behavioral Neuroscience 2018, 12. [Google Scholar] [CrossRef]
  30. Calbi, M.; Langiulli, N.; Siri, F.; Umiltà, M.A.; Gallese, V. Visual exploration of emotional body language: a behavioural and eye-tracking study. Psychological Research 2021, 85, 2326–2339. [Google Scholar] [CrossRef]
  31. Calvo, M.G.; Fernández-Martín, A.; Gutiérrez-García, A.; Lundqvist, D. Selective eye fixations on diagnostic face regions of dynamic emotional expressions: KDEF-dyn database. Scientific reports 2018, 8, 1–10. [Google Scholar] [CrossRef]
  32. Pérez-Moreno, E.; Romero-Ferreiro, V.; García-Gutiérrez, A. Where to look when looking at faces: Visual scanning is determined by gender, expression and tasks demands. Psicológica 2016, 37, 127–150. [Google Scholar]
  33. Schurgin, M.W.; Nelson, J.; Iida, S.; Ohira, H.; Chiao, J.Y.; Franconeri, S.L. Eye movements during emotion recognition in faces. J Vis 2014, 14, 14. [Google Scholar] [CrossRef]
  34. Knox, L.; Douglas, J. Long-term ability to interpret facial expression after traumatic brain injury and its relation to social integration. Brain and Cognition 2009, 69, 442–449. [Google Scholar] [CrossRef]
  35. Wechsler, D. Wechsler abbreviated scale of intelligence. 1999.
  36. Dziobek, I.; Fleck, S.; Kalbe, E.; Rogers, K.; Hassenstab, J.; Brand, M.; Kessler, J.; Woike, J.K.; Wolf, O.T.; Convit, A. Introducing MASC: A Movie for the Assessment of Social Cognition. Journal of Autism and Developmental Disorders 2006, 36, 623–636. [Google Scholar] [CrossRef]
  37. McDonald, S. New Frontiers in Neuropsychological Assessment: Assessing Social Perception Using a Standardised Instrument, The Awareness of Social Inference Test. Australian Psychologist 2012, 47, 39–48. [Google Scholar] [CrossRef]
  38. Brent, J.; Neumann, D.; Hammond, F. Exploring Changes in Social Inferencing and Negative Attributions Following an Intervention for Individuals with Brain Injury. Archives of Physical Medicine and Rehabilitation 2022, 103, e16. [Google Scholar] [CrossRef]
  39. Naz, S.I.; Neumann, D.; Mueid, R.; Christopher, L. Using Machine Learning Classification to Predict Social Inferencing Performance from Eye-tracking Data in Participants with and without Brain Injury. Archives of Physical Medicine and Rehabilitation 2021, 102, e27. [Google Scholar] [CrossRef]
  40. Neumann, D.; Mayfield, R.; Sander, A.M.; Jang, J.H.; Bhamidipalli, S.S.; Hammond, F.M. Examination of Social Inferencing Skills in Men and Women After Traumatic Brain Injury. Archives of Physical Medicine and Rehabilitation 2022, 103, 937–943. [Google Scholar] [CrossRef]
  41. Tobii Technologies Stockholm, Sweden.
  42. Finch, H. Comparison of the performance of nonparametric and parametric MANOVA test statistics when assumptions are violated. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences 2005, 1, 27. [Google Scholar] [CrossRef]
  43. Feng, C.; Wang, H.; Lu, N.; Chen, T.; He, H.; Lu, Y.; Tu, X.M. Log-transformation and its implications for data analysis. Shanghai Arch Psychiatry 2014, 26, 105–109. [Google Scholar] [CrossRef]
  44. Bird, G.; Press, C.; Richardson, D.C. The role of alexithymia in reduced eye-fixation in Autism Spectrum Conditions. J Autism Dev Disord 2011, 41, 1556–1564. [Google Scholar] [CrossRef]
  45. Klin, A.; Jones, W.; Schultz, R.; Volkmar, F.; Cohen, D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Arch Gen Psychiatry 2002, 59, 809–816. [Google Scholar] [CrossRef]
  46. McDonald, S. Neuropsychological Studies of Sarcasm. Metaphor and Symbol 2000, 15, 85–98. [Google Scholar] [CrossRef]
  47. Deliens, G.; Antoniou, K.; Clin, E.; Kissine, M. Perspective-taking and frugal strategies: Evidence from sarcasm detection. Journal of Pragmatics 2017, 119, 33–45. [Google Scholar] [CrossRef]
  48. Turkstra, L.S.; Norman, R.S.; Mutlu, B.; Duff, M.C. Impaired theory of mind in adults with traumatic brain injury: A replication and extension of findings. Neuropsychologia 2018, 111, 117–122. [Google Scholar] [CrossRef]
  49. Alves, N.T. Recognition of static and dynamic facial expressions: a study review. Estudos de Psicologia (Natal) 2013, 18, 125–130. [Google Scholar] [CrossRef]
  50. Barrett, L.F.; Mesquita, B.; Gendron, M. Context in Emotion Perception. Current Directions in Psychological Science 2011, 20, 286–290. [Google Scholar] [CrossRef]
  51. Ibañez, A.; Manes, F. Contextual social cognition and the behavioral variant of frontotemporal dementia. Neurology 2012, 78, 1354–1362. [Google Scholar] [CrossRef]
  52. McCamy, M.B.; Otero-Millan, J.; Di Stasi, L.L.; Macknik, S.L.; Martinez-Conde, S. Highly Informative Natural Scene Regions Increase Microsaccade Production during Visual Scanning. The Journal of Neuroscience 2014, 34, 2956. [Google Scholar] [CrossRef]
  53. Kilts, C.D.; Egan, G.; Gideon, D.A.; Ely, T.D.; Hoffman, J.M. Dissociable Neural Pathways Are Involved in the Recognition of Emotion in Static and Dynamic Facial Expressions. NeuroImage 2003, 18, 156–168. [Google Scholar] [CrossRef]
  54. Trautmann-Lengsfeld, S.A.; Domínguez-Borràs, J.; Escera, C.; Herrmann, M.; Fehr, T. The perception of dynamic and static facial expressions of happiness and disgust investigated by ERPs and fMRI constrained source analysis. PLoS One 2013, 8, e66997. [Google Scholar] [CrossRef]
  55. Kujawa, K.; Żurek, A.; Gorączko, A.; Olejniczak, R.; Zurek, G. Monitoring Eye Movements Depending on the Type of Visual Stimulus in Patients with Impaired Consciousness Due to Brain Damage. Int J Environ Res Public Health 2022, 19. [Google Scholar] [CrossRef]
  56. Blais, C.; Fiset, D.; Roy, C.; Saumure Régimbald, C.; Gosselin, F. Eye fixation patterns for categorizing static and dynamic facial expressions. Emotion 2017, 17, 1107–1119. [Google Scholar] [CrossRef]
  57. Stoesz, B.M.; Jakobson, L.S. A sex difference in interference between identity and expression judgments with static but not dynamic faces. J Vis 2013, 13, 26. [Google Scholar] [CrossRef]
  58. Rothermich, K.; Schoen Simmons, E.; Rao Makarla, P.; Benson, L.; Plyler, E.; Kim, H.; Henssel Joergensen, G. Tracking nonliteral language processing using audiovisual scenarios. Can J Exp Psychol 2021, 75, 211–220. [Google Scholar] [CrossRef]
  59. Arcara, G.; Tonini, E.; Muriago, G.; Mondin, E.; Sgarabottolo, E.; Bertagnoni, G.; Semenza, C.; Bambini, V. Pragmatics and figurative language in individuals with traumatic brain injury: fine-grained assessment and relevance-theoretic considerations. Aphasiology 2020, 34, 1070–1100. [Google Scholar] [CrossRef]
  60. Königs, M.; Engenhorst, P.J.; Oosterlaan, J. Intelligence after traumatic brain injury: meta-analysis of outcomes and prognosis. Eur J Neurol 2016, 23, 21–29. [Google Scholar] [CrossRef]
Table 1. Mean, standard deviations (SD), significance (p), and effect size (Cohen’s d) for demographic variables for TBI and non-TBI groups.
Table 1. Mean, standard deviations (SD), significance (p), and effect size (Cohen’s d) for demographic variables for TBI and non-TBI groups.
Demographic Variable TBI group mean (SD) Non-TBI group mean (SD) p d.
Gender m = 15, f = 3 m = 15, f = 3
Age at test 44.94 (11.69) 43.83 (12.26) .696 0.09
Age at injury 36.44 (13.79)
Post-injury years 8.50 (8.68)
Years of education 14.83 (4.25) 5.56 (3.65) .389 0.18
Verbal IQ 84.06 (18.71) 95.33 (8.66) .007 0.77
Performance IQ 91.00 (17.50) 104.72 (11.64) 0.15 0.94
Full IQ score 90.25 (19.69) 100.06 (10.44) .025 0.65
Note: Two participants with TBI opted out of the Wechsler Abbreviated Scale of Intelligence [35] (IQ test). Due to a technical issue, one participant’s data was corrupted for the non-TBI group SI-M and SI-E but this made little difference to the demographic comparisons.
Table 2. Descriptive statistics for the number of correct responses for the TBI and non-TBI groups for the EET.
Table 2. Descriptive statistics for the number of correct responses for the TBI and non-TBI groups for the EET.
TASIT EET Score TBI Mean (SD) Non-TBI Mean (SD)
Overall Correct 19.67 (3.99) 24.28 (1.60)
Happy 3.11 (1.13) 3.28 (0.67)
Surprised 3.22 (0.81) 3.67 (0.49)
Neutral 2.00 (0.91) 2.72 (0.83)
Sad 2.72 (1.32) 3.50 (0.71)
Angry 3.00 (1.08) 3.56 (0.51)
Anxious 2.89 (1.32) 3.94 (0.24)
Revolted 2.72 (1.02) 3.56 (0.62)
Table 3. Descriptive statistics for the TBI and non-TBI group for fixation duration across the EET in seconds.
Table 3. Descriptive statistics for the TBI and non-TBI group for fixation duration across the EET in seconds.
EET Emotions Groups combined
Mean (SD)		 TBI Mean (SD) Non-TBI Mean (SD)
Angry
 Eyes
 Nose
 Mouth		 3.97 (3.01)
1.15 (1.25)
1.59 (1.80)
1.23 (0.90)		 2.92 (2.12)
0.92 (1.08)
0.93 (1.16)
1.07 (0.90)		 5.03 (3.43)
1.37 (1.40)
2.26 (2.10)
1.40 (0.90)
Revolted
 Eyes
 Nose
 Mouth		 3.76 (3.06)
0.78 (0.97)
1.43 (2.05)
1.54 (1.56)		 3.01 (1.99)
0.72 (0.97)
0.82 (1.04)
1.48 (1.35)		 4.50 (3.76)
0.85 (0.99)
2.05 (2.60)
1.61 (1.78)
Anxious
 Eyes
 Nose
 Mouth		 2.44 (2.74)
0.41 (0.61)
1.43 (1.99)
0.60 (0.87)		 1.53 (2.10)
0.51 (0.74)
0.60 (1.06)
0.42 (0.76)		 3.35 (3.05)
0.32 (0.44)
2.27 (2.36)
0.77 (0.96)
Happy
 Eyes
 Nose
 Mouth		 4.41 (2.35)
0.93 (1.13)
0.86 (1.02)
2.62 (1.70)		 4.04 (2.32)
0.62 (0.96)
0.69 (0.68)
2.73 (1.74)		 4.79 (2.38)
1.24 (1.23)
1.02 (1.28)
2.53 (1.71)
Sad
 Eyes
 Nose
 Mouth		 8.56 (6.68)
3.84 (3.53)
2.16 (3.16)
2.56 (2.60)		 6.92 (5.48)
3.40 (3.91)
1.05 (1.13)
2.47 (2.59)		 10.21 (7.50)
4.28 (3.16)
2.37 (3.99)
2.65 (2.67)
Surprised
 Eyes
 Nose
 Mouth		 2.18 (1.95)
0.63 (1.05)
0.94 (1.22)
0.61 (0.68)		 1.64 (1.77)
0.70 (1.28)
0.51 (0.66)
0.43 (0.46)		 2.72 (2.01)
0.55 (0.79)
1.38 (1.49)
0.79 (0.82)
Neutral
 Eyes
 Nose
 Mouth		 4.84 (2.92)
1.92 (2.22)
1.59 (1.49)
1.33 (1.23)		 4.36 (2.76)
1.85 (2.36)
1.13 (1.23)
1.38 (1.51)		 5.32 (3.08)
1.98 (2.14)
2.06 (1.61)
1.27 (0.90)
Overall (emotions combined) 3.82 (2.82) 2.93 (2.32) 4.69 (3.05)
 Eyes 0.98 (1.21) 0.87 (1.29) 1.57 (1.58)
 Nose 1.37 (1.65) 0.72 (0.89) 1.57 (1.56)
 Mouth 1.47 (1.16) 1.35 (1.16) 1.68 (1.88)
Table 4. Descriptive statistics for the TBI and non-TBI group for fixation count across the EET.
Table 4. Descriptive statistics for the TBI and non-TBI group for fixation count across the EET.
EET Emotions Groups combined
Mean (SD)		 TBI Mean (SD) Non-TBI Mean (SD)
Angry
 Eyes
 Nose
 Mouth		 8.80 (5.52)
2.67 (2.76)
2.98 (2.74)
3.14 (1.98)		 6.54 (3.75)
1.93 (2.14)
1.69 (1.70)
2.92 (1.99)		 11.06 (6.15)
3.42 (3.16)
4.26 (3.01)
3.38 (1.99)
Revolted
 Eyes
 Nose
 Mouth		 7.12 (4.93)
1.68 (1.66)
2.75 (3.00)
2.69 (2.13)		 6.36 (4.07)
1.66 (1.68)
2.02 (2.35)
2.68 (2.02)		 7.89 (5.68)
1.70 (1.69)
3.48 (3.45)
2.71 (2.29)
Anxious
 Eyes
 Nose
 Mouth		 4.21 (4.25)
1.02 (1.24)
2.02 (2.52)
1.17 (1.65)		 3.00 (3.51)
1.12 (1.39)
0.92 (1.13)
0.96 (1.65)		 5.41 (4.68)
0.91 (1.11)
3.11 (3.05)
1.39 (1.65)
Happy
 Eyes
 Nose
 Mouth		 9.22 (4.19)
2.17 (2.12)
2.25 (1.77)
4.79 (2.60)		 8.79 (3.99)
1.42 (1.38)
2.07 (1.65)
5.30 (2.88)		 9.64 (4.45)
2.92 (2.49)
2.43 (1.92)
4.29 (2.26)
Sad
 Eyes
 Nose
 Mouth		 12.81 (8.24)
5.22 (3.77)
3.44 (3.98)
4.15 (3.26)		 11.15 (7.00)
4.96 (4.08)
2.21 (2.51)
3.99 (3.09)		 14.47 (9.21)
5.49 (3.53)
4.67 (4.81)
4.32 (3.49)
Surprised
 Eyes
 Nose
 Mouth		 4.29 (3.28)
1.08 (1.16)
1.87 (2.15)
1.34 (1.29)		 3.37 (2.89)
1.02 (1.19)
1.19 (1.32)
1.16 (1.15)		 5.21 (3.47)
1.13 (1.16)
2.54 (2.60)
1.53 (1.42)
Neutral
 Eyes
 Nose
 Mouth		 9.56 (5.42)
3.75 (3.72)
3.03 (2.46)
2.78 (2.03)		 7.81 (4.34)
2.98 (3.08)
2.12 (2.10)
2.71 (2.29)		 11.31 (5.93)
4.52 (4.21)
3.94 (2.50)
2.84 (1.78)
Overall 7.38 (4.61) 5.82 (3.72) 8.94 (4.97)
 Eyes 2.02 (1.84) 1.57 (1.58) 2.48 (2.01)
 Nose 2.54 (2.44) 1.57 (1.56) 3.51 (2.80)
 Mouth 2.82 (1.80) 2.68 (1.88) 2.96 (1.76)
Table 5. Descriptive statistics for the accuracy of the TBI and control groups for the different conversational constructs during the SI-M.
Table 5. Descriptive statistics for the accuracy of the TBI and control groups for the different conversational constructs during the SI-M.
SI-M Score TBI Mean (SD) Control Mean (SD)
Simple sarcasm 15.28 (3.63) 17.71 (2.11)
Paradoxical sarcasm 15.94 (4.01) 18.18 (2.24)
Sincere 15.39 (3.31) 19.76 (1.48)
Table 6. Descriptive statistics for the accuracy scores of the TBI and non-TBI groups for the four comprehension probes during the SI-M.
Table 6. Descriptive statistics for the accuracy scores of the TBI and non-TBI groups for the four comprehension probes during the SI-M.
SI-M Score TBI Mean (SD) Non-TBI Mean (SD)
Intentions 11.39 (2.73) 13.82 (1.07)
Meaning 11.78 (2.21) 13.83 (0.95)
Beliefs 11.11 (2.47) 13.71 (1.05)
Feelings 12.89 (1.75) 14.00 (0.94)
Table 7. Fixation duration for SI-M across the conversation styles.
Table 7. Fixation duration for SI-M across the conversation styles.
Conversational Style Overall mean (SD) TBI mean (SD) Control mean (SD)
Simple sarcasm
 Eyes
 Nose
 Mouth 		1.70 (1.54)
0.34 (0.54)
0.55 (0.67
0.81 (0.79)		 1.77 (1.47)
0.44 (0.65)
0.51 (0.56)
0.82 (0.75)		 1.63 (1.65)
0.23 (0.37)
0.58 (0.78)
0.81 (0.86)
Paradoxical sarcasm
 Eyes
 Nose
 Mouth		 0.74 (0.66)
0.21 (0.26)
0.29 (0.39)
0.24 (0.28)		 0.80 (0.62)
0.26 (0.30)
0.24 (0.30)
0.30 (0.33)		 0.68 (0.72)
0.15 (0.19)
0.35 (0.48)
0.18 (0.20)
Sincere
 Eyes
 Nose
 Mouth		 2.79 (1.82)
0.59 (0.82)
0.95 (0.92)
1.25 (1.23)		 3.02 (2.24)
0.68 (1.00)
1.01 (0.98)
1.34 (1.51)		 2.53 (1.26)
0.49 (0.59)
0.89 (0.88)
1.15 (0.88)
Table 8. Descriptive statistics for the AOI for the TBI and control groups across the three conversational styles (fixation count).
Table 8. Descriptive statistics for the AOI for the TBI and control groups across the three conversational styles (fixation count).
Conversational Style Overall mean (SD) TBI mean (SD) Control mean (SD)
Sarcasm
 Eyes
 Nose
 Mouth		 4.52 (3.21)
0.95 (1.27)
1.55 (1.46)
2.02 (1.58)		 4.62 (2.74)
1.22 (1.49)
1.48 (1.76)
1.92 (1.43)		 4.41 (3.72)
0.67 (0.96)
1.61 (1.75)
2.13 (1.76)
Paradoxical sarcasm
 Eyes
 Nose
 Mouth		 1.97 (1.48)
0.62 (0.63)
0.75 (0.78)
0.61 (0.50)		 2.12 (1.44)
0.76 (0.71)
0.73 (0.69)
0.63 (0.56)		 1.82 (1.56)
0.46 (0.52)
0.76 (0.89)
0.59 (0.46)
Sincere
 Eyes
 Nose
 Mouth		 5.73 (3.41)
1.67 (2.06)
1.94 (1.56)
2.12 (1.69)		 5.44 (3.71)
1.55 (1.62)
1.91 (1.67)
1.98 (1.90)		 6.04 (3.16)
1.79 (2.49)
1.98 (1.49)
2.27 (1.48)
Table 9. Descriptive statistics for the behavioural data of the TBI and control groups during the sarcastic and lie conditions of the SI-E.
Table 9. Descriptive statistics for the behavioural data of the TBI and control groups during the sarcastic and lie conditions of the SI-E.
SI-E Score TBI Mean (SD) Control Mean (SD)
Sarcastic 22.56 (4.77) 29.41 (3.37)
Lie 25.33 (4.38) 27.76 (3.11)
Table 10. Descriptive statistics for the behavioural data of the TBI and control groups for the four comprehension probes during the SI-E.
Table 10. Descriptive statistics for the behavioural data of the TBI and control groups for the four comprehension probes during the SI-E.
SI-E Score TBI Mean (SD) Control Mean (SD)
Intentions 11.28 (2.49) 14.82 (1.63)
Meaning 11.17 (2.60) 14.47 (1.70)
Beliefs 13.33 (1.81) 14.47 (0.72)
Feelings 12.11 (2.47) 14.12 (2.45)
Table 11. Descriptive statistics for fixation duration to the AOI for the TBI and control groups across the two conversational styles of the SI-E.
Table 11. Descriptive statistics for fixation duration to the AOI for the TBI and control groups across the two conversational styles of the SI-E.
Conversational Style Overall mean (SD) TBI mean (SD) Control mean (SD)
Sarcasm
 Eyes
 Nose
 Mouth		 0.98 (0.92)
0.37 (0.45)
0.35 (0.39)
0.26 (0.25)		 0.85 (0.70)
0.31 (0.37)
0.29 (0.30)
0.25 (0.19)		 1.11 (1.10)
0.43 (0.53)
0.41 (0.47)
0.28 (0.30)
Lie
 Eyes
 Nose
 Mouth		 0.38 (0.57)
0.14 (0.24)
0.11 (0.20)
0.13 (0.22)		 0.25 (0.30)
0.12 (0.19)
0.07 (0.11)
0.06 (0.08)		 0.51 (0.75)
0.17 (0.30)
0.14 (0.25)
0.20 (0.29)
Table 12. Descriptive statistics for fixation counts to the AOI for the TBI and control groups across the two conversational styles of the SI-E.
Table 12. Descriptive statistics for fixation counts to the AOI for the TBI and control groups across the two conversational styles of the SI-E.
Conversational Style Overall mean (SD) TBI mean (SD) Control mean (SD)
Sarcasm
 Eyes
 Nose
 Mouth 		3.90 (2.74)
1.33 (1.25)
1.39 (1.22)
1.18 (0.82)		 3.62 (2.04)
1.21 (1.10)
1.24 (0.83)
1.16 (0.72)		 4.20 (3.37)
1.45 (1.42)
1.55 (1.55)
1.20 (0.95)
Lie
 Eyes
 Nose
 Mouth		 1.48 (1.94)
0.47 (0.63)
0.49 (0.74)
0.52 (0.77)		 1.12 (0.98)
0.44 (0.53)
0.35 (0.39)
0.33 (0.31)		 1.86 (2.58)
0.49 (0.74)
0.65 (0.98)
0.72 (1.03)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated