Prerpints.org logo
Preprint
Article

Facial Pain Detection Using Multilinear Subspace Learning

Altmetrics

Downloads

203

Views

320

Comments

0

Submitted:

18 May 2024

Posted:

30 May 2024

You are already at the latest version

Alerts
Abstract
Detection and measurement of pain are crucial in designing effective pain management interventions to alleviate suffering and prevent functional decline in the medical domain. Traditional methods of pain identification rely on verbal patient reports or observable signs, which can be imprecise or subjective due to variability in patient self-assessments. Hence, there is a pressing need for automated methods to standardize pain measurement procedures. This paper proposes a novel facial expression-based automated pain assessment system as a behavioral indicator for pain evaluation. Our system integrates a fusion structure comprising Convolutional Neural Networks (CNNs) and Tensor-based extended Quantitative Expression Descriptors Analysis (TXQEDA). This innovative approach allows for joint learning of robust pain-related facial expression features from raw facial images, combining RGB appearance with shape-based latent representation. We extensively evaluated our proposed model using the UNBC-McMaster dataset for pain classification and intensity estimation. Our results demonstrate the efficacy of the proposed technique, achieving an impressive accuracy of 99.10% for pain level classification. Furthermore, our system excels in pain intensity estimation, showcasing its potential to provide precise and reliable assessments of pain severity. Overall, our research underscores the importance of automated pain assessment systems in improving patient care and clinical outcomes. By leveraging advanced deep learning techniques and fusion strategies, our proposed model offers a promising approach for standardizing pain measurement procedures and enhancing the quality of pain management interventions.
Keywords: 
Subject: Computer Science and Mathematics  -   Artificial Intelligence and Machine Learning

1. Introduction and Motivation

Understanding human behavior is complex and has been one of the promising areas since the advent of computing machines. Building a system that can accurately distinguish and understand human behavior has played and still plays a crucial role in research and industry. Our study focuses on a specific property of pain behavior, namely the automatic detection of pain by facial expression. Pain is a very unpleasant sensation caused by an illness or injury, or by mental distress or suffering. It is often referred to as the fifth vital sign in health care because pain, like the other critical signs, is now considered an objective sensation in health care. According to the National Centers for Health Statistics, approximately 76.2 million people worldwide suffer from pain. In general, medical abnormalities are difficult to assess and manage and are usually measured by patients by self-report. This and similar procedures are accepted because they are simple and easy to understand and often provide information that validates the level of pain experienced by the individual. However, these methods only work when the patient is sufficiently alert and cooperative, which is not always possible in medicine. But beyond that, pain assessment using self-report measures is a significant challenge. It is not always reliable and valid in critically ill adults, including those who are unable to communicate their pain level, such as people with dementia and certain types of neurological disorders, as well as intensive care unit (ICU) patients who require an oxygen mask to breathe. In addition, it cannot be applied to unconscious patients or newborns. To overcome these limitations, observational and psychological measures have become indispensable. Our hypothesis that the human face is a rich source of non-verbal information that delivers semantic cues for understanding emotions. Wang et al. proposes to objectively measure a patient’s pain level using machine learning and pupillary response algorithms in [1]. We confirmed this rich resource information discriminative can be useful in revealing mental status through social cues. The face texture is rich resource information and a free natural password and a means through which people recognize each other, it is used in the early stages of life by an infant to recognize his surrounding people based on facial features in [4]. We were motivated and inspired by the choice of these CNN architectures given their good performance in different vision tasks, as shown in the ImageNet large-scale visual recognition challenge [7]. An overview of analysis on the subject by comparing some GoogleNet, ResNeXt-50, and MobileNet based on CNN in [2]. Theses research recently demonstrates the importance of deep learning and methods conventionnel for pain estimation by [3] is considered a spontaneous reaction to pain. In plus Othman et al.[43] are demonstrates how to recognize pain intensity from faces in videos, while humans are very adept at this task compared to machines. Variation in facial expression often symbolizes the onset of pain. Physicians place great importance on the credibility of these behaviors and consider them to be consistent and convincing signs of pain. Facial activity has been co-opted as a primary or major component of most multidimensional behavioral checklists or pain rating scales. There is a considerable amount of literature in which the Facial Action Coding System (FACS) [5], has been applied to pain expression in [6]. The principal contributions of this article comprise: The distribution of the degraded spectrum by total variation regularization by Liu et al. [23,28,32,33,34,35,36,37]. model capable of calculating with high precision the identity of the person under analysis separately from the incoming angles. Our contributions can be summarized in the following points:
  • -We propose a new robust approach that mainly explores a newly designed automatic pain detection using facial expressions.
  • -The new framework includes multilinear principal component analysis as a contributing step for face feature extraction and dimension reduction, which increases the detection rate compared to previous studies reported in the literature review to improve the discriminatory power.
  • - Our system performance is evaluated by testing on the UNBC-McMaster Shoulder Pain Expression Archive databases confirming its supremacy over those of the state-of-the-art.
Table 1. Proposed Algorithm by Aliradi et Taleb-Ahmed.
Table 1. Proposed Algorithm by Aliradi et Taleb-Ahmed.
Imput: A face in pain from the UNBC-McMaster, learned space projection matrices by Multilinear Whitened Principal Component Analysis (MWPCA), and a fixed threshold. Initialize Type=(A pair of images)
For i = 0 to 4. λ = 4 × 2 i Calculate G × ( λ , θ ) Compute P S P I t y p e ( i + 1 ) (equation 1) End.
Output: represent an image that contains five scales, with the convergence parameters.
The rest of the paper is organized as follows. Section 2 reviews the current literature in Face Verification Based on Subspace Tensor Transformation. Section 3 presents the framework of the proposed verification system. Section 4 demonstrates the experiments and their settings. Section 5 discusses the obtained results. Finally, the conclusion and future works are introduced in Section 6.

2. Related Works

With the rapid development of computer technology, tensor data increased and extended in many areas such as augmented reality, signal processing, computer vision, medical image analysis, and web data mining. Especially the most famous As the problem of pain detection is not new and much research has been done on this topic, it is very important to have a view of these previous works to have an idea of the approaches used and their limitations as well as the results obtained, so for this, we made a literature review of some articles presented in Table 1. In [8], Ashraf et al. used the Active Appearance Model (AAM) on videos containing pain expressions based on Support Vector Machine (SVM) based machine learning procedure to classify pain and non-pain. With the advantage of representing dynamic alterations in pain-related actions, the best-performing predictive model yielded a success rate of 82 . 00 % . In [9], Lucey et al. Utilized a classifier SVM to develop a system for detecting image-level pain in two ways on patient images: first directly from facial features, and second by fusing individual action unit (AU) detectors. In addition. extended their work as described to detect pain to use an AAM approach on a frame-by-frame. They showed that fusing all AAM representations using linear logistic regression (LLR) provides notable performance for detecting pain and action units in the image. In [10], Kaltwang et al. proposed to use a different form of facial landmarks and appearance features, namely discrete cosine transform (DCT), relevance vector regression (RVR), and local binary model (LBP), and then merged these features and thus showed that merging these features leads to a better estimation of pain level compared to the specific estimation of pain intensity, and they manage to achieve an accuracy of 92 . 00 % . In ref [11], authors used AAM to extract the canonical normalized facial appearance. Finally, an SVM classifier is used to detect the pain level on a frame to obtain an accuracy of 73 % . In ref [12], Sourav and Mrinal utilized Gabor filtering as a contributing step for face feature extraction and dimension reduction using principal component analysis (PCA), which increases the detection rate according to the comparative study. In [17], authors utilized test datasets to compare the target algorithm. They then use the support vector machine as a classifier and achieve 95 . 50 % accuracy on the images, which is the best result in the pain detection problem so far. Still recent research was published in [13] where the authors proposed a fully automatic pain assessment model including a novel convolutional network fusion framework for assessing different pain intensities from raw facial images that achieved an F1-score of 94 . 00 % for pain level classification. In ref [14] Fat et al. present a framework for pain detection/classification that uses a combination of KNN and Adaboost classifiers to obtain an accuracy of 91 % . Alghamdi et al. in [18], developed a new system expressions-called (FEAPAS) to notify a staff when a patient suffers pain by activating an alarm. Plus the intensive care patients in ref [23]. Pedersen et al. [25] proposed a new approach based on the combination of depth information and descriptors such as RGB values, thermal facial images. Ferroukhi et al. proposed a new coding method video based on bandelet transform algorithm in [26]. In addition, Chen et al. [15] detect a pain event and locating in the video. We used CNN for automatic feature extraction from frames in [24]. Often the pain is expressed verbally, but in some cases, traditional patient self-reporting is not efficient in [16]. Ghosh et al. proposed method in the fourth component which utilize the scores due to the statistical and deep feature analysis are fused to ameliorate the performance in [27]. To utilize the prediction models are based on deep feature analysis and statistical procure scores for the facial region’s pain intensities (low-pain, high-pain, and no-pain). X et al. [20] have extensively used the convolutional neural network (CNN) using a different image classification tasks. Recently authors proposes a new architecture that uses discriminative information, which is based on the exponential discriminant analysis (DIEDA) and the projected histograms for each region are scored using the discriminative metric in refs [21]. Mimouna et al. simplifed a heterogeneous multimodal dataset for advanced environment in [42].[38] Al-Shiha et al. [39]. We see through our article that we have used, several hand-crafted features in the pain estimation task. In addition, many classifiers are used to discriminate these features. They implemented the proposed approaches to a sufficiently detailed level, for instance, in refs. [Aliradi], the relationship between AUs and pain have been considered.

2.0.1. Prkachin and Solomon Pain Intensity Metric (PSPI)

PSPI is a pain assessment metric calculated from pain-related action units (AU). Previous studies where it has been observed that pain-related expressions are characterized by certain facial muscles being activated during pain. We note that the activations of these muscles are encoded by a set of numerical values called AU. During the pain episode, sets of AUs are activated, such as eyebrow lowering (AU4), orbital tightening (AU 6 and AU7), levator raising (AUC9 and AU10), and approximation eyes ( i . e . A U 43 ) . All of these AUs are measured using a 6-point scale (i.e., 0 and 5 represent absence and maximum, respectively). These studies confirmed that these AUs consist of pain-related information and are defined by the PSPI score. The PSPI score is calculated based on the following equation: By assigning numerical values to those six AUs, pain can be calculated via the Prkachin and Soloman pain intensity (PSPI) equation,
P S P I = A U 4 + ( A U 6 o r A U 7 ) + ( A U 9 o r A U 10 ) + A U 43
The dataset also contains pain scores for each frame which are based on the PSPI (Prkachin and Solomon pain intensity scale). The PSPI scale is defined in 17 levels using FAC. The calculation is shown in the equation below: The PSPI score is the sum of AU4, AU6, or AU7 (depending on which is higher in intensity), AU9 or AU10 (depending on which is higher in intensity), and AU43. The AUs are: lowering the eyebrows (AU4), puffing of the cheeks (AU6), tightening of the eyelids (AU7), wrinkling of the nose (AU9), the raising of the upper lip (AU10) and closing of the eyes (AU43). Except for AU43(0 = absent, 1 = present), each AU is coded in 6 levels of intensity (0 = absent, 5 = maximum). Figure 1 shows an example of the same face with different PSPI levels. According to the results of strong pain detection (PSPI = 3) would give a more stable result, therefore, the pain data set is preprocessed into NO-PAIN(PSPI=0) and PAIN(PSPI = 3) subset.

2.0.2. UNBC Databaset

We chose in this work a database called the UNBC-McMaster shoulder pain expression archive[19]. The database consists of 200 videos of 25 subjects with spontaneous facial expressions who suffer from shoulder pain. It has in total 48,398 FACS (Facial Action Coding System) coded frames. The subjects performed the physical exercise from affected and unaffected limbs and all activities were recorded by a video camera. In addition, each frame was labeled by the PSPI score [19]. For the PAIN subset, there are about 3000 images from 25 subjects, for the No-PAIN subset, there are about 40 000 images in total. Advantages: The size of the database is it’s main strength, more than 48,000 images are more than enough to train our model with almost all the algorithms we want to use, and better still, the fact that the labels are not in binary, which means that the pain is expressed in levels for each frame of the video (following the formula above), which helps to greatly improve the accuracy of the model, it also gives us the opportunity to try to predict the level of pain felt by the subject instead of a simple binary guess. Another strong point is that the images have a background this time, which means that testing images outside the database (images we capture) will not confuse the model since it is already used to seeing backgrounds before. Limitations: The main weakness of the dataset is the lack of diversity in the subjects, as there are only 25 subjects who all suffer from the same type of pain (shoulder pain), which makes the 48398 images very similar and repetitive, especially the fact that they are taken from a video with consecutive images. The images are also of significantly lower quality which can cause problems for the face extraction model.

3. Proposed of Our Architecture Method

In this section, we initially present an abbreviation of (faces preprocessing), we explain the details of the proposed sustem for multimodal face verification based on tensor representation. As illustrated in Figure 3, the block diagram of the global architecture of our method consists of four essential components: feature extraction, multilinear subspace transformation tensor, dimensionality reduction, and comparison(Cosine similarity), and in conclusion, we prove our architecture. In this section, we details our work, we introduce a system for pain detection/classification that uses a combination of different techniques in face detection, features extraction, at he theory wavelets Gabor which bases this Frames which invented by Gabor Having looked at the dataset as well as the previously mentioned related works, as shown in Figure 1 the block diagram of the global architecture in general design for the system was essential fixed from the beginning with very little wiggle room; as any facial recognition system works, the main steps are: preprocessing the data, extracting the face, and then training the classification model, except in our case we are working with a Multidimensional Subspace Learning algorithm, meaning the data has to go through another step before being parsed through the model, this step is going to fall between the face extraction and the model training. As for the algorithms and hyperparameters used, we first started with an educated guess based on other articles as well as general Computer Vision classification models’ configurations, then we started experimenting more until we reached the desired performance from the model, the system was created around the UNBC Pain Dataset with over 40 thousand images, except due to overfitting issues as well as time and computing performance constraints we decided to limit the data used to just 20 thousand images (which is generally more than enough for binary classification models), we prove performance our architecture.

3.1. Data Preprocessing

We proceed by several stages of technique preprocessing images as with any computer vision system, the extracted images did not meet our requirements for the model, specifically the image resolution as well as the color channels. Since pain is expressed through changes in our facial expressions, the colors would have no meaning on the pain state in question, hence our decision to transform all images to grayscale, not only will this increase the performance of the model through all its training stages, but it will also get rid of any unwanted or unnecessary features that could cause interference. Since the previous face extraction function will return images of different sizes We also decided to lower the resolution of all images used (for both datasets) to size ( 48 × 48 ) pixels, we found that this is the lowest resolution we can use that will still guarantee distinguishable facial features. We must remember that one of the main reasons for using multidimensional subspace learning is not only to increase the accuracy of predictions but especially to reduce learning times. Therefore, we need to make sure that the data we feed to the model is as small as possible (without losing too much detail, of course). The Python library used is OpenCV (via the CV2 interface), which offers easy-to-use and efficient functions for resizing images as well as for changing color channels.

3.2. Feature Extraction

First, we need to extract the faces from each image, this is a crucial step because it not only reduces the size of our images but also removes the background and any unwanted features. Since the facial expressions of pain in humans are expressed by lowering the eyebrows, squeezing the eyes, wrinkling the nose, raising the upper lip, and opening the mouth, these are the only facial features that need to be retained as detailed in [Sri09]. For this, we implemented a function and we decided to use the MTCNN library which allows us to extract the key points we need from the images, namely the right eye, the left eye, and the right corner of the mouth. well as the left corner, the key points are then used to create a rectangular frame around the facial features. The MTCNN library (as the name suggests) uses a CNN classifier to extract landmarks from the face. The model is able to detect faces across different lighting levels and degrees of rotation, with extremely high accuracy of between 94 . 80 % and 100 % . In that article, a novel technique and scheme of fusion have been proposed that uses high-order Multilinear Whitened Principal Component Analysis (MWPCA).

4. Multilinear Subspace Learning

Over the past decade, most of the data has previously been treated just as a 1 D vector which required breaking the fabric of the initial data. In order to enhance the subspace tensor approach to use in our painful estimation. Such as the case of the classical PCA approach for dimensionality reduction which has become unsupportable for the presentation of multidimensional space data for very high dimensionality. For this reason, The main objective of our system is to see the effect of multilinear subspace learning on multidimensional imaging. This approach aims at dimensionality reduction on large data tensors containing either vectors or, in our case, matrices. This approach is also a generalization of the familiar linear subspace learning, usually implemented with PCA, ICA, LDA, or CCA, and to solve all problem by using any features with the TXQEDA method for dimensionality reduction in ref [44]. Multilinear subspace learning encompasses multiple dimensionality reduction algorithms. In our case, we decided to use multilinear principal component analysis (MPCA or M-mode PCA), which is actually an N-dimensional extension of principal component analysis, and works by centering the data tensors to find abnormally large (or abnormally small) values over defined intervals to extract and sort them into new lower-dimensional tensors. The MPCA is an extension of PCA. It allows the projection to a lower subspace. The implementation of the MPCA approach is done in 3 main steps, the first one is the centering of the input tensor (the image essentially), then the projection matrices are initialized by finding the eigenvalues and the eigenvectors according to the image in question, then the Q-Based method is applied, this method works by finding the ratio of the eigenvalues after truncating the number of least important eigenvalues on the sum of the other values, the user defines the Q-value in question (in our case we have used 97 % ) Finally, the MPCA loop is launched, this loop works by applying the Scatter Max method or the Error Minimization method (we chose the first one) on the data tensor with the remaining eigenvalues, which will produce 3 different projection matrices which will then be grouped by the transformation function [18].

5. Training of the Model

Finally, we used SVM as a data classification model, more precisely the Support Vector Classification (SVC) from SkLearn, the data were first divided by 80 % and 20 % for training and validation respectively, the hyper parameters of the SVC model we used were: [ 0.1 , 1 , 10 ] for C and, [ 0.001 , 0.01 , 0.1 ] for Gamma and finally we used RBF, Linear and Poly kernels, the best accuracy results for the combinations of these parameters were automatically chosen by the SVC fitting function.

6. Results Analysis

To evaluate the performance, Using the UNBC database, as shown in Table 2 we managed to achieve an accuracy of around 98 . 80 % only by using one-half of the data available (20,000 images), and although we were able to use the whole database, there was a fear of falling into overfitting especially considering that this data suffers from the variety and that the images are taken from successive video frames which lead to repetitive images. the trained model gives accurate and precise pain status predictions for both test images from the dataset as well as images outside the set (images we took ourselves), the most notable thing about this model was how short the training time was especially for such a high level of accuracy, this was helped by the smaller tensor size that was used during the training phase, which is where we can definitely see the benefit of using MPCA, although it is important to note that the fitting time for the MPCA model itself, as well as the data preprocessing phase was noticeably longer.
Table 2. Accuracy comparison of the different techniques for detecting painful expressions estimation.
Table 2. Accuracy comparison of the different techniques for detecting painful expressions estimation.
Work/Year Methods Dataset Classifier Best Result
Rodriguez etal. [12] Features: VGG-16 UNBC-McMaster Shoulder Pain LSTM MSE 0.74 AUC 93 . 30 % Accuracy 83 . 10 %
Tavakolian et al. [13] Features: CNN UNBC-McMaster Shoulder Pain Deep Binary M S E 0.69 PCC81%
Bargshady et al. [14] Features: VGG-face UNBC-McMaster Shoulder Pain RNN MSE 0.95   A c c u r a c y 75 . 20 %
Wang et al. [39] Features: GA-ANN UNBC-McMaster Shoulder Pain ANN MSE 1.57   A c c u r a c y 81 %
Huang et al. [15] Features: spatiotemporal, and geometric UNBC-McMaster Shoulder Pain Hybrid network MAE 0.40 MSE 0.76   A c c u r a c y 82 %
Fat et al. [43] Features: Gabor-Adaboost UNBC-McMaster Shoulder Pain KNN A c c u r a c y 91 %
El-Morabit et al. [2] Features DEIT-PNP UNBC-McMaster Shoulder Pain SVR Accuracy 84 . 15 %
Semwal et al. [11]. Features: Deep-CNN UNBC-McMaster Shoulder Pain SVM A c c u r a c y 92 %
Ghosh et al. [6] Features:Gabor-PCA UNBC-McMaster Shoulder Pain SVM A c c u r a c y 95 . 50 %
Our system Features SVM UNBC-McMaster Shoulder Pain SVC MAE 2.20 , and A c c u r a c y 98 . 88 %

6.1. Defining the Time Cost

which attains a good performance and represents a record time to compare with other methods.

6.2. Comparison with Existing Methods

we compare our method against other state-of-the-art models under the same metrics. Firstly, we conduct a comparison with traditional hand-crafted methods, as shown in Table 1. As can be seen, our method has obvious advantages against these traditional methods under AUC, ICC, PCC, MAE, and MSE. It can be concluded that in the task of pain estimation, our proposed HDN shows a more promising performance than hand-crafted methods. This is reasonable, as the deep learning method could learn target features according to the task. This makes the current pain research tend to deep methods. So, we also list some deep pain methods in Table 2. From the comparison of Tables 1 and 2, it is apparently observed that deep learning methods have better performances, which indicates the superiority of deep models. However, these methods take coarse input rather than considering the locality and relationship characteristics. In this paper, the proposed HDN leverages local and hierarchical branches to involve the locality of pain and region relationships simultaneously. Therefore, our method outperforms these deep learning methods in most cases. Although the results of PCC are not the best, but it is a good enough result. This is because the methods with better PCC performance are devoted to the video segments (e.g. refs. [18], which naturally predict the trend of pain intensity well. On the other side, for single-frame images, their prediction errors are obviously larger than ours. Since our method is targeted to image frames, our training procedure would reduce the distance between the prediction and label of each frame. It is possible to ignore some changes in pain intensity during this process. Therefore we can conclude that our HDN method achieves impressive performance. Besides, there are some other hierarchical methods whose framework is similar to ours, as shown in Table 3. These researches employed a group of hand-crafted features, e.g. refs. [11]. Some combinations of traditional and deep features are used in ref. [23]. These studies tried to provide some solutions from various aspects. However, there is no exploration of the nature of pain. They neglect the locality of pain and region relationship that are significant pain characteristics. Therefore, compared to our method, these methods present disadvantages in terms of performance to some extent

6.3. Discussions

Our principal goal is to offer more credibility to our approach (Facial pain detection using multilinear) in terms of visibility, scalability. We have compared it with other methods. The obtained results confirm the power of the new face pain detection to take into consideration different variability factors for unimodal face pain cases and the one databases used. We have examined the precision performances obtained in tables 2 and 5 with the classification of the various descriptors during our experiences. Individual descriptors and their merging: To compare individual descriptors’ performance, the results consistently show that system is the best. We evaluated the performance of our new architecture, which we chose because it yields the highest accuracy rate (AR) and the lowest error equal rates (EER) based on the findings obtained and summarized in tables 2 and 5. Tables 2 and 5 report the errors EER and the VR at ( 0.001 FAR) on the UMBC Shoulder databases. Findings are recommended for individual functionality and their merger. We notice technique achieved a 98 . 80 % of F 1 − s c o r e . Moreover, for pain intensity estimation the proposed model has achieved a mean absolute error of 2 . 20 % and an accuracy result of 98 . 80 % that demonstrate performance outperforms other methods the state-of-the-art in areas under-the-curve of the UNBC-McMaster dataset. We can confirm that the discriminative power of face pain images is higher than that achieved by face images to handle different challenges, such as poses, illuminations, and expressions. In summary, we confronted our results with the state of the art for the (controlled) environments. We compared our obtained results with their last nine methods best cited in the state of the art, including: [12, 13, 14, 39, 43, 2, 11, 6] for the same UNBC-McMaster data sets. The performance of our system is easily proven and visible (tables 2 and 5). The accuracy obtained from our system is better compared to the other systems tested.
Preprints 106869 g004
Figure 4. The Result of our system proposed, respectively.
Figure 4. The Result of our system proposed, respectively.
Preprints 106869 g004
Preprints 106869 g005
Figure 5. Accuracy, respectively.
Figure 5. Accuracy, respectively.
Preprints 106869 g005
We synthesize the confusion matrix of the ANN classification results using selected functions is shown in Figure 3. The ANN classifier had good performance for data labeled "B" (baseline) and “H” (strong pain). Of class "B", 98 . 80 % were classified correctly and 21 % were classified as class "L". Among the “H” class, 95 % were classified correctly, and only 5 % were classified as class “L”. However, the ANN had poor performance for "L" data (low pain). Between "L" classes, only about half were classified correctly and 44 % were classified as "H" class, which means that the classifier was not sensitive to "L" data. It can also be seen in Figure 3. The confusion matrix also suggested that the type 1 error (false positive) and type 2 error (false negative) for class ’B’ were 0 and 3 respectively. Type 1 and type 2 errors for class ’L’ were 33 and 732 respectively. Type 1 and type 2 errors for class ’H’ were respectively 3222 and 13.

6.4. Extensive Experimental Comparison

For the UNBC-McMaster shoulder pain database, we have evaluated the performance of this suggested system with some existing state-of-the-art approaches. We employed Gho et al./2016 with Features:(Gabor+Filters+PCA) in [6] , VGG-16 [12], CNN-Deep Binary [13], VGG-face [14], Deep-CNN [11], Huang et al. 2021a [15], Fat et al. /2021b [14], and DEIT-PNP in [2], methods and obtain the performance of the proposed system under the same training–testing protocol as used in the proposed system. The comparison of the proposed system for UNBC database have been reported in Tables 1. From these performances, it has been observed that the proposed system has obtained better performance than the other competing methods concerning both the employed databases.

7. Conclusion and Future Work

In this article, we presented a new system original (Facial pain detection) based on tensor subspace transformation. Firstly, we propose a new discriminative local face descriptor, Hist-Gabor, based on histograms of basic Gabor filters. The Gabor wavelets images are subdivided into several blocks with the objective to build and encode the face structure exactly. Taking advantage of the histogram features which consider the microstructure information such as plane area, edges, and spots, the histograms of the face blocks are concatenated in order to create a robust feature vector. Secondly, to enhance the discrimination of the proposed tensor face representation, we extended two multilinear subspace methods transformation (MWPCA + TEDA). In addition, we have computed the time cost total of the system, which attains a record time. Experimental results on one publicly available face pain datasets have shown that our system method to comparable or better accuracy, and performance and outperform other existing face descriptors and state-of-the-art face verification methods. For perspective, we are interested in applying the proposed; this calls for the research community to cooperate in efforts to add new and more reliable databases and evaluation protocols to advance face verification research. For a fair comparison, the new databases should include the conditions of the image, and possession and discuss the potential allegation of the limitations of the data sets. We give the code available by request to verify the validity of our experiments.

Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

  1. Wang, L. , Guo, Y., Dalip, B., Xiao, Y., Urman, R.D. and Lin, Y., 2022. An experimental study of objective pain measurement using pupillary response based on genetic algorithm and artificial neural network. Applied Intelligence, 52(2), pp.1145-1156.
  2. El Morabit, S. , Rivenq, A., Zighem, M.E.N., Hadid, A., Ouahabi, A. and Taleb-Ahmed, A., 2021. Automatic pain estimation from facial expressions: a comparative analysis using off-the-shelf CNN architectures. Electronics, 10(16), p.1926.
  3. Bendjillali, R.I.; Beladgham, M.; Merit, K.; Taleb-Ahmed, A. Improved facial expression recognition based on dwt feature for deep CNN. Electronics 2019, 8, 324. [Google Scholar] [CrossRef]
  4. Amrane, A. , Mellah, H., Aliradi, R., Amghar, Y. (2014). Semantic indexing of multimedia content using textual and visual information. International journal of advanced media and communication, 5(2-3), 182-194.
  5. Kenneth, M. Prkachin Patrick Lucey Jeffrey F. Cohn. “PAINFUL DATA: The UNBC-McMaster Shoulder Pain Expression Archive Database”. In: (2011).
  6. Patricia, E. Solomon Kenneth M. Prkachina. “The structure, reliability and validity of pain expression:Evidence from patients with shoulder pain”. In: (2009).
  7. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2019, 25, 1097–1105. [Google Scholar] [CrossRef]
  8. Ashraf, A.B., Lucey, S., Cohn, J.F., Chen, T., Ambadar, Z., Prkachin, K., Solomon, P. and Theobald, B.J., 2007, November. The painful face: Pain expression recognition using active appearance models. In Proceedings of the 9th international conference on Multimodal interfaces (pp. 9-14).
  9. Lucey, P. , Cohn, J., Lucey, S., Matthews, I., Sridharan, S. and Prkachin, K.M., 2009, September. Automatically detecting pain using facial actions. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops (pp. 1-8). IEEE.
  10. Kaltwang, S., Rudovic, O. and Pantic, M., 2012. Continuous pain intensity estimation from facial expressions. In Advances in Visual Computing: 8th International Symposium, ISVC 2012, Rethymnon, Crete, Greece, July 16-18, 2012, Revised Selected Papers, Part II 8 (pp. 368-377). Springer Berlin Heidelberg.
  11. Hammal, Z. and Cohn, J.F., 2012, October. Automatic detection of pain intensity. In Proceedings of the 14th ACM international conference on Multimodal interaction (pp. 47-52).
  12. Roy, S.D. , Bhowmik, M.K., Saha, P. and Ghosh, A.K., 2016. An approach for automatic pain detection through facial expression. Procedia Computer Science, 84, pp.99-106.
  13. Semwal, A. and Londhe, N.D., 2021. Computer aided pain detection and intensity estimation using compact CNN based fusion network. Applied Soft Computing, 112, p.107780.
  14. Elgendy, F. , Alshewimy, M. and Sarhan, A.M., 2021. Pain detection/classification framework including face recognition based on the analysis of facial expressions for E-health systems. Int. Arab J. Inf. Technol., 18(1), pp.125-132.
  15. Chen, J. , Chi, Z. and Fu, H., 2017. A new framework with multiple tasks for detecting and locating pain events in video. Computer Vision and Image Understanding, 155, pp.113-123.
  16. Karamitsos, I. , Seladji, I. and Modak, S., 2021. A modified CNN network for automatic pain identification using facial expressions. Journal of Software Engineering and Applications, 14(8), pp.400-417.
  17. Virrey, R.A. , Liyanage, C.D.S., Petra, M.I.B.P.H. and Abas, P.E., 2019. Visual data of facial expressions for automatic pain detection. Journal of Visual Communication and Image Representation, 61, pp.209-217.
  18. Alghamdi, T. and Alaghband, G., 2022. Facial expressions based automatic pain assessment system. Applied Sciences, 12(13), p.6423.
  19. Lucey, P. , Cohn J.F., Prkachin K.M., Solomon P.E., Matthews I. Painful data: The UNBC-McMaster shoulder pain expression archive databaseFace Gesture 2011 (2011), pp. 57-64.
  20. H. Jung, S. Lee, J. Yim, S. Park, J. Kim, Joint fine-tuning in deep neural networks for facial expression recognition, in: Proc. IEEE Int. Conf. Comput. Vis. 2015 Inter, 2015, pp. 2983–2991.
  21. Aliradi, R. , Belkhir, A., Ouamane, A. and Elmaghraby, A.S., 2018. DIEDA: discriminative information based on exponential discriminant analysis combined with local features representation for face and kinship verification. Multimedia Tools and Applications, pp.1-18.
  22. Aliradi, R. , Belkhir, A., Ouamane, A., Aliane, H., Sellam, A., Amrane, A. and Elmaghraby, A.S., 2018, April. Face and kinship image based on combination descriptors-DIEDA for large scale features. In 2018 21st Saudi Computer Society National Computer Conference (NCC) (pp. 1-6). IEEE.
  23. Othman, E.; Werner, P.; Saxen, F.; Al-Hamadi, A.; Gruss, S.; Walter, S. Automatic Vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database. Sensors 2021, 21, 3273. [Google Scholar] [PubMed]
  24. Zhou, J.; Hong, X.; Su, F.; Zhao, G. Recurrent Convolutional Neural Network Regression for Continuous Pain Intensity Estimation in Video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Las Vegas, NV, USA, 27–30 June 2016; pp. 84–92. [Google Scholar]
  25. H. Pedersen, Learning appearance features for pain detection using the unbc-mcmaster shoulder pain expression archive database, in: Int. Conf. Comput. Vis. Syst., 2015, pp. 128–136.
  26. Ferroukhi, M. , Ouahabi, A., Attari, M., Habchi, Y. and Taleb-Ahmed, A., 2019. Medical video coding based on 2nd-generation wavelets: Performance evaluation. Electronics, 8(1), p.88.
  27. Ghosh, A. , Umer, S., Khan, M.K., Rout, R.K. and Dhara, B.C., 2023. Smart sentiment analysis system for pain detection using cutting edge techniques in a smart healthcare framework. Cluster Computing, 26(1), pp.119-135.
  28. Hamouchene, I. and Aouat, S., 2016. Efficient approach for iris recognition. Signal, Image and Video Processing, 10(7), pp.1361-1367.
  29. Zhang, G., and Wang, Y. (2011). Robust 3D face recognition based on resolution invariant features. Pattern Recognition Letters, 32(7), 1009-1019.
  30. Yusoff, Nooraini, and Loai Alamro. "Implementation of feature extraction algorithms for image tampering detection." International Journal of Advanced Computer Research 9.43 (2019): 197-211.
  31. Hoemmen, Mark. Communication-avoiding Krylov subspace methods. University of California, Berkeley, 2010.
  32. Wu Yi, Xiaoyan Jiang, Zhijun Fang, Yongbin Gao, and Hamido Fujita. "Multi-modal 3D object detection by 2D-guided precision anchor proposal and multi-layer fusion." Applied Soft Computing 108 (2021): 107405.
  33. Dutta, K. , Bhattacharjee, D., Nasipuri, M., and Krejcar, O. (2021). Complement component face space for 3D face recognition from range images. Applied Intelligence, 51(4), 2500-2517.
  34. Liu, Tingting, et al. "Flexible FTIR spectral imaging enhancement for industrial robot infrared vision sensing." IEEE Transactions on Industrial Informatics 16.1 (2019): 544-554.
  35. Wang, Ben, Shuhan Chen, Jian Wang, and Xuelong Hu. "Residual feature pyramid networks for salient object detection." The visual computer 36, no. 9 (2020): 1897-1908.
  36. Hai Liu, Hanwen Nie, Zhaoli Zhang, and You-Fi Li. " Anisotropic angle distribution learning for head pose estimation and attention understanding in human-computer interaction."Neurocomputing, Volume 433, April 2021: 310-322.
  37. Almuashi, Mohammed, et al. "Siamese convolutional neural network and fusion of the best overlapping blocks for kinship verification." Multimedia Tools and Applications (2022): 1-32.
  38. Y. Luo, D. Tao, K. Ramamohanarao, C. Xu Tensor canonical correlation analysis for multi-view dimension reduction IEEE Trans. Knowl. Data, 27 (11) (2015), pp. 3111-3124.
  39. Wang, L. , Guo, Y., Dalip, B., Xiao, Y., Urman, R.D. and Lin, Y., 2022. An experimental study of objective pain measurement using pupillary response based on genetic algorithm and artificial neural network. Applied Intelligence, 52(2), pp.1145-1156.
  40. Zhang, J. , Gao, K., Fu, K. and Cheng, P., 2020. Deep 3D Facial Landmark Localization on position maps. Neurocomputing, 406, pp.89-98.
  41. Laiadi, O. , Ouamane, A., Benakcha, A., Taleb-Ahmed, A., and Hadid, A. (2020). Tensor cross-view quadratic discriminant analysis for kinship verification in the wild. Neurocomputing, 377, 286-300.
  42. Guo, Y. , Huang, J., Xiong, M., Wang, Z., Hu, X., Wang, J. and Hijji, M., 2022. Facial expressions recognition with multi-region divided attention networks for smart education cloud applications. Neurocomputing, 493, pp.119-128.
  43. Othman, E. , Werner, P., Saxen, F., Al-Hamadi, A., Gruss, S. and Walter, S., 2021. Automatic vs. human recognition of pain intensity from facial expression on the X-ITE pain database. Sensors, 21(9), p.3273.
  44. Aliradi, R. and Ouamane, A.A., BSIF Features Learning using TXQEDA Tensor Subspace for kinship verification. The First International Conference on Advances in Electrical and Computer Engineering (ICAECE’2023) will be held on May 15-16, 2023, Tebessa, Algeria.
Preprints 106869 g001
Figure 1. Figure 1 shows an example of face in pain from the UNBC-McMaster Shoulder Pain Expression Archive with the corresponding AUs and their intensities. In this example, pain intensity using the PSPI metric is computed as:
Figure 1. Figure 1 shows an example of face in pain from the UNBC-McMaster Shoulder Pain Expression Archive with the corresponding AUs and their intensities. In this example, pain intensity using the PSPI metric is computed as:
Preprints 106869 g001
Preprints 106869 g002
Figure 2. The difference in preprocessing steps to follow for improving the quality of 2D and 3D face images.
Figure 2. The difference in preprocessing steps to follow for improving the quality of 2D and 3D face images.
Preprints 106869 g002
Preprints 106869 g003
Figure 3. Facial actions associated when a person in pain.
Figure 3. Facial actions associated when a person in pain.
Preprints 106869 g003
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated