A Study of Patient’s Pain Assessment Based on Facial Expression: Issues and Challenges

Saddam, Elaf N.; Mutashar, Saad; Ali, Wissam H.

doi:10.30684/etj.v39i10.2079

	A Study of Patient’s Pain Assessment Based on Facial Expression: Issues and Challenges
Engineering and Technology Journal
Article 6, Volume 39, Issue 10, October 2021, Pages 1514-1527 PDF (767.45 K)
Document Type: Research Paper
DOI: 10.30684/etj.v39i10.2079
Authors
Elaf N. Saddam^* ; Saad Mutashar; Wissam H. Ali
Electrical Engineering Dep., University of Technology-Iraq, Alsina’a street,10066 Baghdad, Iraq.
Abstract
Pain is considered as an emotional experience and a restless feeling associated with tissue damage. When the interpretation begins in the brain, the sensation of pain occurs; a signal transmitted to the brain through the nerve fiber. Pain helps the body to stop further damage to the tissues. Since there are numerous ways to convey and feel pain, the perception of pain is special to all. Technology that promotes pain assessment is an urgent need to reduce restless feelings and suffering. This paper aims to demonstrate the issues and challenges facing the patient’s pain assessment based on facial expression. The design and implementation of an automatic pain recognition system and explain the various concepts relevant to it, such as the type of modalities, the procedure of collection and processing data sequentially to reach the classifier. Then presenting clarification for various signals as input data (facial expressions, body movement, and vocalization). This survey would positively help researchers to supplement their efforts towards the expansion of patients' pain assessment based on facial expression.
Highlights
Automatic Pain Recognition (APR) reduces patient overcrowding in healthcare facilities. Especially during the time of epidemics, such as covid-19. If the APR is not accepted by the physicians and the rest of the medical staff then this system will not give any chance to develop and obtain experimental results in real life. Facial expression presented higher acceptability than other indicators of standing alone as an input signal for the automatic pain recognition system. More efforts must be made to collect the data and make it available.
Keywords
Pain assessment; Pain recognition; Facial expressions; Body movement and vocalization

References
[1] J. Watt-Watson et al. An integrated undergraduate pain curriculum, based on IASP curricula, for six Health Science Faculties, Pain, 110, 1–2,2004. 140–148. doi: 10.1016/j.pain.2004.03.019. [2] A. C. de Williams, “Facial expression of pain: an evolutionary account., Behav Brain Sci, 25, 4, 2002,439–455. [3] A. C. de C. Williams and K. D. Craig, Updating the definition of pain, Pain, 157, 11, 2016,2420–2423. [4] A. Mitchell and B. J. Boss, Adverse effects of pain on the nervous system of newborns and young children: a review of the literature, J. Neurosci. Nurs., 34, 5,2002,228. [5] D. C. Turk and R. Melzack, The measurement of pain and the assessment of people experiencing pain., 2011. [6] H. McQuay, A. Moore, and D. Justins, Fortnightly review: treating acute pain in hospital, BMJ, 314, 7093, 1997, 1531. [7] L. Scholl, P. Seth, M. Kariisa, N. Wilson, and G. Baldwin, Drug and opioid-involved overdose deaths—United States, 2013–2017, Morb. Mortal. Wkly. Rep., 67, 51–52, 2019, 1419. [8] J. T. Chibnall and R. C. Tait, “Pain assessment in cognitively impaired and unimpaired older adults: a comparison of four scales,” Pain, 92, 1–2, 2001,173–186. [9] P. Werner, D. Lopez-Martinez, S. Walter, A. Al-Hamadi, S. Gruss, and R. Picard, Automatic recognition methods supporting pain assessment: A survey, IEEE Trans. Affect. Comput., 2019. [10] P. Werner, A. Al-Hamadi, S. Gruss, and S. Walter, Twofold-Multimodal Pain Recognition with the X-ITE Pain Database, 2019 8th Int. Conf. Affect. Comput. Intell. Interact. Work. Demos, ACIIW 2019, no. September, 2019,290–296. doi: 10.1109/ACIIW.2019.8925061. [11] E. E. Benarroch, Pain-autonomic interactions: a selective review, Clin. Auton. Res., 11, 6, 2001,343–349. [12] B. S. Husebo et al., Herausforderungen bei der erfassung und behandlung von schmerzen bei patienten mit demenzieller erkrankung, Zeitschrift fur Neuropsychol., 23, 4,2012,237–246.doi: 10.1024/1016-264X [13] W. Boucsein, Electrodermal activity. Springer Science & Business Media, 2012. [14] M. Benedek and C. Kaernbach, Decomposition of skin conductance data by means of nonnegative deconvolution, Psychophysiology, 47, 4, 2010,647–658. [15] M. Sacco et al., The relationship between blood pressure and pain, J. Clin. Hypertens., 15, 8,2013, 600–605. [16] A. J. Terkelsen, H. Mølgaard, J. Hansen, O. K. Andersen, and T. S. Jensen, Acute pain increases heart rate: differential mechanisms during rest and mental stress, Auton. Neurosci., 121, 1–2, 2005,101–109. [17] B. M. Appelhans and L. J. Luecken, Heart rate variability and pain: associations of two interrelated homeostatic processes, Biol. Psychol., 77, 2, 2008,174–182. [18] C. R. Chapman, S. Oka, D. H. Bradshaw, R. C. Jacobson, and G. W. Donaldson, Phasic pupil dilation response to noxious stimulation in normal volunteers: relationship to brain evoked potentials and pain report, Psychophysiology, 36, 1, 1999,44–52. [19] E. Szabadi, Modulation of physiological reflexes by pain: role of the locus coeruleus, Front. Integr. Neurosci., 6, 2012,94. [20] E. S. D. S. Pinheiro et al. Electroencephalographic patterns in chronic pain: a systematic review of the literature, PLoS One, 11, 2,2016, p. e0149085. [21] B. D. Kussman et al. Capturing pain in the cortex during general anesthesia: near infrared spectroscopy measures in patients undergoing catheter ablation of arrhythmias, PLoS One, 11, 7, 2016,p. e0158975. [22] Y. Chu, X. Zhao, J. Han, and Y. Su, Physiological signal-based method for measurement of pain intensity, Front. Neurosci., 11, 2017, 279. [23] H. J. Kim et al.Racial and ethnic differences in experimental pain sensitivity: systematic review and meta-analysis, Pain, 158, 2, 2017,194–211. [24] I.-P. Tsai, S. Y.-S. Jeong, and S. Hunter, Pain assessment and management for older patients with dementia in hospitals: an integrative literature review, Pain Manag. Nurs., 19, 2018,1, pp. 54–71. [25] T. R. Dawes et al. Objectively measuring pain using facial expression: is the technology finally ready?, Pain Manag., 8, 2, 2018,105–113. [26] K. Karos, A. C. de C. Williams, A. Meulders, and J. W. S. Vlaeyen, Pain as a threat to the social self: a motivational account, Pain, 159, 9, 2018,1690–1695. [27] M.-H. Tessier, C. Gingras, N. Robitaille, and P. L. Jackson, Toward dynamic pain expressions in avatars: perceived realism and pain level of different action unit orders, Comput. Human Behav., 96, 2019,95–109. [28] K. K. Sekhon, S. R. Fashler, J. Versloot, S. Lee, and K. D. Craig, Children’s behavioral pain cues: Implicit automaticity and control dimensions in observational measures, Pain Res. Manag., 2017, 2017. [29] M. A. Rahu, M. J. Grap, J. F. Cohn, C. L. Munro, D. E. Lyon, and C. N. Sessler, Facial expression as an indicator of pain in critically ill intubated adults during endotracheal suctioning, Am. J. Crit. care, 22, 5, 2013,412–422. [30] P. A. Beach, J. T. Huck, M. M. Miranda, K. T. Foley, and A. C. Bozoki, Effects of Alzheimer disease on the facial expression of pain, Clin. J. Pain, 32, 6, 2016, 478–487. [31] L. I. Strand et al. Body movements as pain indicators in older people with cognitive impairment: A systematic review, Eur. J. Pain, 23, 4, 2019,669–685, [32] S. Chow et al. Pain assessment tools for older adults with dementia in long-term care facilities: a systematic review, Neurodegener. Dis. Manag., 6, 6, 2016,525–538. [33] Fry, Margaret, Glenn Arendts, and Lynn Chenoweth,Emergency nurses' evaluation of observational pain assessment tools for older people with cognitive impairment,Journal of clinical nursing 26.9-10 (2017): 1281-1290.‏ [34] L. Cravello et al. Chronic pain in the elderly with cognitive decline: a narrative review, Pain Ther.2019, 1–13. [35] S. D. Subramaniam, B. Doss, L. D. Chanderasekar, A. Madhavan, and A. M. Rosary, Scope of physiological and behavioural pain assessment techniques in children–a review, Healthc. Technol. Lett., 5, 4,2018,124–129. [36] O. Doody and M. E. Bailey, Pain and pain assessment in people with intellectual disability: issues and challenges in practice, Br. J. Learn. Disabil., 45, 3, 2017,157–165. [37] D. J. Crellin, D. Harrison, N. Santamaria, and F. E. Babl, Systematic review of the Face, Legs, Activity, Cry and Consolability scale for assessing pain in infants and children: is it reliable, valid, and feasible for use?, Pain, 156, 11, 2015,2132–2151. [38] S. E. Thrane, S. Wanless, S. M. Cohen, and C. A. Danford, The assessment and non-pharmacologic treatment of procedural pain from infancy to school age through a developmental lens: a synthesis of evidence with recommendations, J. Pediatr. Nurs., 31, 1, 2016,e23–e32. [39] M. Mittal, A. Kumar, D. Srivastava, P. Sharma, and S. Sharma, Pain perception: computerized versus traditional local anesthesia in pediatric patients, J. Clin. Pediatr. Dent., 39, 5,2015,470–474. [40] M. Atee, K. Hoti, R. Parsons, and J. D. Hughes, Pain assessment in dementia: evaluation of a point-of-care technological solution, J. Alzheimer’s Dis., 60, 1, 2017,137–150. [41] D. Tapp et al.Observational pain assessment instruments for use with nonverbal patients at the end-of-life: a systematic review, J. Palliat. Care, 34, 4, 2019,255–266. [42] G. Zamzmi, C.-Y. Pai, D. Goldgof, R. Kasturi, Y. Sun, and T. Ashmeade, Automated pain assessment in neonates, in Scandinavian Conference on Image Analysis, 2017, pp. 350–361. [43] M. Kunz, K. Prkachin, PE. Solomon, S. Lautenbacher, Faces of clinical pain: inter‐individual facial activity patterns in shoulder pain patients, J. Pain, 25, 3,2021,529-540. [44] S. M. G. Zwakhalen, J. P. H. Hamers, H. H. Abu-Saad, and M. P. F. Berger, Pain in elderly people with severe dementia: a systematic review of behavioural pain assessment tools, BMC Geriatr., 6, 1, 2006,1–15. [45] K. Herr, P. J. Coyne, M. McCaffery, R. Manworren, and S. Merkel, Pain assessment in the patient unable to self-report: position statement with clinical practice recommendations, Pain Manag. Nurs., 12, 4, 2011,230–250. [46] A. H. van Dalen-Kok et al.Pain Assessment in Impaired Cognition (PAIC): content validity of the Dutch version of a new and universal tool to measure pain in dementia, Clin. Interv. Aging, 13, 25, 2018. [47] M. Kunz et al. The Pain Assessment in Impaired Cognition scale (PAIC15): A multidisciplinary and international approach to develop and test a meta‐tool for pain assessment in impaired cognition, especially dementia, Eur. J. Pain, 24, 1, 2020,192–208. [48] J. C. Solodiuk, Parent described pain responses in nonverbal children with intellectual disability, Int. J. Nurs. Stud., 50,. 8, 2013,1033–1044. [49] Y. Takai, N. Yamamoto-Mitani, A. Ko, and M. V Heilemann, Differences in Pain Measures by Mini-Mental State Examination Scores of Residents in Aged Care Facilities: Examining the Usability of the Abbey Pain Scale–Japanese Version, Pain Manag. Nurs., 15, 1, 2014, 236–245. [50] F. J. Symons, B. Byiers, R. Tervo, and A. Beisang, Parent reported pain in rett syndrome, Clin. J. Pain, 29, 8,2013, 744. [51] S. Gruss et al. Multi-modal signals for analyzing pain responses to thermal and electrical stimuli, J. Vis. Exp., 2019, 146, 2019, doi: 10.3791/59057. [52] E. Laures, C. LaFond, K. Hanrahan, N. Pierce, H. Min, and A. M. McCarthy, Pain assessment practices in the pediatric intensive care unit, J. Pediatr. Nurs., 48, 2019,55–62. [53] D. S. Tsze, C. L. von Baeyer, V. Pahalyants, and P. S. Dayan, Validity and reliability of the verbal numerical rating scale for children aged 4 to 17 years with acute pain, Ann. Emerg. Med., 71, 6,2018, 691–702. [54] O. Karcioglu, H. Topacoglu, O. Dikme, and O. Dikme, A systematic review of the pain scales in adults: which to use?, Am. J. Emerg. Med., 36, 4, 2018, 707–714. [55] D. Freund and B. Bolick, Assessing a Child ’ s, 119, 5. [56] Z. Alizadeh, A. Paymard, A. Khalili, and H. Hejr, “A systematic review of pain assessment method in children,” Ann. Trop. Med. Public Heal., 10, 4,2017,847. [57] E. J. Kim and M. T. Buschmann, “Reliability and validity of the Faces Pain Scale with older adults,” Int. J. Nurs. Stud., 43, 4, 2006,447–456. [58] H. M. Rostad, I. Utne, E. K. Grov, M. Puts, and L. Halvorsrud, Measurement properties, feasibility and clinical utility of the Doloplus-2 pain scale in older adults with cognitive impairment: a systematic review, BMC Geriatr., 17, 1, 2017,1–28. [59] S. Rijkenberg, W. Stilma, H. Endeman, R. J. Bosman, and H. M. Oudemans-van Straaten, Pain measurement in mechanically ventilated critically ill patients: behavioral pain scale versus critical-care pain observation tool, J. Crit. Care, 30, 1, 2015,167–172 [60] P. Lucey, J. F. Cohn, K. M. Prkachin, P. E. Solomon, and I. Matthews, Painful data: The UNBC-McMaster shoulder pain expression archive database, in 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG), 2011, 57–64. [61] M. A. Haque et al. Deep multimodal pain recognition: a database and comparison of spatio-temporal visual modalities, in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, 250–257. [62] S. Walter et al. The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system, in 2013 IEEE international conference on cybernetics (CYBCO), 2013, 128–131. [63] L. Zhang et al. BioVid Emo DB’: A multimodal database for emotion analyses validated by subjective ratings, in 2016 IEEE Symposium Series on Computational Intelligence (SSCI), 2016, 1–6. [64] P. Werner, A. Al-Hamadi, R. Niese, S. Walter, S. Gruss, and H. C. Traue, Towards pain monitoring: Facial expression, head pose, a new database, an automatic system and remaining challenges, in Proceedings of the British Machine Vision Conference, 2013, pp. 1–13. [65] X. Zhang et al.Bp4d-spontaneous: a high-resolution spontaneous 3d dynamic facial expression database, Image Vis. Comput., 32, 10, 2014,692–706. [66] Z. Zhang et al. Multimodal spontaneous emotion corpus for human behavior analysis, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 3438–3446. [67] S. Brahnam, C.-F. Chuang, F. Y. Shih, and M. R. Slack, SVM classification of neonatal facial images of pain, in International Workshop on Fuzzy Logic and Applications, 2005, 121–128. [68] D. Harrison et al.Too many crying babies: a systematic review of pain management practices during immunizations on YouTube, BMC Pediatr., 14, 1, 2014,1–8. [69] V. K. Mittal, Discriminating the infant cry sounds due to pain vs. discomfort towards assisted clinical diagnosis, in 7th Workshop on Speech and Language Processing for Assistive Technologies, SLPAT 2016, 2016, 2016, 7–42. [70] M. Velana et al. The senseemotion database: A multimodal database for the development and systematic validation of an automatic pain-and emotion-recognition system, in IAPR Workshop on Multimodal Pattern Recognition of Social Signals in Human-Computer Interaction, 2016, pp. 127–139. [71] M. S. H. Aung et al. The automatic detection of chronic pain-related expression: requirements, challenges and the multimodal EmoPain dataset, IEEE Trans. Affect. Comput., 7, 4, 2015,435–451. [72] A. Martínez López, F. A. Pujol, and H. Mora, Application of texture descriptors to facial emotion recognition in infants, Appl. Sci., 10, 3, 2020,1115. [73] M. N. Mansor and M. N. Rejab, A computational model of the infant pain impressions with Gaussian and nearest mean classifier, in 2013 IEEE International Conference on control system, computing and engineering, 2013, 249–253. [74] D. J. Jobson, Z. Rahman, and G. A. Woodell, A multiscale retinex for bridging the gap between color images and the human observation of scenes, IEEE Trans. Image Process., 6, 7, 1997,965–976. [75] G. Zamzmi, D. Goldgof, R. Kasturi, and Y. Sun,“Neonatal pain expression recognition using transfer learning, arXiv Prepr. arXiv1807.01631, 2018. [76] R. Nazari, H. S. Nia, S. P. Sharif, K. Allen, and A. Yaghoobzadeh, Which facial movements are pain indicators of patients with traumatic brain injury?, 2019. [77] C. M. A. Ilyas, M. A. Haque, M. Rehm, K. Nasrollahi, and T. B. Moeslund, Facial expression recognition for traumatic brain injured patients, in International Conference on Computer Vision Theory and Applications, 2018, 522–530. [78] N. L. Gallant and T. Hadjistavropoulos, Experiencing pain in the presence of others: A structured experimental investigation of older adults, J. Pain, 18, 4, 2017,456–467. [79] M. Kunz, D. Meixner, and S. Lautenbacher, Facial muscle movements encoding pain—a systematic review, Pain, 160, 3, 2019,535–549. [80] K. D. Craig, K. M. Prkachin, and R. E. Grunau, The facial expression of pain., 2011. [81] G. Zamzmi, R. Kasturi, D. Goldgof, R. Zhi, T. Ashmeade, and Y. Sun, A review of automated pain assessment in infants: features, classification tasks, and databases, IEEE Rev. Biomed. Eng., 11, 2017,77–96. [82] J. Yan et al. FENP: A Database of Neonatal Facial Expression for Pain Analysis, IEEE Trans. Affect. Comput., 2020. [83] M. Awais et al. Novel framework: face feature selection algorithm for neonatal facial and related attributes recognition, IEEE Access, 8, 2020,59100–59113. [84] J. Egede, M. Valstar, M. T. Torres, and D. Sharkey, .Automatic neonatal pain estimation: An acute pain in neonates database,. in 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), 2019, 1–7. [85] A. Malara et al. Pain assessment in elderly with behavioral and psychological symptoms of dementia, J. Alzheimer’s Dis., 50, 4,2016, 1217–1225. [86] A. Corbett et al. An international road map to improve pain assessment in people with impaired cognition: the development of the Pain Assessment in Impaired Cognition (PAIC) meta-tool, BMC Neurol., 14, 1, 2014, 1–14. [87] E. Kurth, H. P. Kennedy, E. Z. Stutz, A. Kesselring, I. Fornaro, and E. Spichiger, Responding to a crying infant–You do not learn it overnight: A phenomenological study, Midwifery, 30, 6, 2014,742–749. [88] S. Lautenbacher, M. Salinas-Ranneberg, O. Niebuhr, and M. Kunz, Phonetic characteristics of vocalizations during pain, Pain Manag., 9, 6, 2019,569–582. [89] J. V Pergolizzi, R. B. Raffa, A. Paladini, G. Varrasi, and J. A. LeQuang, Treating pain in patients with dementia and the possible concomitant relief of symptoms of agitation, Pain Manag., 9, 6, 2019,569–582. [90] L. M. L. Helmer et al. Crying out in pain—A systematic review into the validity of vocalization as an indicator for pain, Eur. J. Pain, 24, 9,2020,1703–1715.
Statistics Article View: 440 PDF Download: 326