Citation :

P.R. Ajitha, 2023. "YOL-SFV2: An Effective Deep Learning Technique to Detect and Classify the Human Face Action in Thermal Images" ESP International Journal of Advancements in Computational Technology (ESP-IJACT)  Volume 1, Issue 2: 13-28.

Abstract :

Facial expression recognition (FER), a computer vision problem, tries to identify and classify the many expressions of emotion that can be detected on a person's face. One of the largest challenges to face recognizing and identification is the extraordinary variety of human faces in terms of size, shape, position, illumination, expression, and occlusion. In this research, propose a novel deep learning technique to detect and classify the human face expression in thermal images. Initially, remove noise from the input image using median filter, and then normalize the data using min-max normalization method to improve the performance. Next, use the Improved Principal Component Algorithm (IPCA) to extract the pertinent features, such as texture, shape, and location. Then, using YOLOv8 technique to detect the face expression by extracted feature. Finally, employ a new deep learning technique of ShuffleNetV2 to classify the actions into their categories. To improve the classification performance, employs the Enhanced Golden Jackal Optimization algorithm. The performance of propose methodology is evaluated on publicly available datasets are Terravic facial IR database and IRIS Thermal/Visible Face Database. State-of-the-art methodologies were used to compare the performance of the proposed approach, and the proposed approach produced higher categorization accuracy while using less processing time.

References :

[1] Prabhu, K., Kumar, S. S., Sivachitra, M., Dineshkumar, S., & Sathiyabama, P. (2022). Facial Expression Recognition Using Enhanced Convolution Neural Network with Attention Mechanism. Computer Systems Science & Engineering, 41(1).

[2] Wang, K., Lian, Z., Sun, L., Liu, B., Tao, J., & Fan, Y. (2022, October). Emotional reaction analysis based on multi-label graph convolutional networks and dynamic facial expression recognition transformer. In Proceedings of the 3rd International on Multimodal Sentiment Analysis Workshop and Challenge (pp. 75-80).

[3] Jeong, J. Y., Hong, Y. G., Kim, D., Jung, Y., & Jeong, J. W. (2022). Facial expression recognition based on multi-head cross attention network. arXiv preprint arXiv:2203.13235.

[4] Savchenko, A. V., Savchenko, L. V., & Makarov, I. (2022). Classifying emotions and engagement in online learning based on a single facial expression recognition neural network. IEEE Transactions on Affective Computing, 13(4), 2132-2143.

[5] Wang, Y., Sun, Y., Huang, Y., Liu, Z., Gao, S., Zhang, W., ... & Zhang, W. (2022). Ferv39k: A large-scale multi-scene dataset for facial expression recognition in videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 20922-20931).

[6] Sadeghi, H., & Raie, A. A. (2022). HistNet: Histogram-based convolutional neural network with Chi-squared deep metric learning for facial expression recognition. Information Sciences, 608, 472-488.

[7] Ruan, D., Mo, R., Yan, Y., Chen, S., Xue, J. H., & Wang, H. (2022). Adaptive deep disturbance-disentangled learning for facial expression recognition. International Journal of Computer Vision, 130(2), 455-477.

[8] Moung, E. G., Wooi, C. C., Sufian, M. M., On, C. K., & Dargham, J. A. (2022). Ensemble-based face expression recognition approach for image sentiment analysis. Int. J. Electr. Comput. Eng, 12(3), 2588-2600.

[9] Ma, T., Tian, W., & Xie, Y. (2022). Multi-level knowledge distillation for low-resolution object detection and facial expression recognition. Knowledge-Based Systems, 240, 108136.

[10] Bodapati, J. D., Srilakshmi, U., & Veeranjaneyulu, N. (2022). FERNet: a deep CNN architecture for facial expression recognition in the wild. Journal of The institution of engineers (India): series B, 103(2), 439-448.

[11] Zhu, Q., Mao, Q., Jia, H., Noi, O. E. N., & Tu, J. (2022). Convolutional relation network for facial expression recognition in the wild with few-shot learning. Expert Systems with Applications, 189, 116046.

[12] Nan, F., Jing, W., Tian, F., Zhang, J., Chao, K. M., Hong, Z., & Zheng, Q. (2022). Feature super-resolution based Facial Expression Recognition for multi-scale low-resolution images. Knowledge-Based Systems, 236, 107678.

[13] Gan, C., Xiao, J., Wang, Z., Zhang, Z., & Zhu, Q. (2022). Facial expression recognition using densely connected convolutional neural network and hierarchical spatial attention. Image and Vision Computing, 117, 104342.

[14] Liu, Y., Wang, W., Feng, C., Zhang, H., Chen, Z., & Zhan, Y. (2023). Expression snippet transformer for robust video-based facial expression recognition. Pattern Recognition, 138, 109368.

[15] Zou, W., Zhang, D., & Lee, D. J. (2022). A new multi-feature fusion based convolutional neural network for facial expression recognition. Applied Intelligence, 52(3), 2918-2929.

[16] Nan, Y., Ju, J., Hua, Q., Zhang, H., & Wang, B. (2022). A-MobileNet: An approach of facial expression recognition. Alexandria Engineering Journal, 61(6), 4435-4444.

[17] Boughida, A., Kouahla, M. N., & Lafifi, Y. (2022). A novel approach for facial expression recognition based on Gabor filters and genetic algorithm. Evolving Systems, 13(2), 331-345.

[18] Litvin, A., Nasrollahi, K., Escalera, S., Ozcinar, C., Moeslund, T. B., & Anbarjafari, G. (2019). A novel deep network architecture for reconstructing RGB facial images from thermal for face recognition. Multimedia Tools and Applications, 78, 25259-25271.

[19] Nayak, S., Nagesh, B., Routray, A., & Sarma, M. (2021). A Human–Computer Interaction framework for emotion recognition through time-series thermal video sequences. Computers & Electrical Engineering, 93, 107280.

[20] Said, Y., & Barr, M. (2021). Human emotion recognition based on facial expressions via deep learning on high-resolution images. Multimedia Tools and Applications, 80(16), 25241-25253.

[21] Zhu, Y., Zhu, C., & Li, X. (2018). Improved principal component analysis and linear regression classification for face recognition. Signal Processing, 145, 175-182.

[22] Talaat, F. M., & ZainEldin, H. (2023). An improved fire detection approach based on YOLO-v8 for smart cities. Neural Computing and Applications, 1-16.

[23] Yang, R., Lu, X., Huang, J., Zhou, J., Jiao, J., Liu, Y., ... & Gu, P. (2021). A multi-source data fusion decision-making method for disease and pest detection of grape foliage based on ShuffleNet V2. Remote Sensing, 13(24), 5102.

[24] Zhang, J., Zhang, G., Kong, M., & Zhang, T. (2023). Adaptive infinite impulse response system identification using an enhanced golden jackal optimization. The Journal of Supercomputing, 1-26.

Keywords :

Face Expression, Thermal Images, Shufflenetv2, Deep Learning, Classification, YOLOv8, Detection.