Skip to main content
SpringerLink
Log in

Transfer Learning Based Face Emotion Recognition Using Meshed Faces and Oval Cropping: A Novel Approach

  • Published:
Optical Memory and Neural Networks Aims and scope Submit manuscript

Abstract

The potential applications of emotion recognition from facial expressions have generated considerable interest across multiple domains, encompassing areas such as human-computer interaction, camera and mental health analysis. In this article, a novel approach has been proposed for face emotion recognition (FER) using several data preprocessing and Feature extraction steps such as Face Mesh, data augmentation and oval cropping of the faces. A transfer learning using VGG19 architecture and a Deep Convolution Neural Network (DCNN) have been proposed. We demonstrate the effectiveness of the proposed approach through extensive experiments on the Cohn-Kanade+ (CK+) dataset, comparing it with existing state-of-the-art methods. An accuracy of 99.79% was found using the VGG19. Finally, a set of images collected from an AI tool that generates images based on textual description have been done and tested using our model. The results indicate that the solution achieves superior performance, offering a promising solution for accurate and real-time face emotion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.

REFERENCES

  1. Keltner, D., Sauter, D., Tracy, J., and Cowen, A., Emotional expression: Advances in basic emotion theory, J. Nonverbal. Behav., 2019, vol. 43, no. 2, pp. 133–160.

    Article  Google Scholar 

  2. Jasuja, A. and Rathee, S., Emotion recognition using facial expressions, Int. J. Inf. Retrosp. Res., 2021, vol. 11, no. 3, pp. 1–17.

    Google Scholar 

  3. Kumari, A., Tanwar, S., Tyagi, S., and Kumar, N., Fog computing for Healthcare 4.0 environment: Opportunities and challenges, Comput. Electr. Eng., 2018, vol. 72, pp. 1–13.

    Article  Google Scholar 

  4. Zhang, L., Verma, B., Tjondronegoro, D., and Chandran, V., Facial expression analysis under partial occlusion: A survey, ACM Comput. Surv., 2018, vol. 51, no. 2, pp. 1–49.

    Article  Google Scholar 

  5. Ko, B.C., A brief review of facial emotion recognition based on visual information, Sensors (Switzerland), 2018, vol. 18, no. 2, p. 401.

    Article  Google Scholar 

  6. Canedo, D. and Neves, A.J.R., Facial expression recognition using computer vision: A systematic review, Appl. Sci., 2019, vol. 9, no. 21, p. 4678.

    Article  Google Scholar 

  7. Dagher, I., Dahdah, E., and Al Shakik, M., Facial expression recognition using three-stage support vector machines, Vis. Comput. Ind. Biomed. Art, 2019, vol. 2, no. 1.

  8. Dino, H.I. and Abdulrazzaq, M.B., Facial expression classification based on SVM, KNN and MLP classifiers, in 2019 International Conference on Advanced Science and Engineering, ICOASE 2019, Zakho – Duhok, 2019, pp. 70–75.

  9. Boughida, A., Kouahla, M.N., and Lafifi, Y., A novel approach for facial expression recognition based on Gabor filters and genetic algorithm, Evol. Syst., 2022, vol. 13, no. 2, pp. 331–345.

    Article  Google Scholar 

  10. Murugappan, M., Mutawa, A.M., Sruthi, S., Hassouneh, A., Abdulsalam, A., Jerritta, S., and Ranjana, R., Facial expression classification using KNN and decision tree classifiers, in 4th International Conference on Computer, Communication and Signal Processing, ICCCSP 2020, Chennai, 2020, pp. 1–6.

  11. Sokolova, A.D. and Savchenko, A.V., Computation-efficient face recognition algorithm using a sequential analysis of high dimensional neural-net features, Opt. Mem. Neural Networks, 2020, vol. 29, no. 1, pp. 19–29.

    Article  Google Scholar 

  12. Varma, S., Shinde, M., and Chavan, S.S., Analysis of PCA and LDA features for facial expression recognition using SVM and HMM classifiers, in Techno-Societal 2018, Maharashtra, 2020, pp. 109–119.

  13. Fallahzadeh, M.R., Farokhi, F., Harimi, A., and Sabbaghi-Nadooshan, R., Facial expression recognition based on image gradient and Deep Convolutional Neural Network, J. AI Data Min., 2021, vol. 9, no. 2, pp. 259–268.

    Google Scholar 

  14. Cai, J., Meng, Z., Khan, A.S., Li, Z., Oreilly, J., and Tong, Y., Island loss for learning discriminative features in facial expression recognition, in Proceedings – 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018, Xi’an, 2018, pp. 302–309.

  15. Kusuma, G.P., Jonathan, A., and Lim, P., Emotion recognition on FER-2013 face images using fine-tuned VGG-16, Adv. Sci. Technol. Eng. Syst., 2020, vol. 5, pp. 315–322.

    Article  Google Scholar 

  16. Zheng, H., Wang, R., Ji, W., Zong, M., Wong, W.K., Lai, Z., and Lv, H., Discriminative deep multi-task learning for facial expression recognition, Inf. Sci. (Ny), 2020, vol. 533, pp. 60–71.

    Article  Google Scholar 

  17. Mukhopadhyay, M., Dey, A., and Kahali, S., A deep-learning-based facial expression recognition method using textural features, Neural Comput. Appl., 2023, vol. 35, pp. 6499–6514.

    Article  Google Scholar 

  18. Chowdary, M.K., Nguyen, T.N., and Hemanth, D.J., Deep learning-based facial emotion recognition for human–computer interaction applications, Neural Comput. Appl., 2023, vol. 35, pp. 23311–23328.

    Article  Google Scholar 

  19. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I., The extended cohn-kande dataset (CK+): A complete facial expression dataset for action unit and emotion specified expression, in Proceedings of the Third International Workshop on CVPR for Human Communicative Behavior Analysis (CVPR4HB 2010), San Francisco, 2010, pp. 94–101.

  20. MediaPipe Library. https://developers.google.com/mediapipe.

  21. Handa, A., Agarwal, R., and Kohli, N., Incremental approach for multi-modal face expression recognition system using deep neural networks, Int. J. Comput. Vis. Robot., 2021, vol. 11, no. 1, pp. 1–20.

    Article  Google Scholar 

  22. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R., Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res., 2014, vol. 15, no. 56, pp. 1929–1958.

    MathSciNet  Google Scholar 

  23. Nelsoncruz, 3D_Image_Generator, 2024.

Download references

Funding

This work was supported by ongoing institutional funding. No additional grants to carry out or direct this particular research were obtained.

Author information

Authors and Affiliations

  1. Laboratory of Process Engineering, Computer Science and Mathematics (LIPIM), National School of Applied Science—USMS (ENSA-KH), 25000, Khouribga, Morocco

    Ennaji Fatima Zohra

  2. SMARTE Systems and Applications (SSA), National School of Applied Science—UCA (ENSA-M), 40000, Marrakesh, Morocco

    El Kabtane Hamada

Authors
  1. Ennaji Fatima Zohra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. El Kabtane Hamada
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ennaji Fatima Zohra or El Kabtane Hamada.

Ethics declarations

The authors of this work declare that they have no conflicts of interest.

Additional information

Publisher’s Note.

Allerton Press remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ennaji Fatima Zohra, El Kabtane Hamada Transfer Learning Based Face Emotion Recognition Using Meshed Faces and Oval Cropping: A Novel Approach. Opt. Mem. Neural Networks 33, 178–192 (2024). https://doi.org/10.3103/S1060992X24700073

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S1060992X24700073

Keywords:

Search

Navigation