What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature review
Emotional engagement is essential in human communication, and the meaning of emotions often entails multimodal relationships. Besides language, multimodality and emotions are semiotic systems that have increasingly attracted the attention of researchers, especially concerning their contribution to t...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2024-12-01
|
| Series: | Cogent Social Sciences |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/23311886.2024.2376309 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1839645422491009024 |
|---|---|
| author | Reny Rahmalina Wawan Gunawan |
| author_facet | Reny Rahmalina Wawan Gunawan |
| author_sort | Reny Rahmalina |
| collection | DOAJ |
| description | Emotional engagement is essential in human communication, and the meaning of emotions often entails multimodal relationships. Besides language, multimodality and emotions are semiotic systems that have increasingly attracted the attention of researchers, especially concerning their contribution to the meaning-making process in communication. However, research trends that address how emotions and multimodal components (hereafter multimodal emotions) complement each other in the meaning-making process have not been extensively researched. Hence, this research aims to identify research trends on multimodality and emotions studied from 2018 to 2022. The method used in this research was systematic literature review (SLR) to review, identify, evaluate and interpret all existing research on this topic. Data were obtained from IEEE Xplore, Science Direct and Emerald journals which were then sorted based on the PRISMA method. This analysis acquired comprehensive information on the multimodal components of emotions, data acquisition technologies, datasets and emotional focus and trends. This SLR demonstrated that the research of emotions involving multimodality was conducted with a focus on audio-visual mode with the use of machines namely EEG and LSTM sourced from the IEMOCAP dataset and focused on positive and negative emotions, with anger being the largest focus. Moreover, the findings also showed the interconnectedness of each multimodal component of emotions. Based on these findings, this study suggests that multimodal emotion research should focus on identifying and investigating the meanings generated from emotions to produce good communication. Lack of subject privacy and unmanageable bias in research are other things to consider for future research directions. |
| format | Article |
| id | doaj-art-c38ba813a1b24d079d596ebb3cdae074 |
| institution | Matheson Library |
| issn | 2331-1886 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Cogent Social Sciences |
| spelling | doaj-art-c38ba813a1b24d079d596ebb3cdae0742025-07-01T17:33:09ZengTaylor & Francis GroupCogent Social Sciences2331-18862024-12-0110110.1080/23311886.2024.2376309What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature reviewReny Rahmalina0Wawan Gunawan1Linguistics Study Program, Postgraduate School, Universitas Pendidikan Indonesia, Bandung, IndonesiaLinguistics Study Program, Postgraduate School, Universitas Pendidikan Indonesia, Bandung, IndonesiaEmotional engagement is essential in human communication, and the meaning of emotions often entails multimodal relationships. Besides language, multimodality and emotions are semiotic systems that have increasingly attracted the attention of researchers, especially concerning their contribution to the meaning-making process in communication. However, research trends that address how emotions and multimodal components (hereafter multimodal emotions) complement each other in the meaning-making process have not been extensively researched. Hence, this research aims to identify research trends on multimodality and emotions studied from 2018 to 2022. The method used in this research was systematic literature review (SLR) to review, identify, evaluate and interpret all existing research on this topic. Data were obtained from IEEE Xplore, Science Direct and Emerald journals which were then sorted based on the PRISMA method. This analysis acquired comprehensive information on the multimodal components of emotions, data acquisition technologies, datasets and emotional focus and trends. This SLR demonstrated that the research of emotions involving multimodality was conducted with a focus on audio-visual mode with the use of machines namely EEG and LSTM sourced from the IEMOCAP dataset and focused on positive and negative emotions, with anger being the largest focus. Moreover, the findings also showed the interconnectedness of each multimodal component of emotions. Based on these findings, this study suggests that multimodal emotion research should focus on identifying and investigating the meanings generated from emotions to produce good communication. Lack of subject privacy and unmanageable bias in research are other things to consider for future research directions.https://www.tandfonline.com/doi/10.1080/23311886.2024.2376309Multimodalitymultimodal emotionsemotionssystematic literature reviewGuangchao Charles Feng, School of Communication, Hong Kong Baptist University, Hong KongInterpersonal Communication |
| spellingShingle | Reny Rahmalina Wawan Gunawan What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature review Cogent Social Sciences Multimodality multimodal emotions emotions systematic literature review Guangchao Charles Feng, School of Communication, Hong Kong Baptist University, Hong Kong Interpersonal Communication |
| title | What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature review |
| title_full | What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature review |
| title_fullStr | What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature review |
| title_full_unstemmed | What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature review |
| title_short | What multimodal components, tools, dataset and focus of emotion are used in the current research of multimodal emotion: a systematic literature review |
| title_sort | what multimodal components tools dataset and focus of emotion are used in the current research of multimodal emotion a systematic literature review |
| topic | Multimodality multimodal emotions emotions systematic literature review Guangchao Charles Feng, School of Communication, Hong Kong Baptist University, Hong Kong Interpersonal Communication |
| url | https://www.tandfonline.com/doi/10.1080/23311886.2024.2376309 |
| work_keys_str_mv | AT renyrahmalina whatmultimodalcomponentstoolsdatasetandfocusofemotionareusedinthecurrentresearchofmultimodalemotionasystematicliteraturereview AT wawangunawan whatmultimodalcomponentstoolsdatasetandfocusofemotionareusedinthecurrentresearchofmultimodalemotionasystematicliteraturereview |