A comparative annotator-agreement analysis of EMOLA (Thai) and IEMOCAP (English) emotional speech corpora

Name: Piyawat Sukhummek

keyword: Annotator-agreement analysis

; IEMOCAP corpus

; EMOLA corpus

; HMM-based emotion recognition

; Agreement-reliability assessment

Abstract: Understanding of human emotions is one of the essential problems in the cognitive science because emotion recognition technologies can improve the communication between humans and machines. In the process of developing these technologies, emotional utterance corpora are required to construct and validate models used for recognizing emotions. However, evaluation of an emotion recognition model strongly depends on the corpus used in the evaluation. Specifically, the number of emotionally ambiguous utterances in corpora strongly affects evaluation result of the emotion recognition models. Therefore, this work aims to propose an efficient method for removing those emotionally ambiguous utterances from a corpus such that the refined corpus can be used to construct and evaluate an emotion recognition model effectively. We propose three methods for removing or filtering out ambiguous utterances: the filtering based on the first label preference and majority vote, the filtering based on full consensus, and the filtering based on the first label preference and full consensus. We investigate two corpora, which are Interactive Emotional Dyadic Motion Capture Database (IEMOCAP) and Emotional Tagged Corpus on Lakorn (EMOLA). The first corpus is an English language corpus whereas the second one is a Thai language corpus, and both are annotated by six annotators. We primarily study only four emotions, which are anger, happiness, neutral, and sadness. The experimental results show that, once the emotionally ambiguous utterances are removed from a corpus by the proposed methods, and then the corpora are used in training and testing emotion recognition models, the accuracy results improve considerably compared with those of emotion recognition models trained and tested by the original corpora. In the best case, the accuracy improves by 37.47 percents. Although we can achieve the better performance via a reduced corpus using information from emotion labels assigned by annotators, there is still room for improvement. For example, some features of utterances can be combined with information from annotators to remove more emotionally ambiguous utterances from a corpus

Thammasat University. Thammasat University Library

Address: BANGKOK

Email: preserv@tu.ac.th

Name: Thanaruk Theeramunkong

Role: advisor

Name: Chai Wutiwiwatchai

Role: co-advisor

Created: 2016

Modified: 2564-05-19

Issued: 2021-05-19

วิทยานิพนธ์/Thesis

application/pdf

DOI: 10.14457/TU.the.2016.669

eng

DegreeName: Master of Engineering

Level: Master's Degree

Descipline: Information and Communication Technology for Embedded Systems

Grantor: Thammasat University

©copyrights Thammasat University

ลำดับที่.	ชื่อแฟ้มข้อมูล	ขนาดแฟ้มข้อมูล	จำนวนเข้าถึง	วัน-เวลาเข้าถึงล่าสุด
1	2411piyawat.pdf	2.04 MB	1	2022-12-12 20:49:02

ใช้เวลา

0.033834 วินาที