Sirapob Sangnark. Multi-modal deep learning for diagnosis of dyssynergic defecation using abdominal X-ray images and symptom questionnaires. Master's Degree(Computer Engineering). Chulalongkorn University. Office of Academic Resources. : Chulalongkorn University, 2569.
Multi-modal deep learning for diagnosis of dyssynergic defecation using abdominal X-ray images and symptom questionnaires
Abstract:
Dyssynergic defecation (DD) is a specific type of chronic constipation necessitating a specialized diagnostic test. However, these tests are exclusively available in tertiary care settings due to the unavailability of the required devices in primary and secondary care settings. In this thesis, we present a multi-modal deep learning model that can preliminary diagnose patients who suffer from DD. This model utilizes cost-effective and accessible data in small hospitals, such as abdominal X-ray images and symptom questionnaires. Cross-modal attention is applied to assist the multi-modal model in identifying significant relationships between the two types of data. A convolution block attention module (CBAM) and masking augmentation are employed to capture crucial semantic and spatial features from the X-ray images. Furthermore, gradient-weighted class activation mapping (Grad-CAM) is used to highlight important parts in the images, and deep shapley additive explanation (DeepSHAP) is used to explain the impact of symptom data. In our experiment, we compared the performance of the multi-modal model against single-modal models and human experts. The results indicate that our multi-modal model outperforms both the single-modal models and human experts, achieving an 82.17% F1-score and 82.27% accuracy. Thus, this model can effectively support human experts in diagnosing DD.