Post-Model Fusion of Speech, Drawing, and Movement Data to Classify Parkinson’s Disease

Abstract

Parkinson’s disease is one of the most common movement disorders that is not curable according to recent clinical knowledge. An accurate and early diagnosis can help to maintain the quality of life. Since the symptoms at the early stage are heterogeneous, the detection is challenging. To solve this, several modalities have been investigated to support the recognition of the disease. Speech, drawings, and specific movements are studied, and promising performances are shown to assist the doctors. The joint usage of these modalities can further decrease the misclassification of the disease. In this study, acceleration data from 6 movements and X-Y coordinates from Archimedean spiral drawings were processed in image and time-series representations. MobileNet and time-series-based features were used to describe them. Speech was also processed by x-vector technology. Support Vector Machine, Random Forest, and k-nearest Neighbors algorithms were trained and tested to classify 33 Parkinson’s disease patients and 47 healthy controls. Next to the single modalities, post-model fusions were examined with different combinations of the modalities. Mann-Whitney U test was used to compare the performances of the models next to a 0.05 significance level. The speech significantly outperformed the drawing and movement activities. Furthermore, any combination of the modalities resulted in significantly better balanced accuracy than using movements and drawings alone. However, the speech achieved a significantly not different performance than any combination of the modalities. In conclusion, the combination of drawing and movements improves the detection of the disease. Speech gives similar results to the combination of the other two modalities.