Model Development and Validation for Repetition Severity Assessment in Stuttered Speech Using Clinical Speech Datasets

Abstract

Stuttering disrupts the forward flow of speech through involuntary repetitions, prolongations, and blocks, with repetitions being the most common and clinically telling of the three. Quantifying how severe those repetitions are matters for diagnosis, therapy planning, and tracking whether treatment is actually working. The catch is that severity has long been judged by ear, by trained speech-language pathologists counting disfluent events and assigning ratings on standardised scales. That process is slow, variable between clinicians, and constrained by who is available. This paper builds and tests a computational model that grades repetition severity from clinical speech recordings. The model draws on a multimodal feature set combining acoustic, prosodic, temporal, and spectral descriptors, evaluated on 480 audio samples from 60 adult speakers with persistent developmental stuttering. Two certified clinicians labelled each sample as mild, moderate, or severe, with strong inter-rater agreement. Recursive feature elimination trimmed the feature set to a compact, discriminative subset, and five classifiers were trained under stratified ten-fold cross-validation repeated five times. A Gradient Boosted Trees model reached 91.46 percent mean accuracy, a macro F1-score of 0.90, and a Cohen kappa of 0.86, ahead of Random Forest, a Support Vector Machine, a Multilayer Perceptron, and Logistic Regression. Per-class scores were strong for mild and severe cases and weaker for moderate, where acoustic characteristics overlap with both neighbouring classes. Friedman and Nemenyi tests confirmed the top model's lead was significant at the 0.05 level. The pipeline is reproducible and the results support its use as a clinical decision-support tool.