Scalable Non-Contact Stress Detection Using Hybrid Multimodal Intelligence

Abstract

Scalable digital health and human-computer interaction systems require non-contact stress detection methods, where wearable sensors and self-reports are impractical. This paper introduces a new reliability-aware hybrid multimodal stress detection scheme for stress detection based on speech characteristics and multimodal cues such as facial cues, eye/pupil features, keyboard dynamics, and handwriting behavior. The model includes encoders specialized for each modality, temporal learning and adaptive reliability-aware fusion to mitigate noisy or missing modalities. TExperiments conducted in a multimodal setting derived from ForDigitStress achieved 94.30% accuracy, 94.00% precision, 93.20%  recall,  93.60%  F1-score,  0.967  AUROC,  0.036 calibration error and 38.40 ms inference latency per window. Results show strong, interpretable and scalable stress detection in real-world digital environments.