A Hybrid Multimodal AI Framework for BI-RADS Breast Cancer Risk Prediction and Report Generation: Radiologist-Mimetic Cognitive Knowledge Graph Injection and Cognitive Prompt Engineering

Abstract

Automated BI-RADS breast cancer risk stratification from multimodal clinical data remains limited by two persistent gaps: the inability of purely discriminative architectures to resolve fine-grained BI-RADS 4 subcategory boundaries with calibrated uncertainty, and the absence of a mechanism to translate multimodal evidence into reasoning-transparent structured radiology reports. We propose a hybrid discriminative-generative framework integrating deep learning encoders for mammography, ultrasound, and DCE-MRI; XGBoost-Random Forest ensembles for structured clinical, pathological, and environmental data; and ClinicalBERT for free-text narratives, unified through Transformer cross-modal attention. Two novelties ground the framework in radiologist cognitive reasoning: the Radiologist-Mimetic Cognitive Knowledge Graph (RM-CKG), an OWL 2 DL formally specified, three-round expert-validated ontology encoded by a Graph Attention Network and injected as a patient-specific cognitive prior at the fusion layer; and Radiologist-Mimetic Cognitive Prompting (RMCP), which serializes the RM-CKG subgraph traversal into a five-layer reasoning narrative conditioning a LoRA fine-tuned Vision-Language Model for structured report generation. Evaluated on 11,847 patients, the framework achieves a macro-averaged AUC of 0.931 and QWK of 0.836, with the RM-CKG producing its largest isolable gain at the BI-RADS 4A/4B subcategory boundary (ΔAUC = +0.117); the RMCP-conditioned reports achieve a blinded radiologist Clinical Accuracy Score of 4.31/5.00 and a BI-RADS Consistency Rate of 0.962.