Research on Cognitive Augmented Real-Time Decision-Making Technology Based on Cross-Modal Biological Signal Fusion

DOI: https://doi.org/10.62517/jbdc.202601218

Author(s)

Jiayi Shen, Luyan Shen, Sitong Ruan, Zhihui Xu, Jingyi Xu*

Affiliation(s)

Artificial Intelligence College, Zhejiang Dongfang Polytechnic, Wenzhou, Zhejiang, China *Corresponding Author

Abstract

Aiming at the industrial pain points in the current rehabilitation medical field, including the shortage of therapists, the lack of real-time professional guidance for patients during independent home training, non-standard training movements, and inconsistent rehabilitation effects, this paper proposes a cognitive augmented real-time decision-making technology based on cross-modal biological signal fusion. Relying on VR intelligent rehabilitation equipment, this technology integrates three heterogeneous biological and motion perception signals, namely IMU inertial sensing, eye tracking, and brain-computer interface signals, to construct a multi-dimensional and high-precision patient rehabilitation training environment and state perception system. Based on massive standardized patient rehabilitation training datasets provided by hospitals, special fine-tuning and training of AI large models are completed, endowing the models with full-process intelligent capabilities including rehabilitation movement recognition, error deviation judgment, real-time correction guidance, and training effect evaluation, thereby forming a mature real-time intelligent feedback, error correction and evaluation function. Experimental and application results show that the proposed technology can accurately identify movement deviations, state abnormalities and insufficient cognitive cooperation of patients during rehabilitation training without on-site guidance from professional therapists, and output standardized correction decision instructions in real time. It significantly improves the accuracy and standardization of patients’ independent training, effectively solves the problem of unbalanced supply and demand of rehabilitation medical resources, and provides a new technical support for intelligent, inclusive and normalized home-based rehabilitation and out-of-hospital continuous rehabilitation.

Keywords

Cross-Modal Signal Fusion; Biological Signal Perception; AI Large Model; Real-Time Intelligent Error Correction; Rehabilitation Training; Cognitive Augmentation

References

[1] Liu C L, Jin Y R, Wang M X. Research on Intelligent Health Perception Technology Based on Multi-Modal Physiological Signal Fusion. Information Fusion, 2026(02):112-120. [2] Wu K, et al. BrainFusion: Standardization Framework of Multi-Modal Biological Signals and Its Application in Intelligent Rehabilitation. Advanced Science, 2025(05):89-97. [3] Zhang M, Li R. Multi-Modal Physiological Signal Feature Fusion Algorithm Based on Attention Mechanism. Computer Engineering and Applications, 2025(18):134-141. [4] Wang H Y, Chen X. Research Progress of AI-Driven Real-Time Feedback Technology for Intelligent Rehabilitation. Chinese Medical Equipment Journal, 2024(09):45-50. [5] Li Y T, et al. Review of Cross-Modal Biosensing and AI Decision-Making in Rehabilitation Medical Applications. Journal of Biomedical Engineering, 2026(01):210-218. [6] Zhang Yangzhou, Jiang Ziyi, Shi Na. Research on the Long-term Mechanism of the “Five Integrations” Textbook Construction in Higher Vocational Colleges in the New Era. Journal of Zhejiang Industry & Trade Vocational College, 2023(03): 47-51. [7] Liu Fen, Li Yajing. Current Situation, Direction and Strategies of the Construction of New “Paper-Digital Integration” Textbooks in Vocational Education. Journal of Hubei Open Vocational College, 2024, 37(22): 52-54. [8] Ji Zhaohua. Exploration and Practice of Digital Textbook Development for Computer Specialty Courses in Vocational Education. E-Learning, 2024, (12): 72-74. [9] Kong Ling, Yuan Lihui. Research on the Construction of Digital Textbooks for Computer Networks in Application-oriented Undergraduate Education. High-Technology & Industrialization, 2024, 30(09): 124-125. [10]Liu Chao, Huang Ronghuai, Wang Hongyu. Exploration on the Construction and Application Path of New Textbooks Based on Knowledge Graph. China University Teaching, 2023(8): 10-16.