Abstract:With the development and popularity of smartphone products, a large number of bowed-head tribes have emerged who play mobile phones at any time regardless of the occasion; for the frequent occurrence of traffic accidents caused by bowed-head tribes′ dependence on mobile phones, a multimodal bowed-head tribes′ hazard perception and warning system based on mobile phones is proposed. First, gravity acceleration on the mobile phone side is used to monitor behaviors in real time based on fuzzy control rules, including Walking and looking at the mobile phone, Walking up and down stairs, Looking at the mobile phone at rest, Walking with the mobile phone in hand, Walking with the mobile phone in pocket; and then the user′s environment is described in real time using the mobile phone′s rear view camera images based on the grouping of fast spatial pyramids pooled in the lightweight YOLO network, including: stairs, crosswalks, low-light environments, puddles, and normal road surfaces. Finally, a state-environment-multimodal hazard detection model is constructed for the Android system; and based on the detection results, audible, visual, and tactile three-dimensional warning signals are given to the bowed tribe by using sound, image, and vibration signals to reduce the potential hazards of the bowed tribe such as fall injury and collision. Online experiments show that the proposed multimodal threat perception model for mobile phones is highly accurate, robust, and real-time, and is able to achieve effective proactive warning for the common threat states of bowed heads.