Recently, Huawei issued the patent of “Traffic Distress and Road Rage Detection Method”. The patent provides a system for determining traffic distress of a driver of a vehicle, including multiple sensors: an in-vehicle image sensor, an in-vehicle audio sensor, a vehicle data sensor, and a global positioning system (GPS) data sensor; also includes a or multiple processors to receive input from multiple sensors and process the received input to obtain driver thermal change estimates, driver expression estimates, driver posture estimates, on-board diagnostic system estimates, and GPS estimates.

Road rage patients, also known as “aggressive drivers”, are defined by NHTSA as: “A behavior caused by the driver’s personal misconduct that leads to vehicle safety accidents and casualties.” According to A survey of 900 drivers showed that 35% of drivers had “road rage”. According to another survey, 80% of Chinese drivers suffer from road rage to varying degrees, and nearly 20% of traffic accidents are caused by road rage.

Previous road rage detection systems for vehicle drivers have focused on invasive systems such as blood pressure and heart rate monitoring, as well as non-invasive systems that primarily use images and audio recordings. Huawei has designed a more complete solution, which uses cameras and video and audio analysis to monitor the driver’s various modes in real time (such as facial expressions, gestures, vehicle speed, etc.) to detect changes in the driver’s temper.

In the scheme, there are many means to detect drivers, including but not limited to thermal imaging of drivers, voice, visual information and other modalities, such as driving behavior and gestures, which are used to monitor drivers or passengers to detect traffic distress and road rage. And send notifications when traffic distress or road rage is detected.

Due to the characteristics of each driver’s personality and driving state, the manifestation of road rage also varies from person to person. Huawei engineers first grade road rage and assess the risk of drivers’ escalating angry behaviors. According to Huawei’s standards, road rage can be divided into four levels.

Level 1, when a driver is annoyed by someone, they will often initiate non-threatening gestures or facial expressions to show annoyance. At the second level, angry drivers show dissatisfaction by aggravating the situation by honking the horn, flashing lights, maliciously braking, tailgating, blocking the car, etc. At level three, aggressive drivers may curse, yell and threaten other drivers. At level four, the driver may hit the vehicle with an object, chase the vehicle, or run off the road.

Then, according to a multimodal approach (eg, multiple data streams of images, audio, vehicle data, etc.), each modality can be used to detect features that help the system understand the driver’s level of traffic distress and road rage. There are many indicators of the degree of traffic distress and road rage of drivers, mainly including:

● Identify the driver and passengers in the vehicle;

● Identify the hand and face of the driver and passenger, track the hand and face posture, thermal state;

● Identify the state of the environment, such as traffic conditions, objects near the vehicle, sounds around the vehicle (eg, other vehicles honking), road conditions, and speed limits;

● Driving behavior performance data, such as acceleration, speed, steering angle and other embedded sensor data;

The above indicators, status and data are combined to determine whether the driver is irritated or in danger. After summarizing all kinds of data, deep learning is carried out so that the system can adapt to and understand the ways in which different drivers may show anger and frustration expressions, and determine how to issue warnings and driving assistance to meet the driver’s preferences.