Non-contact, non-intrusive machine vision-based in-vehicle distraction sensor (mDrive)

According to EU data [1], road safety is a major societal issue. In 2011, more than 30,000 people died on EU roads. One of the major problems in modern vehicles is driver distraction - drivers need to keep their attention focused on surrounding traffic at all times. The problem of driver distraction is well studied issue in road safety [2]. However, increasing efforts to mitigate the consequences of distraction, such as Automatic Emergency Braking (AEB) and Forward Collision Warning (FCW) may even lead to behavioural adaptation of the drivers. Additionally, despite widely known restrictions on using mobile phones while driving, the effects are doubtful - reportedly an outage of Blackberry services caused observable drop in traffic accidents and fatalities [2].

Driver distraction is a very complex phenomenon and difficult to define unambiguously. Mobile phones and satnavs are major sources of distraction. Smoking, eating and operating the radio or CD player are also common distractions, but this list is far from complete. However, it can be observed that here are three basic types of distraction [1]:

  1. Visual – things that make drivers take their eyes off the road. This can be addressed only by observing the direction of driver’s gaze
  2. cognitive – causing drivers to think about other things, and it is very difficult to detect objectively.
  3. manual – things that cause drivers take their hands off the wheel. This can be easily detected, provided that one can detect whether driver’s hands are at correct position at the wheel

Often, a driver is distracted by a combination of the three. In the TTP, we rely on this fact to detect driver’s inattention or distraction.

We observe manual distraction, which is observable within current technological limits, to detect events that cause the driver to take its hands of the wheel or holds them in position inconsistent with the rules of safe driving. Given that a distraction often comes as a combination of multiple factors which are extremely varied and therefore difficult to model individually, we use the hand position as a proxy for distraction and/or driver inattention.

Machine Vision Laboratory at Faculty of Electrical Engineering at University of Ljubljana has developed a set of algorithms and a lab prototype of a system for detecting the position of driver’s hands on the steering wheel, based on problem specification by the industrial partner.

The hardware platform consists of inexpensive, mass-produced and widely used Linux hardware platform as a processing module, and accessory hardware to capture the necessary video data. The system consists of processing board using quad-core ARM processor with integrated GPU, camera and compatible illumination, GPS module to measure velocity and turns of the vehicle, communication subsystem, and control module to adapt the system to the rough automotive environment. Software consists of specially developed image processing/machine vision software to process acquired data in real time, and cloud component to aggregate and present data to the vehicle owner.

Figure 1: 3rd prototype system board (cover and some components are not shown), side view System is to be mounted to vehicle ceiling.

TTP tasks:  Study of minimum technical requirements (technical standards). To reduce the cost to a marketable levels, minimum necessary hardware specs were defined. Study of legal requirements, especially from the privacy viewpoint. We were eager to avoid the need to declare the sensor as “video surveillance”. Privacy Impact Assessment submitted to the Slovenian Information Commissioner, which provided non-binding positive opinion regarding implemented and planned privacy safeguards. Manual annotation of a variety of captured video, comparing the actual output of the sensor with manual annotations. Downsizing the prototype with inexpensive, mass produced components to fit it within marketable target price. Production of a number of prototypes (11) which will be used by focus group of users to judge design and ergonomics of the system, and later handed over as samples to prospective customers and prospective investors in large-scale manufacturing effort.

The developed sensor cannot reach 100% accuracy, and accordingly, it cannot be used as an enforcement tool - the enforcement of driving rules is explicitly not the purpose of this technology. The initial market will be corporate clients with large fleets of professionally used vehicles, since the problem of driver distraction is especially acute for professional drivers, which spend many hours on a road with sometimes mentally unchallenging conditions. Therefore, the sensor will be marketed as positive reinforcement tool, allowing vehicle owners (the clients) to reward drivers if they drive responsibly. The solution makes perfect sense for vehicle owners: reckless and distracted driving results not only in lost lives and injuries, but ultimately in higher insurance costs.

Given the nature of the system, this project enabled a long-term partnership. We expect the technology transfer process to continue. Availability of the hardware platform, once installed in the vehicle, will offer the opportunity to continue with development and production of new and improved algorithms that would possibly detect wide range of specific driving behaviours (e.g. presence of a mobile phone, gestures, road rage, etc), which which will guarantee long-term revenue stream for the company.

[1] EU on distraction:

[2] Road user distraction study,

PDF icon PosterPDF icon Abstract