Unlocking the potential of automotive 3D ToF imaging
By Gaetan Koers, Melexis
The automobile industry is placing ever greater emphasis on protecting vehicle occupants and pedestrians from harm and ensuring that elevated safety levels are realized. At the same time, customers expect an intuitive user experience while driving - interacting with infotainment and connectivity functions in their car. Meanwhile, gradually higher degrees of driving automation are expected to be incorporated into vehicle designs.
There is, however, still quite some way to go yet to transition from the capabilities of current advanced driver assistance system (ADAS) implementations to what could be considered truly autonomous operation. An important part of this evolutionary process will be how seamlessly the switch from autonomous mode to manual mode can be carried out in specific scenarios. If it is to be fully effective, this change needs to be done in a straightforward and user-friendly manner. The vehicle’s ADAS will require access to accurate, highly detailed information of the driver’s bio-mechanical and cognitive state on an ongoing basis.
Figures recently published by the World Health Organisation (WHO) estimate that there are approximately 1.25 million fatalities every year through road accidents.
It also identifies driver distraction as one of the major causes of such accidents - with calling/texting people on mobile phones, checking wing/overhead mirrors, talking to passengers, eating/drinking, reading instrumentation dials (speedometer, fuel gauge, etc.), dealing with disruptive children and operating infotainment/navigation systems all being prospective sources of risk.
With driver distraction often proving to be the reason (or at least contributing) to road accidents, the value of ADAS systems like automatic emergency braking and collision avoidance maneuvering is clear. However, when the driver is aware and bio-mechanically able to react to a hazardous situation, an overly abrupt intervention of the ADAS system can be annoying or even more dangerous. On the contrary, if the ADAS can ascertain the full driver’s state, it can then more intelligently judge whether or not it needs to take over from the driver in response to a hazardous situation - such as an imminent collision. Real-time monitoring of the driver’s position and movements is crucial if this is to be accomplished.
Through employment of 3D imaging data, the driver’s body pose, their head position and hand positioning can all be accurately ascertained. It can be confirmed, for example, that the driver’s attention is on the road and their hands placed on steering wheel. The 3D information can be used to estimate the bio-mechanical reaction time of the driver to re-engage, and compared with the event horizon that the vehicle has calculated to gauge the safety margin. Then if the driver is not adequately engaged to react quickly enough, the ADAS is aware that should a potentially dangerous situation arise it may be required to step in.
Until now, unfortunately, optical 3D imaging technologies have been difficult to implement within automotive settings. The main reason for this is that they struggle to cope with large and sudden variations in ambient light levels that occur in common driving scenarios. In addition, the supporting electronics must be suited to harsh operational conditions and support the ASIL B/C functional safety standards - something that few suppliers have expertise in. There are also issues relating to integration within the vehicle that need to be addressed. Often optical sensor implementations are very specific to the vehicle, with any differences in the size of the cabin or the orientation of the central console/seating having serious implications for the sensor system’s effectiveness.
Time of Flight offering for automotive industry
Melexis has taken the optical Time-of-Flight (ToF) technology that has been used for several years for 3D imaging in applications like gaming machines and applied it to the automobile industry. Here an infrared light source emits a wide angle beam which upon striking obstacles is reflected back towards the ToF sensor. The sensor then detects the reflected infrared signal and compares it to a reference signal. By determining the phase shift that has occurred, the distance to a particular obstacle can be calculated and detailed 3D images constructed.
Melexis’ MLX75023 AEC-Q100 qualified, high dynamic range image sensor is highly optimized for automotive use, with proprietary pixel demodulation technology allowing it to deliver the necessary sunlight robustness. Furthermore, it supports an operational temperature range covering -40°C to +105°C. The ToF imaging mechanism is much easier to install into a vehicle roof module or dashboard compared to other 3D imaging technologies.
To date automotive ToF has been focused on human machine interfaces (HMIs) - providing a capacity for gesture recognition (with the first models utilizing it now entering the market). Through this, different vehicle functions can be carried out, via hand swipes and suchlike, without the driver having to look at the console (accepting/rejecting calls, raising/lowering the volume on the music system, etc.). This is really just the beginning though. There is potential for it to go way beyond these current parameters. By increasing the sensitivity and field-of-view, the ToF imaging system can be used to map an entire driver’s upper body in 3D. It will also permit data to be garnered on the exact location of other vehicle occupants and their size (adult/child), so that e.g. air bags can be deployed where necessary and with the correct amount of force.
Melexis is leading the way forward for automotive ToF sensing. Through the implementation of proprietary technology, and its continuous interest in new forging strategic partnerships, the engineering team is pushing the technology envelope, so that performance levels are heightened and cost effectiveness increased. This will ensure that the full scope of ToF within the automobile application environment is realized.