Enabling the full potential of automotive 3D ToF imaging

By Gualtiero Bagnuoli, Melexis

Time-of-Flight (ToF) has been considered for a while a niche and exotic technology within the automotive industry. Lack of automotive proven sensors and more in general a not yet mature ecosystem, resulting in high costs, were preventing from wider adoption of this technology.

Time-of-Flight interiorAnother limiting factor was the limited resolution of the ToF sensor itself, resulting into a limited set of use cases due to either a narrow Field-of-View (FoV) or not enough spatial resolution with wider Field-of-View.

Recently ToF technology has gained a new momentum because of new active safety standards (e.g. NCAP) and new features required by level 4 and 5 autonomous vehicles, going beyond in-cabin use. ToF technology is now evaluated for exterior use cases like short range cocooning because it can combine high resolution with accurate depth information at short range, complementing long range sensor like FWC camera, Radar or Lidar.

It is clear that the variety of use cases implies the availability of sensor with specific characteristics to enable the industry to choose the most appropriate solution and optimize performance/cost ratio.

Time-of-Flight can combine high resolution with accurate depth information at short range, complementing long range sensor like FWC camera, Radar or Lidar.

While few year ago the emphasis was on availability of automotive proven ToF sensor, today the industry needs a complete portfolio. Let’s take two typical use cases: gesture recognition and in-cabin monitoring.

Gesture recognition

Gesture recognition is usually limited to a restricted interaction area, usually around the central console. Field-of-View is quite narrow, distance range is limited and depth precision is very crucial. Usually QVGA resolution (320 x 240 px) is enough and cost-effective LED illumination can deliver the required performances. With the same resolution, it is possible to use a wider Field-of-View in order to cover both front seat. Tracking of arm, head and in general larger object will be still possible but not fingers.

A ToF sensor with higher resolution will be needed for covering a wide Field-of-View with enough spatial resolution required to properly detect and track fingers.

In-cabin monitoring

Similar consideration are relevant for in-cabin monitoring.
People counting and position of the people in the vehicle can be carried out with QVGA sensor (320 x 240 px).

When it comes to collect detailed information of the driver’s bio-mechanical and cognitive state higher resolution and depth precision become mandatory.

Through employment of 3D imaging data, the driver’s body pose, their head position and hand positioning can all be accurately ascertained. It can be confirmed, for example, that the driver’s attention is on the road and their hands placed on steering wheel. The 3D information can be used to estimate the bio-mechanical reaction time of the driver to re-engage, and compared with the event horizon that the vehicle has calculated to gauge the safety margin. Then if the driver is not adequately engaged to react quickly enough, the ADAS is aware that should a potentially dangerous situation arise it may be required to step in.

The use case here above described requires both higher and depth precision. Whenever high depth precision is required, then cost-effective LEDs have to be replaced by VCSELs, which can operate at modulation frequency well beyond the typical maximum operating frequency of LEDs (typ 30-40 MHz) and the sensor must be able to operate without significant drop of its characteristics (e.g. demodulation contrast).

Time-of-Flight offering for automotive industry

Melexis has been pioneering in optical Time-of-Flight technology applied to the automobile industry.

Melexis’ MLX75023 AEC-Q100 qualified, high dynamic range image sensor is on the road since 2015, focused on Human Machine Interfaces (HMIs) - providing a capacity for gesture recognition.

Last year the second generation of QVGA chipset has been released (MLX75024 + MLX75123), focused on in-cabin monitoring.

Automotive VGA Time-of-Flight sensor - MelexisThe new MLX75024 ToF sensor has the same resolution (320 x 240 px) and maintains industry-leading ambient light robustness while offering double the sensitivity of the previous generation, allowing operation in lower light levels, or requiring 30% less illumination power. System efficiency is further enhanced by a 50% reduction in current consumption and the resulting lower heat generation allows the design of more compact cameras. Supporting the ToF sensor is the new MLX75123BA ToF companion chip with three times lower front-end noise than its predecessor. The companion chip is able to configure parameters such as pixel gain and now supports pixel binning to simplify hardware and software for lower resolution applications. Additionally, the MLX75123BA supports two MLX75024 sensors, further reducing system cost, complexity and size for the dual-head ToF camera.

In March 2019, our third generation VGA ToF sensor MLX75027 has been released.

Key features of the MLX75027 include support for modulation frequencies up to 100MHz, which allows manufacturers to take advantage of the full potential of VCSEL to achieve high distance accuracy. In addition, the device supports a frame rate of up to 135 FPS, enabling the detection and tracking of fast moving objects. It combines low power dissipation of just 230 mw at 30 FPS with a power supply scheme that only requires three voltage domains (2.7V, 1.8V and 1.2V), which simplifies system design and helps lower the cost of the power supply unit.

Table: Melexis Time-of-Flight (ToF) sensor portfolio

Melexis automotive ToF portfolio is today the most complete, allowing customers to select the optimal sensor for their system in terms of precision, resolution and integration level.

A common footprint across the generations allows existing customers to switch easily while a comprehensive set of evaluation kit reduces time-to-market for designers developing solutions for new projects and enables significant cost savings.

Table: Melexis Time-of-Flight (Tof) evaluation kits

Inspired Engineering