High resolution automotive ToF sensor: beyond 100 Kpixel
Gualtiero Bagnuoli is a Product Marketing Manager with Melexis. He has a background in microprocessor and related areas of computer design complemented with extensive experience in the automotive industry. Gualtiero joined Melexis eight years ago as a System Marketing Manager before moving to his present role of Marketing Manager Optical sensors. Intrepid Delta, the company organizing the event, spoke with Gualtiero at the Intuitive Vehicles Conference on March 12, 2019 in Berlin, Germany.
Intrepid Delta: What do you see as the greatest challenges facing the automotive industry in terms of the vehicle development and technology of the future?
Gualtiero Bagnuoli: I would say there are two challenges from my point of view. The first is related to autonomous or semi-autonomous vehicles - which type of sensor technology will ensure reliable operation and a good user experience. The second challenge is what kind of propulsion will replace today’s combustion technology. There is a lot of emphasis on the electric drive train right now.
Regarding both of these challenges, it's not yet clear what the final outcome will be. Fully autonomous vehicles will most likely appear as a kind of robo-taxi focused on mobility as a service. With regard to electric drive trains, a few years ago, everybody was expecting to see a migration to electric cars happening pretty quickly. Although it's true that at present there are quite a few different offerings from the various car makers with some having a 300-kilometer range, it’s unclear if that’s enough for the end user. A big push could come from China because they're starting from a completely different approach. They need a range of 500 or 600 kilometers or else there is no use case for them. If they achieve that, it may help the transition.
ID: At the conference, Melexis presented potential use cases for QVGA versus VGA Time-of-Flight sensors. Would you discuss some of these use cases as it relates to the feature differences?
GB: We are currently in mass production with the first generation of Time-of-Flight (ToF) 3D sensing technology, which is available since 2015. Thanks to the interaction and feedback from our customers, we developed a second generation. The first generation was aimed to gesture recognition. With skin, there is a lot of reflectivity - 50% of the signal comes back. But with clothes, in dark conditions, only 10% of the signal is reflected. Therefore, the sensor must be able to work with very low signal. Key focus of second generation QVGA ToF chipset was on improving the ability to capture much lower reflected signals. Moreover, since the camera can be in the line of sight of passengers the sensor must be able to use 940 nm non visible infrared illumination. This becomes all the more important when you consider that some OEMs create customizable ambient lighting to give users the option of setting the atmosphere within the car.
What we are launching with the third generation technology is very relevant to the intuitive vehicle. The higher resolution (VGA) means that it is possible to learn more about what’s going on in the vehicle in general, even in the rear seat or to be able to gain a sense of the surrounding environment. As an example, I'm driving and suddenly point to something outside the car. It’s important for the system to be able to determine that I’m looking at or pointing to something which is outside the car. The sensor therefore needs to be able to cover a wide field of view with still high enough resolution.
An interesting aspect of Time-of-Flight technology is that it does not need very high resolution. Our third generation VGA sensor is meant to cover a wider area than QVGA rather than being able to capture fine details about people. End users may be sensitive about cameras in terms of privacy but what you get with time of flight technology is a point cloud instead of a color picture. So, it's a 3D model of an object, you cannot really see the person’s face.
The key advantage of a Time-of-Flight sensor versus a conventional 2D image sensor is the combination of distance information and sunlight robustness. These characteristics enable reliable sensing either when it is about in-vehicles monitoring or exterior short-range cocooning. For example if you open the sunroof of your car, the ToF sensor will continue to deliver reliable information. This makes our ToF sensor well suited for automotive and more in general for use cases with challenging light conditions.
ID: Is there any concern that the newest VGA sensor could compete with your second generation QVGA sensor that was just released last year?
GB: No, it’s a completely different segment. We do expect that VGA’s system cost will be 30% or 40% higher than the QVGA one because of the wide field of view. That means that it is necessary to spend more on better optics and on providing a homogeneous illumination over the area to be observed. Last but not least the processor has to compute four times more data.
So, the greater resolution implies the additional cost. We see this VGA sensor as a solution for premium vehicles where the goal is covering a wide area inside the vehicle, while the QVGA can be a solution for vehicles at a lower price point with less demanding use cases.
I do expect that some of the new players in automotive will try to provide a full-blown experience in order to distinguish themselves from the traditional OEMs. But in general, I expect an incremental step-by-step adding of new features. The hardware will be capable of supporting the new features, but they can be added over time through software upgrades.
ID: Beyond automotive, what other markets and applications do you see as applicable for this sensor?
GB: We see quite some interesting use cases in logistics with this kind of sensor. A customer can keep track of how many boxes are loaded into a truck, their size, whether there's still volume available, and so on, which can help a lot. Other areas could be optimizing inventory in a shop, or people counting in transportation and even security. With the 3D information, you cannot easily cheat the camera and as already mentioned one of the beauties of Time-of-Flight technology is that changing lighting conditions do not drastically change the performance of the sensor.
Melexis has had a collaboration with Sony since 2015. Sony released its own VGA sensor - the IMX456 - recently. Is this distinct from the Melexis MLX75027?
GB: Allow me to say we are proud of this collaboration with Sony. The IMX456 is the consumer version of the Time-of-Flight sensor we co-developed. Our MLX75027 sensor is based on the automotive version of the IMX456. The product architecture is the same, the underlying technology, pixel architecture, and so on are the same. But for automotive functions the sensor undergoes additional testing and AECQ-100 qualification of course.
ID: Could you give us some insight into what is on the horizon for Melexis regarding new products or areas in which you are focusing?
GB: Cost-efficiency is of course important for the automotive industry, so we are working on even more cost-effective solutions, especially for applications where you don't need full VGA resolution. Greater integration to make the camera much smaller is also something we are working on. In the future, we will put effort into providing the industry with a solution suitable for very high integration with the capability to take full advantage of laser illumination.
We are thinking of a down-scaled version of the VGA sensor. Our second generation QVGA chipset is well suited to work with LEDs, which are cost-effective and reliable solutions for several applications. The chipset can work with laser, but with limitations on the maximum modulation frequency. Therefore it will make sense to complete our product portfolio with a sub-VGA sensor capable to take full advantage of high modulation frequency provided by VCSEL illumination.
One final area that we are working on is determining the direction we should go on with regard to the ISO26262 functional safety guideline.