SOPHIA ANTIPOLIS, France – August 16, 2022 │ Since 2010 and the end of the crisis, the automotive industry has been undergoing veritable revolutions, with the dawn of electric and autonomous vehicles. While electrification is already well established with the release of hybrid and fully electric vehicles, the launch of robot taxis and fully autonomous vehicles (AV) is taking a little more time. The development of AV redefines the use of automotive technology in our day-to-day lives. Many safety and legal challenges need to be addressed. From the automakers’ point of view, developing fully autonomous cars (Advanced Driver-Assistance Systems (ADAS) level 5) or robot taxis means improving and integrating new sensors and related computing methods. As with EV, Toyota Group is emerging as a pioneer and leader in the development of ADAS and AV, especially in the field of sensor technologies for autonomous vehicles. Lately, the group has announced a new milestone with the release of the flagship Crown model that will include an automated driving system with hands-free capability on the highway. According to Nikkei’s newspaper, Toyota aims to provide an autonomous driving system by 2025.
Toyota’s current investments to develop AV could be linked to the group’s recent patenting activity. With a CAGR of +16% between 2011 and 2020, reaching a figure of 900+ new inventions published in 2020, the IP activity of Toyota Group related to automotive ADAS and AV technology is among the most dynamic of the group (Figure 1). Such an increase in R&D activity is necessary to be able to compete against the other automakers like Volkswagen, Audi, Mercedes and Ford; but also against the robot taxi making newcomers, especially in the US (Waymo, Uber, Zoox, etc.). Furthermore, strengthening and enlarging its IP portfolio allows Toyota to secure its products and offers the group a set of tools to fight against its competitors, who have also strongly reinforced their patent portfolios.
Today, Toyota’s portfolio related to automobile ADAS and autonomous systems is almost equally divided between patents related to sensors and patents describing control methods, algorithms and systems that can manage the great variety of data and execute an efficient action at vehicle level.
Fig.1: Time evolution of Toyota Group’s patent family publications related to ADAS/autonomous vehicles.
As for many automakers, Toyota’s AV system includes several sensors controlled by a single operating system that ensures fast, secure, and reliable information management regardless of the road conditions, traffic and weather. However, the whole system relies on data transmitted by a set of sensors placed on the specific position of the vehicle. Cameras, radar and lidar are the three main sensors combined to sense the car environment and thus constitute the core of the autonomous vehicle system. Toyota Group has been one of the earliest automakers to massively invest in these autonomous vehicle sensor technologies. Indeed, since the late 90s, Denso and Toyota Motor have filed numerous patent applications related to radar, cameras and lidar for ADAS (level 1 and 2) applications for automotive technology. Since 2012, providing a fully autonomous vehicle has become a new challenge for automakers. Since then, Toyota Group has expanded its patenting activity to protect not only the sensors but also their integration in a vehicle and their operation in a real environment. With such an approach, the company is now able to hamper the freedom-to-operate of its competitors from device-making to use and operating methods in a vehicle. Accordingly, Toyota Group can control the different levels of the supply chain, from the sensor components to the final system. This IP position is well aligned with Toyota’s recent partnership with Aurora and the development of a new operating system that will be available by 2025. Toyota will thus benefit from the extensive know-how of the US startup, co-founded by a former Google self-driving car team leader, that has recently acquired Uber’s self-driving unit.
Fig.2: Denso sensing system for ADAS operations (source: Denso).
Singular IP and market strategies
In its recent communication, Toyota has announced its will to achieve fully autonomous vehicles using cameras instead of more expensive and less developed lidar devices (see here). The announcement comes after Tesla’s decision to train its self-driving car sensor technology by using less expensive cameras. Such a position echoes the group strategy related to electric vehicles (EV), where the automaker partnered with Tesla to provide a new model of EV. Such a move attests to Toyota’s belief that cameras and radars combined with an adapted computing system may be enough to compensate for the technology’s drawbacks (low-distance vision, poor sensitivity in complex weather conditions, etc.), thereby avoiding the use of lidar. Toyota’s announcement also confirms its belief and will to invest a lot in computing and operating systems. This strategy is confirmed by Toyota’s IP activity.
From lidar IP leader to quitter
Many competitors have strongly increased their patenting activity related to lidar, attesting to their strong investments and R&D developments. This is not the case for Toyota. The Group, and especially Denso, has slowly reduced its number of new inventions. Indeed, with just a little more than 20 new inventions per year related to lidar systems, Toyota Group’s patenting activity is far from the level demonstrated by its main IP competitors. However, the company still has a noticeable IP position thanks to its activity in the early 2000s. Indeed, Denso released its first lidar device in the late 90s and has been developing MEMS-based scanning lidars since the early 2000s. The activity peaked in 2012 with the release of Denso’s last lidar product. Today, most of the patents protecting these early technologies have expired, and Toyota Group’s IP position finds itself increasingly under threat (Figure 4) from other tier one suppliers in Europe (Bosch, Valeo, Continental), as well as from pure lidar IP players in China (Robosense or Hesai Technology). Furthermore, Toyota’s IP strength is mainly related to low-level ADAS applications, meaning lidar for parking assistance, anti-collision systems, etc. Advanced applications such as 3D mapping remain a small part of Toyota’s IP activity.
Fig.3: Evolution of the IP landscape related to lidar from 2018 to 2021.
Toyota’s strategy on autonomous vehicles is quite different from what we are used to seeing in its activity related to EV or thermal vehicles: the technology is developed almost uniquely in-house. No major collaborations exist at IP level. Indeed, less than 10 companies have co-filed inventions with Toyota Group. As a comparison, more than 600 inventions have been co-filed by Toyota Group in the fuel cell field, and more than 300 inventions in the battery field.
Fig.4: Time evolution of Toyota Group’s patent family publications related to sensors for ADAS or autonomous vehicle applications, split by main sensing technology.
Toyota Group’s patents were mainly filed by Denso, which has filed more than 100 inventions that claim a lidar system. Toyota Motor has also filed more than 90 inventions related to lidar mounting as well as data processing. Up to now, the company’s IP strategy has focused on only two countries: USA and Japan. Toyota has just started to extend its IP activity to China in recent years. On the other hand, we note that Toyota has not shown a specific interest in developing its IP in Europe. Such a strategy could be explained by the late adoption of AV in Europe, where tests on autonomous vehicles in real conditions are far less advanced than in the US and China, where they are already circulating.
Fig.5: Geographical coverage of Toyota Group’s alive documents related to radar, lidar or camera technology.
A strong prior art that still impacts the patenting activities of competitors
In addition, Toyota and Denso patents describe lidar systems based on a scanning mirror placed on a rotating shaft and orienting an edge-emitting laser (EEL) beam to illuminate the scene. Such an approach is the most common nowadays. Historically, this technology was the first to be developed and protected by players. Accordingly, the prior art is large and very dense, which makes it harder for established and new IP players to file new inventions. Yet in a recent patent analysis of the domain, Knowmade highlighted that:
- The IP competition remains fierce: IP players are still actively filing patents describing this technology, among which many newcomers and pure players entering the patent landscape.
- Their IP activity mostly relates to incremental innovation rather than disruptive innovation: to obtain a new patent, IP players must focus on very specific systems and methods, for instance to drive or correct the system to enhance the sensor performances.
- Due to the narrow scope of the protection provided by such patents, it will be more difficult for the players to enforce their patent portfolio against competitors.
Toyota Group follows this same trend and its recent patents related to lidar technology describe systems that can detect dirt, snow or any other type of dirt that may be deposited on the lidar and obstruct the beam. For instance, since 2019, Toyota Group’s patents have been describing solutions to reduce the number of data points that must be processed (JP2019158894, US20200386894); solutions to detect snow (JP2020012716) or foreign bodies (JP2020012759) on the lidar, or road condition (US20210396887).
Fig.6: Denso lidar system as described in patent US20200355805.
Camera and low-cost sensors as the solution for Toyota autonomous vehicles
The third generation of Denso’s Global Safety Package uses the combined performance of a millimeter-wave radar sensor and vision sensor. The millimeter-wave radar sensor detects the shapes of road objects, such as vehicles and guardrails, while the vision sensor uses a camera to detect the environment ahead of the vehicle. The combination of these low-cost technologies is the strategy chosen by Toyota to provide high-level ADAS systems for automobiles and potentially self-driven cars.
Stable IP activity related to radars
From an IP point of view, we clearly see two different dynamics. On the one hand, Toyota Group has just maintained its patenting activity related to radar. Like with lidar, Toyota Group can count on its former patenting activity. Being a historical player, Toyota has been able to file patents protecting several key designs and architectures. For instance, the company filed more than 20 inventions describing frequency modulated continuous wave radar using digital beamforming operation mode. Recently, the company’s patenting activity has focused on suppressing interference thanks to a specific unit as well as reducing power consumption (JP2021189005). Thus, completing its portfolio with inventions related to systems using radar.
Cameras will become Toyota vehicles’ eyes
On the other hand, Toyota Group’s patenting activity on cameras is by far the most intense and fast-growing segment. Regarding cameras for ADAS vehicles or 3D mapping, Toyota’s recent patenting activity is mainly related to methods of mounting cameras (US20220084230) and processing the data. Toyota Group’s patenting activity demonstrates the automaker’s desire to develop the sensor infrastructure and related algorithms and software. For instance, Toyota’s most recent activity describes methods of modelling a car’s surroundings using a neural camera network. Regarding the recent development of the camera itself, Denso has filed one patent (US20210399034, fig 2) describing a camera in which a fluid is placed in the support to dissipate the heat. Alongside this patent, Toyota has recently accelerated its activity related to in-vehicle cameras. Measuring the driver state as well as the car interior appear to be a key part of Toyota Group’s recent R&D activities. Such an activity attests to the importance of cameras in Toyota’s future vehicles. What’s more, it is likely that some developments made for in-vehicle cameras will be used for other scenarios and applications.
Fig.7: Camera design described in patent US20210399034.
Conclusion
Toyota is adopting a singular approach to address the AV market. While most of its competitors are investing fiercely and enlarging their lidar portfolio, the corporation has chosen to reduce its lidar R&D activity, as shown by the patenting activity slow-down. Today, Toyota’s efforts are targeting solutions to integrate and control multiple sensors, especially cameras in a vehicle. The company’s IP position further down the supply chain is becoming stronger, ensuring its position as an integrator and operating system provider. On the upper part of the supply chain, Toyota Group benefits from its past IP activity. Once again, Toyota’s patenting activity shows a will to control and protect all key links in the supply chain. In the EV industry, the strategy is to encompass everything from the material to the system with a balanced portfolio. For the autonomous vehicle, the company strategy is to maintain its IP position on devices and sensor technologies thanks to a smaller but well-targeted patent portfolio, while pushing the development and protection of operating systems and methods with advanced computing technologies such as artificial intelligence (AI).
Related MEMS patent landscape reports by KnowMade.
Press contact
contact@knowmade.fr
Le Drakkar, 2405 route des Dolines, 06560 Valbonne Sophia Antipolis, France
www.knowmade.com
About our analysts
Paul works for Knowmade in the fields of RF technologies, Wireless communications and MEMS sensors. He holds a PhD in Micro and Nanotechnology from the University of Lille (France), in partnership with IEMN in Villeneuve-d’Ascq and CRHEA-CNRS in SophiaAntipolis (France). Paul previously worked in innovation strategy consulting firm as Consultant.
About Knowmade
Knowmade is a Technology Intelligence and IP Strategy consulting company specialized in analysis of patents and scientific information. The company helps innovative companies and R&D organizations to understand their competitive landscape, follow technology trends, and find out opportunities and threats in terms of technology and patents.
Knowmade’s analysts combine their strong technology expertise and in-depth knowledge of patents with powerful analytics tools and methodologies to turn patents and scientific information into business-oriented report for decision makers working in R&D, Innovation Strategy, Intellectual Property, and Marketing. Our experts provide prior art search, patent landscape analysis, scientific literature analysis, patent valuation, IP due diligence and freedom-to-operate analysis. In parallel the company proposes litigation/licensing support, technology scouting and IP/technology watch service.
Knowmade has a solid expertise in Compound Semiconductors, Power Electronics, Batteries, RF Technologies & Wireless Communications, Solid-State Lighting & Display, Photonics, Memories, MEMS & Solid-State Sensors/Actuators, Semiconductor Manufacturing, Packaging & Assembly, Medical Devices, Medical Imaging, Microfluidics, Biotechnology, Pharmaceutics, and Agri-Food.