For many years now drivers have relied on cameras, lidar, and radar to detect the environment around passenger cars.
However, the perception of external sounds has been lacking, even regarding automated driving. That is until now as German company, Fraunhofer IDMT, is researching the integration of acoustic sensor technology in its KI4BoardNet project with The Hearing Car.
The company has now equipped research vehicles with new microphones and specially developed measurement technology. To test both, the team recently travelled to a test site in Sweden.
Every day, we humans perceive different sounds. At an intersection, an electric vehicle drives past pedestrians, a dog barks on the other side of the street, and a cyclist rings his bell. An ambulance approaches in the distance. External microphones on cars are intended to help automated vehicles recognise emergency vehicles in traffic. However, other scenarios are also conceivable. For example, acoustic sensor technology can be combined with speech recognition systems to interact with the vehicle from outside via voice commands.
Robust technologies for use in wet, frosty, or hot conditions
A test drive in Sweden was conducted to determine how the newly developed and integrated measurement technology performs in terms of reliability and usability. The researchers wanted to find out how the microphones perform under different weather conditions and degrees of contamination.
“Together with CARIAD, the Volkswagen Group's automotive software company, we are testing and developing algorithms and microphone hardware at Fraunhofer IDMT in Oldenburg to prepare acoustic sensor technology for series production,” project manager for The Hearing Car at the Fraunhofer Institute for Hearing, Speech and Audio Technology HAS, Moritz Brandes, explained.
The researchers are applying their expertise in acoustic pattern recognition and sensor data fusion, as well as important domain knowledge. In addition to cold weather testing, regular tests were also carried out in southern Europe to investigate the influence of heat.
Some developments are part of the larger research project ‘KI4BoardNet’, which defines and further develops acoustic environment detection and board network architecture in modern vehicles, among other things.
“Our tests show that exterior microphones can change the way we interact with our cars and, in combination with optical sensors, improve safety for automated driving functions,” product owner exterior acoustic perception at CARIAD, Hagen Jaeger, said.
Initial analyses of data from road tests under various conditions – from snow and ice to heat and wet weather – show positive results.
Fraunhofer IDMT is now publishing documentary videos on its LinkedIn channel which can be found here at regular intervals to document the journey of The Hearing Car to Sweden.
The vehicle electrical system as an intelligent component
The aim of the ‘KI4BoardNet’ project is to develop the architectures, components, and design tools for the vehicle electrical system of the future. The project involves joint research and implementation of dynamic, AI-supported design processes and maximum automation in the development and production of vehicle electrical systems.
This is necessary because vehicle design increasingly involves digital components. The data for these components usually has to be processed in real time, which places high demands on the vehicle electrical system. At the same time, the number of electrical consumptions is increasing, as the requirements for the quality of the power supply.
Fraunhofer IDMT is helping to define requirements for AI-enabled control units and the energy system for supplying power to AI-driven sensor technology. Vehicle information, such as the power consumption of an electric vehicle or its speed, will be combined with the recorded microphone data to open up new areas of application.
With in-house expertise in the development of hardware and software systems for audio system technology and signal enhancement, the employees at the Oldenburg site are responsible for transferring scientific findings into practical, customer-oriented solutions.