Based on advanced audio management solutions, like Wwise Automotive, Continental can now integrate the control over all sound sources in a vehicle: The Functional Audio solution manages all data sources that have potential relevance for driver information and uses that data for a tailored sound design. As a true automotive system Functional Audio combines high fidelity with the functional level of an acoustic human machine interface channel plus the necessary automotive system reliability.

In a vehicle sound has many levels: While the first thing that may come to a driver’s mind is the audio quality of music and radio programs, there is also the added level of information that may directly relate to the driving task. Currently this human-machine interface level of sound is treated separately, which results in a situation where different types of sound are generated and handled by several discrete components or electronic control units in the vehicle. Examples are basic vehicle sounds like the turn signal, obviously the radio/USB, incoming messages and telephone calls, infotainment applications and of course important driver information like navigation instructions, and audible alerts coming from, e.g., blind spot assist or parking aide systems.

Using sound to communicate information to the driver is advocated by the good ergonomics of sound in the vehicle: Not only is sound a very fast sensory channel supporting 3D orientation, it is also a way of limiting the dominance of the visual channel. Ever more dense traffic and additional visual information can quickly lead to a sensory overload. In recent years, the automotive industry has therefore improved the user-friendly presentation of information. Until quite recently, the focus has been on visual displays. However, auditory messages make a valuable contribution to a multimodal approach – a term that denotes a human-machine interface or interaction concept, which uses different sensory channels (visual, acoustic, haptic) for different types of message and levels of urgency.

On top of avoiding an overload on a single channel, this holistic concept facilitates a stepped escalation which serves to attract and direct the driver’s attention according to the driving situation and instantaneous driver workload. This is one key characteristic of the Functional Audio all-software solution: The “sonification” of data [1] helps to increase the level of driving safety by tailoring sound design to a specific situation and requirement. 

Holistic approach to sound

A holistic interaction concept will also be required as an answer to global automotive trends such as automated driving, electrification or vehicle fleets for smart cities. New types of information are required to make the transition from automated mode to manual mode as seamless as possible, for example. In an electric vehicle sound can be used differently thanks to the lack of engine noise. The many discrete sound sources in present cars make it difficult to prioritize sounds outside the bounds of their own domains. Sound remains largely subject to specific functions and it is not possible to manage audible messages at a superordinate level. New vehicle architectures, however, are now making it necessary for all sources to be integrated. The Functional Audio software provides the technical solution for mastering this challenge.

Figure 1: Central audio management for head units or amplifiers offers new functions and freedom in design.

Functional Audio (Figure 1) implements sound design functions that were previously impossible in the automotive sector. With the aid of a central audio management, a driver’s attention can be drawn to a specific situation. For example, if another vehicle approaches from the right, an audible warning signal from the right-hand side of the vehicle intuitively shifts the driver’s focus of attention to where it needs to be – while other sounds are faded out. Functional Audio is a solution that builds on the Integrated Interior Platform (IIP) from Continental (Figure 2):  

Figure 2: Functional Audio builds on Continental´s Integrated Interior Platform.

The IIP and its architecture combine the functions of infotainment and the instrument cluster within shared hardware. Hypervisor technology ensures that safety-relevant systems are capable of running at all times, separated from the infotainment system. The software architecture guarantees a smooth and non-domain-specific flow of information appropriate to the situation for all modes. Functional Audio uses this to process all information and manage the output of all audio sources. The software also makes it possible to customize sounds in the vehicle and to prioritize or intensify existing noises. For example the audible alert can be more intensive when a person walks behind the vehicle than an alert indicating an inanimate object. 

Sonification of data from many sources

Functional Audio can be connected to advanced driver assistance systems or cloud applications, such as the eHorizon from Continental. The system creates or modifies different sounds based on the data received. For instance, sonification can be applied to collision warning systems [2]. If the driver has already noticed a hazard, an early warning is not necessary. However, if the hazard is not within the driver’s line of sight, an early warning is issued to enable suitable countermeasures to be taken. Camera-based driver monitoring helps to evaluate the line of sight and thus the level of awareness. The system processes the data for a special sound design and reproduces it in the form of an auditory display [3].

Thanks to the spatial audio output, the sound source can be placed almost anywhere around the vehicle and be played from any speaker – either alone or over other sounds – to attract the attention of the driver. The driver can then narrow down the direction of the audible signal to an accuracy of up to eight angular degrees. Converting data into sound can help drivers during navigation (Figure 3).

Figure 3: Live data connection enables a sonification of the navigation.

If the vehicle is approaching a suggested turn-off point, an intensifying sound alerts the driver. Thus, the driver is no longer compelled to read the distance indicator on the display or to estimate distances.

Other driving situations also illustrate the significance of the modified audio output. For instance, the system can enhance the sound of the turn signal with additional information. If the driver wishes to overtake and activates the turn signal to change the lane, the system takes on the role of a blind spot assistant. An audible warning will overlay the sound of the turn signal, if a car approaches from behind. The sound of the turn signal returns to normal only when the relevant lane is clear. Collision warnings follow a similar principle (Figure 4).

Figure 4: Collision warning with spatial audio alerts to inform the driver in time. 

The system informs and warns drivers audibly of obstacles and hazards. The connected vehicle’s live data connection means that the system can warn of a potential collision even if the hazard is after the next bend or behind the crest of a hill. 

Plug-in-based system

Sound algorithms from third-party providers, such as text to speech engines or 3D sound enhancements from Auro, can be used flexibly as plug-ins in Functional Audio. All functions and settings can be configured using the Wwise sound design tool with its graphical user interface. Developers can simulate different sound designs and settings in advance and test, change and save them in the vehicle in real time. Increased efficiency means that the respective sound effects take less time to develop.

The central audio management system includes all the conventional and familiar functions. For example, it can stream content from mobile devices and prioritize sound playback. This does not require any modifications to the hardware, which means that the software can run on the Integrated Interior Platform, for instance. The carmakers can decide which sound they wish to be played in which situation and how the various sounds are to be prioritized. They can provide their own sounds or have the sounds developed externally. Continental is taking the list of requirements and converting it into a reliable system.

Although various functions benefit from high-quality, multi-channel speaker systems, fundamentally, Functional Audio works if a vehicle has two speakers – meaning that it can be used in all vehicle classes.  By the secure connectivity of the vehicle to the cloud it is possible to update or modify the sound design. This makes it possible for drivers to install new functions using apps or even enables to download their customized sound design into their current vehicle in a car fleet.

Literature

[1] Thomas Hermann, Andy Hunt: The Sonification Handbook, first edition, 2011, Logos Verlag, page 10.

[2] Jaka Sodnik, Sašo Tomažič: Spatial Auditory Human-Computer Interfaces, first edition, 2015, Springer Verlag, page 62.

[3] P. Bazilinskyy: Auditory Displays for Automated Driving, 2017. www.researchgate.net/publication/312940263_Auditory_displays_for_automated_driving

[4] Continental: Integrated Interior Platform, 2017. Link: https://www.continental-automotive.com/en-gl/Passenger-Cars/Interior/Display-Systems/Integrated-Interior-Platform.

[5] Continental: 3D-sound improves driver assistance systems in cars. Link: https://www.continental-corporation.com/en/press/press-releases/2018-02-26-predictive-connectivity-122554.