Introduction

My name is Natsuo Koda from Miraisens. I am the CEO of Miraisens, engaged in R&D and business development in haptics. 

There has been an increase in number of haptics applications in mobile devices, automobiles, and games, and there are more opportunities for exposure at exhibitions and articles about haptics. However, discussions around haptics editing often extend from sound editing. To help you deepen your understanding of haptics, I am writing this article.

In this blog post, I will explain what haptics is, particularly focusing on how to implement better haptics in games. I also want to introduce the steps on how to easily integrate our haptics design tool (AMPTIX) with Wwise.

 

Table of Contents

What is haptics?
　　Why do we need haptics?
　　Who are we, and where are we going?
　　Reality Devices
　　Let's create rich haptics with AMPTIX
Introduction to the integration between Wwise and AMPTIX
　　Timeline  Playback Process
　　RTPC Playback Process
Conclusion

 

What is haptics?

Why do we need haptics?

In today’s rapidly evolving gaming landscape, we’re accustomed to graphics that rival reality and immersive surround sound. But what about haptics?

Haptics generally refers to the technology that provides cutaneous feedback by applying force, vibration, or movement to the user. In the gaming world, the history of haptics started with simple controller vibrations. Back then, discussions often revolved around statements like, “Controller vibration? I turn it off because it distracts me from the game,” or “Whether it’s there or not, it doesn’t significantly impact gameplay.” 

Recently, as controller capabilities have improved, perceptions of realistic haptics have gradually shifted. Still, there remains a lack of well-established know-how on creating realistic vibrations.

As a company primarily focused on haptics research and development, we aim to make a significant impact in this evolving landscape. Our goal is to provide haptics solutions that allow creators to fully utilize their intuition and senses to achieve whatever they want. Let’s work together to create a world where “games without haptics are considered dull and uninteresting.”

 

Who are we and where are we going?

Miraisens has entered the world as an R&D-type startup specializing in haptics.

As a subsidiary of Murata Manufacturing Co., Ltd., we are committed to advancing the practical implementation of realistic haptics from both hardware and software perspectives.

 

Allow me to touch briefly on the history of haptics here. From an academic standpoint, haptics has already yielded decades of research results, providing humanity with a wealth of knowledge relating to tactile sensations.

Source: http://wiki.ahuman.org/index.php/MindTracts 

On the other hand, while applications of haptics from the 1980s VR era have expanded significantly, widespread development and popularization of general-purpose haptic technology is still work in progress. Haptics can be categorized into several schools of thought, but broadly, there are two main approaches: "Those that reproduce reality as it is: real devices" and "Things that are not real but feel like reality: reality devices."

［Example of a real device］

Source: https://www.dextarobotics.com 

 

［Example of a reality device］

Reality Devices

If you consider applying it to games, the former "reproducing reality as it is" is like a shooting game in an arcade. There tends to be a lack of large-scale versatility, such as actually hitting a gun and enjoying the recoil. 
The latter approach, "not real, but feels real," is highly compatible with general-purpose games, as each game creates a different feel while using the same controller.
At Miraisens, we are exploring the latter approach, the "reality device”.

These devices aim to evoke realistic sensations, even if they don’t directly mimic reality. As we delve into this field, we have observed a shift from traditional “rhythm (Resonant frequency) type” haptics to more sophisticated “rich (broadband) type” haptics.

Our research aligns with this evolution. While resonant frequency skin stimulation feels like mere vibration, the nonlinear waveforms produced by rich-type devices lead to perceptions more akin to specific textures or sensations. As a company, our ultimate goal is to continue advancing the performance of rich-type hardware and create devices that evoke even more advanced sensory experiences.

Let's create rich haptics with AMPTIX

Let's say you play an electric guitar. We connect it to an amplifier and effects pedals to create a more realistic sound. By adjusting the distortion, we can achieve a clearer or grittier tone. When working with sound, we often fine-tune the waveform while listening to achieve our desired sound.

Now, let’s explore haptics. Suppose we want to create audio haptics using sound effects. One initial approach might involve applying a low-pass filter to sound effects to extract the low-frequency components and use them as tactile sensations. While this method works reasonably well when no specialized tools are available, it often results in simplistic sensations (like a simple “buzzing” feeling).

But how do we create richer sensations? How can we generate tactile feedback that we’ve never experienced before? Sometimes you may be able to create an interesting feeling by chance, but more often, it requires persistent trial and error. You will probably question yourself: How should we distort these waveforms? How can we add a touch of high-frequency components?

Miraisens' haptic editing tool takes the approach of creating haptic sensations with synthesizers based on sound effects. We blend textures like roughness or smoothness directly into the sensation library rather than manipulating waveforms. It’s about editing sensations, not just waveforms, to achieve the desired haptic experience.

For example, when you create a sense of the afterglow of the bazooka with a synthesizer in a DAW. 

• Between particle sizes → Vibration frequency
• Sharpness → Harmonic components 
• Randomness → Adds fluctuation to both frequencies and harmonic components

These concepts allow for an intuitive approach to achieving the desired effect.

In addition, by using Miraisens’ proprietary technology, "Directional Force," you can even incorporate unique sensations into the vibrations of the controller. These sensations include: 

• Sense of force that feels like a heavy thump
• Sense of force being swung from side to side

However, these sensations cannot be conveyed unless you actually feel it, and I want to convey it to game developers, those who are skeptical of this article, and game players who will enjoy the game in the future.

［Example of Timeline Editing］

［Example of Tactile Editing］

* It is possible to check multiple surface materials' textures while feeling it.

 

Introduction to Integration Features between Wwise and AMPTIX

Wwise is an indispensable tool for those involved in sound editing and development. However, integrating Wwise with other tools, including AMPTIX, is an issue that directly affects productivity.

AMPTIX offers a plug-in to Wwise, which provides a unique advantage: even sound designers and game designers who routinely use Wwise can implement rich haptics functionality with minimal understanding of the integration process. The following image shows the state of integrating AMPTIX plugins into Wwise. Let’s take a detailed look at the steps to integrate Wwise and AMPTIX. 

Introduction to Timeline Playback Process

First, to import the audio data in the Wwise project, open the Wwise project folder and select the WAV files you want to edit. Then, click the ‘RUN’ button to import those data to AMPTIX. You can send the WAV data both individual file and multiple ones to edit haptics. To edit haptics, you can use the ‘AutoGenerate’ features to create haptics-edited data automatically based on the audio waveform. Additionally, manual editing is available while previewing haptics in AMPTIX.

［Import Wwise data to AMPTIX］

［AutoGenerate Feature］

［Edit haptics with AMPTIX］

［Send back Haptics data to Wwise］

When you send haptics data back to the Wwise project, it becomes playable for haptics on the Wwise side.

In Wwise, you connect devices and busses, as shown in the following diagram.

For example, the steps mentioned in #4 would result in the following configuration: When the AMPTIX plugin is integrated into Wwise, the AMPTIX Bus will appear in the menu. You can select the bus to use AMPTIX features.

Once the connection is completed, you will be able to play haptics. Connect the target controller, and when you press the Play button in the screenshot below, the controller will vibrate.

 

Introduction to the RTPC Playback Process

Let’s delve into real-time RTPC playback for haptics. Within Wwise, you can achieve rich Force (Directional Force) and Tactile (Surface Texture) playback using AMPTIX.

To use the feature, you select the AmptixForce (or AmptixTactile) from the New Child and assign parameters to them.

You select the RTPCs and press the Play button to playback the haptics on the controller. While vibrating, you can modify parameters in real-time to adjust the vibration.

 

Conclusion

As mentioned earlier, our company has been researching and developing haptics for many years, and we are thrilled about the recent expansion of haptics. With the integration features between Wwise and AMPTIX, haptic editing becomes more straightforward and accessible. We take pride in contributing to the creation of an environment where rich haptics can be delivered to the market as quickly as possible.

If you are interested, please contact us by following URL and try out the integration between Wwise and AMPTIX.

miraisens.com 
info@miraisens.com