PlayStation 5's DualSense uses neural networks to generate haptics

Sony has developed neural network-based algorithms to power the PS5 DualSense controller's haptics.

2

Sony is making it very easy for developers to utilize the DualSense controller's advanced haptics. The PlayStation 5 controller features programmable adaptive triggers, which allows devs to do nifty things like locking the triggers when your gun jams or simulate nuanced tactile sensations like pulling a bowstring or driving through mud. A big part of this immersion is how the DualSense can turn sound waveforms into rumble feedback.

To streamline this process for devs, Sony has made a system that uses neural network algorithms to automatically create rumble and vibration effects based on sounds and in-game effects. This gives developers more time to focus on other things like ray tracing optimization.

2

Sony outlined how this will work in an R&D technology brief from July 31, 2020:

"Although DualSense will allow for more realistic game experiences, creators will also need more time and know-how to create high quality vibrations. To reduce this burden, we have created a haptic vibration waveform design environment that anyone can use easily," Sony R&D engineer Yukari Konishi said in the report.

"In this way, we have not only developed a tool that allows game creators to design an impactful, natural and comfortable vibration waveform in fewer steps, but also created a method of almost automatically generating vibration patterns from a game's sound effects.

"The focus of our development was deep neural network technology, but the problem was that there were few past studies that applied this technology to the sense of touch rather than to images and sounds.

"In the beginning, we spent many days just agonizing over the data. Later, we studied and examined different algorithms while getting advice from experts. This allowed us to automate the generation of high-quality vibration waveforms to a certain extent, making it look as if they were created manually by the creators."

DualSense specifications and features

• USB Type-C charging
• Adaptive triggers with improve actuators/haptic feedback
• Built-in Mic lets you chat without a headset
• New Create button
• Touchpad
• Lightbars on the sides of the touchpad
• Improved 1560mAh battery
• Built-in speaker

Read Also: PlayStation 5's Dualsense 'offers players more ways to make content'