Sony's new VR HMD patent is pretty weird. It features some interesting add-ons like eye-tracking tech, but it can also monitor the words you say--or words that are spoken to you--to help avoid "ill health effects". The aim here is to tackle different forms of virtual reality sickness and it may be a huge focus for Sony's next-gen PlayStation VR headset.
The patent, which was originally filed in 2017 and published this month, is largely focused on making VR safer and more healthy for users by tackling a huge accessibility barrier: discomfort. The HMD can be outfitted with various biometric sensors that monitor key signals and warn users if they're doing unhealthy things while in VR. It's also linked to an external remote device that processes data and can send it to a smartphone, or via on-screen HMD indicators.
Eye-tracking cameras can adjust in-game images based on your eye movements or pupils, or flash a warning message on the HUD if your posture starts to slouch. There's even mention of the HMD being complimented with other sensors that monitor blood pressure, glucose levels, breathing, and even neural activity.
Perhaps the strangest part of the patent is mention of a "natural language processor" which can shut off your headset if you're making weird noises or perhaps saying naughty words online.
"The present invention generally concerns head-mounted display technologies. More particularly, the present invention concerns mitigation of ill health effects on users of head-mounted displays based on biometric sensor measurements and natural language processing," reads the patent.
A head-mounted display (HMD) system for illness mitigation, the head-mounted display (HMD) system comprising:
- a body to be worn on a head of a user;
- a display integrated with the body;
- a memory to store instructions;
- a processor coupled to the memory, wherein execution of the instructions by the processor causes the head-mounted display (HMD) system to:
- receive a baseline sensor measurement value from a health sensor, the baseline sensor measurement value measuring a health characteristic of the user,
- generate a health threshold value based on the baseline sensor measurement,
- receive an active sensor measurement value from the health sensor, the active sensor measurement value measuring the health characteristic of the user following generation of the health threshold value,
- identify that the active sensor measurement value has reached or crossed the health threshold value,
- identify a health concern associated with the user based on the active sensor measurement value reaching or crossing the health threshold value, and
- display a visual indicator on a display integrated with the body in response to the identification of the health concern.
Given the wide breadth of health issues tackled in the patent, I'm thinking this may not just be for a new PSVR headset. Sony makes lots of health-oriented products and this could be a new kind of VR headset used in clinical environments for rehabilitation.
But at the same it's likely at least some of this tech would find its way in a new console-powered VR headset.
Users of head-mounted displays (HMDs) often experience nausea and other symptoms often referred to a "virtual reality sickness." Various health and biometric sensors are employed to gather baseline sensor measurements. The HMD or a device associated with it can then generate a range or threshold such that any future sensor measurements that fall outside of the range or cross the threshold suggest a health concern for the user of the HMD.
The HMD can also be coupled to a microphone, and natural language processing (NLP) is employed to detect negative words or noises made by the user that could also suggest a health concern for the user. After detecting the health concern, the HMD can warn its user, can shut off certain functions, can communicate with an external device, or can trigger an alarm.
Check below for some interesting diagrams on the patent: