Sony patents banana controller, Donkey Kong would approve

A newly published Sony patent uses unique AR sensor technology to create controllers and peripherals out of everyday objects--including bananas.

3

Sony's new patent is rather bizarre and incredibly innovative. The patent is quite technical, but the gist of it is centered around VR/AR sensor device that essentially scans an object and uses machine learning not only to identify it, but use six-dimensional pose estimation to translate it into a peripheral. Theoretically this would allow gamers to use everyday household things like bananas to interact with and control video games.

3

FIG. 3 shows schematically an example of a video image of a banana being used as a video games controller;

The sensor would detect the object and track its movement and positioning as the game reacted accordingly. This has far-reaching implications, but if the patent ever coalesces (which it probably won't), don't expect a banana to replace a DualSense controller. The applications should be fairly limited to mini-games and other novel experiences inside of VR and not actual blockbuster PS5 games.

"It would be desirable if a user could use an inexpensive, simple and non-electronic device as a video game peripheral. The present disclosure seeks to address or at least alleviate some of the above-identified problems," reads the patent's background.

The abstract also highlights the purpose of the patent: "A system for generating video game inputs is provided. The system comprises an input unit operable to obtain images of a passive non-luminous object being held by a user as a video games controller. The system also comprises an object detector and object pose detector for detecting the object and its respective pose in the obtained images."

3

FIG. 6 shows schematically an example of a video image of two bananas being used as a video games controller;