Researchers train an AI to identify keystrokes on a keyboard by sound alone

Generative AI is going to be used to cause havok with online security, this new research paper shows how AI can track what you're typing using audio.

1 minute & 28 seconds read time

"A Practical Deep Learning-Based Acoustic Side Channel Attack on Keyboards" is a new research paper out of Cornell showing how AI can accurately predict keystrokes being pressed on a keyboard through sound alone. The AI model was trained on a specific keyboard using the conferencing app Zoom and achieved 93% accuracy in predicting keystrokes as they were being entered.

Researchers train an AI to identify keystrokes on a keyboard by sound alone 02

It's impressive and scary stuff, thanks in part to the brand-new world of generative AI being used for malicious purposes, but the good news (at least for now) is that the system deployed by researchers Joshua Harrison, Ehsan Toreini and Maryam Mehrnezhad, required the use of a specific keyboard. This is unlikely to change as different keyboards and keyboard styles feature different sound profiles.

Using sound, the AI model analyses waveforms to recognize the subtle differences between different keys on a keyboard, even when pressed multiple times. Being able to hit a 93% accuracy in predicting keystrokes over a Zoom conference call is an impressive achievement.

Using audio to predict keystrokes is not new, but achieving this level of accuracy with the help of AI is. "When trained on keystrokes recorded by a nearby phone, the classifier achieved an accuracy of 95%, the highest accuracy seen without the use of a language model," the research paper writes. "When trained on keystrokes recorded using the video-conferencing software Zoom, an accuracy of 93% was achieved, a new best for the medium."

Using sound as the basis does have limitations for using this tech for malicious purposes outside of being limited to a specific subset of keyboards. The findings note that switching to touch-type as a means of input dramatically reduces the accuracy to around 40 - 64% - which makes you wonder if that recent loud, clicky mechanical keyboard purchase was a good idea.

Then there's AI-based noise reduction software like NVIDIA's RTX Broadcast that removes keyboard sounds when you're on a video stream. Fighting AI with more AI is very cool.

To check out the new research paper in full, head here.

Buy at Amazon

Razer BlackWidow V4 Pro Wired Mechanical Gaming Keyboard

TodayYesterday7 days ago30 days ago
* Prices last scanned on 9/25/2023 at 9:06 am CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission.

Kosta might be a relatively new member of TweakTown, but he’s a veteran gaming journalist that cut his teeth on well-respected Aussie publications like PC PowerPlay and HYPER back when articles were printed on paper. A lifelong gamer since the 8-bit Nintendo era, it was the CD-ROM-powered 90s that cemented his love for all things games and technology. From point-and-click adventure games to RTS games with full-motion video cut-scenes and FPS titles referred to as Doom clones. Genres he still loves to this day. Kosta is also a musician, releasing dreamy electronic jams under the name Kbit.

Newsletter Subscription

Related Tags