Researchers train an AI to identify keystrokes on a keyboard by sound alone

"A Practical Deep Learning-Based Acoustic Side Channel Attack on Keyboards" is a new research paper out of Cornell showing how AI can accurately predict keystrokes being pressed on a keyboard through sound alone. The AI model was trained on a specific keyboard using the conferencing app Zoom and achieved 93% accuracy in predicting keystrokes as they were being entered.

2

It's impressive and scary stuff, thanks in part to the brand-new world of generative AI being used for malicious purposes, but the good news (at least for now) is that the system deployed by researchers Joshua Harrison, Ehsan Toreini and Maryam Mehrnezhad, required the use of a specific keyboard. This is unlikely to change as different keyboards and keyboard styles feature different sound profiles.

Using sound, the AI model analyses waveforms to recognize the subtle differences between different keys on a keyboard, even when pressed multiple times. Being able to hit a 93% accuracy in predicting keystrokes over a Zoom conference call is an impressive achievement.

Using audio to predict keystrokes is not new, but achieving this level of accuracy with the help of AI is. "When trained on keystrokes recorded by a nearby phone, the classifier achieved an accuracy of 95%, the highest accuracy seen without the use of a language model," the research paper writes. "When trained on keystrokes recorded using the video-conferencing software Zoom, an accuracy of 93% was achieved, a new best for the medium."

Using sound as the basis does have limitations for using this tech for malicious purposes outside of being limited to a specific subset of keyboards. The findings note that switching to touch-type as a means of input dramatically reduces the accuracy to around 40 - 64% - which makes you wonder if that recent loud, clicky mechanical keyboard purchase was a good idea.

Then there's AI-based noise reduction software like NVIDIA's RTX Broadcast that removes keyboard sounds when you're on a video stream. Fighting AI with more AI is very cool.

To check out the new research paper in full, head here.