Researchers hacked a robotic vacuum cleaner to remotely record words and music

A team of researchers has shown that popular home robotic vacuum cleaners can be remotely hacked to act as microphones.

The researchers, including Nirupam Roy, an assistant professor in the University of Maryland Department of Computer Science, gathered information from the laser-based navigation system in a popular vacuum robot and applied signal processing and deep learning techniques to retrieve the word and identify the television programs being played. in the same room as the device.

The research demonstrates the potential for any device that uses light sensing and sensing technology (Lidar) to be manipulated for sound collection, despite not having a microphone. This work, which is a collaboration with Assistant Professor Jun Han at the University of Singapore, was presented at the Association for Computing Machinery’s Conference on Embedded Networked Sensor Systems (SenSys 2020) on November 18, 2020.

“We welcome these devices into our homes and don’t think about it,” said Roy, who holds a joint appointment at the University of Maryland’s Institute of Advanced Computer Studies (UMIACS). “But we have shown that even if these devices don’t have microphones, we can reuse the systems they use for navigation to spy on conversations and potentially reveal private information.”

Lidar navigation systems in home vacuum robots project a laser beam around a room and perceive the reflection of the laser as it bounces off nearby objects. The robot uses reflected signals to map the room and avoid collisions as it moves through the house.

Privacy experts have suggested that the maps created by vacuum robots, which are often stored in the cloud, pose potential privacy breaches that could allow advertisers to access information on things like the size of the house, which suggests the level of income and other lifestyle information. Roy and his team wondered whether these robots ‘Lidar could also pose potential safety risks as sound recording devices in users’ homes or businesses.

Sound waves cause objects to vibrate, and these vibrations cause slight variations in the light bouncing off an object. Laser microphones, used in espionage since the 1940s, are able to convert those variations back into sound waves. But laser microphones rely on a focused laser beam that reflects off very smooth surfaces, such as glass windows.

A Lidar vacuum cleaner, on the other hand, scans the environment with a laser and detects scattered light back from objects of irregular shape and density. The diffuse signal received by the vacuum sensor provides only a fraction of the information needed to recover the sound waves. Researchers weren’t sure if a robot vacuum’s Lidar system could be manipulated to function as a microphone and if the signal could be interpreted into meaningful sound signals.

First, the researchers hacked a robot vacuum to show that they could control the position of the laser beam and send the detected data to their laptops via Wi-Fi without interfering with the device’s navigation.

Subsequently, they conducted experiments with two sound sources. One source was a human voice reciting numbers played over computer speakers and the other was the audio of a variety of television programs played through a TV soundbar. Roy and his colleagues then captured the laser signal detected by the vacuum navigation system as it bounced off a variety of objects placed close to the sound source. Items included a garbage can, cardboard box, take-out container, and polypropylene bag, items that would normally be found on a typical floor.

The researchers relayed the received signals through deep learning algorithms that were trained to match human voices or to identify musical sequences from television programs. Their computer system, which they call LidarPhone, identified and matched the spoken numbers with 90% accuracy. It also identified one-minute recording television shows with greater than 90% accuracy.

“This type of threat may be more important than ever when you consider that we all order food over the phone and have computer meetings, and often talk about our credit card or bank information,” Roy said. “But what’s even more troubling to me is that it can reveal a lot more personal information. This kind of information can tell you about my lifestyle, how many hours I’m working, other things I’m doing. And what we watch on TV can reveal. our political orientations. This is crucial for someone who might want to manipulate political elections or deliver very specific messages to me. “

The researchers point out that vacuum cleaners are just one example of a potential vulnerability to Lidar-based spying. Many other devices could be subject to similar attacks such as smartphone infrared sensors used for facial recognition or passive infrared sensors used for motion detection.

“I believe this is significant work that will make manufacturers aware of these possibilities and prompt the security and privacy community to find solutions to prevent these types of attacks,” Roy said.

---

---

More information:
The research paper, “Spying with Your Robot Vacuum Cleaner: Lidar Sensor Interceptions”, Sriram Sami, Yimin Dai, Sean Rui Xiang Tan, Nirupam Roy and Jun Han, was presented on November 18, 2020 at the Association for Computing Machinery, SenSys 2020.