UW Security Researchers Show That Google’s AI Tool for Video Searching Can Be Easily Deceived
April 4, 2017 | University of WashingtonEstimated reading time: 2 minutes
University of Washington researchers have shown that Google’s new tool that uses machine learning to automatically analyze and label video content can be deceived by inserting a photograph periodically and at a very low rate into videos. After they inserted an image of a car into a video about animals, for instance, the system returned results suggesting the video was about an Audi.
Google recently released its Cloud Video Intelligence API to help developers build applications that can automatically recognize objects and search for content within videos. Automated video annotation would be a breakthrough technology, helping law enforcement efficiently search surveillance videos, sports fans instantly find the moment a goal was scored or video hosting sites weed out inappropriate content.
Google launched a demonstration website that allows anyone to select a video for annotation. The API quickly identifies the key objects within the video, detects scene changes and provides shot labels of the video events over time. The API website says the system can be used to “separate signal from noise, by retrieving relevant information at the video, shot or per frame” level.
In a new research paper, the UW electrical engineers and security researchers, including doctoral students Hossein Hosseini and Baicen Xiao and professor Radha Poovendran, demonstrated that the API can be deceived by slightly manipulating the videos. They showed one can subtly modify the video by inserting an image into it, so that the system returns only the labels related to the inserted image.
The same research team recently showed that Google’s machine-learning-based platform designed to identify and weed out comments from internet trolls can be easily deceived by typos, misspelling offensive words or adding unnecessary punctuation.
“Machine learning systems are generally designed to yield the best performance in benign settings. But in real-world applications, these systems are susceptible to intelligent subversion or attacks,” said senior author Radha Poovendran, chair of the UW electrical engineering department and director of the Network Security Lab. “Designing systems that are robust and resilient to adversaries is critical as we move forward in adopting the AI products in everyday applications.”
As an example, a screenshot of the API’s output is shown below for a sample video named “animals.mp4,” which is provided by the API website. Google’s tool does indeed accurately identify the video labels.
The researchers then inserted the following image of an Audi car into the video once every two seconds. The modification is hardly visible, since the image is added once every 50 video frames, for a frame rate of 25.
The following figure shows a screenshot of the API’s output for the manipulated video. As seen below, the Google tool believes with high confidence that the manipulated video is all about the car.
“Such vulnerability of the video annotation system seriously undermines its usability in real-world applications,” said lead author and UW electrical engineering doctoral student Hossein Hosseini. “It’s important to design the system such that it works equally well in adversarial scenarios.”
“Our Network Security Lab research typically works on the foundations and science of cybersecurity,” said Poovendran, the lead principal investigator of a recently awarded MURI grant, where adversarial machine learning is a significant component. “But our focus also includes developing robust and resilient systems for machine learning and reasoning systems that need to operate in adversarial environments for a wide range of applications.”
The research is funded by the National Science Foundation, Office of Naval Research and Army Research Office.
Suggested Items
Cadence, TSMC Collaborate on Wide-Ranging Innovations to Transform System and Semiconductor Design
04/25/2024 | Cadence Design SystemsCadence Design Systems, Inc. and TSMC have extended their longstanding collaboration by announcing a broad range of innovative technology advancements to accelerate design, including developments ranging from 3D-IC and advanced process nodes to design IP and photonics.
Siemens’ Breakthrough Veloce CS Transforms Emulation and Prototyping with Three Novel Products
04/24/2024 | Siemens Digital Industries SoftwareSiemens Digital Industries Software launched the Veloce™ CS hardware-assisted verification and validation system. In a first for the EDA (Electronic Design Automation) industry, Veloce CS incorporates hardware emulation, enterprise prototyping and software prototyping and is built on two highly advanced integrated circuits (ICs) – Siemens’ new, purpose-built Crystal accelerator chip for emulation and the AMD Versal™ Premium VP1902 FPGA adaptive SoC (System-on-a-chip) for enterprise and software prototyping.
Taiyo Circuit Automation Installs New DP3500 into Fuba Printed Circuits, Tunisia
04/25/2024 | Taiyo Circuit AutomationTaiyo Circuit Automation is proud to be partnered with Fuba Printed Circuits, Tunisia part of the OneTech Group of companies, a leading printed circuit board manufacturer based out of Bizerte, Tunisia, on their first installation of Taiyo Circuit Automation DP3500 coater.
Vicor Power Orders Hentec Industries/RPS Automation Pulsar Solderability Testing System
04/24/2024 | Hentec Industries/RPS AutomationHentec Industries/RPS Automation, a leading manufacturer of selective soldering, lead tinning and solderability test equipment, is pleased to announce that Vicor Power has finalized the purchase of a Pulsar solderability testing system.
Lockheed Martin Successfully Transitions Long Range Discrimination Radar To The Missile Defense Agency
04/23/2024 | Lockheed MartinThe Long Range Discrimination Radar (LRDR) at Clear Space Force Station in Clear, Alaska, completed DD250 final acceptance and was officially handed over to the Missile Defense Agency in preparation for an Operational Capability Baseline (OCB) decision and final transition to the Warfighter. In addition, prior to this transition, the system has started Space Domain Awareness data collects for the United States Space Force.