Privacy Respecting Video Surveillance

Motivation

Privacy-respecting video surveillance introduces a conceptual framework for lawful, smart video surveillance that enforces privacy-related constraints based on the current threat situation and the type of incident that has been detected. It constitutes the first step towards situation-dependent smart video surveillance and helps to increase security while protecting the observed peoples’ privacy. Video surveillance systems are a powerful tool for improving public security and the prosecution of crime. So-called smart video surveillance systems support the operator in evaluating video data. Image processing algorithms process captured live data and search for critical events, e.g., violence.

Systems such as these are making video analysis algorithms for activity recognition, tracking, and biometric identification of persons a reality. However, while these developments aim to improve the effectiveness and efficiency of video surveillance, they coincidentally increase the diversity and intensity of intrusions into the observed individuals’ privacy and create new potential for misuse. Legal scholars agree that applicable data protection laws, i.e. the German Federal Data Protection Act (BDSG) or the European General Data Protection Regulation (GDPR), do not sufficiently cover the technological evolution of smart video surveillance and demand further regulations. In this sense, they expect that the lawful operation of smart video surveillance will require effective mechanisms for protecting privacy and preventing misuse.

Privacy in security applications

To cope with the privacy risk while still increasing security, Fraunhofer IOSB has developed a generic privacy-aware video surveillance architecture that operates in three distinct modes. The ‘Default Mode’ is active most of the time. In this mode, no critical activity is observed and therefore the impact on privacy must be limited. The operator has only limited access to information and functionality, e.g., all video data is artificially pixelated. ‘Assessment Mode’ is activated once an algorithm, e.g., violence detection, or the operator raises an alarm. In this mode, the first indicators for a critical incident have been detected. The operator has access to extended information, e.g., high quality video streams, to assess the situation. ‘Investigation Mode’ is activated once a suspected incident is confirmed. In this mode, a given event has been assessed as critical. The operator may therefore use the highest level of functionality to prevent further damage. Algorithms with a high privacy impact, e.g., automatic person tracking and biometric identification become available. Once the incident is resolved, the system changes back to ‘Default Mode’.

This generic architecture has been adapted to different scenarios. In the security-related scenario “Unattended Luggage” a smart video surveillance system is deployed at an airport. In ‘Default Mode’, video is captured and stored in a 24-hour archive that is not accessible by the operator. Additionally, an algorithm searches for unattended pieces of luggage. If one is detected, the algorithm activates the ‘Assessment Mode’ and alerts the operator. If the operator confirms the alarm, the ‘Investigation Mode’ is unlocked. Here, the operator has access to the video recordings of the last 15 minutes and can use them to find the person who dropped the piece of luggage. Once the operator has found the person, he activates a tracking algorithm that reconstructs where the owner went after abandoning the luggage. Depending on the results, the operator decides if the area must be evacuated, e.g. the owner left the area in a hurry, or if a security officer should inform the owner about his left behind luggage, e.g. in case the owner went to a nearby kiosk. Once the incident has been resolved, the system is set back into its ‘Default Mode’.

Privacy in safety applications

The same concept can also be used in safety applications. In the safety-related “NurseEye” scenario, the system operates in nursing homes or hospitals. In ‘Default Mode’, an algorithm designed to detect people falling to the floor processes all video data. If no fall occurs, the system deletes the video data immediately. If the system believes a fall has occurred, it activates the ‘Assessment Mode’. First, the nearest member of the nursing staff receives an alarm on his smartphone. When he accepts the alarm, the staff member is granted access to an anonymized video so he can check whether there is an actual emergency. If the alarm is confirmed, the system enters ‘Investigation Mode’. Here, the nurse has full access to the live video data to assess the current situation and decide on the optimum way to handle it, i.e. he decides if special equipment is needed. Once the faller has received help, the system returns to ‘Default Mode’ and deletes all the video data.