Video surveillance systems play an important role in public safety systems by allowing the monitoring of larger areas with fewer personnel. The goal of this proposal is to integrate advances in wireless networking and computer vision research to improve public surveillance systems. Existing camera-based surveillance systems typically rely on statically installed cameras to capture videos streams, which are then manually reviewed by law enforcement personnel. This is not only a relatively slow process, but also sometimes ineffective, since static cameras generally cannot be dynamically positioned to capture ad hoc events like flash mobs, street parties, and so on. Furthermore, most of the video cameras used in surveillance systems may not perform well in environments such as during the night, during inclement weather, and so on, due to poor lighting conditions. Our project proposes a surveillance system that uses 3D cameras affixed to moving vehicles, such as police squad cars. High speed wireless broadband networks are used to transmit the collected data for real time video processing and QoS adjustment. 3D cameras, unlike their 2D counterparts, perform far better under poor light environments. While individual components like wireless broadband networks and 3D video cameras already exist, the combination of these technologies to address a practical problem still requires addressing challenging problems. Local campus police department has agreed to provide domain expertise and facilitate testing of our prototype system.

Intellectual Merit:

Designing a real time mobile surveillance with 3D video cameras requires integrating research from wireless networks and computer vision to address challenging problems. This include (1) practical QoS adjustment (e.g. bit rate, video encoding rate, etc) that balances the requirements for timeliness of data transmission and quality needed for public safety requirements; (2) practical multi-radio handoffs between high speed broadband wireless networks (LTE/WiMAX) and local WiFi networks. The former has better performance has better coverage and speeds across large areas, whereas the latter often perform well in localized buildup areas, e.g. lots of buildings; (3) extending video processing algorithms, e.g. feature detection, target tracking, etc., to accommodate 3D video streams, which carry rich additional depth information not present in 2D video streams; and (4) prototype deployment and testing that can satisfy real world surveillance requirements.

Broader Impact:

The project has the potential to improve public safety surveillance systems, especially those deployed on large university campuses, since the underlying network infrastructure we rely on (e.g. WiMAX network, WiFi networks, and backend cluster computers) is generally available on most large campuses. Beyond impacts to public safety, this project will also have significant education impact. The PIs will train students in aspects to high speed broadband wireless networking and computer vision through inclusion of research into the undergraduate and graduate curriculums. In addition, the project include outreach efforts to underrepresented minorities through an existing REU Site. The PIs of this proposal are also PIs in an REU Site on Next Generational Networking Technologies at Temple University.

Results of video summarization