Modeling Objects, Concepts, Aesthetics and Emotions in Images

James Z. Wang
Professor, College of Information Sciences and Technology
The Pennsylvania State University
Wachman 447
Wednesday, November 13, 2013 - 11:00
Automated annotation of digital pictures has been a highly challenging open research problem. In our work, by advancing statistical modeling and optimization techniques, we can train computers about hundreds of semantic concepts using example pictures. The ALIPR (Automatic Linguistic Indexing of Pictures - Real Time) system of automatic and high-speed annotation for online pictures has been constructed. Large collections of pictures from Internet photo-sharing sites, unrelated to the source of those pictures used in the training process have been tested. The experimental results show that a single computer processor can suggest annotation terms in real-time and with good accuracy. The talk will also introduce our related research efforts on modeling visual aesthetics and emotions. A comprehensive system has been developed to enhance the aesthetic quality of the photographs captured by the mobile consumers. The system, named OSCAR, has been designed to provide on-site composition and aesthetics feedback through retrieved examples.