Visualizing Paired Image Similarity in Transformers
Transformers have shown promise for a wide range of computer vision tasks, including image embedding. We introduce a new method for producing interpretable visualizations that, given a pair of images encoded with a Transformer, show which regions contributed to their similarity. (Github)
Deep Randomized Ensembles for Metric Learning
We propose a generalizable and fast method to define a family of embedding functions that can be used as an ensemble to for deep metric learning. Each embedding function is learned by randomly bagging the training labels into small subsets. We show experimentally that these embedding ensembles create effective embedding functions on CUB-200-2011, Cars-196, In-Shop Clothes Retrieval and VehicleID. (Github)
Multi-Camera Human Activity Analysis
For the problem of human activity analysis using distributed camera networks, we designed computationally-efficient algorithms for multiple tasks in the analysis pipeline, including detection, tracking, action recognition. Our approach, dynamic camera selection, uses low-level cues in the scene to select the best camera(s) for prediction. For a variety of tasks, dynamic camera selection outperforms (in terms of accuracy and efficiency) related methods that integrate information from multiple cameras in the system.
Biological Trajectory Matching
We developed a rank-based distance metric learning method that combines user input and a new set of biologically-motivated features for biological trajectory matching. With a small or amount of user effort, our method outperforms existing trajectory matching methods on the problem of finding biologically-relevant instances of particular cell motion patterns. Matlab source code is available.
Image Manifold Visualization
This (Matlab) code can be used to interactively display 2-dimensional image manifolds. It was used as we wrote the paper: "A Survey of Manifold Learning for Images", Robert Pless and Richard Souvenir, IPSJ Transactions on Computer Vision and Applications, vol. 1 pp. 83-94, 2009.
Viewpoint Manifolds for Action Recognition
We model classes of atomic actions (e.g., punching, kicking, waving) as functions of the camera viewpoint. This allows us to cast action recognition as an optimization problem and also provide viewpoint estimates for single-view examples.
k-Manifolds: Nonlinear Subspace Clustering
We developed an iterative algorithm for clustering high-dimensional data points which are sampled from multiple, intersecting low-dimensional manifolds. This work extends manifold learning to classify and parameterize data which lie on multiple, intersecting manifolds. Details can be found in our ICCV 2005 paper and Matlab source code is available.
This project presents a hybrid model of data analysis targeted at complex, 3D motion. We created a framework blending interactive visual analysis with automated techniques (e.g., nonlinear dimensionality reduction and clustering). Click here for more information.