Research

Visualizing Paired Image Similarity in Transformers

Transformers have shown promise for a wide range of computer vision tasks, including image embedding. We introduce a new method for producing interpretable visualizations that, given a pair of images encoded with a Transformer, show which regions contributed to their similarity. (Github)

Deep Randomized Ensembles for Metric Learning

We propose a generalizable and fast method to define a family of embedding functions that can be used as an ensemble to for deep metric learning. Each embedding function is learned by randomly bagging the training labels into small subsets. We show experimentally that these embedding ensembles create effective embedding functions on CUB-200-2011, Cars-196, In-Shop Clothes Retrieval and VehicleID. (Github)

Multi-Camera Human Activity Analysis

For the problem of human activity analysis using distributed camera networks, we designed computationally-efficient algorithms for multiple tasks in the analysis pipeline, including detection, tracking, action recognition. Our approach, dynamic camera selection, uses low-level cues in the scene to select the best camera(s) for prediction. For a variety of tasks, dynamic camera selection outperforms (in terms of accuracy and efficiency) related methods that integrate information from multiple cameras in the system.

Biological Trajectory Matching

We developed a rank-based distance metric learning method that combines user input and a new set of biologically-motivated features for biological trajectory matching. With a small or amount of user effort, our method outperforms existing trajectory matching methods on the problem of finding biologically-relevant instances of particular cell motion patterns. Matlab source code is available.

Image Manifold Visualization

This (Matlab) code can be used to interactively display 2-dimensional image manifolds. It was used as we wrote the paper: "A Survey of Manifold Learning for Images", Robert Pless and Richard Souvenir, IPSJ Transactions on Computer Vision and Applications, vol. 1 pp. 83-94, 2009.

Viewpoint Manifolds for Action Recognition

We model classes of atomic actions (e.g., punching, kicking, waving) as functions of the camera viewpoint. This allows us to cast action recognition as an optimization problem and also provide viewpoint estimates for single-view examples.

k-Manifolds: Nonlinear Subspace Clustering

We developed an iterative algorithm for clustering high-dimensional data points which are sampled from multiple, intersecting low-dimensional manifolds. This work extends manifold learning to classify and parameterize data which lie on multiple, intersecting manifolds. Details can be found in our ICCV 2005 paper and Matlab source code is available.

Biomechanical Visualization

This project presents a hybrid model of data analysis targeted at complex, 3D motion. We created a framework blending interactive visual analysis with automated techniques (e.g., nonlinear dimensionality reduction and clustering). Click here for more information.

Selecting Multiple Degenerate PCR Primers

This work (described in this paper) describes an algorithm for finding a set of degenerate primers which cover a set of target locations. A C++ implementation is available.