Visualizing Deep Similarity Networks

• NCJ Number: 254594
• Author(s): Abby Stylianou; Richard Souvenir; Robert Pless
• Date Published: 2019
• Length: 9 pages
• Grant Number(s): 2018-75-CX-0038
• Agencies: NIJ-Sponsored
• Publication Type: Research (Applied/Empirical)
• Award: An Object-Centric Approach for Image Analysis to Combat Human Trafficking

Annotation

For convolutional neural network models that optimize an image embedding, this article proposes a method to highlight the regions of images that contribute most to pairwise similarity.

Abstract

This work is a corollary to the visualization tools developed for classification networks, but applicable to the problem domains better suited to similarity learning. The visualization shows how similarity networks that are fine-tuned learn to focus on different features. This approach is generalized to embedding networks that use different pooling strategies and provides a simple mechanism to support image similarity searches on objects or sub-regions in the query image. (publisher abstract modified)