Concept-based query expansion and interactive visualization for web image search

Abstract

Even though Web image search queries are often ambiguous, traditional search engines promote not only the most common interpretations of the query, which makes it difficult for the searcher to find the desired images. For addressing this problem, a concept-based query expansion technique is used to generate a diverse range of images covering multiple interpretations of the query. Then, a multi-resolution extension of a Self-Organizing Map is used to group conceptually and visually similar images. The resulting interface, named concept-based image search system (CBIS), allows the search to interactively highlight and filter images based on the concepts and zoom into an area within the image space to show additional images that are similar. Finally, a query refinement approach is proposed, which enables enhancement of queries based on explicitly selecting concepts or extracting concepts from example images. A series of evaluations illustrate the potential benefits of the system.