AiSearch enables flexible image and video search with user feedback
AiSearch: Interactive Multi-Modal Search with VLMs
Artificial IntelligenceComputer Vision and Pattern Recognition
Summary
Finding the right pictures or videos can be hard and often requires going back and forth to get better results. The authors created AiSearch, a tool that lets people search using natural language and also improve their search results by giving feedback. AiSearch works with different AI models trained to understand both pictures and words, helping users pick the model that fits their needs best. This makes searching visual content faster and more interactive.
What this means in practice
- •For digital media teams: Improve visual content search workflows by allowing interactive filtering and choosing the best AI models for different media projects.
- •For social media platform developers: Integrate real-time multimodal search with user feedback to enhance content discovery and user engagement.$Commercial implications: Enables platforms to offer advanced, marketable search features that improve user experience and retention at scale.
Authors
Ali Koksal, Mei Chee Leong, Vicky Sintunata, Ching Ling Chin, Wee Teck Fong
Abstract
Modern retrieval systems must both be automated and interactive, allowing users to search and refine results in real time. We present AiSearch, a flexible multimodal retrieval framework that leverages the zero shot capabilities of Vision Language Models (VLMs) for natural language search over images and videos. AiSearch supports interactive search refinement through user feedback to tailor results to the user's intent, and allows visual benchmarking across multiple VLMs, enabling users to select the most suitable model for their task.