ROSE Object Search PlatformAPIs provided by ROSE
Compact Descriptor Visual Search (CDVS) API Test this API
CDVS API is an image retrieval api base on compact descriptor visual search standards.With the rising adoption of smart phones and tablet PCs, mobile multimedia services such as visual search and augmented reality are now becoming popular. To meet the increasing demands for searching landmarks on mobile devices, we have built up a low-bit rate mobile visual search system on a million-scale database of landmark images.This system can extract compact visual descriptors as low as 512 bytes on mobile clients and conduct efficient landmark retrieval on this million-scale database.
One highlight of the system is the compact descriptor, which is proposed to address the challenges of data transmission, power consumption, computing capability and memory limitation on mobile platform. In a mobile visual search scenario, it may take several seconds to directly send a JPEG query image over a slow wireless link and the power consumption will be high. To improve the user experience, we propose to directly extract compact visual descriptors on the mobile end. The main advantages of using the low bit rate descriptor are two-fold. First, the descriptors are expected to be compact and discriminative to reduce overall query delivery latency. Second, the extraction of the compact descriptor is surprisingly efficient and ensures impressive power saving. Therefore, the compact descriptor is the best choice for mobile search systems.
Recent research, development and standardization of compact descriptors for visual search involve numerous industry efforts from STMicroelectronics, Samsung, Qualcomm, Huawei, NEC, etc. In particular, this topic relates to an ongoing MPEG CDVS (Compact Descriptor for Visual Search) standardization. Our PKU team plays as a leading contributor and one of the draft editors of the standard.
Visual Object Search API Test this API
Visual object search in large-scale image and video datasets is gaining increasing traction from industry and academia in recent years. The research team led by Prof. Yuan Junsong has built an advanced visual object search system on a one million image dataset. Given a query object such as a logo, the search system is designed to efficiently retrieve all images containing the same logo from millions of images, as well as to accurately locate the object in these retrieved images (even when the logo is partially obscured or distorted). Such a search system is of great interest to many applications, including product search and recommendation, context-aware advertisement, etc. For example, the system can help the user to find a person's favorite product in the online shopping sites by simply snapping a picture of the product through mobile phone. Moreover, the system can also enable intelligent video surveillance applications such as quickly finding a suspicious person or vehicle in surveillance videos.
Shoe Search API Test this API
The Product Search team deals with the topic of visual fashion analysis for handbags and shoes. The researchers devote efforts in exploring the prior knowledge of these two types of products and capturing the fine details from the product image, to design search algorithms that perform favorably in recognizing/retrieving these products.
With the rapid growth of online shoe market, demand for effective shoe search systems has never been greater. This shoe search API is to find exactly the same shoes from the online shop (shop scenario), given a daily shoe photo (street scenario). We address the significant visual differences of these two scenarios in the triplet-based convolutional neural network (CNN). Specifically, we propose 1) the weighted triplet loss to reduce the feature distance between the same shoe in different scenarios; 2) attribute-based hard example mining process to distinguish fine-grained differences; 3) a novel viewpoint invariant loss to reduce ambiguous feature representation from different views.
Fashion Recommendation API Test this API
Recently, there is an increasing demand for fashion recommendation in the form that by specifying a query item and an occasion, the system is expected to return the rest matching items.
This fashion recommendation API is to return the matched clothing and accessories given a query fashion item. The most challenging thing is to model the compatibility relationships between different categories of the fashion items. We propose to investigate the LSTM model for modelling such matching relationships between varieties of fashion items. We consider the outfit as a collection of fashion item images in order (from the top to bottom body), which is utilized for the LSTM model training.