10740927
Photo-to-search: using multimodal queries to search the web from mobile devices
Nowadays, mobile phones with the digital camera are getting more and more popular. With necessary technologies, they are possible to become a powerful tool to search the Web on the go. Most Web search engines only support text queries. Therefore, users have to convert their information needs into words. However, it is sometimes difficult to describe the needs in text and the text input is inconvenient on small devices. To solve the problem, we propose a system named Photo-to-Search which allows users to input multimodal queries. Particularly, we study queries with captured images and optional text messages in this paper. For example, the user can simply take a photo of the flower and input a few terms like "flower". Textually relevant Web images are retrieved according to the query terms. Afterwards, the snapped picture is compared with these images by the CBIR (Content Based Image Retrieval) method. According to the context of the visually similar images, related key phrases are extracted. Finally, the search results are returned in multiple forms. Our system can also search for very similar images on the Web, such as movie posters or photos of film stars, to find related information. Experimental results on the large scale data showed our system achieved satisfactory efficiency and performance.
18031163
Browsing large pictures under limited display sizes
Pictures have become increasingly common and popular in mobile communications. However, due to the limitation of mobile devices, there is a need to develop new technologies to facilitate the browsing of large pictures on the small screen. In this paper, we propose a set of novel approaches which are able to aid or automate common image browsing tasks on mobile devices. All of these approaches are based on an image attention model which is employed to illustrate the information structure within an image. An efficient algorithm to generate the optimal browsing path based on the information foraging theory is presented. Experimental evaluations of the proposed mechanism indicate that our approach is an effective way for viewing large images on small displays
7137580
Adapting Web pages for small-screen devices
We propose a page-adaptation technique that splits existing Web pages into smaller, logically related units. To do this, we must first solve two technical problems: how to detect an existing Web page's semantic structure, and how to split a Web page into smaller blocks based on that structure. To date, we've implemented our technique in Web browsers for mobile devices, in a proxy server for adapting Web pages on the fly, and as an authoring tool plug-in for converting existing Web pages. The Web page can then be adapted to form a two-level hierarchy with a thumbnail representation at the top level for providing a global view and an index to a set of subpages at the bottom level for detailed information.
11709445
Learning user interest for image browsing on small-form-factor devices
Mobile devices which can capture and view pictures are becoming increasingly common in our life. The limitation of these small-form-factor devices makes the user experience of image browsing quite different from that on desktop PCs. In this paper, we first present a user study on how users interact with a mobile image browser with basic functions. We found that on small displays, users tend to use more zooming and scrolling actions in order to view interesting regions in detail. From this fact, we designed a new method to detect user interest maps and extract user attention objects from the image browsing log. This approach is more efficient than image-analysis based methods and can better represent users' actual interest. A smart image viewer was then developed based on user interest analysis. A second experiment was carried out to study how users behave with such a viewer. Experimental results demonstrate that the new smart features can improve the browsing efficiency and are a good compliment to traditional image browsers.
37379077
Improving Web Browsing on Small Devices Based on Table Classification
Browsing large web pages on small screens is still very inconvenient due to their limited display sizes. A straightforward solution is eliminating the annoying horizontal scrolling requirement, i.e. present all the contents into a single narrow column, usually named one-column view. It is implemented by deleting the layout that will cause horizontal scrolling. However, after deleting the layout structure, some structural information becomes unreadable, like data tables. In this paper, we propose an approach to improve web browsing on small devices by classifying HTML tables into data tables and layout tables. For data tables, we try to preserve the original structural information in the final presentation. Therefore, the browsing experience can be improved for the one-column view. Experimental results show that our approach can achieve a satisfactory performance.
10639273
Maximizing information throughput for multimedia browsing on small displays
As a great many new devices with diverse capabilities are having a population boom, their limited display sizes become the major obstacle that has undermined their usefulness for information access. We introduce our recent research on adapting multimedia content, including images, videos and Web pages, for browsing on small-form-factor devices. A theoretical framework as well as a set of novel methods for presenting and rendering multimedia under limited screen sizes is proposed to improve the user experience. The content modeling and processing are provided as subscription-based Web services on the Internet. Experiments show that our approach is extensible and able to achieve satisfactory results with high efficiency.
9393077
Spike-based Learning Rules and Stabilization of Persistent Neural Activity
We analyze the conditions under which synaptic learning rules based on action potential timing can be approximated by learning rules based on firing rates. In particular, we consider a form of plasticity in which synapses depress when a presynaptic spike is followed by a postsynaptic spike, and potentiate with the opposite temporal ordering. Such differential anti-Hebbian plasticity can be approximated under certain conditions by a learning rule that depends on the time derivative of the postsynaptic firing rate. Such a learning rule acts to stabilize persistent neural activity patterns in recurrent neural networks.
307226
Selectively Grouping Neurons in Recurrent Networks of Lateral Inhibition
Winner-take-all networks have been proposed to underlie many of the brain's fundamental computational abilities. However, notmuchisknown about how to extend the grouping of potential winners in these networks beyond single neuron or uniformly arranged groups of neurons. We show that competition between arbitrary groups of neurons can be realized by organizing lateral inhibition in linear threshold networks. Given a collection of potentially overlapping groups (with the exception of some degenerate cases), the lateral inhibition results in network dynamics such that any permitted set of neurons that can be coactivated by some input at a stable steady state is contained in one of the groups. The information about the input is preserved in this operation. The activity level of a neuron in a permitted set corresponds to its stimulus strength, amplified by some constant. Sets of neurons that are not part of a group cannot be coactivated by any input at a stable steady state. We analyze the storage capacity of such a network for random groupsthe number of random groups the network can store as permitted sets without creating too many spurious ones. In this framework, we calculate the optimal sparsity of the groups (maximizing group entropy). We find that for dense inputs, the optimal sparsity is unphysiologically small. However, when the inputs and the groups are equally sparse, we derive a more plausible optimal sparsity. We believe our results are the first steps toward attractor theories in hybrid analog-digital networks.
22043660
Colour tonality inspection using eigenspace features
In industrial quality inspection of colour texture surfaces, such as ceramic tiles or fabrics, it is important to maintain a consistent colour shade or tonality during production. We present a multidimensional histogram method using a novelty detection scheme to inspect the surfaces. The image noise, introduced by the imaging system, is found mainly to affect the chromatic channels. For colour tonality inspection, the difference between images is very subtle and comparison in the noise dominated chromatic channels is error prone. We perform vector-ordered colour smoothing and extract a localised feature vector at each pixel. The resulting histogram represents an encapsulation of local and global information. Principal component analysis (PCA) is performed on this multidimensional feature space of an automatically selected reference image to obtain reliable colour shade features, which results in a reference eigenspace. Then unseen product images are projected onto this eigenspace and compared for tonality defect detection using histogram comparison. The proposed method is compared and evaluated on a data set with groundtruth.
10241359
Localising surface defects in random colour textures using multiscale texem analysis in image eigenchannels
A novel method is presented to detect defects in random colour textures which requires only a very few normal samples for unsupervised training. We decorrelate the colour image by generating three eigenchannels in each of which the surface texture image is divided into overlapping patches of various sizes. Then, a mixture model and EM is applied to reduce groupings of patches to a small number of textural exemplars, or texems. Localised defect detection is achieved by comparing the learned texems to patches in the unseen image eigenchannels.
1308804
Texture Exemplars for Defect Detection on Random Textures
We present a new approach to detecting defects in random textures which requires only very few defect free samples for unsupervised training. Each product image is divided into overlapping patches of various sizes. Then, density mixture models are applied to reduce groupings of patches to a number of textural exemplars, referred to here as texems, characterising the means and covariances of whole sets of image patches. The texems can be viewed as implicit representations of textural primitives. A multiscale approach is used to save computational costs. Finally, we perform novelty detection by applying the lower bound of normal samples likelihoods on the multiscale defect map of an image to localise defects.