1448018
Generating velocity tuning by asymmetric recurrent connections
Asymmetric lateral connections are one possible mechanism that can account for the direction selectivity of cortical neurons. We present a mathematical analysis for a class of these models. Contrasting with earlier theoretical work that has relied on methods from linear systems theory, we study the network's nonlinear dynamic properties that arise when the threshold nonlinearity of the neurons is taken into account. We show that such networks have stimulus-locked traveling pulse solutions that are appropriate for modeling the responses of direction selective cortical neurons. In addition, our analysis shows that outside a certain regime of stimulus speeds the stability of this solutions breaks down giving rise to another class of solutions that are characterized by specific spatio-temporal periodicity. This predicts that if direction selectivity in the cortex is mainly achieved by asymmetric lateral connections lurching activity waves might be observable in ensembles of direction selective cortical neurons within appropriate regimes of the stimulus speed.
1014578
Learning Curves for Stochastic Gradient Descent in Linear Feedforward Networks
Gradient-following learning methods can encounter problems of implementation in many applications, and stochastic variants are sometimes used to overcome these difficulties. We analyze three online training methods used with a linear perceptron: direct gradient descent, node perturbation, and weight perturbation. Learning speed is defined as the rate of exponential decay in the learning curves. When the scalar parameter that controls the size of weight updates is chosen to maximize learning speed, node perturbation is slower than direct gradient descent by a factor equal to the number of output units; weight perturbation is slower still by an additional factor equal to the number of input units. Parallel perturbation allows faster learning than sequential perturbation, by a factor that does not depend on network size. We also characterize how uncertainty in quantities used in the stochastic updates affects the learning curves. This study suggests that in practice, weight perturbation may be slow for large networks, and node perturbation can have performance comparable to that of direct gradient descent when there are few output units. However, these statements depend on the specifics of the learning problem, such as the input distribution and the target function, and are not universally applicable.
9393077
Spike-based Learning Rules and Stabilization of Persistent Neural Activity
We analyze the conditions under which synaptic learning rules based on action potential timing can be approximated by learning rules based on firing rates. In particular, we consider a form of plasticity in which synapses depress when a presynaptic spike is followed by a postsynaptic spike, and potentiate with the opposite temporal ordering. Such differential anti-Hebbian plasticity can be approximated under certain conditions by a learning rule that depends on the time derivative of the postsynaptic firing rate. Such a learning rule acts to stabilize persistent neural activity patterns in recurrent neural networks.
7144646
Learning Winner-take-all Competition Between Groups of Neurons in Lateral Inhibitory Networks
It has long been known that lateral inhibition in neural networks can lead to a winner-take-all competition, so that only a single neuron is active at a steady state. Here we show how to organize lateral inhibition so that groups of neurons compete to be active. Given a collection of potentially overlapping groups, the inhibitory connectivity is set by a formula that can be interpreted as arising from a simple learning rule. Our analysis demonstrates that such inhibition generally results in winner-take-all competition between the given groups, with the exception of some degenerate cases. In a broader context, the network serves as a particular illustration of the general distinction between permitted and forbidden sets, which was introduced recently. From this viewpoint, the computational function of our network is to store and retrieve memories as permitted sets of coactive neurons.
2399803
Exact solution of the nonlinear dynamics of recurrent neural mechanisms for direction selectivity
Abstract Different theoretical models have tried to investigate the feasibility of recurrent neural mechanisms for achieving direction selectivity in the visual cortex. The mathematical analysis of such models has been restricted so far to the case of purely linear networks. We present an exact analytical solution of the nonlinear dynamics of a class of direction selective recurrent neural models with threshold nonlinearity. Our mathematical analysis shows that such networks have form-stable stimulus-locked traveling pulse solutions that are appropriate for modeling the responses of direction selective cortical neurons. Our analysis shows also that the stability of such solutions can break down giving raise to a different class of solutions (“lurching activity waves”) that are characterized by a specific spatio-temporal periodicity. These solutions cannot arise in models for direction selectivity with purely linear spatio-temporal filtering.
307226
Selectively Grouping Neurons in Recurrent Networks of Lateral Inhibition
Winner-take-all networks have been proposed to underlie many of the brain's fundamental computational abilities. However, notmuchisknown about how to extend the grouping of potential winners in these networks beyond single neuron or uniformly arranged groups of neurons. We show that competition between arbitrary groups of neurons can be realized by organizing lateral inhibition in linear threshold networks. Given a collection of potentially overlapping groups (with the exception of some degenerate cases), the lateral inhibition results in network dynamics such that any permitted set of neurons that can be coactivated by some input at a stable steady state is contained in one of the groups. The information about the input is preserved in this operation. The activity level of a neuron in a permitted set corresponds to its stimulus strength, amplified by some constant. Sets of neurons that are not part of a group cannot be coactivated by any input at a stable steady state. We analyze the storage capacity of such a network for random groupsthe number of random groups the network can store as permitted sets without creating too many spurious ones. In this framework, we calculate the optimal sparsity of the groups (maximizing group entropy). We find that for dense inputs, the optimal sparsity is unphysiologically small. However, when the inputs and the groups are equally sparse, we derive a more plausible optimal sparsity. We believe our results are the first steps toward attractor theories in hybrid analog-digital networks.
27158603
A simulation-based optimization method for production-distribution network design
Production-distribution network design is a critical decision that has significant impacts on a supply chain's long-term performance. Supply chain dynamics, such as demand fluctuation, and transportation instability, are omitted in most mathematical models due to tractability. We present a simulation-based optimization method in this paper for multi-criteria production-distribution network design. The method consists of a multiobjective optimizer and a simulation module. The optimizer, based on a multiobjective genetic algorithm, is used to direct the search for compromised solutions regarding to various criteria. Candidate solutions are evaluated by the discrete-event simulation module, developed in a flexible manner that enables automatic simulation of various supply chain structures without human intervention. The method is applied to a case study from automotive industry. A set of Pareto-optimal solutions are obtained, including decisions on the open/close decisions on facilities, order splitting ratios and inventory control policies.
1308804
Texture Exemplars for Defect Detection on Random Textures
We present a new approach to detecting defects in random textures which requires only very few defect free samples for unsupervised training. Each product image is divided into overlapping patches of various sizes. Then, density mixture models are applied to reduce groupings of patches to a number of textural exemplars, referred to here as texems, characterising the means and covariances of whole sets of image patches. The texems can be viewed as implicit representations of textural primitives. A multiscale approach is used to save computational costs. Finally, we perform novelty detection by applying the lower bound of normal samples likelihoods on the multiscale defect map of an image to localise defects.
6641983
An adaptive web page layout structure for small devices
The limited display size of current small Internet devices is becoming a serious obstacle to information access. In this paper, we introduce a Document REpresentation for Scalable Structures (DRESS) to help information providers make composite documents, typically web pages, scalable in both logic and layout structure to support effective information acquisition in heterogeneous environments. Through this novel document representation structure based on binary slicing trees, the document can dynamically adapt its presentation according to display sizes by maximizing the information throughput to users. We discuss the details of this structure with its key attributes. An automatic approach for generating this structure for existing web pages is also presented. A branch-and-bound algorithm and a capacity ratio-based slicing method are proposed to select proper content representation and aesthetic document layouts respectively. A set of user study experiments have been carried out and the results show that compared with the thumbnail-based approach, the DRESS-based interface can reduce browsing time by 23.5%.
8828427
Detecting geographic locations from web resources
The rapid pervasion of the web into users' daily lives has put much importance on capturing location-specific information on the web, due to the fact that most human activities occur locally around where a user is located. This is especially true in the increasingly popular mobile and local search environments. Thus, how to correctly and effectively detect geographic locations from web resources has become a key challenge to location-based web applications. In our previous work, we proposed to explicitly distinguish three types of locations for web resources, namely provider location, content location and serving location. Provider location is the physical location of the provider who owns the web resource; content location is the geographic location described in the web content; while serving location is the geographic scope that a web resource can reach. In this paper, we present a system that comprehensively employs a set of algorithms and different geographic sources by extracting geographic information from the web content, and mining hyperlink structures as well as user logs. As the result, only relevant geographic sources, rather than all of possible ones are used in computation of each category of web location. Finally, experimental results on large samples of web data show that our solution outperforms previous approaches.
10678156
Detecting The Sufficient Display Resolution For Image Browsing
In image browsing, the resolution greatly affects user’s experience. If an image is down-scaled too much, a considerable amount of information within it will be lost. In this paper, we studied the problem of "What is a sufficient display resolution or scale for an image or an image region?" This problem arises in many real-life applications including image browsing on mobile devices, image adaptation and progressive image delivery. Kullback-Leibler (K-L) distance is employed to measure the information loss and the sufficient display scale is selected based on the information loss curve during image down-sampling. Since the images are presented to viewers finally, some visual characteristics are also taken into account to ensure the precision of the measurement. A user study was carried out to evaluate the performance of our approach. Experimental results show that the approach is in good accord with human perception.