You are an expert at summarizing long articles. Proceed to summarize the following text: 
many neural network models@xcite@xcite ,  in addition to their potential applications to computation , robotics and artificial intelligence , constitute intriguing dynamical systems in their own right , showing unusual manifestations of the statistical mechanics phenomena of order , disorder and frustration . the connection between neural networks and statistical mechanics became especially clear with the introduction of the hopfield @xcite@xcite model ,  which furnishes a model of associative memory , or the recall of a memorized pattern from an incomplete stimulus . this model has a well - defined energy function and is closely related to the sherrington - kirkpatrick spin glass model @xcite@xcite .    in this paper we consider a hopfield - like network of @xmath0 bistable elements , the bistable gradient network or bgn , previously introduced in @xcite . a closely related model was also discussed in @xcite and suggested as a model for the so - called `` bistability of perception '' in the interpretation of ambiguous visual stimuli @xcite . the network s dynamics consists of a continuous gradient descent described by the coupled differential equations @xmath1 where @xmath2 are continuous - valued real state variables associated with the @xmath3 nodes of the network and the hamiltonian or energy function is given by @xmath4 the quantities @xmath5 are a symmetric matrix of coupling strengths , and the quantities @xmath6 are bias terms or external inputs to the network . for the remainder of this paper we will set all @xmath7 unless otherwise stated ;  we include them here only for the sake of generality . @xmath8 is a control parameter determining the strength of the internode couplings relative to the local terms . the variables @xmath9 can be viewed as components of an @xmath3-dimensional state vector @xmath10 . we define a normalized inner product between two state vectors @xmath11 and @xmath12 by @xmath13 . the first term in the hamiltonian represents a local double - well potential for each node , making each node individually bistable . this local potential constitutes the main difference between the bgn and the hopfield model . the classical hopfield network ( hn ) which we consider by way of comparison is described by the hamiltonian @xmath14 where the @xmath9 are now _ discrete _ state variables restricted to the values @xmath15 . although continuous versions of  the hn have also been studied ,  these generally lack the bistability property , and their behaviour is essentially similar to that of the discrete version @xcite . the variables @xmath9 can be thought of as the outputs of proccessing units or neurons . their dynamical equations can be written as @xmath16 where @xmath17 is the input to the neuron from internal and external connections . by analogy with ising spin systems , we also refer to @xmath18 as a magnetic field . the steady - state output for a given input is a solution of the fixed - point equation @xmath19 when @xmath20 , there are stable fixed points at @xmath21 and an unstable fixed point at @xmath22  an applied field shifts the positions of the fixed points . a saddle - node bifurcation occurs when @xmath23 so that for larger values of the field there is only one equilibrium , aligned with the field ( @xmath24 and @xmath25 have the same sign ) . @xmath24 is in principle unbounded ;  the output does not truly saturate when the input is large . the double - valuedness and the lack of saturation are the principal differences between the input - output relation for the bgn and that of the hopfield model , including its continuous  versions . numerous experimental studies have been made on intriguing chemical analogs of the bgn ( see , e.g. @xcite , @xcite ) . these studies involved networks of bistable chemical reactors coupled either electrically or through mass exchange . previous computational work on small bgn s @xcite  suggested that under some conditions the network might permit the storage of a larger number of patterns than in a hn of the same size , without any modification of the basic hebb learning rule . it was noted , however , that the stability of a particular attractor can depend on the control parameter @xmath8 . some dependence of pattern stability on the coupling strength had also been noted in the experiments on the mass - coupled chemical networks @xcite .     in this paper we focus on the behaviour of the network in the case where the number of nodes is large and the number of memorized patterns is small .  using both analytical techniques and numerical simulations fell below a convergence criterion which for most examples was taken as 0.001 . our hn simulations , which were used for comparisons , were performed using asynchronous updating in random order . unless otherwise indicated , all simulations were done on an @xmath26 network with @xmath27 stored memory patterns . ] , we examine the retrieval of stored patterns and classify the attractors that occur . we find that there are three types of attractors . in addition to _ memory _ or _ retrieval states _ , there are spurious attractors in which no pattern is fully recognized . these include the _ mixture _ or _ spin glass _ _ states _ familiar from hn studies , as well as an additional category specific to the bgn which we refer to as _ uncondensed states_.  we examine how the attractors and their basins of attraction change as the control parameter @xmath8 is changed .  throughout the paper , we compare our model to the zero - temperature or deterministic discrete hopfield model . it is hoped that these results can illuminate some of the novel behavior of the bgn and clarify its relation to the hn . the behavior of the bgn under higher memory loading and the question of its maximum storage capacity will be addressed elsewhere . as in previous work on hopfield networks@xcite@xcite@xcite , we define the task of associative memory as follows . we are given a set of  @xmath28 distinct @xmath3-dimensional vectors or _ memory patterns _  @xmath29 @xmath30 which are to be recognized by the network . the patterns should correspond to attractors of the network s dynamics . we will refer to these attractors as _ retrieval states . _  input is given by imposing a particular initial condition on the network .  if that initial condition is sufficiently close to one of the memorized patterns , then the network s state should converge to the correct nearby attractor , and we say that the network has recognized or retrieved the pattern . in this paper we follow the hn literature in considering the case where the patterns are random and uncorrelated strings of @xmath31 s and @xmath32 s . we read the output of the network according to the _ signs _ of the @xmath9 . thus we say that the network has recalled pattern 1 , for example , if @xmath33 for all @xmath34 . although variations in the magnitude of @xmath9 can be important to the dynamics ,  we will for the moment ignore them for the purpose of reading the output .  as we will see below , the retrieval states in general do not have @xmath35 even though the patterns have @xmath36  we focus here on the limiting case @xmath37 , @xmath38 ,  or large networks with low memory loading . ( strictly speaking , we take @xmath3 to infinity while @xmath28 remains finite . )  in this case the inner product of a pair of patterns * *  * * @xmath39 behaves as a gaussian random variable with zero mean and variance @xmath40 ,  so that in the @xmath37 limit the pattern vectors are nearly orthogonal to each other  and form a basis for a @xmath28-dimensional subspace of the @xmath3-dimensional configuration space .    as in the hn , we construct the coupling matrix from the stored patterns according to the hebb @xcitelearning rule : @xmath41 the term @xmath42 is included to make all diagonal elements of the coupling matrix zero . non - zero diagonal entries would have the effect of adding an additional quadratic self - interaction term . following the usual practice we omit them here so that the quadratic term is contained only in the local potential .  for the case @xmath43 , however , the effect of the diagonal elements is negligible and we can substitute the simpler learning rule @xmath44    a useful set of order parameters are the overlaps @xmath45 , which are inner products of the network s state with each of the stored patterns : @xmath46 for the discrete hn , these variables take values @xmath47 ,  while for the bgn any real values are possible .  it will be useful to define another set of variables , which we will call `` bit overlaps , '' by should not be confused with the biases or external fields @xmath6 in ( [ hamiltonian ] ) , which are set to zero for the remainder of this paper .  ] @xmath48  the bit overlap is simply related to the hamming distance by  @xmath49 where the hamming distance @xmath50 between two vectors is defined as the number of elements for which their signs differ , or the number of positions @xmath34 such that @xmath51 . unlike @xmath52 the bit overlaps always obey @xmath53 . they encode information about sign agreements but not about magnitudes of the outputs @xmath9 . the definitions of the overlap variables allow us to rewrite the hamiltonian and the dynamical equations in useful forms . @xcite  in particular , if the synaptic matrix @xmath54 is given by the simplified hebb rule ( [ simplehebb ] ) , then the interaction term of the hamiltonian can be rewritten in terms of @xmath45 as follows : @xmath55 and the net input to a given node from the other nodes is given by @xmath56      to show that the network functions properly as an associative memory , we exhibit the attractor states corresponding to the stored patterns , demonstrate their stability , and show that a pattern can be retrieved from an initial condition which lies close to the pattern but differs from it by one or more incorrect signs . consider the state @xmath57 where @xmath58 is a scalar and @xmath59 is a particular one of the stored patterns . we will show that for a suitable value of @xmath58 this state represents a stable fixed point of the dynamics and is therefore the retrieval state we seek . in this state , @xmath60 , @xmath61 , and all other overlaps are small . the field acting on the @xmath34-th node can be written as follows:@xmath62 the sum over patterns @xmath63 is called the crosstalk term . for the overlaps with these other patterns we have@xmath64 since the patterns are random , each of these overlaps is of order @xmath65 . the number of patterns remains finite as @xmath37 , so the sum in the crosstalk term vanishes in this limit and @xmath66 .  a stationary state must satisfy the fixed point condition ( [ fixedpt ] ) for each node , which leads to a self - consistency condition on @xmath58 : @xmath67 in the last step , we have used the fact that @xmath68 when @xmath69 and then divided out the common factor of @xmath70 . solutions to this condition are an unstable equilibrium @xmath71 and two stable equilibria@xmath72 the two stable solutions represent perfect retrieval of pattern @xmath73 and its mirror state , respectively . the doubled state is a consequence of the @xmath74 symmetry in the hamiltonian .  since the overlap @xmath75 is equal to @xmath58 and all other overlaps vanish in the thermodynamic limit , the energy of this retrieval state is easily calculated using expression ( [ mham ] ) for the energy in terms of the overlap variables , giving:@xmath76 note that this energy expression is extensive ( proportional to @xmath3 ) and a monotonically decreasing function of @xmath8 . having identified the state @xmath77 as an equilibrium state , we now demonstrate its stability using a linear stability analysis of the dynamical equations@xmath78 evaluating the jacobian @xmath79 at the fixed point ( [ retrievalstate ] ) we obtain :   @xmath80 where @xmath81 is the kronecker delta . the fixed point is linearly stable if and only if the jacobian has no positive eigenvalues . this depends in turn on the eigenvalues of the synaptic connection matrix @xmath54 . but in the limit where all of the stored patterns @xmath29 are mutually orthogonal , the stored patterns are themselves eigenvectors spanning a degenerate subspace with eigenvalue 1 , while the complement of this subspace has eigenvalue 0 . ( the hebb rule ( [ simplehebb ] ) itself gives a spectral decomposition of @xmath54 . ) since the maximum eigenvalue of @xmath54 is 1 ,  we see that the jacobian at the retrieval fixed point has no positive eigenvalues and so the retrieval state is linearly stable for any value of @xmath8 . in fact , all eigenvalues become more negative as @xmath8 increases . @xmath82 we reiterate that this result is valid in the ideal limit of large @xmath3 and low loading where the stored patterns are orthogonal .  for finite - sized networks with finite overlaps among the patterns ,  it is possible for the memory states to be destabilized by the crosstalk terms . this issue will be examined elsewhere . numerical results for a network with @xmath26 nodes and @xmath27 random patterns agree excellently with the above description .  to study a retrieval state numerically , we initialized the network to the state @xmath83 ( arbitrarily choosing the first pattern ) .  starting at @xmath84 ,  we increased @xmath8 by small steps to @xmath85 . at each step , we integrated the dynamical equations until they converged . this procedure allows us to examine the evolution of a state under quasistatic changes in the control parameter @xmath86  we verified that @xmath87 remained equal to 1 over the whole range @xmath88 indicating that the retrieval state is stable . the measured values of @xmath89 and @xmath90 were within 1% of the theoretical expressions ( [ retrievalm ] ) and ( [ retrievale ] ) respectively . linear stability analysis has shown that the retrieval states are stable against _ infinitesimal _ perturbations for any value of @xmath8 , but this does not guarantee their stability against the flipping of signs of one or more nodes .  in order to function as an associative memory , a network must be capable of dynamically correcting sign errors .  when presented with an input at a small , nonzero hamming distance from one of the stored patterns ( i.e. , differing from it by a few reversed signs ) it must be able to flip the reversed signs and restore the correct pattern . we will now show that there is a critical value @xmath91 above which the correction of sign - flip errors can occur .  for smaller values of @xmath8 the bgn does not correct sign errors and thus does not truly function as an associative memory , but as @xmath8 increases above @xmath92 the retrieval states develop increasingly large basins of attraction .     consider a state of the network which is a slightly corrupted retrieval state :  all node variables have the values @xmath93 with the exception of one or possibly some number @xmath94 of nodes which may be misaligned . in such a state the few misaligned nodes make only a small contribution to the overlap sums , so we have @xmath95 and @xmath96 . the field acting on each node is therefore @xmath97 the misaligned bits experience a field opposite to their signs . if the field becomes larger than the critical value @xmath98 then there is only one stable equilibrium for each node , and the misaligned nodes will flip to conform with the stored pattern . error correction therefore occurs if @xmath99 the critical value occurs when the equality @xmath100 holds , or at @xmath91 . if the pattern is more strongly corrupted ( a significant number of bits misaligned ) then the situation is more complicated , because the presence of a larger number of misaligned bits may reduce the value of @xmath75 and thus the magnitude of the field . the misaligned bits have a significant back - reaction on the ones with the correct sign . the correction of larger numbers of sign errors requires higher values of @xmath8 . we will return to this point later ; the basic result is that when @xmath8 is only slightly above the threshold of @xmath101 the memory states have rather small basins of attraction , but these basins grow as @xmath8 increases . in the case of the hn , the hebb learning rule results in a large number of `` spurious '' attractors in addition to the retrieval states .  the energy function defines a rugged landscape , and a trajectory which does not start sufficiently close to one of the stored patterns may become trapped in one of the spurious local minima instead of one corresponding to a recalled pattern . it is possible to suppress the spurious minima by introducing thermal noise which allows trajectories to jump out of the shallower basins of attraction into deeper ones .     at low levels of loading , the hn possesses spurious attractors which are nonlinear combinations of the stored patterns . there is a hierarchy of symmetric _ mixture states _ of the form@xcite@xcite@xmath102 these states overlap equally with @xmath103 different patterns :  @xmath104 .  for the hn , only the mixtures with odd @xmath103 are stable . the @xmath105 mixtures have the lowest energy in this category , and the energies increase with @xmath106 asymptotically approaching @xmath107 . as the number of stored patterns @xmath28 increases , these spurious states proliferate exponentially ; their number is of order @xmath108 . there are also non - symmetric mixtures . the proliferation of spurious states is associated with spin glass type behaviour in the hn . accordingly we also refer to these mixed states somewhat loosely as _ spin glass states_.     we will show here that the bgn possesses mixture states analogous to those of the hn , but their structure is slightly more complex . let us focus on the @xmath105 symmetric mixture state with positive signs . for the hn , this state is given by @xmath109 this state is stable against individual sign flips because each node is subject to a non - zero magnetic field which maintains its alignment .  to see this , note that there are two possibilities for each bit either all three patterns agree at that particular site ( @xmath110 or one of the patterns has the opposite sign from the other two , for example @xmath111  when all three agree , we say that the @xmath34-th bit is a unamimous  bit . if the patterns are random , then each @xmath112 is @xmath15 with equal probability ,  giving a probability of @xmath113 that a given bit is unanimous . the mixture state has equal overlaps with all three of the patterns .  since for a given @xmath34 there is a probability of @xmath114 that @xmath115 , we have @xmath116 , and likewise @xmath117  the field acting at the @xmath34-th site is @xmath118 ( consistent with the low - loading , large-@xmath3 limit , we ignore all other overlaps which are of order @xmath40 . )  this gives @xmath119 for unanimous bits and @xmath120 for the others .  in any case , each node of the hn experiences a field which stabilizes its alignment . we will now show that the bgn , like the hn , possesses a mixture state in which the sign of @xmath9 is given by the majority vote of three of the stored patterns : @xmath121 this state has a more complicated structure , however , because the magnitude of @xmath9 at a given node depends significantly on the local field at that node . since the field at a unanimous bit is stronger than the field at a non - unanimous bit , we expect the magnitude of @xmath9 to be larger for a unanimous bit .  therefore , we make the _ _ ansatz__@xmath122{ll}% a\operatorname{sign}(\xi_{i}^{\mu_{1}}+\xi_{i}^{\mu_{2}}+\xi_{i}^{\mu_{3 } } ) & \text{if \ } \xi_{i}^{\mu_{1}}=\xi_{i}^{\mu_{2}}=\xi_{i}^{\mu_{3}}\\ d\operatorname{sign}(\xi_{i}^{\mu_{1}}+\xi_{i}^{\mu_{2}}+\xi_{i}^{\mu_{3 } } ) & \text{otherwise}% \end{array } \right . \label{bgnthreemix}%\ ] ] where @xmath123 and @xmath124 are real numbers .  the dynamical equations for the network give a pair of self - consistency equations which can be solved numerically for @xmath123 and @xmath125  first , we need an expression for the overlap of the mixture state with one of the three patterns , say , pattern @xmath126 . if the @xmath34-th bit is a unanimous bit , then @xmath127 has magnitude @xmath123 and agrees in sign with @xmath128 . on the other hand , if it is a non - unanimous bit , then @xmath127 has magnitude @xmath124 and has a @xmath129 probablity of agreeing in sign with @xmath128 . the result is that @xmath130 note that in the special case @xmath131 , the above expression reduces to the hopfield value @xmath132 , as it should . again , all three overlaps have the same size : @xmath133  .  the total energy of the network in this state is given by@xmath134 we can think of this as an an energy function on the restricted family of states parametrized by ( [ bgnthreemix ] ) . a necessary condition for the mixture state ( [ bgnthreemix ] ) to be a fixed point is that @xmath135 . this gives us two self - consistency conditions for the parameters @xmath123 and @xmath124 : @xmath136 these are the equations of  the nullclines of the energy function ( [ bgnthreee ] ) . alternatively , the above equations could be derived directly from the dynamical equations for each node and the expressions for @xmath18 instead of using the energy function .  for a _ stable _ fixed point , @xmath137 must be a local minimum of the energy function ( [ bgnthreee ] ) . the self - consistency equations ( [ selfconsist ] ) can be rewritten as follows:@xmath138 \nonumber\\ a   &   = \frac{4}{\gamma}\left [   d^{3}-(1+\frac{\gamma}{4})d\right ]   .\end{aligned}\ ] ] the graphs of these two cubic equations are plotted for @xmath139 in figure [ nullclinesboth]a . solutions are points where the two nullclines intersect . note that the slope , @xmath140 , of the first curve at the origin is @xmath141 and for the second curve @xmath142 . these two inequalities satisfied by the slopes ensure that the curves always intersect in at least 5 points . the five solutions can be classified by looking at the energy function and its gradient . we see that the solution @xmath143 is an unstable node ( maximum of the energy ) , the two solutions in the 2nd and 4th quadrants are saddle points , and the two solutions in the first and third quadrants are the stable solutions we seek . ( there are two because of the @xmath74 sign reversal symmetry  one is a mirror state of the other . ) [ ptb ]    fig1.eps       the self - consistency equations for @xmath123 and @xmath124 were solved numerically for a range of @xmath8 values using a gradient descent algorithm . these values were in turn substituted into ( [ bgnthreee ] ) to find the energy as a function of @xmath8 . the results are plotted in figure [ mixcompare ] where they are also compared with numerical results from dynamical simulations of a bgn with @xmath26 . we studied the mixture state numerically by initializing a network to the state @xmath144 and incrementing @xmath8 beginning at 0 , much as was done for the retrieval states . figure [ mixcompare]a shows the magnitudes of  unanimous and non - unanimous bits in the mixture state as functions of @xmath86 the solid lines show the solutions of the self consistency equations for @xmath123 and @xmath124 . the symbols show the observed magnitudes @xmath145 for @xmath146 in the simulated network state . two of these first six bits are unanimous while the other four are not . there is good agreement between the observed values of @xmath147and the values obtained from the self - consistency equations .  for comparison , @xmath148 is plotted as a dotted line on the same axes . ( recall that this is the value of all @xmath149 in a pure retrieval state . ) figure [ mixcompare]b shows a corresponding comparison of the observed and theoretical energies . finally , figure [ mixcompare]c  compares the @xmath105 mixture state with the retrieval state by plotting the ratios  @xmath150 and @xmath151 as well as the ratio of the mixture state energy @xmath152to that of a retrieval state @xmath153 . all three of these ratios appear to approach constant asymptotic values as @xmath8 increases . asymptotically ,  @xmath154 ,  while for the hn the corresponding ratio is 0.75 . the strength of the field acting on each unanimous bit , @xmath155 ,  and that acting on the non - unanimous bits , @xmath156 , both increase as @xmath8 increases . the mixture state is stable against single sign flips of the unanimous bits when @xmath157 and stable against any single sign flip when @xmath158 . thus as @xmath8 increases , the mixture state begins to develop a non - trivial basin of attraction of its own . [ ptb ]    fig2.eps       it is interesting to note that at @xmath159 a saddle - node bifurcation occurs and for @xmath160 two aditional stable solutions to the self - consistency equations ( [ selfconsist ] ) appear in the second and fourth quadrants , at @xmath161 and @xmath162 . ( see figure [ nullclinesboth]b . ) these are states with @xmath163  in which @xmath164  and the net field acting on each node exactly cancels . they are not stable against sign flips , and become completely destabilized when @xmath165 . as we will see in the following section , they do not properly belong to the category of mixture or spin glass states , but rather to another class of spurious attractors present only in the bgn at low values of @xmath8 . here we examined only the @xmath105 mixture state ,  but similar methods may be used to characterize higher - order mixtures . in general , they are more complex as there are more possibilities for the size of the majority by which the sign is determined . the magnitudes of @xmath9 then take a greater number of distinct values . in section [ errorcorrect ] , we noted that for values of @xmath8 not far above @xmath92 ,  it is possible that a state may have a significant overlap with one of the stored patterns but that the field acting on the nodes may nonetheless not be strong enough to overcome the potential barrier and correct the sign errors . indeed if @xmath8 is below @xmath101 then even a single sign error may go uncorrected . this consequence of the bistability of the bgn units contrasts with the behavior of the hn . consider first the case of the hn . a typical random initial state has small but nonzero overlaps with the memorized patterns ,  @xmath166 , resulting in fields @xmath167 which are random with zero mean and variance of order @xmath168 .  typically , for approximately half of the nodes @xmath9 and @xmath18 will initially have opposite signs . since there is no potential barrier against sign flips , those nodes will change their signs , and the sign flips will continue until the field experienced by every node is aligned with @xmath9 .  every sign flip will increase the magnitude of one or more of the overlap variables . if , for example ,  one overlap @xmath75 is larger than all of the others , then most nodes will experience fields which tend to align them with pattern @xmath73 . every sign flip further increases the value of @xmath75 , and eventually @xmath75 will be fully retrieved even though the initial overlap may have been quite small .  however , if one overlap does not clearly dominate the others , then the trajectory may arrive at a spurious attractor which has roughly equal overlaps with several patterns , instead of at a single one of the memory states . even a state which is initially orthogonal to all of the memorized patterns can be rendered unstable by changing a single sign :  even a single sign flip will create a small but nonzero field affecting the other nodes , resulting in further sign flips , and so on .  in summary , for the hn essentially any initial condition converges under the dynamics to an attractor lying in or close to the @xmath28-dimensional subspace of the patterns .    for the bgn , on the other hand , the situation is different due to the presence of potential barriers .  just as with the hn , given any state @xmath10 every node @xmath34 experiences a field @xmath18 , which may be aligned with or opposed to @xmath9 .  however , the antiparallel local fields may not be strong enough to flip their nodes into the parallel direction . if most @xmath18 are well below the threshold @xmath169 then the flipping of one or a few nodes will not change the field enough to cause any further flips . thus there might be a large number of initial conditions which remain stuck with low overlaps , far away from any of the patterns . we refer to such states with sub - threshold fields as uncondensed  states , because in those states none of the order paramenters @xmath45 are condensed .  however , we will show below that states with low overlaps can not remain stable for @xmath165 ,  and thus for higher values of @xmath8 the behavior is hopfield - like , with all trajectories collapsing toward the pattern subspace . consider a hypothetical state which is strictly orthogonal to all memory patterns , so that @xmath170 for all @xmath171 . ( the extra solutions appearing in the self - consistency equations for the mixture state when @xmath160 are examples of such states . ) if @xmath170 for all @xmath171 , then @xmath172 for all @xmath34 .  in this case , the steady state of each node is @xmath173 . proceeding with linear stability analysis as above , we find the relevant jacobian@xmath174 the equilibrium is unstable if @xmath175 , where @xmath176 is the largest eigenvalue of the coupling matrix . @xmath177 is at least 1 . therefore , if @xmath165 , states orthogonal to the stored patterns are all unstable .  if all of the @xmath28 stored patterns are mutually orthogonal ( which is approximately true in the limit @xmath37 )  then for @xmath165 there are @xmath28 unstable eigenvalues .     we can examine one of those unstable directions , say , the one associated with the @xmath73-th pattern , more closely by making an explicit _ ansatz . _  let us denote the orthogonal , zero - overlap state by @xmath178 ; by assumption @xmath179 for all @xmath171 including @xmath180 .  for half of the nodes @xmath181 ; for the other half @xmath182 . let us then consider a family of states described by the _ _ ansatz__@xmath183{ll}% a\xi_{i}^{\bot}=a\xi_{i}^{\nu } & \text{if } \xi_{i}^{\bot}=\xi_{i}^{\nu}\\ b\xi_{i}^{\bot}=-b\xi_{i}^{\nu } & \text{if } \xi_{i}^{\bot}=-\xi_{i}^{\nu}% \end{array } \right . \label{unstable}%\ ] ] this parametrizes a 2-dimensional subspace of the state space which contains @xmath184 and one of its unstable eigenvectors .  using the methods of section [ mixture ] , one obtains a pair of cubic self - consistency equations for @xmath123 and @xmath185:@xmath186  for all @xmath8 there are two solutions @xmath187 which correspond to the retrieval states @xmath188 .  for @xmath160 there are additional stable solutions @xmath189 corresponding to @xmath190 @xmath191 . a bifurcation occurs at @xmath159 and these solutions become saddle points . thus ,  we see that there is an absolute upper limit for the existence of stable uncondensed states . in fact , @xmath159 turns out to be a high upper bound . the example of a state with all @xmath45 equal to zero is a sort of worst - case scenario .  for a finite - sized network the typical random initial condition has small but nonzero overlaps . in addition , if the patterns are truly random then they will not be exactly orthogonal but have small overlaps and so the largest eigenvalue of the synaptic matrix will be slightly larger than unity .  because of these factors the typical uncondensed state becomes unstable at values of @xmath8 lower than 2 ; in  numerical simulations we found that for the case @xmath192 most become unstable between @xmath193 and @xmath194 . figures [ dyna05]-[dyn12b ] , we show numerical results for the fate of a typical random initial condition of the bgn with @xmath26 , @xmath27 .  figures [ dyna05]-[dyna12 ] show the dynamical evolution of the same initial condition , at different values of @xmath8 . the initial condition was a random string of @xmath15 values . we plot the energy per node , all five overlap variables @xmath45 and all five bit overlap variables @xmath195 as functions of time . recall that @xmath195 contain information about sign agreements only .  for @xmath196 ( figure [ dyna05 ] ) the state changes very little before convergence occurs . the energy per node remains very close to @xmath197 . the overlaps @xmath45 increase slightly in magnitude , but the bit overlaps do not change at all , indicating that no sign flips occur .  for @xmath193 ( figure [ dyna10 ] )  the trajectory is similar , except that the small initial overlaps are amplified to a greater extent ( we will explain this effect below . )  the bit overlaps still do not change .  when @xmath198 ,  however , the trajectory changes qualitatively ( figure [ dyna12 ] ) . the magnitudes of the overlaps @xmath45 grow slowly until at @xmath199 the resulting field becomes strong enough to begin flipping some signs . at this point the bit overlaps begin to change , the energy drops significantly and the trajectory moves close to the pattern subspace .  after some further evolution , the system converges to a mixture state which overlaps with several patterns . a different random initial condition , followed again at @xmath198 , leads instead to a memory state ( figure [ dyn12b ] ) . here one of the five overlaps becomes dominant and the others shrink away . in this case the mirror state of one of the five patterns is retrieved . these trajectories are typical examples representing descent on a rugged energy landscape . different initial conditions lead to different attractors , of which some are memory states and some are mixtures . frequently the trajectory lingers at one or several states before settling at its asymptotic attractor . [ ptb ]    fig3.eps    [ ptb ]    fig4.eps    [ ptbptb ]    fig5.eps    [ ptbptbptb ]    fig6.eps    figures [ dyna05]-[dyna10 ] illustrated that for sufficiently small values of @xmath8 ,  the dynamics amplifies small initial overlaps without flipping the sign of any node .  for more insight into this phenomenon , consider an initial state in which all @xmath9 are @xmath15 but somewhat more nodes are aligned parallel with one particular pattern @xmath200 than are antiparallel . in other words , @xmath201 is nonzero but less than unity .  for simplicity let us neglect all other overlaps . initially , each node experiences a small field given by @xmath202  this field will push @xmath9 to larger magnitudes @xmath203 for those nodes which are aligned with pattern @xmath200 ,  and it will push the others to smaller magnitudes @xmath204 .  this adjustment in turn increases the value of @xmath75 , until an equilibrium is reached with @xmath205 . we might think of this as a kind of  subliminal  recognition of the pattern .  the effect becomes stronger as @xmath8 increases . clearly it has a nonlinear dependence on both @xmath8 and @xmath201 . when the field becomes large enough it will exceed the threshold for sign flips and the state will be attracted toward the pattern retrieval state . the larger @xmath8 is , the smaller the initial @xmath201 that is necessary to fully retrieve the pattern @xmath200 .  in other words , the basins of attraction of the patterns expand as @xmath8 increases . in this section , we provide numerical support for the three - way classification of attractors into retrieval , spin glass and uncondensed states and we show how the respective attractor basins change with the control parameter @xmath206 . we observe an interpolation between two different regimes . as we showed above , for @xmath207 there are no stable uncondensed states .  for lower values of @xmath8 , on the other hand , uncondensed states are numerous . recall that uncondensed states are characterized by local fields too weak to overcome the potential barriers against sign flips ,  and so their dynamics is dominated by the local potential .  in the extreme case @xmath84 , there are of course no magnetic fields at all and only the local potential is present . the classification of attractors is very clearly reflected in the energy spectrum .  to explore attractors and their basins , we seeded  the network with 500 random initial conditions ( taken with several different realizations of the five random patterns ) , integrated the dynamical equations until they converged , and constructed a histogram of the final energies ( figure [ seedhist ] ) .  for the case @xmath193 ( figure [ seedhist]a ) , there are three clearly separated clusters of attractors .  those with the lowest energies are retrieval states ,  while the states clustered at @xmath208 are the uncondensed states , and those in the intermediate range are the glassy states . the picture is qualitatively similar at the slightly larger value @xmath209 ( figure [ seedhist]b ) ,  but the peak at @xmath208 has shrunk relative to the other two . note that the energies of the retrieval and spin glass states change with @xmath8 ,  while the uncondensed states remain at nearly the same energy because their dynamics is dominated by the local potential . for @xmath159 , ( figure [ seedhist]c )  on the other hand , the cluster at @xmath208 is absent as there are no stable uncondensed states . the histogram for a hn ( figure [ seedhist]d ) resembles that for the bgn with @xmath159 . one quantitative difference is that the retrieval state peak is slightly higher for the bgn with @xmath159 , while the glassy states are comparatively suppressed . we performed this experiment at a range of values of @xmath8 .  in all cases the classification of states was clear from the energy spectrum and was verified by examining the final values of @xmath195 . figure [ attractorprobs ] shows the probabilities of convergence to each of the three types of attractors from a random initial condition as functions of @xmath8 .  at @xmath196 the landscape is dominated by the uncondensed states . even though @xmath196 lies above the threshold of @xmath92 and the retrieval states have non - trivial basins of attraction , these basins still occupy a very small fraction of the total configuration space volume . the patterns can be retrieved only if the initial overlaps are relatively high , and the probability of a _ random _ initial condition being sufficiently close is very low . the retrieval probability becomes significant only as @xmath8 approaches 1 .  as @xmath8 increases from 1 to 1.5 , basins for the memory and spin glass states grow at the expense of of the uncondensed states until the latter disappear . the retrieval state basins grow faster than those of the spin glass states .  beyond @xmath210 , the probability of retrieving a memory state saturates at approximately a @xmath211 higher value than in the hopfield case , and the probability of falling into a spin glass state is correspondingly lower . [ ptb ]    fig7.eps    [ ptbptb ]    fig8.eps      in an attempt to map the attractor basins in more detail , we generated configurations at specified initial hamming distances from particular memory patterns . this was done by starting with a pattern @xmath59 and flipping the signs of a specified number of randomly chosen bits .  using an ensemble of such initial conditions , we measured the probability of retrieval of the target pattern @xmath59 as a function of the initial distance from it .  as a rule , the probability of recognizing the pattern is high if only a few signs are flipped , but drops sharply if a certain threshold hamming distance is exceeded . we are interested in learning where this threshold lies , and thus answering the question of how close an initial condition must be to a pattern in order to be attracted to it . we are also interested in the fate of states lying just outside the boundaries of a basin of attraction . in other words , does the basin share a boundary with the basins of other patterns , or only with spurious attractors ? the results are presented in figure [ boundaries ] for a bgn with @xmath26 and @xmath27 , for the three values @xmath212 and @xmath213 , and also for the hn .  in each of these cases , we generated an ensemble of initial conditions at a particular initial value of @xmath201 for some pattern . each initial condition was allowed to evolve under the dynamics and the resulting attractor was classified as :  a )  the target pattern @xmath214 b )  one of the other patterns @xmath215  c ) a spin glass spurious state or  d )  an uncondensed state .  in this case , we classified a state as uncondensed  if no sign flips occured during the dynamical evolution . the probabilities of each of these four outcomes were averaged over several realizations of the random patterns and plotted as functions of the initial bit overlap @xmath201 . [ ptb ]    fig9.eps    consider first the hn data from figure [ boundaries]d . a pattern can evidently be retrieved even if the initial overlap is fairly small ; the probability is close to unity if @xmath216 . if the target pattern is not retrieved , then either a spurious attractor or one of the other patterns may be retrieved . there is a range of @xmath217 over which all three probabilities are significant , indicating that the basins for the memory states border on each other as well as those of spurious states .  for an @xmath26 network , the expected magnitude of the overlap of a random state with any given one of the stored patterns is @xmath218 ,  which is not much smaller than the apparent threshold of @xmath219  this is consistent with the view that for the hn ,  a pattern is likely to be retrieved as long as the initial overlap with that pattern is significantly larger than all of the other overlaps . the @xmath159 bgn ( figure [ boundaries]c )  shares the qualitative features of the hn . note however that the probability of becoming trapped in a spurious state is smaller for the bgn , consistent with the results in figure [ attractorprobs ] . a contrasting case is the @xmath196 bgn ( figure [ boundaries]a ) .  in this case retrieval of the target pattern requires an initial overlap of more than 0.5 . although this represents a significant basin , it is highly unlikely that a _ random _ initial condition will have such a large overlap , thus explaining why random initial conditions almost never flow to a memory state .  the basins of the memory states are bordered only by spurious states , not by other memory states . interestingly , the states which lie adjacent to the basin of a memory pattern are not all uncondensed some are spurious states of the mixture or spin glass type . examination of the states retrieved near the boundary shows that these are typically _ asymmetric _ mixture states with one large overlap and two or more smaller ( but greater than random ) overlaps . finally , for @xmath193 ( figure [ boundaries]b ) ,  the basins of the memory state are almost as large as in the hn case ,  and near the boundaries there is a small but nonzero probability of retrieving one of the other memory patterns , indicating that the basins of different memory states almost touch each other .       taken together , the above results suggest a qualitative , schematic picture of the energy landscape illustrated in figures [ landscape05]-[landscape2 ] . the representation of the configuration space by two dimensions is not to be taken literally , since it is of course @xmath3-dimensional .  at low values of @xmath8 , such as 0.5 , the energy landscape is dominated by uncondensed states , which form a series of shallow basins , each limited to roughly a single orthant of @xmath3-dimensional space . these are represented in the diagram by a series of shallow pits . the basins of attraction for the retrieval and mixture states form isolated depressions in this pitted plateau . they occupy nontrivial volumes but do not lie adjacent to each other ( with the exception of certain spin glass states which lie near the retrieval states ) . at intermediate values @xmath220 ,  the basins of attraction of the retrieval states are much larger and in some places almost touch each other ,  but significant islands of uncondensed states remain .  by @xmath221 , however , the uncondensed states have disappeared and the basins of attraction for the other two types of states occupy the entire energy landscape and share boundaries with each other . [ ptb ]    fig10.eps    [ ptbptb ]    fig11.eps    [ ptbptbptb ]    fig12.eps we have studied the behaviour of the bistable gradient network in the thermodynamic low - loading limits @xmath37 , @xmath38 . we described and classified the attractors of the dynamics and also observed the effectiveness of pattern retrieval as a function of the coupling parameter @xmath8 . we found that states corresponding to perfect retrieval of the stored patterns are linearly stable at all values of @xmath8 , and have an energy that decreases monotonically with @xmath8 .  above the threshold @xmath222 the retrieval states become stable against sign flips of one or more nodes , and the network begins to function as an associative memory .  if @xmath8 is not far above this threshold , then the basins of attraction of the retrieval states are small and input must be very close in hamming distance to a pattern for recognition to occur . the basins of attraction of the retrieval states grow as @xmath8 increases . there are two regimes of behaviour , distinguished by the types of attractors that occur .  at low @xmath8 the configuration space is dominated by the uncondensed states , or states in which no node experiences a field strong enough to overcome its potential barrier .  in these states , @xmath145 remains close to 1 for all nodes ,  and the energy remains close to @xmath197 . each of these states occupies a basin of attraction confined to approximately a single orthant .  in the limit @xmath84 there are @xmath223 such states , all degenerate in energy .  as @xmath8 increases above the threshold @xmath91 , the retrieval states and the mixture or spin glass states at first occupy small isolated basins among the many uncondensed states .  however ,  as @xmath8 increases further , these basins grow until they lie adjacent to each other .  at some value of @xmath8 ( observed to lie between 1 and 1.5 ) , the uncondensed states disappear and there is a transition to a hopfield - like regime where the basins of attraction for retrieval  and spin glass states cover the whole configuration space . as @xmath8 increases still further ,  the retrieval basins grow at the expense of the spin glass states , so that the latter can be noticeably suppressed compared to the deterministic hopfield case . this suppression of the spurious states occurs without thermal noise or a modification of the hebb learning rule . the uncondensed states represent a phase which is neither `` ferromagnetic '' ( i , e , strongly ordered and correlated with one pattern ) nor `` glassy '' in the sense that frustration is an important effect ,  yet they can not properly be described as `` paramagnetic , ''  as paramagnetism is characterized by spins which are able to flip freely from one orientation to the other . a few words on the application of such networks to practical problems of associative memory are in order . the goal of associative memory is to reconstruct a pattern from a more or less corrupted version or from a fragment of the pattern , without becoming trapped in a spurious local minimum .  from this point of view , it appears that increasing @xmath8 improves the performance of the network expanding the basins of attraction for the retrieval states and suppressing the spurious states . the low-@xmath8 regime , on the other hand , may be suited to applications where the goal is a _ selective _ associative memory ,  one which only recognizes a pattern from a fairly close approximation and thus avoids false recognition .  in the low-@xmath8 regime , if the input is not close to one of the stored patterns , then the network is likely to remain in an uncondensed state . these can in general be distinguished clearly from other states ( especially retrieval states ) by their relatively high energy ( @xmath224 )  or by the fact that the magnitudes of all @xmath149 remain close to 1 . the magnitudes of the outputs can therefore be read as a signal of whether recognition has occurred . persistence in an uncondensed state corresponds to an i do nt know  or nonrecognition response .     in a subsequent publication , we will examine the behavior of the bgn when the loading level @xmath225 is of order unity , and we will demonstrate another performance trade - off .  specifically , we will show that the maximum storage capacity of the network decreases as @xmath8 increases .  for a low-@xmath8 regime , it is possible to stabilize more memorized patterns than in the hopfield case , while at higher @xmath8 , even though the low - loading fault tolerance is increased , the storage capacity decreases . w. hohmann , m. krauss and f.w . schneider , j. phys . a * 103 * , 7606 ( 1999 ) ;  j phys . chem a 102 , 3103 ( 1998 ) ;  j. phys . chem a 101 , 7364 ( 1997 ) . g. dechert , k .- zeyer , d. lebender and f.w . schneider , j. phys chem a 100 , 19043 ( 1996 ) . laplante , m. pemberton , a. hjelmfelt and j. ross , j. phys . chem 99 , 10063 ( 1995 ) . v. booth , t. erneux and j .- p . laplante , j. phys . chem 98 , 6537 ( 1994 ) . a. hjelmfelt and j. ross , j. phys . 97 , 7998 ( 1993 ) . amit , h. gutfreund and h. sompolinsky , phys.rev . * a32 * , 1007 ( 1985 ) ; phys . 55 * , 1530 ( 1985 ) ;  ann . phys . * 173 * , 30 ( 1987 ) . a. chrisanti , d.j . amit and h. gutfreund , europhys . * 2 * , 337 ( 1986 ) .
we examine the large - network , low - loading behaviour of an attractor neural network , the so - called bistable gradient network ( bgn ) . 
 we use analytical and numerical methods to characterize the attractor states of the network and their basins of attraction . 
 the energy landscape is more complex than that of the hopfield network and depends on the strength of the coupling among units .  at weak coupling , 
 the bgn acts as a highly selective associative memory ;  the input must be close to the one of the stored patterns in order to be recognized . 
 a category of spurious attractors occurs which is not present in the hopfield network . 
 stronger coupling results in a transition to a more hopfield - like regime with large basins of attraction . 
 the basins of attraction for spurious attractors are noticeably suppressed compared to the hopfield case , even though the hebbian synaptic structure is the same and there is no stochastic noise .