You are an expert at summarizing long articles. Proceed to summarize the following text: 
let @xmath2 be a @xmath3-variate random variable following a gaussian mixture model ( gmm ) with density @xmath4 where the @xmath5 s are the mixing proportions , with @xmath6 @xmath7 and @xmath8 , and the component @xmath9 represents the density of a @xmath3-variate normal distribution with mean vector @xmath10 and covariance matrix @xmath11 ; furthermore , let us indicate the set of model parameters with @xmath12 and the parameter space with @xmath13 where the symbol @xmath14 refers to lwner ordering on symmetric matrices and , in this case , is equivalent to requiring that @xmath11 be positive definite . the gmm is frequently used to classify a sample of observations . the idea is to consider the sample as drawn from a heterogeneous population where each sub - population is described by one component of the mixture . in other terms , each observation is assumed to come from one of the @xmath15 different groups characterized by the mixture components . the observations are classified into the groups by computing the posterior probabilities @xmath16 and assigning each observation to the group with the largest posterior probability . the parameters of the gmm are generally unknown and estimated from the data . given a sample of i.i.d . observations @xmath17 , the estimation is usually done by maximizing the likelihood @xmath18 the likelihood in equation ( [ like ] ) is known to be unbounded and it is cursed by the presence of several local maxima . as a consequence , the em algorithm may fail to converge , leading to such degenerate solutions . to face degeneracy , several methods have been proposed by the literature in which constraints or penalties are added to the log - likelihood . their main objective is to keep the eigenvalues of the class conditional covariance matrices bounded away from zero . this paper considers the sufficient condition formulated by ingrassia ( 2004 ) such that hathaway s ( 1985 ) constraints hold : we propose a generalization that enforces the equivariance with respect to linear affine transformations of the data . the idea is to shrink the class conditional covariance matrices towards a pre - specified matrix @xmath0 we investigate possible data - driven methods for choosing the matrix @xmath1 when _ a priori _ information on the group - specific covariance structure is not available , and we let the data determine the optimal amount of shrinkage . the equivariance property the method possesses is a key feature for twofold reasons . first , it means that irrespective of the kind of standardization performed on the data , the final clustering will be the same . second , whatever the scale of the data is as they come in , there will be no _ best _ pre - processing of the data ensuring a _ better _ result , as the final clustering is not affected by changes in scale . the plan of the paper is the following . section [ degen ] gives insights on the notion of degeneracy for multivariate gmm , and section [ remdegen ] reviews some of the workarounds proposed by the existing literature . in section [ invar ] we state the property of equivariance of gmm and we show , in section [ constr1 ] , that the property holds in the constrained approach of hathaway ( 1985 ) , whereas it does not hold in the sufficient condition provided by ingrassia ( 2004 ) . in section [ constr2 ] we illustrate how these constraints can be generalized to become equivariant under linear affine transformations of the data , and how their configuration can be tuned from the data ( section [ crossvalid ] ) . section [ alg ] summarizes the algorithm . the proposal is evaluated through a simulation study ( section [ simulation ] ) and an empirical application ( section [ wineapp ] ) . section [ concl ] concludes with a final discussion . in the univariate case , the likelihood function increases without bound if some variances tend to zero and one of the components mean coincides with a sample observation ( kiefer and wolfowitz , 1956 ; day , 1969 ) . biernacki and chrtien ( 2003 ) showed that if mixture parameters are close to a degenerate solution , then the em is attracted by it and the divergence is extremely fast . although kiefer ( 1978 ) proved that maximum likelihood does not fail , as there exists a local maximizer strongly consistent and asymptotically efficient , several local maximizers can exist for a given sample . that is , some local maximizers are spurious , i.e. with a high likelihood but of little practical use because highly biased . they are characterized by some component variances and mixing proportions very small relative to the others ( day , 1969 ; mclachlan and peel , 2000 ) . detecting the desired solution , among the many available , can therefore be a complicated task . the same problems hold in the multivariate case ( as an example , see ingrassia and rocci , 2011 , for an extension of biernacki and chrtien , 2003 ) , with additional complications . to notice how unboundedness is caused , first of all let us express the density of the @xmath19-th observation on the @xmath20-th component as follows @xmath21 where @xmath22 is the square @xmath23 matrix whose @xmath24th column is the eigenvector @xmath25 of @xmath26 and @xmath27 is the diagonal matrix whose diagonal elements are the corresponding eigenvalues @xmath28 ordered such that @xmath29 equation can be rewritten as @xmath30[\mathbf{q}_{jg}'(\mathbf{x}_{i } - \boldsymbol{\mu}_{g } ) ] \right\ } \\ & = \frac{1}{\sqrt{2\pi \prod_{j=1}^{j}\lambda_{jg } } } \exp\left\ { -\frac{1}{2 } \sum_{j=1}^{j}\lambda_{jg}^{-1}[(\mathbf{x}_{i } - \boldsymbol{\mu}_{g})'\mathbf{q}_{jg}]^{2 } \right\}. { \addtocounter{equation}{1}\tag{\theequation}}\label{eq : li2parts}\end{aligned}\ ] ] as policiello ( 1981 ) argued , the likelihood can be written as the sum of non negative terms . among them , it is possible to isolate the product of the density of the @xmath19-th observation on the @xmath20-th component - equation - and the densities of the other observations on the other components and the corresponding mixing proportions . if observation @xmath19 is such that @xmath31 then , as @xmath32 there would be no exponential term involving @xmath33 who can attenuate the effect of @xmath34 in words , the sample likelihood diverges when in one component the covariance matrix is close to singularity and the projection of the component s mean on the eigenvector corresponding to the smallest eigenvalue coincides with the projection of one of the observations on the same eigenvector . the easiest way to handle degeneracy is to initialize the em algorithm from several starting points until a local maximum is found ( biernacki and chrtien , 2003 ) . mclachlan and peel ( 2000 ) proposed monitoring the local maximizers by inspecting the relative size of the estimated mixing proportions and component variances . this leads , in practice , to performing maximum likelihood estimation by looking for the correct local maximum and discarding those that seem to be spurious . further methods exploit constraints on the covariance matrices . this approach is based on the seminal work of hathaway ( 1985 ) , where he studied how to avoid the divergence of the likelihood in the univariate case by imposing a lower bound , say @xmath35 , to the ratios of the scale parameters . in this way the variances can not be arbitrarily different . hathaway proved the boundedness of the likelihood and the consistency of the ml estimator under such constraints . in the multivariate case , the lower bound is imposed on the generalized eigenvalues of each pair of covariance matrices and the ml estimator results to be equivariant under linear affine transformations of the data . this implies that , as in the unconstrained case , if the data are linearly transformed , the estimated posterior probabilities do not change and the clustering remains unaltered ( see sections [ invar ] and [ constr1 ] ) . an important issue is the choice of the constant @xmath36 which controls the strength of the constraints . in the context of univariate mixtures of gaussians or linear regression models , some authors have shown that the maximum likelihood constrained estimator is consistent if @xmath35 decreases to zero at a certain rate as the sample size increases to infinity ( e.g. tanaka and takemura ( 2006 ) , tan et al . ( 2007 ) , xu et al . ( 2010 ) ) . nevertheless , finite - sample sensible choice of @xmath35 is still an open issue . hathaway s constraints are very difficult to apply within iterative procedures like the em algorithm . to solve this problem , ingrassia ( 2004 ) proposed to simplify the constraints by putting bounds on the eigenvalues of the covariance matrices . although putting lower bounds on the group conditional covariance matrices was already common practice , ingrassia ( 2004 ) found a way to reconcile hathaway s contribution with the common practice : his bounds on the eigenvalues give a sufficient condition such that hathaway s constraints are satisfied . the simplification is such that the constraints can be easily implemented within the em algorithm , preserving its monotonicity property ( as shown in ingrassia and rocci , 2007 ) . several authors extended the constrained setup of ingrassia ( 2004 ) . greselin and ingrassia ( 2013 ) applied this setup to mixtures of factor analyzers . they proposed a tuning procedure for selecting the bounds for the eigenvalues of the covariance matrices , based on the final likelihood over a set of runs . ingrassia and rocci ( 2011 ) modified the constrained algorithm , allowing for stringent constraints which are lifted during the iterations . browne et al . ( 2013 ) combined the ideas in ingrassia and rocci ( 2007 , 2011 ) , constraining dynamically the smallest eigenvalue , the largest eigenvalue and both the smallest and the largest ones . all of these proposals share the drawback of not being affine equivariant . gallegos and ritter ( 2009a ; 2009b ) , and ritter ( 2014 ) applied hathaway s constraints to robust clustering . they proposed to obtain all local maxima of the trimmed likelihood and , for each solution , investigate the value of @xmath35 such that it fulfills the constraints . the idea is to choose , _ a posteriori _ , the solution with the highest trade - off between scale balance ( @xmath35 ) and fit ( log - likelihood ) . this approach can be viewed as a refined version of what was proposed in mclachlan and peel ( 2000 ) . garcia - escudero et al . ( 2008 ) , from the same strand of literature , introduced the tclust algorithm , based on controlling the relative sizes of the eigenvalues of the cluster scatter matrices . the tclust algorithm implies solving several complex optimization problems . fritz et al . ( 2013 ) and garcia - escudero et al . ( 2014 ) proposed further improvements to the algorithm in order to make it more efficient . the constraints considered therein are not affine equivariant . seo and kim ( 2012 ) pointed out that singular and spurious solutions overfit random localized patterns composed of few observations in the dataset . such observations have a strong influence on the formation of the likelihood - based solutions . their proposal was to take out such , say , @xmath37 observations with the highest likelihood ( likelihood - based @xmath37-deleted method ) , or with the highest value for a score - based statistic ( score - based @xmath37-deleted method ) . in this way the likelihood of the reduced samples is evaluated at each local maximizer previously found : the root they suggested to select is the one with the highest @xmath37-deleted likelihood . kim and seo ( 2014 ) show that their score - based method can be fairly well approximated with the computationally more efficient gradient - based version of the @xmath37-deleted method . the degeneracy problem may also be addressed by adding a penalty to the log - likelihood ( penalized approach ) . ciuperca et al . ( 2003 ) have shown the consistency of the penalized likelihood estimators proposed in ridolfi and idier ( 1999 , 2000 ) for univariate gmm . chen and tan ( 2009 ) extended the consistency result for the multivariate case . in this framework , the penalty term on the component covariance is added to the log - likelihood ( snoussi and djafari , 2001 ; chen et al , 2008 ) . this penalty can be interpreted as the log of the prior distribution in a maximum - a - posteriori estimation setup . yet , the penalized methods are not affine equivariant , unless the prior s hyperparameters are suitably transformed . map estimation , with an _ a priori _ distribution for the covariance matrices , is what fraley and raftery ( 2007 ) suggested to use , instead of maximum - likelihood , to circumvent the issues of degeneracy and spurious solutions . the maximum likelihood estimators ( mle ) of equation ( [ md ] ) are equivariant with respect to linear affine transformations of the data . that is , if the data are linearly transformed , the mle are transformed accordingly . this property is particularly important in classification because it implies that linear affine transformations of the data do not change the posterior estimates ( kleinberg , 2002 ; ritter , 2014 ) . the equivariance property can be shown in the following way . let us define a linear affine transformation @xmath38 , where @xmath39 is non singular . it is well known that @xmath40 where @xmath41 and @xmath42 . this implies that , denoting the likelihood of the original data with @xmath43 and the likelihood of the transformed data with @xmath44 , we have , with obvious notation @xmath45 it follows that there exists a one to one correspondence among the local maxima of @xmath43 and @xmath44 . in particular , if @xmath46 is a local maximizer for @xmath47 then @xmath48 will be a local maximizer for @xmath44 . analogously , if @xmath49 is a local maximizer for @xmath44 , then @xmath50 will be a local maximizer for @xmath43 . it is interesting to note that every pair of local maximizers produces the same estimates of the posterior probabilities , that is @xmath51    the above equality proves that the classification obtained via the gmm model is invariant under the group of linear affine transformations on the data @xmath52 . this property is crucial when dealing with practical applications as it implies that the clustering does not depend on the choice of a particular method of data standardization - which could instead affect the inference . hathaway ( 1985 ) proposed to impose the following restrictions on the covariance matrices @xmath53 where @xmath54 is the @xmath55-th eigenvalue of @xmath39 and @xmath56 . this prevents the likelihood from diverging and reduces the number of spurious maximizers . however , the method is difficult to implement and a correct choice of @xmath35 is not simple in practice . a value of @xmath35 close to @xmath57 could exclude the correct solution , whereas a value too close to @xmath58 is likely to increase the chance of converging to a spurious maximizer . ingrassia ( 2004 ) simplified hathaway s constraints as @xmath59 it is easy to show that ( [ nc ] ) implies hathaway s constraints ( [ ch ] ) while the reverse is not necessarily true ( ingrassia , 2004 ) . this ensures a bounded likelihood , and a reduction in the number of spurious maximizers . the constraints are easy to implement , as shown in rocci and ingrassia ( 2007 ) ; however , choosing an optimal _ c _ is still an issue . it is important to check if the above constrained approaches offer equivariant estimators under linear affine transformations . the property can be shown to hold for hathaway s approach as follows . let @xmath17 be a sample of i.i.d . the estimates are computed as the solution of the optimization problem @xmath60 given the transformation @xmath61 the maximand in equation ( [ eq : maxcon1 ] ) can be rewritten ( see section [ invar ] ) as @xmath62 where @xmath41 and @xmath42 . noting that @xmath63 we can equivalently write the optimization problem in equation ( [ eq : maxcon1 ] ) as @xmath64 it follows that if @xmath46 is a maximizer for ( [ eq : maxcon1 ] ) , then @xmath65 is a maximizer for ( [ eq : maxcon12 ] ) and vice - versa , and the two maximization problems are equivalent . as in the unconstrained case , every pair of local maximizers produces the same estimates of the posterior probabilities . this property does not hold for the constraints given in ( [ nc ] ) . that is , if @xmath66 is a constrained local maximizer for @xmath43 subject to ( [ nc ] ) , @xmath67 does not necessarily satisfy ( [ nc ] ) . as an example , let us suppose that @xmath68 where @xmath69 is the largest singular value of @xmath39 . in this case , for a given @xmath70 @xmath71 we conclude that @xmath72 does not satisfy the constraints in ( [ nc ] ) because @xmath73 , and then it can not be a constrained local maximizer for @xmath44 . constrains in ( [ nc ] ) are such that there is no one to one correspondence between the set of local maximizers of @xmath43 and @xmath44 . thus , the method suffers the disadvantage that the clustering depends on the choice of matrix @xmath39 . to fix this , data standardization is not the best way to go for two main reasons . first , the standardization requires a choice for the matrix @xmath39 and , second , there is no single best approach to data standardization ( milligan and cooper , 1988 ; doherty _ et al _ , 2007 ) . it is now clear that affine equivariance is not just a desirable property . it is one of the basic requirements of any clustering method , which should not be sensitive to the changes in the units of measurement of the data . in the next section , our goal will be that of deriving a new set of constraints that are affine equivariant . with an affine equivariant clustering method , researchers and practitioners shall not be concerned anymore with choosing what method to adopt to standardize their data . our proposal is to generalize the constraints ( [ nc ] ) by @xmath74 where @xmath75 is a symmetric positive definite matrix representing our _ prior _ information about the covariance structure . clearly , is equal to ( [ nc ] ) when @xmath76 . it can be shown that the above constraints imply hathaway s constraints . it is known that ( anderson and gupta , 1963 ) @xmath77 where @xmath39 is a positive semi - definite matrix and @xmath78 and @xmath79 are positive definite matrices . now , if holds , then @xmath80 thus , implies ( [ ch ] ) .    furthermore , it can be shown that is invariant under linear and affine transformations provided that @xmath75 is replaced by @xmath81 . if @xmath66 is a constrained local maximizer for @xmath43 subject to , then @xmath82 is a local maximizer for @xmath44 subject to for @xmath83 . we have that @xmath84    in words , if a linear affine transformation is performed on the data , @xmath75 must be changed accordingly . this scheme of transforming @xmath75 ensures the equivariance of the method . the constraints have the effect of shrinking the covariance matrices to @xmath1 and the level of shrinkage is given by the value of @xmath85 note that for @xmath86 , @xmath87 whereas for @xmath88 @xmath89 equals the unconstrained ml estimate . furthermore we can show that the stein s discrepancy - known as stein s loss ( james and stein , 1961 ) - between the matrices @xmath89 and @xmath75 goes to zero as @xmath35 approaches one . the stein s discrepancy between the matrices @xmath89 and @xmath75 is @xmath90 let us rewrite equation as follows . @xmath91 using the constraints in , we can derive the following majorizing function @xmath92 which is decreasing in @xmath85 this can be shown by noting that the first derivative of the right - hand side of with respect to @xmath35 is equal to @xmath93 and is negative when @xmath56 . this implies that the function is decreasing when @xmath35 increases within the interval @xmath94 $ ] . intuitively , the constraints provide with a way to obtain a model in between a too restrictive model , the homoscedastic , and an ill - conditioned model , the heteroscedastic . issues arise when _ a priori _ information about the structure of the class conditional covariance matrices is not available . in that case , @xmath75 and @xmath35 have to be selected from the data . from the previous discussion , for a given @xmath36 every @xmath89 can not be too far from @xmath75 in terms of stein s discrepancy . thus @xmath75 can be seen as the barycenter of the cloud of the @xmath89 s : the _ average _ conditional covariance matrix . therefore , the most natural choice is to estimate such _ average _ as the within covariance matrix of the homoscedastic gaussian model . how close the final clustering will be to the homoscedastic model will depend on the value of the tuning constant : for values of @xmath35 close to @xmath58 , the resulting clustering will be close to that of the heteroscedastic mixture model , whereas @xmath95 implies a clustering close to that of the homoscedastic mixture model .    other possible choices of @xmath1 which guarantee the equivariance of the constraints , are available : the sample covariance matrix , which is computationally faster and is frequently used as hyperparameter in bayesian gaussian mixtures ( for instance , see fraley and raftery , 2007 ) , or the within covariance matrix of a homoscedastic mixture of student-_t_. to motivate this , let us recall that a random vector conditionally distributed as a multivariate gaussian , given wishart inverse covariance matrix , has a multivariate student-@xmath96 distribution ( dawid , 1981 ; dickey , 1967 ) . using similar arguments as in peel and mclachlan ( 2000 ) , if @xmath97 is a gmm , and @xmath98 are i.i.d . wishart random variables , the marginal distribution of @xmath2 is a homoscedastic mixture of student-@xmath96 s . the choice of @xmath35 is crucial . a value of @xmath35 too large could exclude the right solution , whereas a too small value of @xmath35 is likely to increase the chance to converge to spurious local maxima : such solutions overfit random localized pattern composed of few data points being almost co - planar ( ritter , 2014 ; seo and kim , 2012 ) . hence , selecting @xmath35 jointly with the mixture parameters by maximizing the likelihood on the entire sample would trivially yield a scale balance approaching zero . a practical alternative would be to split the data into a training set , where model parameters are estimated , and a test set , where the log - likelihood is evaluated for a given value of @xmath85 the optimal tuning parameter @xmath35 would then be selected such that the test set log - likelihood is maximized . the use of the test set log - likelihood as a model selection tool is advocated by smyth ( 1996 ; 2000 ) , in the context of estimating the number of mixture components . the motivation behind its use is that it can be showed to be an unbiased estimator ( within a constant ) of the kullback - leibler divergence between the _ truth _ and the model under consideration ( smyth , 2000 ) . this means that , even under a misspecified model , the procedure renders a @xmath35 such that the kullback - leibler divergence is minimized .    in spite of the usual unavailability of large independent test sets , a valid alternative is to use the _ cross - validated _ log - likelihood in order to estimate the test set log - likelihood . this consists in repeatedly partitioning the data into training and test sets and , for a given @xmath36 estimate the mixture parameters on the training sets . the model fit is then measured summing the log - likelihoods of the test sets evaluated at the parameters computed on the training sets , obtaining the so - called _ cross - validated _ log - likelihood . the constant @xmath35 is chosen such that the _ cross - validated _ log - likelihood is maximized . this can be viewed as a function of @xmath35 only ( smyth , 1996 ) , and would solve the issue of overfitting as training and test sets are independent ( arlot and celisse , 2010 ) .    in details , let us partition @xmath99 times the full data set @xmath100 into two parts , a training set @xmath101 and a test set @xmath102 with @xmath103 and @xmath104 for the @xmath37-th partition , let @xmath105 be the constrained maximum likelihood estimator based on the training set @xmath106 furthermore , let @xmath107 $ ] be the log - likelihood function evaluated at the test set @xmath108 the _ cross - validated _ log - likelihood is defined as the sum of the contributions of each test set to the log - likelihood @xmath109.\ ] ] the best @xmath35 is chosen as the maximizer of @xmath110 . further details on the choice of the number of random partitions @xmath99 and of the sizes of training and test sets are given in section [ simulation ] . the objective is to maximize ( [ like ] ) under the constraints . thanks to the equivariance property of the constraints , we can act any linear affine transformation to the data . this is useful since it will suffice to transform the data so to have @xmath111 and the existing algorithm of ingrassia and rocci ( 2007 ) can be applied on the transformed data . the transformation is @xmath112 where @xmath113 is the singular value decomposition of @xmath0 this leads to @xmath114 .    for sake of completeness , we recall briefly the updates of the algorithm proposed by ingrassia and rocci ( 2007 ) . +   + * update * @xmath115 , @xmath116 @xmath10 + as in the case of a normal mixture , the updates are @xmath117 @xmath118 @xmath119 * update * @xmath11 + compute @xmath120 and set @xmath121 where @xmath122 is the diagonal matrix of the eigenvalues in non decreasing order of @xmath123 and @xmath124 its singular value decomposition . letting @xmath125 , the update of @xmath11 is given by @xmath126 in this section we perform a simulation experiment in order to compare the performance of the proposed methods with respect to some existing approaches in the literature . in particular we consider the following seven algorithms :    1 . unconstrained 1 . homoscedastic normal ( homn ) , within covariance matrix @xmath127 2 . heteroscedastic normal ( hetn ) , @xmath128 to prevent degeneracy and numerical instability ; 3 . homoscedastic student _ _ ) , scale matrix @xmath129 @xmath130 ( mclachlan and peel , 1998 ) ; 4 .   gradient - based @xmath37-deleted mle ( kdel ; kim and seo , 2014 ) , @xmath128 to prevent degeneracy and numerical instability , k@xmath131 . 2 .   constrained 1 . sample covariance ( con@xmath132 ) , @xmath133 2 . normal ( conn ) , @xmath134 3 .   student _ t _ ( con__t _ _ ) , @xmath135    for each sample , we randomly split the data @xmath136 times into a training set @xmath137 and a test set @xmath138 choosing how many times to partition the full data set is a trade - off between variability of the estimates and computational burden . as smyth ( 2000 ) argues in the context of model selection for probabilistic clustering using cross - validation , the larger the value of @xmath139 the less the variability in the log - likelihood estimates . in practice - the author argues - values of @xmath99 between 20 and 50 appear adequate for most applications .    the choice of the size of the test set must be such that the training set has all components represented . if one component is not represented in the test set , but the parameters are correctly estimated using the training set , the test set log - likelihood will correctly display the fit of the model . by contrast , if one component is not represented in the training set , although estimation of the other components parameters can be correct , the fit displayed by the test set log - likelihood will be poor . van der laan , dudoit , and keles ( 2004 ) found , in their simulation study , that the likelihood - based cross - validation procedure is performing equally well with any choice of the relative size of the test set between 0.1 and 0.5 . as argued in kearns ( 1996 ) , the importance of choosing an optimal size for the training set increases as the target function becomes more complex relative to the sample size . bearing this in mind , we choose to consider a training set of size @xmath140 and a test set @xmath141 of size @xmath142    then the cross - validation scheme , as described in section [ crossvalid ] , is applied and the optimal @xmath35 is chosen by using a line search with six function evaluations . the sample data have been generated from @xmath15-class mixtures of heteroscedastic @xmath3-variate normal distributions with :    * @xmath143 @xmath144 @xmath145 * @xmath146 @xmath147 * @xmath148 @xmath149 * prior probabilities @xmath150 @xmath151 @xmath152 * component means @xmath153 , independent ; * eigenvalues of the covariance matrices @xmath154 , independent with @xmath155 ; * eigenvectors of the covariance matrices generated by orthonormalizing matrices generated independently from a standard normal distribution . this yields a total of @xmath156 simulation conditions .    for each simulation condition , we generate 250 data sets , each with different means and covariance matrices . it is well known that the em for gmm is sensitive to the initial position , especially in the multivariate context ( among others , mclachlan and peel , 2000 ) . we choose to adopt the standard _ multiple random starts _ strategy . that is , for each data set , 10 random initial partitions are generated : these are used as starting values for the m - step of all the seven algorithms under analysis . for conn , cons , and cont , a constrained algorithm with arbitrary lower and upper bounds of respectively @xmath157 and @xmath158 is run in order to exclude degenerate ( and some spurious ) solutions , and the estimated clustering is used to initialize the cross - validation scheme . the alternative option of directly generating 10 different starts for each training set - within the cross - validation scheme - would have added little in terms of accuracy of the final estimates .    concerning the root selection criterion , for the unconstrained algorithm , we select the roots yielding the highest likelihood , whereas for the constrained algorithms we select the roots based on the cross - validated likelihood .    the performance of the different techniques has been analyzed in terms of :    * mad ( mean absolute deviation ) : @xmath159 * arand ( adjusted rand index ; hubert and arabie , 1985 ) ; * computational time needed to analyze a single data set ; * the value of the calibrated constant @xmath35 ( for the constrained approach only ) .    the mad is computed evaluating the above expression for all possible permutations of the estimated classes . the final mad reported refers to the permutation which yields the lowest difference , and measures inaccuracy of estimated _ fuzzy _ classification - whereas arand measures accuracy of estimated _ crisp _ classification . in addition , we tested the robustness of the results with respect to changes in 1 ) the cross - validation settings , and 2 ) the level of class separation . in order to test robustness with respect to cross - validation settings , we considered a subset of the above simulation conditions as follows . 250 samples , of 50 , 100 , and 200 observations , were generated from a 3-group 8-variate heteroscedastic gaussian mixture model , with prior class membership probabilities of 0.1 , 0.4 , and 0.5.-deleted method within @xmath3 and @xmath160 , as they suggested , on this reduced set of simulation conditions . on average , k=@xmath3 yielded the best clustering . the full set of results is available from the corresponding author upon request . ] the same setting was used in order to count how many different local maxima each algorithm converged to over the 10 random initializations considered . this serves the purpose of providing some information on the likelihood surface . class separation has been manipulated by controlling the dispersion of the group conditional covariance matrices eigenvalues ( through the above sep value ) : higher dispersion levels correspond to overlap between the classes . considering the above full simulation as corresponding to fixed moderate separation ( @xmath155 ) , this final setup compares results for low , moderate , and high separation levels - respectively @xmath161 @xmath162 and @xmath163 . the subset of simulation conditions considered is as follows . 250 samples , of 50 observations each , were generated from a 3-group and 5-group heteroscedastic gaussian mixture model , with prior class membership probabilities of respectively 0.2 , 0.3 , and 0.5 ; 0.1 , 0.4 , and 0.5 ; 0.1 , 0.1 , 0.2 , 0.3 , and 0.3 . table [ tabsimcon ] summarizes the conditions explored in all testing setups . .cross - table of simulation condition and simulation type . [ cols="<,^,^,^,^,^,^,^,^ " , ]     the homoscedasticity assumption seems to fit well the data . the constrained approach conn equals homn in terms of arand , whereas cont yields an arand of 0.93 , compared to 0.87 of homt . confirming the results obtained in the simulation study , cont seems to be the most accurate approach among the ones considered in this work . interestingly , however , all of the constrained approaches improve upon the unconstrained heteroscedastic approach and the gradient - based @xmath37-deleted method . in this paper we have proposed affine equivariant constraints for the class conditional covariance matrices of multivariate gmm in order to circumvent the well - known issue of degenerate and spurious solutions in ml estimation . our approach generalizes the sufficient condition for hathaway ( 1985 ) s constraints to hold as formulated by ingrassia ( 2004 ) . previous constrained approaches lacked affine equivariance and suffered the choice of an optimal finite - sample scale balance ( @xmath35 ) . the setup we propose is such that the class specific covariance matrices are shrunk towards a pre - specified matrix @xmath0 we have been able to show that this yields a clustering method which is equivariant with respect to linear affine transformations of the data .    a natural choice for the shrinkage target matrix , whenever _ a priori _ information on the covariance structure of the components is not available , seems to be the covariance matrix of a homoscedastic mixture of normals . for a given choice of the target matrix , we let the data decide , through the constant @xmath35 , how close to the target the final clustering will be . the tuning constant @xmath35 is chosen by cross - validation . we have also shown that , given a matrix @xmath1 our constrained ml estimate can be computed by applying the algorithm of ingrassia and rocci ( 2007 ) to the data appropriately linearly transformed . this allows us to interpret our proposal as a way to decide how to standardize the data before applying ingrassia ( 2004 ) s constraints . the validity of the proposal has been assessed through a simulation study and an empirical example . all constrained approaches yield more accurate estimates than the unconstrained ones . more specifically , cont has been shown to be the best among the constrained approaches this work has been concerned with . this is not surprising , since a random vector conditionally distributed as a gaussian mixture , given random covariance matrices , has a marginal homoscedastic mixture of student @xmath96 s distribution . however , different choices of @xmath75 can as well be considered , according to the data specificity . the equivariant method developed in gallegos and ritter ( 2009a ; 2009b ) and extended in ritter ( 2014 ) requires to obtain all local maxima of the trimmed likelihood . our method has the virtue of being easily implementable with a minimal extra computational effort , as we have shown in the simulation study and in the empirical example .    by comparing our results on the gradient - based @xmath37-deleted method with those obtained by kim and seo ( 2014 ) , we have observed that the @xmath37-deleted method suffered our choice of focusing on multiple random start strategy only . indeed , the @xmath37-deleted method , being a root selection method , shows its full value with a more vast and diversified set of initialization strategies ( kim and seo , 2014 ) . our research aim was rather to find a data - driven equivariant approach leading to an optimal compromise between the homoscedastic and the heteroscedastic model , irrespectively of the kind of initialization strategy employed . there are cases where the clustering model might assume a specific structure on the relationship between the variables , like local independence ( within - cluster diagonal matrices ) . such a model is not affine equivariant because some ( non diagonal ) affine transformations on the data might destroy the local independence . in cases like these , the affine equivariance property of the constraints is not required . yet our approach can be applied using a diagonal matrix as target . this would prevent the likelihood from degenerating , still improving upon the unconstrained algorithm thanks to the cross - validation strategy we have proposed . clearly , when all variables in a data set are measured in a common scale , non equivariant constraints are a competitive choice . an additional issue , pointed out by both the simulation study and the empirical example , is the computational time cross - validation requires to select an optimal @xmath85 whether different cross - validation schemes can speed up the constrained routines can be a topic for future research . biernacki , c. , celeux , g. , govaert , g. ( 2003 ) . choosing starting values for the em algorithm for getting the highest likelihood in multivariate gaussian mixture models . _ computational statistics and data analysis , 41(3 ) , 561 - 575 . _                    di mari , r. , oberski , d.l . , vermunt , j.k . bias - adjusted three - step latent markov modeling with covariates . _ structural equation modeling : a multidisciplinary journal_. doi:10.1080/10705511.2016.1191015 . ridolfi a. , idier j. ( 2000 ) . penalized maximum likelihood estimation for univariate normal mixture distributions . _ bayesian inference and maximum entropy methods _ , maxent workshops . gif - sur - yvette , france , july 2000 . smyth , p. ( 1996 ) . _ clustering using monte - carlo cross validation . _ in _ proceedings of the second international conference on knowledge discovery and data mining , menlo park , ca , aaai press , pp . 126133 . _      snoussi h. , mohammad - djafari a. ( 2001 ) . _ penalized maximum likelihood for multivariate gaussian mixture . _ in r.l . fry editor _ maxent workshops : bayesian inference and maximum entropy methods , 36 - 46 , aug . 2001 . _    tan x. , chen j. , zhang r. ( 2007 ) . consistency of the constrained maximum likelihood estimator in finite normal mixture models . proceedings of the american statistical association , american statistical association , alexandria , va , 2007 , pp . 2113 - 2119 . [ cd - rom ] . tanaka k. , takemura a. ( 2006 ) strong consistency of the maximum likelihood estimator for finite mixtures of location  scale distributions when the scale parameters are exponentially small . _ bernoulli , 12 ( 6 ) , 1003 - 1017_.
maximum likelihood estimation of gaussian mixture models with different class - specific covariance matrices is known to be problematic . 
 this is due to the unboundedness of the likelihood , together with the presence of spurious maximizers . 
 existing methods to bypass this obstacle are based on the fact that unboundedness is avoided if the eigenvalues of the covariance matrices are bounded away from zero . 
 this can be done imposing some constraints on the covariance matrices , i.e. by incorporating _ a priori _ information on the covariance structure of the mixture components . 
 the present work introduces a constrained equivariant approach , where the class conditional covariance matrices are shrunk towards a pre - specified matrix @xmath0 data - driven choices of the matrix @xmath1 when _ a priori _ information is not available , and the optimal amount of shrinkage are investigated . 
 the effectiveness of the proposal is evaluated on the basis of a simulation study and an empirical example . + * keywords * : model based clustering , gaussian mixture models , equivariant estimators