I've asked this question at [stat-exchange](https://stats.stackexchange.com/questions/20805/how-is-the-proof-that-the-quartimax-varimax-rotation-converges) and at the "Semnet"-mailing list of professionals in statistics. The reference to some articles in Psychometrica (for instance ten Berge 1995, Jennrich 2001) were insofar helpful, that I saw, that reasons for nonsuccessful rotation were handled - but the explicite statement, whether the iteration with plane-wise rotations using the Varimax- or Quartimax-criterion is always converging /can always be made converging was not made/referred to. (This is different for instance for the case of using the principal components-criterion, for which I think I've come across an explicite proof ~20 years ago). 


The reason to ask this is an approach to the Hadamard-matrix-problem: the Hadamard-matrix is defined to consist of entries +1 and -1 only and being orthogonal.  

Being orthogonal means to have the form of a rotation-matrix, simply scaled by one scalar factor such that all entries have the desired value -1 or +1. That means, the *variance* of the squared entries is zero. So if I begin with any initial rotation-matrix, and rotate to *minimal* (instead of maximal) variance using the inverse of the "varimax" or of the "quartimax"-criterion, say "varimin" or "quartimin", I should arrive at a proper Hadamard-matrix, scalar scaled by the reciprocal of the square of the number of rows/columns. 


Indeed this works for small sizes (up to n=20, so nxn=20x20) and "sometimes" for n=24, but then the Hadamard-form is found extremely seldom.  

Ten Berge (1995) gave insight in the rotation procedure, noting, that sometimes convergence is not reached because of systematic missing of optimization of some planes in the iterative process, but which might be overcome by some workaround. The global convergence question however was not explicitely mentioned as solved. 


Question: is a proof for the convergence to the global optimum by the iterative, plane-wise rotation using the quartimax/varimax-rotation known? Or is it known, that it does not always converge?) (If it is simple enough it might be done here, or else a reference were good. I've it -surprisingly- not seen in the monographies on factor analysis of for instance S.Mulaik(74) or K.Überla(68) . 


(This "varimin"-rotation-approach to the Hadamard-matrix problem has its further charme, because it is also applicable to matrices which have other dimensions than n=4\*m, and is thus some more general formulation/notion for that problem: "the lower bound for the variance of the squared entries of orthogonal matrices of size nxn where n=4m is zero" )




---


Examples 



```
//commands in "Matmate" to reproduce the approach
n=8
A = gettrans(randomu(n,n),"drei")  // creates a random rotation-matrix in A
A = sqrt(n) * A                    // rescales it with the scalar  sqrt(n)
T = gettrans(A,"-varimax")         // gets the rotationmatrix T which rotates A
                                   //     using the optimization criterion "Varimin"
H = A * T                          // gets the Hadamardmatrix of size 8x8 
                                   //     by column-rotation 

```

$\qquad \small \text{ A =} \begin{array} {rrrrrrrr} 
0.95& 0.24& 1.32& 0.34& -0.87& -1.08& 1.54& -0.94\\\
1.45& 0.06& -1.03& -0.57& 1.71& -0.41& 0.05& -1.19\\\
1.29& -0.91& -0.50& 1.41& -0.98& 1.29& -0.54& -0.59\\\
1.02& 1.14& -1.36& 0.36& -0.67& -0.75& 0.35& 1.60\\\
0.53& -0.28& 1.21& 1.36& 1.14& -0.97& -1.14& 0.86\\\
0.20& 1.76& 0.72& 0.57& 0.87& 1.69& 0.62& 0.13\\\
0.64& 1.26& 0.62& -1.03& -0.92& -0.11& -1.84& -0.57\\\
1.26& -1.03& 0.85& -1.48& 0.08& 0.78& 0.36& 1.31
 \end{array}
$ 


and the Hadamardmatrix ***H*** 


$\qquad \small 
 \text{ H =} \begin{array} {rrrrrrrr} 
 -1.00& -1.00& 1.00& 1.00& -1.00& -1.00& 1.00& -1.00\\\
1.00& -1.00& -1.00& -1.00& 1.00& -1.00& 1.00& -1.00\\\
1.00& -1.00& -1.00& 1.00& -1.00& 1.00& -1.00& -1.00\\\
1.00& 1.00& -1.00& 1.00& -1.00& -1.00& 1.00& 1.00\\\
1.00& -1.00& 1.00& 1.00& 1.00& -1.00& -1.00& 1.00\\\
1.00& 1.00& 1.00& 1.00& 1.00& 1.00& 1.00& -1.00\\\
1.00& 1.00& 1.00& -1.00& -1.00& -1.00& -1.00& -1.00\\\
1.00& -1.00& 1.00& -1.00& -1.00& 1.00& 1.00& 1.00
 \end{array}
$ 


For a dimension not a multiple of *4* we do not get the variance of the
 squares of the matrix-entries to equal zero but I think, we get the solution 
 of *minimal* variance, and the squares of the entries are not *1*.  

 Example n=6:


$\qquad \small 
 \text{ A =} \begin{array} {rrrrrr} 
1.00& 1.29& 1.32& -1.10& 0.33& 0.52\\\
0.48& 0.23& 1.14& 1.56& -1.39& -0.22\\\
1.51& -1.51& -0.08& 0.09& 0.12& 1.18\\\
1.38& 0.62& -1.39& -0.40& -0.85& -0.94\\\
0.74& -0.32& 0.49& 0.63& 1.55& -1.52\\\
0.21& 1.22& -0.88& 1.34& 0.89& 1.04
 \end{array}
$  

and  

$\qquad \small 
 \text{ H =} \begin{array} {rrrrrr} 
0.00& 1.10& 1.10& -1.10& 1.10& 1.10\\\
1.10& 1.10& 1.10& 1.10& -1.10& 0.00\\\
1.10& -1.10& -0.00& -1.10& -1.10& 1.10\\\
1.10& 1.10& -1.10& -1.10& 0.00& -1.10\\\
1.10& -1.10& 1.10& 0.00& 1.10& -1.10\\\
1.10& -0.00& -1.10& 1.10& 1.10& 1.10
 \end{array}
 $ 


where the nonzero entries are exactly $\small \pm \sqrt{6 / 5} $  



---


[update] For the n=6-case I've got one alternate solution, which is not optimal but which is stationary under the varimin/quartimin-criterion. This is  

$\qquad \small 
 \text{ H =} \begin{array} {rrrrrr} 
0.75580438&0.75580439&1.32408152&1.12540822&-1.12540822&-0.75580438\\\
1.32408152&-1.12540822&-0.75580438&0.75580439&-0.75580438&1.12540822\\\
1.12540822&-1.32408152&0.75580439&-0.75580438&0.75580438&-1.12540822\\\
1.12540822&1.12540822&0.75580438&-0.75580438&0.75580439&1.32408152\\\
0.75580439&0.75580438&-1.12540822&-1.32408152&-1.12540822&-0.75580438\\\
0.75580438&0.75580438&-1.12540822&1.12540822&1.32408152&-0.75580439
 \end{array}
$  

This is just plain vanilla-"varimin", I did not yet include the ten Berge-workaround here. 



For an approach to the Hadamard-matrix-problem: is there a proof, that the iterative plane-wise orthogonal rotations (Quartimax/Varimax) converge to global maximum?
*(I posted this answer initially at the SSE-original question)* 


Working with othogonal eigenvector matrices **M** (created as random rotation matrices) a sequence of experiments suggested, that "varimin"-rotation (which is just minimizing the same criterion which "varimax" maximizes) can run into local extrema and miss the global minimum, and does this more often as the matrix-size is increased. 


I used *4x4* up to *22x22* matrices **M** and produced *1000* and more randomly created examples of the same size, trying to rotate them to "minimal variance of the squared matrix-entries". 


* For sizes *n=4k* the "varimin" rotation found always the "Hadamard"-matrix versions.
* For sizes *n=4k+2* that rotation found sometimes the "Weighing-matrices" with entries from $(1,0,-1)$ (when appropriately rescaled). However, when the dimension went to *18x18* I got that ideal solutions only in roughly 2 percent of the random examples; and for *22x22* matrices that was even less frequent.  

Here I checked the improvement of the rotation-criteria for each single iteration and stopped, when the improvement became just marginal but the distance from the optimal versions was still far away.
* For sizes *n x n* with *n=2k+1* the rotations found sometimes versions which reminded of Hadamard- resp. Weighing- matrices, however had *4,5* or more different entries. For instance for *n=13* I got in *80* percent of all examples rotated matrices with only *4* siginificantly different entries in the form $(1,a,-a,-1)$ but for *n=11* that result occured only in 1 to 2 percent of examples.


Well, I considered then, that possibly such examples, which did not rotate to the "simple" structure of Hadamard or Weighing matrix having $(1,-1)$ or $(1,0,-1)$ or at least having few-categories-solutions like $(1,a,-a,-1)$ , just are incompatible with suchoptimal solutions. But for the matrix-sizes ( *n=4k+2* ) for which I got both types of results, also the bad results could be transformed to an ideal solution by just (the correct) rotation. 


The last observation let's me thus conjecture, that indeed "varimin" is ***not*** guaranteed to find the solution with the smallest variance in the squared matrix-entries, and the same should then be valid for the "varimax"-rotation in their intention to find the maximal variance. 


***update 1.11.2018*** Just found one example of a *12x12* matrix, which by `varimin`-rotation converged first to a non-hadamard-matrix (which is *not* optimal in the terms of the `varimin`-criterion), and when an additional random-rotation was applied, then with a new `varimin`-rotation appended a true hadamard-matrix occured. 


* ***So there is proof,*** that `varimin`- rotation does not always find the global optimum (here: of zero-variance)


*(old text continued...)* This is in contrast to one remark of S. Mulaik in his book on "foundations of factoranalysis", which I cite directly from a comment in SSE:  

*Mulaik (2010, p. 304) said, for Quartimax algorithm, "No proof has ever been offered that such an iterative procedure will always converge, but with empirical structure matrices it always has".* This is the only remark on this problem (whether varimax converges to the global optimum) which I've come across so far; funny ... 


---


Well, this experiments were done on orthogonal random-matrices, which are also eigenvector matrices for correlation-matrices. Rotation in factor-/component-analysis is usually done on *loadings*, which are rescaled entries of the eigenvectors - but there are also software packets which indeed rotate on the eigenvectors.


