Ranks and Pseudo-Ranks
- Paradoxical Results of Rank Tests -

Edgar Brunner

arXiv:1802.05650v1 [] 15 Feb 2018

Department of Medical Statistics, University of Göttingen, Germany

Frank Konietschke
Department of Mathematical Sciences, University of Texas at Dallas, U.S.A.

Arne C. Bathke
Department of Mathematics, University of Salzburg, Austria
and

Markus Pauly
Institute of Statistics, Ulm University, Germany

Abstract
Rank-based inference methods are applied in various disciplines, typically when procedures relying on standard normal theory are not justifiable, for example when data are not symmetrically
distributed, contain outliers, or responses are even measured on ordinal scales. Various specific
rank-based methods have been developed for two and more samples, and also for general factorial designs (e.g., Kruskal-Wallis test, Jonckheere-Terpstra test). It is the aim of the present
paper (1) to demonstrate that traditional rank-procedures for several samples or general factorial designs may lead to paradoxical results in case of unbalanced samples, (2) to explain why
this is the case, and (3) to provide a way to overcome these disadvantages of traditional rankbased inference. Theoretical investigations show that the paradoxical results can be explained
by carefully considering the non-centralities of the test statistics which may be non-zero for
the traditional tests in unbalanced designs. These non-centralities may even become arbitrarily
large for increasing sample sizes in the unbalanced case. A simple solution is the use of socalled pseudo-ranks instead of ranks. As a special case, we illustrate the effects in sub-group
analyses which are often used when dealing with rare diseases.

2

1

Paradoxical Results of Rank Tests

Introduction

If the assumptions of classical parametric inference methods are not met, the usual recommendation is to apply nonparametric rank-based tests. Here, the Wilcoxon-Mann-Whitney and
Kruskal-Wallis (1952) tests are among the most commonly applied rank procedures, often utilized as replacements for the unpaired two-sample t-test and the one-way ANOVA, respectively. Other popular rank methods include the Hettmansperger-Norton (1987) and JonckheereTerpstra (1952, 1954) tests for ordered alternatives, and the procedures by Akritas et al. (1997)
for two- or higher-way designs. In statistical practice, these procedures are usually appreciated as robust and powerful inference tools when standard assumptions are not fulfilled. For
example, Whitley and Ball (2002) conclude that “Nonparametric methods require no or very
limited assumptions to be made about the format of the data, and they may, therefore, be preferable when the assumptions required for parametric methods are not valid.” In line with this
statement, Bewick et al. (2004) also state that “the Kruskal-Wallis, Jonckheere-Terpstra (..)
tests can be used to test for differences between more than two groups or treatments when the
assumptions for analysis of variance are not held.”
These descriptions are slightly over-optimistic since nonparametric methods also rely on
certain assumptions. In particular, the Wilcoxon-Mann-Whitney and Kruskal-Wallis tests postulate homoscedasticity across groups under the null hypothesis, and they have originally only
been developed for continuous outcomes. In case of doubt, it is nevertheless expected that rank
procedures are more robust and lead to more reliable results than their parametric counterparts.
While this is true for deviations from normality, and while by now it is widely accepted that
ordinal data should rather be analyzed using adequate rank-based methods than using normal
theory procedures, we illustrate in various instances that nonparametric rank tests for more than
two samples possess one noteworthy weakness. Namely, they are generally non-robust against
changes from balanced to unbalanced designs. In particular, keeping the data generating processes fixed, we provide paradigms under which commonly used rank tests surprisingly yield
completely opposite test decisions when rearranging group sample sizes. These examples are
in general not artificially generated to obtain paradoxical results, but even include homoscedastic normal models. This effect is completely undesirable, leading to the somewhat heretical
question
• Are nonparametric rank procedures useful at all to handle questions for more than two
groups?
In order to comprehensively answer this question, we carefully analyze the underlying nonparametric effects of the respective rank procedures. From this, we develop detailed guidelines
for an adequate application of rank-based procedures. Moreover, we even state a simple solution
for all these problems: Substituting ranks by closely related quantities, the so-called pseudoranks that have already been considered by Kulle (1999), Gao and Alvo (2005a, b), Gao, Alvo,

Paradoxical Results of Rank Tests

3

Chen, and Li (2008), and in more detail by Thangavelu and Brunner (2007), and by Brunner,
Konietschke, Pauly and Puri (2017). It should be noted that the motivation in these references
was different, and that their authors had not been aware of the striking paradoxical properties
that may arise when using classical rank tests. These surprising paradigms only appear in case
of unbalanced designs since all rank procedures discussed below coincide with their respective
pseudo-rank analogs in case of equal sample sizes. Pseudo-ranks are easy to compute, share the
same advantageous properties of ranks and lead to reliable and robust inference procedures for
a variety of factorial designs. Moreover, we can even obtain confidence intervals for (contrasts
of) easy to interpret reasonable nonparametric effects. Thus, resolving the commonly raised
disadvantage that “nonparametric methods are geared toward hypothesis testing rather than
estimation of effects” (Whitley and Ball, 2002).
The paper is organized as follows. Notations are introduced in Section 2. Then in Section 3
some paradoxical results are presented in the one-way layout for the Kruskal-Wallis test and
for the Hettmansperger-Norton trend test by means of certain tricky (non-transitive) dice. In
the two-way layout, a paradoxical result for the Akritas-Arnold-Brunner test in a simple 2 × 2design is presented in Section 4 using a homoscedastic normal shift model. The theoretical
background of the paradoxical results is discussed in Section 5 and a solution of the problem by
using pseudo-ranks is investigated in detail. Moreover, the computation of confidence intervals
is discussed and applied to the data in Section 4. Section 6 provides a cautionary note for the
problem of sub-group analysis where typically unequal sample sizes appear.
The paper closes with some guidelines for adequate application of rank procedures in the
discussion and conclusions section. There, it is also briefly discussed that the use of pairwise
and stratified rankings would make matters potentially worse.

2

Statistical Model and Notations

P
For d > 2 samples of N = di=1 ni independent observations Xik ∼ Fi = 21 [Fi− + Fi+ ], i =
1, . . . , d, k = 1, . . . , ni , the nonparametric relative effects which are underlying the rank tests are
commonly defined as
pi =

Z

d
1X
HdFi , where H =
nr F r
N r=1

(1)

denotes the weighted mean distribution of the distributions F1 , . . . , Fd in the design. Here we
use the so-called normalized version of the the distribution Fi to cover the cases of continuous,
as well as non-continuous distributions in a unified approach. Thus, the case of ties does not
require a separate consideration. This idea was first mentioned by Kruskal (1952) and later
considered in more detail by Ruymgaart (1980). Akritas, Arnold, and Brunner (1997) extended
this approach to factorial designs, while Akritas and Brunner (1997) and Brunner, Munzel, and
Puri (1999) applied this technique to repeated measures and longitudinal data.

4

Paradoxical Results of Rank Tests

Easily interpreted, pi = P(Z < Xi1 ) + 12 P(Z = Xi1 ) is the probability that a randomly selected
observation Z from the weighted mean distribution H is smaller than a randomly selected observation Xi1 from the distribution Fi plus 21 times the probability that both observations are equal.
Thus, the quantity pi measures an effect of the distribution Fi with respect to the weighted
mean distribution H. In the case of two independent random variables X1 ∼ F1 and X2 ∼ F2 ,
Birnbaum and Klose (1957) had called the function L(t) = F2 [F1−1 (t)] the “relative distribution
function” of X1 and X2 , assuming continuous distributions. Thus, its expectation
Z 1
Z ∞
tdL(t) =
F1 (s)dF2 (s) = P(X1 < X2 )
0

−∞

is called a “relative effect” with an obvious adaption of the notation. In the same way, the
quantity pi = P(Z < Xi1 ) + 12 P(Z = Xi1 ) is called a “relative effect” of Xi1 ∼ Fi with respect
to theRweighted mean Z ∼ H. This effect pi is a linear combination of the pairwise effects
wri = Fr dFi . In vector notation, Equation (1) is written as
p =

Z


 w11 , · · · wd1

..
..
HdF = W 0 n =  ...
.
.

w1d , · · · wdd

 
  n1 /N
  .
 ·  ..
 
nd /N







 = 



Here, F = (F1 , . . . , Fd )0 denotes the vector of distribution functions, and


Z
 w11 , · · · w1d 

.. 
..
W =
F0 dF =  ...
.
. 


wd1 , · · · wdd

p1
..
.
pd




 .


(2)

(3)

is the matrix of pairwise effects wri . Note that wii = 21 and wir = 1 − wri which follows from
integration by parts. The relative effects pi are arranged in the vector p = (p1 , . . . , pd )0 and can
be estimated consistently by the simple plug-in estimator
Z


b F
bi = 1 Ri· − 1 .
b
pi =
Hd
(4)
2
N
bi denotes the (normalized) empirical distribution of Xi1 , . . . , Xini , i = 1, . . . , d, and
Here, F
P
b = 1 dr=1 nr F
br their weighted mean. Finally, Ri· = 1 Pni Rik denotes the mean of the ranks
H
k=1
N
ni
d

Rik

n

r
1
1 XX
b
=
+ N H(Xik ) = +
c(Xik − Xr` ),
2
2 r=1 `=1

(5)

where the function c(u) = 0, 1/2, 1 for u <, = or > 0, respectively, denotes the count function.
The estimators b
p1 , . . . , b
pd are arranged in the vector
Z


b F
b = 1 R· − 1 1d ,
b
p =
Hd
(6)
2
N

Paradoxical Results of Rank Tests

5

b1 , . . . , F
bd )0 is the vector of the empirical distributions, R· = (R1· , . . . , Rd· )0 the
b = (F
where F
vector of the rank means Ri· , and 1d = (1, . . . , 1)0d×1 denotes the vector of 1s.
In the following sections we demonstrate that for d ≥ 3 groups, rank tests may lead to
paradoxical results in case of unequal sample sizes. In particular, for factorial designs involving
two or more factors, Rthe nonparametric main effects and interactions (defined by the weighted
relative effects pi j = HdFi j ) may be severely biased.

3

Paradoxical Results in the One-Way Layout

To demonstrate some paradoxical results of rank tests for d ≥ 3 samples in the one-way layout,
we consider the vector p = W 0 n in (2) of the nonparametric effects pi , which are all equal
P
P
to their mean p· = d1 di=1 pi iff di=1 (pi − p· )2 = 0 or in matrix notation p0 T d p = 0. Here,
T d = Id − d1 J d denotes the centering matrix, Id the d-dimensional unit matrix, and J d = 1d 10d
R
b F
b denote the plug-in estimator of b
the d × d-dimensional matrix of 1s. Let b
p = Hd
p defined
in (6).
√ In order to detect whether the the pi are different, we study the asymptotic distribution
p. This is obtained from the asymptotic equivalence theorem (see, e.g., Akritas et al.,
of NT d b
1997; Brunner and Puri, 2001, 2002 or Brunner et al., 2017),

 √
√
√
.
NT d b
p =.
NT d Y · + Z · − 2p + NT d p,
(7)
R
R
.
b are
b and Z · = HdF
where the symbol =. denotes asymptotic equivalence. Here, Y · = Hd F
vectors of means of independent random vectors
with expectation
E(Y · ) = E(Z · ) = p. It follows


√
from the central limit theorem that NT d Y · + Z · − 2p has, asymptotically, a multivariate nor
√
mal distribution with mean 0 and covariance matrix T d ΣN T d , where ΣN = Cov N[Y · + Z · ]
has a quite involved structure
√ (for details see Brunner et al., 2017). Obviously, the multivariate
distribution is shifted by NT d p from the origin 0. Therefore, we call T d p the “multivariate non-centrality”, and a “univariate non-centrality” may be quantified by the quadratic form
c p = p0 T d p. In particular, we have c p = 0 iff T d p =√0. The actual (multivariate) shift of the
distribution, depending on the total sample size N, is NT d p, and the corresponding univariate
non-centrality (depending on N) is then given by N · c p . From these considerations, it should
become clear √
that N · c p → ∞ as N → ∞ if T d p , 0. This defines the consistency region of a
test based on NT d b
p.
Below, we will demonstrate that for the same vector of distributions F, the non-centrality
c p = p0 T d p may be 0 in case of equal sample sizes, while c p may be unequal to 0 in case of
unequal sample sizes. Under H0F : T d F = 0, tests based on b
p (such as the Kruskal-Wallis
test) reject the hypothesis H0F with approximately the pre-assigned type-I error probability α.
If, however, the strong hypothesis H0F is not true then the non-centrality c p = p0 T d p may be
0 or unequal to 0 for the same set of distributions F1 , . . . , Fd , since c p depends on the relative
samples sizes n1 /N, . . . , nd /N. This means that for the same set of distributions F1 , . . . , Fd

6

Paradoxical Results of Rank Tests

and unequal sample sizes the p-value of the test may be arbitrary small if N is large enough.
However, the p-value for the same test may be quite large for the same total sample size N in
case of equal sample sizes. Some well-known tests which have this paradoxical property are,
for example, the Kruskal-Wallis test (1952), the Hettmansperger-Norton trend test (1987), and
the Akritas-Arnold-Brunner test (1997).
As an example, consider the case of d = 3 distributions where straightforward calculations
show that
1. in case of equal sample sizes,
p1 = p2 = p3 ⇐⇒

w21 = w32 = 1 − w31 = w,

(8)

w21 = w32 = w31 = 12 .

(9)

2. in general, however,
p1 = p2 = p3 ⇐⇒

This means that c p = p0 T d p = 0 in case of equal sample sizes, but c p , 0 in case of unequal
sample sizes if w21 = w32 = 1 − w31 = w , 12 .
We note that (9) follows under the strict hypothesis H0F : F1 = F2 = F3 . However, this
null hypothesis is not a necessary condition for (9) to hold. For example,
if Fi , i = 1, 2, 3 are
R
symmetric distributions with the same center of symmetry then wri = Fr dFi = 12 for i = 1, 2, 3.
Thus, in this case, c p = 0 is also true for all samples sizes.
An example of discrete distributions generating the nonparametric effects wri in (8) is given
by the probability mass functions
• f1 (x) =

1
6

if x ∈ {9, 16, 17, 20, 21, 22} and f1 (x) = 0 otherwise,

• f2 (x) =

1
6

if x ∈ {13, 14, 15, 18, 19, 26} and f2 (x) = 0 otherwise,

• f3 (x) =

1
6

if x ∈ {10, 11, 12, 23, 24, 25} and f3 (x) = 0 otherwise,

which are derived from some tricky dice (see, e.g., Peterson, 2002). For the distribution functions Fi (x) defined by fi (x), i = 1, 2, 3 above, it is easily seen that
Z
w21 = P(X2 < X1 ) =
F2 dF1 = 7/12
(10)
Z
w13 = P(X1 < X3 ) =
F1 dF3 = 7/12
(11)
Z
w32 = P(X3 < X2 ) =
F3 dF2 = 7/12.
(12)
Thus, w21 = w13 = 1 − w31 = w32 = w and the vector of the weighted relative effects is given
by
 1


 n1 + n3 + (n2 − n3 )w
 p1 
1  2


0
p =  p2  = W n =
 n1 + 12 n2 + (n3 − n1 )w


N 
p3
n2 + 12 n3 + (n1 − n2 )w




 .


(13)

Paradoxical Results of Rank Tests

7

Table 1: Ratios of relative sample sizes ni/N , weighted relative effects pi , and the non-centralities for the example
of the tricky dice where w = 7/12 and the distributions F1 , F2 , and F3 are fixed.

Setting

n1/N

n2/N

n3/N

(A)
(B)
(C)

1/3

1/3

1/3

2/3

1/12

1/4

1/4

2/3

1/12

p1

p2

p3

0.5
0.5
0.5
0.4861 0.4653 0.5486
0.5486 0.4861 0.4653

p·

cp

0.5 0
0.5 0.00376
0.5 0.00376

The weighted relative effects pi and the resulting non-centralities c p are listed in Table 1 for
equal and some different unequal sample sizes.
Since for unequal sample sizes one obtains c p , 0, it is only a question of choosing the total
sample size N large enough to reject the hypothesis H0F : F1 = F2 = F3 by the Kruskal-Wallis
test with a probability arbitrary close to 1 while in case of equal sample sizes for N → ∞ the
probability of rejecting the hypothesis remains constant equal to α∗ (close to α) since in this
case, c p = 0. It may be noted that in general α∗ , α since the variance estimator of the KruskalWallis statistic is computed under the strong hypothesis H0F : F1 = F2 = F3 , which is obviously
not true here. Thus, the scaling is not correct, and the Kruskal-Wallis test has a slightly different
type-I error α∗ .
For the Hettmansperger-Norton trend test, the situation gets worse since for different ratios
of sample sizes the nonparametric effects p1 , p2 , and p3 may change their order. In setting
(B) in Table 1 we have p2 < p1 < p3 , while in setting (C) we have p3 < p2 < p1 . Now
consider the non-centrality of the Hettmansperger-Norton trend test which is a linear rank test.
Let c = (c1 , . . . , cd )0 denote a vector reflecting the conjectured pattern. Then it follows from (7)
that
 √
√ 0 
√ 0
.
N c Td b
p =.
N c T d Y · + Z · − 2p + N c0 T d p,
(14)
where cHN = c0 T d p is a univariate non-centrality. If T d F = 0 then it follows that T d p = 0
and cHN = c0 T d p = 0. If, however, T d F , 0 then cHN < 0 indicates a decreasing trend
and cHN > 0 an increasing trend. In the above discussed example, we obtain for setting (B)
and for a conjectured pattern of c = (1, 2, 3)0 for an increasing trend the non-centrality cHN =
P3
1
1
i=1 ci (pi − 2 ) = /16 > 0, indeed indicating an increasing trend. For setting (C) however, we
1
obtain cHN = − /12, indicating a decreasing trend. In case of setting (A) (equal sample sizes),
cHN = 0 since p1 = p2 = p3 = 21 , and thus indicating no trend. Again it is a question of the total
sample size N to obtain the decision of a significantly decreasing trend for the first setting (B)
of unequal sample sizes and for the second setting (C) the decision of a significantly increasing
trend with
F1 , F2 , and F3 . In the fist
√ a probability arbitrary close to 1 for the same distributions
√
case, N · cHN → ∞ for N → ∞ and in the second case, N · cHN → −∞ for N → ∞. In case
of equal sample sizes, the hypothesis of no trend is only rejected with a type-I error probability
α∗∗ . Regarding α∗∗ , α, a similar remark applies as above for the Kruskal-Wallis test.

8

4

Paradoxical Results of Rank Tests

Paradoxical Results in the Two-Way Layout

In the previous section, paradoxical decisions by rank tests in case of unequal sample sizes were
demonstrated for the one-way layout using large sample sizes and particular distributions leading to non-transitive decisions. In this section, we will show that in two-way layouts paradoxical
results are already possible with rather small sample sizes and even in simple homoscedastic
normal shift models. To this end, we consider the simple 2 × 2-design with two crossed factors
A and B, each with two levels i = 1, 2 for A and j = 1, 2 for B. The observations Xi jk ∼ Fi j ,
k = 1, . . . , ni j , are assumed to be independent.
The hypotheses of no nonparametric effects in terms of the distribution functions Fi j (x) are
expressed as (see Akritas et al., 1997)
(1) no main effect of factor A - H0F (A) : F11 + F12 − F21 − F22 = 0
(2) no main effect of factor B - H0F (B) : F11 − F12 + F21 − F22 = 0
(3) no interaction AB - H0F (AB) : F11 − F12 − F21 + F22 = 0,
where in all three cases, 0 denotes a function which is identical 0.
Let F = (F11 , F12 , F21 , F22 )0 denote the vector of the distribution functions. Then the three
hypotheses formulated above can be written in matrix notation as H0F (c) : c0 F = 0, where
c = cA = (1, 1, −1, −1)0 generates the hypothesis for the main effect A, c = cB = (1, −1, 1, −1)0
for the main effect B, and c = cAB = (1, −1, −1, 1)0 for the interaction AB.
For testing these hypotheses, Akritas et al. (1997) derived rank procedures based on the
statistic
T N (c) =

√

1
N c0 b
p = √ c0 R· ,
N

(15)

where R· = (R11· , R12· , R21· , R22· )0 denotes the vector of the rank means Ri j· within the four
samples. They showed that under the hypothesis H0F (c), the statistic T N (c) has, asymptotically,
a normal distribution with mean 0 and variance
σ20

2 X
2
X
N 2
=
σ ,
n ij
i=1 j=1 i j

(16)

where the unknown variances σ2i j (see Akritas et al., 1997, for their explicit form) can be consistently estimated by
ni j

X
1 2
1
S
=
(Ri jk − Ri j· )2
N2 i j
N 2 (ni j − 1) k=1

(17)

Paradoxical Results of Rank Tests

9

Table 2: Comparison of the empirical distributions in the four factor level combinations A1 B1 , A2 B1 , A1 B2 , and

A2 B2 sampled from homoscedastic normal distributions with µ11 = 10, µ12 = µ21 = 9, µ22 = 8, and standard deviation τ = 0.4. Within each factor level combination Ai B j , i, j = 1, 2, the unadjusted p-values of a t-test comparing
the location of balanced and unbalanced samples, more specifically testing µi j (balanced) = µi j (unbalanced), are
listed in the last column.
Level

Means

Standard Deviations

t-Tests

Combinations

Balanced

Unbalanced

Balanced

Unbalanced

p-Values

A1 B1
A2 B1
A1 B2
A2 B2

10.07
9.04
9.05
8.07

9.91
8.95
8.99
8.02

0.311
0.380
0.408
0.371

0.314
0.313
0.480
0.359

0.178
0.409
0.684
0.562

P P
and b
σ20 = N 2i=1 2j=1 S i2j /ni j . For small sample sizes, the null distribution of LN (c) = T N (c)/b
σ0
can be approximated by a t f -distribution with estimated degrees of freedom
S 04

b
f = P2 P2
i=1

2
2
j=1 (S i j /ni j ) /(ni j

− 1)

,

(18)

P P
where S 02 = 2i=1 2j=1 S i2j /ni j . The non-centrality of T N (c) is given by cT = c0 p, and under the
restrictive null hypothesis H0F (c) : c0 F = 0 it follows that c0T p = 0.
To demonstrate a paradoxical result, we assume that the observations Xi jk are coming from
the normal distributions N(µi j , τ2 ) with equal standard deviations τ = 0.4 and expectations
µ = (µ11 , µ12 , µ21 , µ22 )0 = (10, 9, 9, 8)0 . From the viewpoint of linear models, there is a main
effect A of c0A µ = µ11 + µ12 − µ21 − µ22 = 2, a main effect B of c0B µ = µ11 − µ12 + µ21 − µ22 = 2,
and no AB-interaction since c0AB µ = µ11 − µ12 − µ21 + µ22 = 0. Since this is a homoscedastic
linear model, the classical ANOVA should reject the hypotheses H0µ (cA ) : c0A µ = 0 and H0µ (cB ) :
c0B µ = 0 with a high probability if the total sample size is large enough. In contrast to that, the
hypothesis H0µ (cAB ) : c0AB µ = 0 of no interaction is only rejected with the pre-selected typeI error probability α. The non-centralities are given by cµA = c0A µ = 2, cµB = c0B µ = 2, and
cµAB = c0AB µ = 0. The following two settings of samples sizes ni j are considered:
(1) n11 = 10, n12 = 20, n21 = 20, n22 = 50,
(2) n11 = n12 = n21 = n22 = 25,

- (unbalanced)

- (balanced).

First we demonstrate that the empirical characteristics of the two data sets, which are sampled from the same distributions, are nearly identical. Thus, potentially different results could
not be explained by substantially different empirical distributions obtained by an “unhappy randomization”. The results of the comparisons are listed in Table 2.
We apply the classical ANOVA F-statistic and the rank statistic LN (c) = T N (c)/b
σ0 to the
same simulated data sets from Table 2 and compare the results for testing the three hypotheses

10

Paradoxical Results of Rank Tests

Table 3: Comparison of the results obtained by an ANOVA and by the rank test LN (c) in case of a balanced
(left) and an unbalanced (right) 2 × 2-design. Surprising is the fact that in the balanced case, the decisions of both
procedures coincide while in the unbalanced case the decisions for testing the interaction AB are totally opposite.
Balanced
n11 = n12 = n21 = n22 = 25
ANOVA
Effect
A
B
AB

Unbalanced
n11 = 10, n12 = 20, n21 = 20, n22 = 50

Rank Test

ANOVA

Rank Test

F

p-Value

L2N (c)

p-Value

F

p-Value

L2N (c)

p-Value

184.43
182.60
0.12

< 10−4
< 10−4
0.7317

168.19
169.32
0.00

< 10−4
< 10−4
0.9541

120.99
111.48
0.01

< 10−4
< 10−4
0.9112

150.07
143.55
7.99

< 10−4
< 10−4
0.0065

H0F (A), H0F (B), and H0F (AB). We note that H0F (c) : c0 F = 0 ⇒ H0µ (c) : c0 µ = 0 and c0 µ ,
0 ⇒ c0 F , 0. The decisions for H0F (A), H0F (B), and H0F (AB) obtained by the ANOVA as
well as by the rank tests based on LN (c) are identical in all cases in the balanced setting. In the
unbalanced setting, all decisions obtained by the parametric ANOVA are comparable to those
in the balanced setting. However, the decision on the interaction AB based on the rank test is
totally different from that obtained by the parametric ANOVA, as well as that obtained for the
rank test in the balanced setting. The results are summarized in Table 3.
On the surface, the difference of the decisions in the unbalanced case could be explained
by the fact that the nonparametric hypothesis H0F (AB) and the parametric hypothesis H0µ (AB)
are not identical and that this particular configuration of normal distributions falls into the gap
between H0F (AB) and H0µ (AB). That is, here H0µ (AB) is true, but H0F (AB) is not. It is surprising,
however, that this explanation does not hold for the balanced case. The difference of the two
p-values 0.9541 and 0.0065 in Table 3 calls for an explanation.
The reason becomes clear when computing the vector p in (2) for this particular example of
the 2 × 2-design. To avoid fourfold indices we re-label the distributions F11 , F12 , F21 , and F22
as F1 , F2 , F3 , and F4 , respectively, and the sample sizes accordingly as n1 , n2 , n3 , and n4 . In the
2
2
2
example, F1 = N(10,
R τ ), F2 = F3 = N(9, τ ), and F4 = N(8, τ ), where τ = 0.4. Thus, the
probabilities wri = Fr dFi of the pairwise comparisons are

w = w12 = w13 = w24 = w34
w23 = w32
v = w14

!
−1
= 0.0392,
= Φ √
τ 2
1
=
,
2 √ 
 − 2 
 ≈ 0.
= Φ 
τ

Paradoxical Results of Rank Tests
Finally, by observing wri = 1 − wir , we obtain
 1
 2 n1 + n2 + n3 + n4 − (n2 + n3 )w − n4 v
 1
 2 (n2 + n3 ) + n4 + (n1 − n4 )w
1

p = W0 n =
N  12 (n2 + n3 ) + n4 + (n1 − n4 )w

1
n + (n2 + n3 )w + n1 v
2 4

11





 ,




and the nonparametric AB-interaction is described by
p
cAB
= c0AB p = p1 − p2 − p3 + p4

n1 − n4  1
−
2w
+
v
.
=
2
N

(19)

p
In this example, we
obtain for
equal samples sizes cAB
= 0, while for n = (10, 20, 20, 50)


√
p
p
2 1
we obtain cAB = − 5 2 − 2w + v ≈ −0.1686 and NcAB ≈ −1.686. This explains the small
p-value for unequal sample sizes in the example.

5

Explanation of the Paradoxical Results

5.1

Unweighted Effects and Pseudo-Ranks

The simple reason for the paradoxical results is the fact that even when all distribution functions underlying the observations are specified, the consistency regions C0 p , 0 of the rank
tests based on C0 b
p are not fixed. Indeed, the consistency regions are defined by the weighted
nonparametric relative effects pi which are not fixed model quantities by which hypotheses
could be formulated or for which confidence intervals could be reasonably computed since the
pi themselves generally depend on the sample sizes ni .
Thus, it appears reasonable to define different nonparametric effects which are fixed model
1 Pd
quantities not depending on sample
R sizes. To this end let G = d r=1 Fr denote the unweighted
mean distribution, and let ψi = GdFi . Easily interpreted, this nonparametric effect ψi measures an effect of the distribution Fi with respect to the unweighted mean distribution G and is
P
therefore a “fixed relative effect”. As ψi = d1 dr=1 wri is the mean of the pairwise nonparametric
effects w1i , . . . , wdi , it can be written in vector notation as the vector of row means of W 0 , that is,




Z
 w11 , · · · wd1 
 ψ1 

..  · 1 1 =  ..  .
..
ψ =
GdF = W 0 · d1 1d =  ...
(20)
 . 
.
.  d d




w1d , · · · wdd
ψd
The fixed relative effects ψi can be estimated consistently by the simple plug-in estimator
Z


b F
bi = 1 Rψi· − 1 ,
b
ψi =
Gd
(21)
2
N

12

Paradoxical Results of Rank Tests

b=
where G
and

ψ
Ri·

=

1
ni

1
d

Pd
r=1

br denotes the unweighted mean of the empirical distributions F
b1 , . . . , F
bd ,
F

ψ
k=1 Rik

Pni

the mean of the so-called pseudo-ranks

ps-rank(Xik ) =

Rψik

nr
d
1 NX 1 X
1
b
+ N G(Xik ) = +
c(Xik − Xr` ).
=
2
2 d r=1 nr `=1

(22)

Finally, the estimators b
ψi are arranged in the vector



b = 
ψ

ψ

ψ

b
ψ1
..
.
b
ψd



Z
 ψ


b F
b = 1 R· − 1 1d ,
 =
Gd
2

N

ψ

(23)

ψ

where R· = (R1· , . . . , Rd· )0 is the vector of the pseudo-rank means Ri· .
It may be noted that the pseudo-ranks Rψik have similar properties as the ranks Rik . The
properties given below follow from the definitions of the ranks and pseudo-ranks and by some
straightforward algebra.
P
Lemma 1 Let Xik denote N = di=1 ni observations arranged in i = 1, . . . , d groups each involving ni observations. Then, for i, r = 1, . . . , d and k = 1, . . . , ni , ` = 1, . . . , nr ,
1. If Xik < Xr` then Rik < Rr` and Rψik < Rψr` .
2. If Xik = Xr` then Rik = Rr` and Rψik = Rψr` .
3.

1
N

Pd Pni
i=1

k=1

Rik =

1
d

Pd

1
i=1 ni

Pni
k=1

Rψik =

N+1
.
2

4. If m(u) is a strictly monotone transformation of u then,
(a) Rik = rank(Xik ) = rank(m(Xik ))
(b) Rψik = ps-rank(Xik ) = ps-rank (m(Xik )).
5. 1 ≤ Rik ≤ N.
6.

d+1
1
N
1
N
d−1
≤ +
≤ Rψik ≤ N + −
≤N+
2d
2 2dni
2 2dni
2d

7. If ni ≡ n, i = 1, . . . , d, then Rik = Rψik
8. Let Xik , i = 1, . . . , d; k = 1, . . . , ni be independent and identically distributed random
variables, then
 
N+1
E (Rik ) = E Rψik =
.
2

Paradoxical Results of Rank Tests

5.2

13

Consistency Regions of Pseudo-Rank Procedures

As a solution to the paradoxical results discussed in Sections 3 and 4, we demonstrate that
replacing the ranks Rik with the pseudo-ranks Rψik leads to procedures that do not have these
undesirable properties. The main reason is that pseudo-rank procedures are based on the (unweighted) relative effects ψi which are fixed model quantities by which hypotheses can be formulated and for which confidence intervals can be derived. In case of equal sample sizes ni ≡ n,
i = 1, . . . , d, we do not obtain paradoxical results since in this case ranks and pseudo-ranks coincide, Rik = Rψik (see Lemma 1).
Pseudo-rank based inference procedures are obtained in much the same way as the common
rank procedures, by using relations (21) and (22), which generally means substituting ranks
by the corresponding pseudo-ranks. Only in the computation of the confidence intervals (see
Section 5.3), there is a minor change in the variance estimator. For details we refer to Brunner,
Bathke, and Konietschke (2018, Result 4.16 in Section 4.6.1).
Below, we examine the behavior of pseudo-rank procedures in the situations where the use
of rank tests led to paradoxical results.
1. For testing the hypothesis H0F : F1 = F2 = F3 ⇐⇒ T 3 F = 0 in case of the tricky dice in
the example in Section 3, one obtains for the non-centrality of the Kruskal-Wallis statistic,
when the ranks Rik are replaced with the pseudo-ranks Rψik , the value cψ = ψ0 T 3 ψ = 0.
This is easily seen from (20), since in this case it follows from (10), (11), and (12) that
ψ = W 0 31 13 = 21 13 and cψ = ψ0 T 3 ψ = 12 103 T 3 12 13 = 0 by noting that T 3 is a contrast
matrix and thus, T 3 13 = 0.
2. The non-centrality for the Hettmansperger-Norton trend test when substituting ranks with
pseudo-ranks becomes cψHN = c0 T 3 ψ = c0 T 3 W 0 31 13 = 0 for all trend alternatives
c = (c1 , c2 , c3 )0 .
3. In the two-way layout, we reconsider the example of the four shifted normal distributions
we obtain the unweighted relative effects ψi j

ψ = W0 ·

1
1
4 4

 7
 /2 − 2w − v
1  2
=

4  2
1/2 + 2w + v








and the non-centrality of the statistic for the interaction
cψAB = ψ1 − ψ2 − ψ3 + ψ4 = 0.

(24)

In the analysis of the unbalanced data in Table 3 using pseudo-ranks, one obtains for the
interaction the statistic L2N (cAB ) = 0.08 and the resulting p-value is 0.7832 which agrees
with the result for the balanced sample from the same distributions in Table 3.

14

Paradoxical Results of Rank Tests

In all cases, paradoxical results obtained by changing the ratios of the sample sizes cannot
occur since the non-centralities cψ , cψHN , and cψAB are equal to 0 for all constellations of the
relative sample sizes.
R
In case of d = 2 samples, it is easily seen that p2 − p1 = ψ2 − ψ1 = p = F1 dF2 which does
not depend on sample sizes. Thus, paradoxical results for rank-based tests – such as presented
in the previous sections – can only occur for d ≥ 3 samples.

5.3

Confidence Intervals

Here we briefly explain the details on how to compute confidence intervals for fixed nonparametric effects which have an intuitive and easy to understand interpretation. This shall be
demonstrated by means of the example involving the four shifted normal distributions considered in Section 4.
The quantity b
ψi estimates the probability that a randomly drawn observation from the mean
P
distribution G = 14 4i=1 Fi is smaller than a randomly drawn observation from distribution Fi
(plus 12 times the probability that they are equal). The estimator is obtained from (21), and the
limits of the confidence interval are obtained from formula (25) in Brunner et al. (2017) or
Result 4.16 in Chapter 4 of Brunner, Bathke, and Konietschke (2018). The results are listed in
Table 4.
Table 4: Estimates and two-sided 95%-confidence intervals [ψi,L , ψi,U ] for the (unweighted) nonparametric rel-

ative effects ψi of the two data sets considered in Table 3. The results for the balanced case (upper part) and the
unbalanced case (lower part) are quite similar.
Equal Sample Sizes n1 = n2 = n3 = n4 = 25
Sample

Lower Limit ψi,L

Estimator ψi

Upper Limit ψi,U

1
2
3
4

0.84
0.45
0.45
0.12

0.86
0.50
0.50
0.14

0.88
0.55
0.55
0.16

Unequal Sample Sizes n1 = 10, n2 = n3 = 20, n4 = 50
Sample

Lower Limit ψi,L

Estimator ψi

Upper Limit ψi,U

1
2
3
4

0.85
0.45
0.43
0.13

0.86
0.51
0.48
0.15

0.88
0.57
0.53
0.16

Obviously, the results for equal and unequal sample sizes are nearly identical. Let samples
1 and 2 refer to the factor level combinations A1 B1 and A2 B1 within level B1 and samples 3 and

Paradoxical Results of Rank Tests

15

4 to the factor level combination A1 B2 and A2 B2 within level B2 , respectively. The differences
b
ψ1 − b
ψ2 and b
ψ3 − b
ψ4 between factor levels A1 and A2 of the estimates in factor level B1 and in
factor level B2 are identical in the balanced case (b
ψ1 − b
ψ2 = 0.36, b
ψ3 − b
ψ4 = 0.36) and nearly
identical (b
ψ1 − b
ψ2 = 0.35, b
ψ3 − b
ψ4 = 0.33) in the unbalanced case. These results do not indicate
any interaction. This finding agrees with the analysis of the balanced data in Table 3 when using
ranks.
For the weighted relative effects pi in (1), the estimators b
pi based on ranks, as well as 95%intervals for pi are listed in Table 5. The 95%-intervals for pi , however, cannot strictly be
interpreted as confidence intervals since the quantities pi depend on the relative sample sizes,
unless the design is balanced. In case of equal sample sizes we have pi = ψi . Therefore only
the results for the unbalanced case are listed in Table 5. The 95%-intervals for pi are obtained
from formulas (1.14) - (1.17) in Brunner and Puri (2001) and formulas (2.2.33) - (2.2.35) in
Section 2.2.7 in Brunner and Munzel (2013).
Comparing the differences of the estimates within the two factor levels B1 and B2 using the
usual ranks, we obtain b
p1 − b
p2 = 0.25 and b
p3 − b
p4 = 0.41 which are quite different. Such
a difference indicates an interaction effect, which agrees with the results of the analysis of
the unbalanced data in Table 3. Moreover, the corresponding 95%-intervals for the weighted
effects pi are totally different from the 95%-confidence intervals for the unweighted effects ψi
in Table 4. They even do not overlap with the confidence intervals for the ψi . Thus, only the
nonparametric effects ψi offer the possibility of computing estimates and confidence intervals
for fixed model quantities with an intuitive interpretation.
Table 5: Estimates and 95%-intervals [pi,L , pi,U ] for the weighted effects pi of the data set in the unbalanced case
in Table 2. The results for the balanced case are identical to those in Table 4 since pi = ψi for equal sample sizes.
Unequal Sample Sizes n1 = 10, n2 = n3 = 20, n4 = 50

6

Sample

Lower Limit pi,L

Estimate b
pi

Upper Limit pi,U

1
2
3
4

0.93
0.62
0.63
0.25

0.94
0.69
0.68
0.27

0.95
0.74
0.72
0.28

Cautionary Note for Sub-Group Analysis

The fact that the (weighted) relative effects in (2), which are estimated by the ranks, depend on
the relative sample sizes ni /N is important in sub-group analyses, for example in clinical trials.
Here, typically, the total group of patients is quite large while the sub-group may be small. The
design of such a trial can be considered as a 2 × 2-design where F11 and F12 are the distributions

16

Paradoxical Results of Rank Tests

of the outcome for the patients who received treatment 1 (e.g., standard treatment with distribution F11 ) or treatment 2 (experimental treatment with distribution F12 ), without the particular
sub-group of interest. The corresponding larger sample sizes, n11 and n12 are approximately the
same when using an appropriate randomization procedure. The smaller sample sizes n21 and n22
in the sub-group may be equal or quite different depending on whether or not the randomization
was also stratified for the sub-group.
The main question regarding the sub-group in this design is whether the treatment effect is
the same as in the population of patients without the sub-group. Here we consider only the case
where the sub-group is known in advance and is not identified on the basis of the data. In order
to demonstrate the advantage of using the unweighted relative effects in (20) estimated by the
pseudo-ranks, we consider the homoscedastic two-way design from Section 4. To this end, let
cAB = (1, −1, −1, 1)0 denote the contrast vector for the interaction in the 2 × 2-design as given
in Section 4 and let p denote the vector of the weighted relative effects estimated by b
p using the
b using
ranks. Moreover, let ψ denote the vector of the unweighted relative effects estimated by ψ
pseudo-ranks, and finally let µ denote the vector of the expectations. Then the corresponding
p
non-centralities are given by cAB
= c0AB p, cψAB = c0AB ψ, and cµAB = c0AB µ. The total sample size N
is increased such that the sample sizes n21 = n22 = 50 in the sub-group remain constant while the
samples sizes n11 = n12 obtained from the population without the sub-group increases from 50
to
For
√ the interaction term of the procedure based on ranks, we also consider the quantity
√ 2000.
p
NcAB
= N c0AB p describing the shift of the asymptotic multivariate normal distribution. It
√ p
is noteworthy that with increasing total sample size N not only NcAB
is increasing but also
p
the basic non-centrality cAB increases while the corresponding non-centralities cψAB and cµAB of
the two other procedures remain constant equal to 0. Note that for equal sample sizes, all
non-centralities are 0. The results are listed in Table 6.
Analyses of this type are very common in biostatistical practice. In particular, the subgroup’s proportion of the total sample size is often very small, as, for example. in case of rare
disease studies with large control groups. The above example further illustrates the need for
changing the usage of classical rank procedures to more favorable pseudo-rank procedures.

7

Discussion and Conclusions

We have demonstrated that inference based on ranks may lead to paradox results in the case
of at least three samples and an unbalanced design. These results may occur in rather natural
situations and are not restricted to artificial configurations of the population distributions.
The reason for this behavior of rank-based tests (including some classical and frequently
used tests) can be explained by the non-centralities of the test statistics which are functions of
weighted nonparametric relative effects. Moreover, it is worth to note that neither pairwise nor
stratified rankings provide a solution to the problem. In fact, it would only cause new problems.
For a discussion of these types of rankings see Brunner et al., (2018). As a remedy, the use
of unweighted nonparametric relative effects, and thus, the use of tests based on pseudo-ranks
instead of ranks, is recommended.

Paradoxical Results of Rank Tests

17

p
Table 6: Non-centralities cAB
, cψAB , and cµAB of the interaction statistics in the 2 × 2-design with increasing sample

sizes in only one stratum. This reflects the situation of a sub-group analysis where the sample size
the sub-group
√ in
p
is also listed
is small compared to the total population. For the procedure based on ranks, the quantity NcAB
to demonstrate the shift of its asymptotic multivariate normal distribution generated only by the unequal sample
sizes in the two strata. The non-centralities cψAB and cµAB of the two statistics based on the pseudo-ranks or on the
expectations are both equal to 0, independently of the sample sizes.

Sample Sizes

Non-Centralities

n11 = n12

n21 = n22

cµAB

50
100
200
300
400
500
1000
2000

50
50
50
50
50
50
50
50

0
0
0
0
0
0
0
0

cψAB

p
cAB

0
0
0
0
0
0
0
0

0
0.071
0.127
0.151
0.165
0.173
0.191
0.201

√

p
NcAB

0
1.22
2.84
4.00
4.94
5.74
8.77
12.89

A completely different approach to circumventing the above mentioned problem could be
given by so-called aligned rank procedures (see Hodges and Lehmann, 1962 or Puri and Sen,
1985). However, these procedures are restricted to semi-parametric models and are not applicable to ordinal data since the hypotheses are formulated in terms of artificially introduced
parameters of metric data. In addition, the typical invariance under strictly monotone transformations of the data is lost by the two different transformations.
On the contrary, the proposed pseudo-rank procedures share the typical invariance of rankbased procedures and do not lead to the paradoxical results described in Sections 3 and 4. They
can additionally be used to construct confidence intervals for meaningful nonparametric effect
sizes as described in Section 5.3. In case of factorial designs with independent observations
pseudo-rank procedures are already implemented in the R-package rankFD by choosing the
option for unweighted effects.

18

8

Paradoxical Results of Rank Tests

References

Akritas, M. G., Arnold, S. F. and Brunner, E. (1997). Nonparametric hypotheses and rank
statistics for unbalanced factorial designs. Journal of the American Statistical Association
92, 258-265.
Akritas, M. G. and Brunner, E. (1997). A unified approach to ranks tests in mixed models.
Journal of Statistical Planning and Inference 61, 249–277.
Bewick, V., Cheek, L., and Ball, J. (2004). Statistics Review 10: Further Nonparametric
Methods. Critical Care 8, 196-199.
Birnbaum, Z. W. and Klose, O. M. (1957). Bounds for the Variance of the Mann-Whitney
Statistic. Annals of Mathematical Statistics 28, 933–945.
Brunner, E., Bathke, A. C., and Konietschke, F. (2018). Rank- and Pseudo-Rank Procedures
in Factorial Designs - Using R and SAS - Independent Observations. Lecture Notes in
Statistics, Springer, Heidelberg.
Brunner, E., Konietschke, F., Pauly, M., and Puri, M.L. (2017). Rank-Based Procedures in
Factorial Designs: Hypotheses about Nonparametric Treatment Effects. Journal of the
Royal Statistical Society Series B, 79, 1463–1485.
Brunner, E. and Munzel, U. (2000). The Nonparametric Behrens-Fisher Problem: Asymptotic Theory and a Small-Sample Approximation. Biometrical Journal 42, 17–25.
Brunner, E. and Munzel, U. (2013). Nichtparametrische Datenanalyse. 2nd edition, Springer,
Heidelberg.
Brunner, E. and Puri, M.L. (2001). Nonparametric Methods in Factorial Designs. Statistical
Papers 42, 1–52.
Brunner, E. and Puri, M. L. (2002). A class of rank-score tests in factorial designs. Journal
of Statistical Planning and Inference 103, 331–360.
Gao, X. and Alvo, M. (2005a). A Nonparametric Test for Interaction in Two-Way Layouts.
The Canadian Journal of Statistics 33, 529–543.
Gao, X. and Alvo, M. (2005b). A Unified Nonparametric Approach for Unbalanced Factorial
Designs. Journal of the American Statistical Association, 100, 926–941.
Govindarajulu, Z. (1968). Distribution-free confidence bounds for Pr{X < Y}. Annals of the
Institute of Statistical Mathematics 20, 229–238.
Hettmansperger, T. P. and Norton, R. M. (1987). Tests for patterned alternatives in k-sample
problems. Journal of the American Statistical Association 82, 292–299.

Paradoxical Results of Rank Tests

19

Jonckheere, A. R. (1954). A Distribution-free k-sample Test Against Ordered Alternatives.
Biometrika 41, 133–145.
Konietschke, F. (2009). Simultane Konfidenzintervalle für nichtparametrische relative Kontrasteffekte. PhD thesis, Inst. of Math. Stochastics, University of Göttingen.
Konietschke, F. Hothorn, L. A., and Brunner, E. (2012). Rank-based multiple test procedures
and simultaneous confidence intervals. Electronic Journal of Statistics 6, 737–758. DOI:
10.1214/12-EJS691
Kruskal, W. H. (1952). A Nonparametric Test for the Several Sample Problem. Annals of
Mathematical Statistics 23, 525–540.
Kruskal, W. H. and Wallis, W. A. (1952). The use of ranks in one-criterion variance analysis.
Journal of the American Statistical Association 47, 583–621.
Kulle, B. (1999). Nichtparametrisches Behrens-Fisher-Problem im Mehrstichprobenfall. Diploma
Thesis, Inst. of Math. Stochastics, University of Göttingen.
Peterson, I. (2002). Tricky Dice Revisited. Science News 161,
https://www.sciencenews.org/article/tricky-dice-revisited
Puri, M. L. (1964). Asymptotic efficiency of a class of c-sample tests. Annals of Mathematical
Statistics 35, 102–121.
Puri, M. L. and Sen, P. K. (1985). Nonparametric methods in general linear models. Wiley,
New York.
Ruymgaart, F. H. (1980). A unified approach to the asymptotic distribution theory of certain
midrank statistics. In: Statistique non Parametrique Asymptotique, 1–18, J.P. Raoult
(Ed.), Lecture Notes on Mathematics, No. 821, Springer, Berlin.
Sen, P. K. (1967). A note on asymptotically distribution-free confidence intervals for Pr(X <
Y) based on two independent samples. Sankhya, Series A 29, 95–102.
Thangavelu, K. and Brunner, E. (2007). Wilcoxon-Mann-Whitney test for stratified samples
and Efron’s paradox dice. Journal of Statistical Planning and Inference 137, 720–737.
Terpstra, T. J. (1952). The asymptotic normality and consistency of Kendall’s test against
trend, when ties are present in one ranking. Indagationes Mathematicae 14, 327–333.
Whitley, E., and Ball, J. (2002). Statistics Review 6: Nonparametric Methods. Critical Care,
6(6), 509-513.
Zaremba, S. K. (1962). A generalization of Wilcoxon’s tests. Monatshefte für Mathematik 66,
359–70.

