We know that the Riemann tensor is antisymmetric with respect to the first two vectors (the vectors that we parallel transport the third vector around the parallelogram made by their integral curves).


But what is the geometric interpretation?
Why do we have to expect that by changing the order of parallel transportation we will have the same vector with opposite orientation?



Interpretation of Riemann tensor antisymmetry
This is a re-flavouring of Alexander's answer but in a language I prefer. 


Take two vectors $v,w \in T\_p N$, and consider the `rectangle' $exp(xv+yw)$ where $0 \leq x \leq a$ and $0 \leq y \leq b$. The holonomy around the boundary of this rectangle is an orthogonal transformation of $T\_p N$, and it looks approximately like


$$ Hol \simeq Id\_{T\_p N} + ab R(v,w) $$


where the approximation indicates the 2nd order taylor expansion of the holonomy with respect to the variables $a$ and $b$. Here $R(v,w)$ is the Riemann curvature tensor. 


So from this point of view, the reason why it's anti-symmetric in the variables $v,w$ is that if you switch $v$ and $w$ you are essentially reversing the orientation of the rectangle you're computing the holonomy over, so the corresponding linear transformation (holonomy) is the inverse, which corresponds to negating the linear and quadratic parts of the Taylor expansion. 


This is also the reason why the Riemann curvature tensor is skew-symmetric:


$$\langle R(v,w)z, y \rangle + \langle z, R(v,w)y \rangle = 0 $$


since the tangent space to the orthogonal group is consists of all the skew-symmetric linear transformations. 


