Let $X$ be a complex manifold and $g$ a hermitian metric on $X$. Consider the Riemannian exponential $\exp\_p: T\_p X \to X$.  

If $\exp\_p$ is holomorphic for every $p \in X$, then $(\exp\_p)^{-1}$, suitably restricted, provide holomorphic normal coordinates near $p$, with respect to which the metric osculates to order 2 to the standard metric at the origin. This shows that $g$ is a Kähler metric.  



However, Kähler is not sufficient to ensure that $\exp\_p$ is holomorphic: take $X$ a curve of genus $g \geq 2$. If $\exp\_p:T\_pX \to X$ is holomorphic, then it lifts to a holomorphic map from $T\_pX$ to the universal cover $\widetilde{X} = \Delta$, giving a holomorphic map $T\_pX \simeq \mathbb{C} \to \Delta$, which must be constant by Liouville's theorem. In fact, one can see that $\exp$ cannot be holomorphic if $X$ is Kobayashi hyperbolic.  



This leaves the question: What are the hermitian manifolds/metrics whose exponential map is holomorphic?



Complex manifolds in which the exponential map is holomorphic
NB: I've had a little time to think about this and can now improve my answer, in particular, removing the real-analytic assumption, which, as I suspected, was not necessary. Here is the improved answer:


If the metric $g$ is Kähler, then having the exponential map from a point $p\in M$ be holomorphic makes it flat in a neighborhood of $p$.


Suppose that $\exp\_p:T\_pM\to M$ is holomorphic near $0\_p\in T\_pM$ (where we use the natural holomorphic structure on the complex vector space $T\_pM$). Let $z:T\_pM\to\mathbb{C}^n$ be a complex linear isometry, so that the hermitian metric on $T\_pM$ is just $|z|^2$ in the usual sense. Let $Z$ be the holomorphic 'radial' vector field on $\mathbb{C}^n$, whose real part is the standard radial vector field on $\mathbb{C}^n$. 


Then
$$
{\exp\_p}^\*g = g\_{i\bar j}(z)\ dz^i\ d\overline{z}^j
$$
for some functions $g\_{i\bar j}$ on a neighborhood of $0\in\mathbb{C}^n$. Since $g$ is Kähler, there is a function $f$ defined on a neighborhood of $0\in\mathbb{C}^n$ such that
$$
g\_{i\bar j} = \frac{\partial^2f}{\partial z^i\ \partial\overline{z}^j}.
$$


Now, the condition that $z$ furnish Gauss normal coordinates for ${\exp\_p}^\*g$ is easily seen to be that  

$$
\mathcal{L}\_Z\bigl(\bar\partial f\bigr) = \bar\partial\bigl(|z|^2\bigr).
$$
In particular, $ \bar\partial\bigl(\mathcal{L}\_Z(f - |z|^2)\bigr) = 0$, 
so $\mathcal{L}\_Z(f - |z|^2) = h$ for some holomorphic function $h$ on a neighborhood of $0$. This $h$ must vanish at $0$, so it is easy, by adding the real part of the appropriate holomorphic function to $f$ (which won't change $g$) to arrange that $h\equiv0$ and, moreover, that $f(0) = 0$. But this now implies that the real-valued function $f-|z|^2$ vanishes at the origin and also is constant along the radial vector field. Thus, $f = |z|^2$, and the metric $g$ is flat in these coordinates.


