This is probably going to lead nowhere, but maybe it be possible to use the matrix logarithm to invert matrices?
For positive definite matrices, we have that the logarithm exists and
$$
\log(A^{-1})= - \log(A) 
$$
So very brutally applying the first taylor approximation everywhere (assuming this works for matrices)
$$
A^{-1} = \exp(-\log(A)) \approx \exp(- (A- \mathbb{I}) \approx \mathbb{I} + (\mathbb{I} - A) = 2\mathbb{I} -A
$$
This is likely to only work well for matrices where the eigenvalues are all close to $1$, which doesn't really fit my use case so I don't really want to pursue this further. But maybe this is useful somewhere?


Has this concept come up before? Are there more intelligent approximations than the first taylor approximation?



Inverting a matrix using the Matrix logarithm
Numerically, inverting a matrix by computing matrix exponentials and logarithms doesn't really work well, because (1) typically methods to compute matrix exponentials and logarithms are much more expensive than methods to compute the inverse, and (2) there are branch points in the logarithm which may create stability pitfalls.


However, the first-order approximation you note is used in practice. Typically, you see it in the equivalent form
$$
(I+E)^{-1} = I - E + O(\|E\|^2).
$$
Note indeed that $I-E = 2I - (I+E)$. This is a truncated Neumann series; see for instance on [Wikipedia](https://en.wikipedia.org/wiki/Invertible_matrix#By_Neumann_series).


