Is there a known characterization of such spaces? 


An example: the space of $n \times n$ matrices spanned by $I$ and $J$ (the identity and all-ones matrices, respectively) is inverse closed by the Sherman-Morrison formula. 


A possible question of interest would be the maximum dimension of a non-trivial inverse-closed space.


I am reminded here of the work on spaces of matrices of bounded rank about which not a little is known but maybe it's a very false analogy.


EDIT: After reading Denis Serre's neat answer I started thinking what happens in the singular case. The Moore-Penrose inverse of $A$ in general is **not** polynomial in $A$. It does not even always commute with $A$. But in a paper by Edward Wong, *Does the Generalized Inverse of A Commute with A?* (Mathematics Magazine, Vol. 59, No. 4 (Oct., 1986), pp. 230-232) it was shown that $A^{\dagger}$ is a polynomial in $A$ if and only if $A$ and $adj(A)$ have the same row-reduced echelon form, so that's some answer to the generalized question.



inverse-closed matrix spaces
Such matrix subspaces have been characterized: They are Jordan algebras.
A bit more generally, linear structure is preserved in inversion if a matrix subspaces is equivalent to a Jordan algebra. 


(Jordan algebra is closed under the product MN+NM.) Equivalence 
means that you are allowed to multiply with an invertible matrix from the left
and with an invertible matrix from the right.


Reference: (Marko Huhtanen, Differential geometry of matrix inversion. Math. Scand., 107, pp. 267-284, 2010.)


