Is there an ordinal $\alpha$ such that $ZF$ believes that $V\_{\alpha}$ is a model of $ZF$? (If it is problematic to state this since we have to check infinitely many axioms at once, formalize logic in $ZF$. ) If $\alpha > \omega$ is a limit ordinal, then $V\_{\alpha}$ is a model of $ZF - R$, where $R$ stands for the axiom of replacement. The reflection principle tells us that $ZF$ knows that the set $S$ of ordinals $\alpha$ such that $R$ holds relative to $V\_{\alpha}$ is unbounded. So is there some limit ordinal in $S$, thus answering the question? What is the smallest $\alpha$ such that $V\_{\alpha}$ is a model of $ZF$, if it exists? I already know that if $\kappa$ is a strongly inaccessible cardinal, then $V\_{\kappa}$ is a model of ZF, but the existence of such $\kappa$ is independent from $ZF$.



Replacement in von Neumann hierarchy of sets
You cannot prove that there is such an ordinal, but (under a suitable large cardinal assumption) it is consistent that there is such an ordinal. 


If you could prove that there was such an ordinal, then you will have proved Con(ZF) in ZF, contrary to the incompleteness theorem. 


Another way to see it is: if there were such an ordinal, let $\alpha$ be the least ordinal with $V\_\alpha\models$ZF. Thus, $V\_\alpha$ is a model of ZF having no $\beta$ with $V\_\beta\models$ZF, since the $V\_\beta$ of $V\_\alpha$ is the same as the $V\_\beta$ of $V$.


However, if $\kappa$ is an inaccessible cardinal, then $V\_\kappa\models$ZFC. In fact, there are many smaller $\alpha\lt\kappa$ with $V\_\alpha\models$ZFC, and so the consistency strength of having an $\alpha$ with $V\_\alpha\models$ZFC is strictly lower than an inaccessible, if it is consistent.


Your remark about using the Reflection Theorem to get $\alpha$ with $V\_\alpha$ with Replacement is not quite right. The Replacement Axiom is a *scheme* of axioms, an infinite list of axioms, and the Reflection idea will only produce $\alpha$ with $V\_\alpha$ satisfying any one (or finitely many) of them. But we cannot get a model of the whole scheme this way. 


Lastly, it is interesting to note that every nonstandard model $M$ of ZF, having a nonstandard $\omega$, will have a $V\_\alpha$ that is a model of ZF as viewed from outside $M$. The reason is that for any finite collection of the ZF axioms, we may apply the Reflection Theorem as you indicated to get a $V\_\alpha^M$ satisfying them, but then since the $\omega$ of $M$ is nonstandard and $M$ cannot identify its standard cut, it follows by overspill that there must be some nonstandard finite set of ZF axioms in $M$ that $M$ thinks is satisfied in one of its $V\_\alpha^M$. But since this includes all the standard axioms, we have thus obtained a $V\_\alpha^M$ satisfying the true ZF as viewed externally.


