Hi,


I have a question related to Serre Duality. 


First, I would like to recall some preliminar definitions.


**Definition 1.** A variety $X$ is $\mathbb{Q}$-Gorenstein if it is Cohen-Macaulay and $\omega\_X^{\otimes m}$ is a line bundle for some $m>0$, where $\omega\_X$ is the dualizing sheaf of $X.$ In particular, we shall say that $X$ is Gorenstein if $m=1.$


**Definition 2.** A coherent sheaf $\mathcal{E}$ on $X$ is pure if any non-trivial subsheaf of $\mathcal{E}$ has the same dimension of $\mathcal{E}.$


**Question.** Let $X$ be a $n$-dimensional proper $\mathbb{Q}$-Gorenstein variety, $\mathcal{E}$ and $\mathcal{F}$ pure coherent sheaves on $X.$ Is it true that


$$\mathrm{Ext}^i(\mathcal{E},\mathcal{F})\cong \mathrm{Ext}^{n-i}(\mathcal{F},\mathcal{E}\otimes \omega\_X)^\* ?$$


**Remark.** As it is explained in Appendix C.1 of the book "Bartocci, Bruzzo, Ruipérez - Fourier-Mukai and Nahm Transforms in Geometry and Mathematical Physics", my question has positive answer under the following assumptions


* $X$ smooth projective variety,


or


* $X$ is Gorenstein and $\mathcal{E}$ is of finite Tor-dimension.


Thank you


P.S.: I would like to thank Karl Schwede for suggesting me to re-edit the question.



Ext groups and Serre duality for proper $\mathbb{Q}$-Gorenstein varieties.
**EDIT:** After some clarification from the questioner with respect to the $Q$-Gorenstein definition, let me point out that additional hypotheses are needed I think.


I don't think there's any hope that this can be true without additional hypotheses. I'll start by claiming that I can pick $\mathcal{E, F}$ such that $\text{Ext}^i(\mathcal{E,F}) \neq 0$ for some $i > n = \dim X$, even if $X$ is $Q$-Gorenstein. 


For example, since $X$ is not Gorenstein, $O\_X$ does not have finite injective dimension, and fix a pure $\mathcal{E'}$ such that $\mathcal{E}xt^i(\mathcal{E'}, O\_X) \neq 0$ for some $i > n$ (we can find such modules that are pure for $i - 1 > n$ , write any such module with that nonvanishing in a short exact sequence as a quotient by a free module, the kernel should do the job and will be pure). By setting $\mathcal{E} = \mathcal{E}' \otimes L^{-m}$ for some ample $L$ and $m \gg 0$ and setting $\mathcal{F} = O\_X$, we also obtain $\text{Ext}^i(\mathcal{E,F}) \neq 0$ for some $i > n$ by Serre vanishing.


But $\text{Ext}^{n-i}(\mathcal{F}, \mathcal{E}\otimes \omega\_X) = 0$ for $i > n$ for obvious reasons. 


**ORIGINAL Answer:**


First let me clarify something, when you say that $\mathcal{E}$ and $\mathcal{F}$ are *pure*, you mean sheaves of pure dimension? If so, then you are saying that every subsheaf of $\mathcal{E, F}$ is of the same dimension. In particular, line bundles on a variety $X$ are always pure? 


Now, when you say $\omega\_X$ is a *dualizing sheaf*, this actually doesn't really make sense unless $X$ is Cohen-Macaulay, because otherwise there is no dualizing sheaf, but there is a *dualizing complex*. If $X$ is normal, by $\omega\_X$ you probably mean the *canonical sheaf* $i\_\* \Omega\_{X^{\text{reg}}/k}$ where $i : X^{\text{reg}} \to X$ is the inclusion of the regular locus. 


**Definition:** Suppose that $K\_X$ is a divisor such that $O\_X(K\_X) \cong \omega\_X$ then we say that $X$ is $Q$-Gorenstein if $nK\_X$ is a Cartier divisor for some $n > 0$.


In this case, we now run into a problem with this *very* unfortunately terminology.


**Fact:** When people say the word $Q$-Gorenstein, they do *NOT* typically require that $X$ is Cohen-Macaulay. This is in start constrast to the term *Gorenstein* which includes in its definition the fact that $X$ is Cohen-Macaulay (Gorenstein means Cohen-Macaulay + $K\_X$ being Cartier).


There are some exceptions to this, but they are rare in algebrabraic geometry.


Now we come to your statement. 


**Answer**: *Your statement holds for all* $\mathcal{E,F}$ line bundles *on a projective variety $X$ if and only if $X$ is Cohen-Macaulay.* 


You are probably already familiar with the section in Hartshorne on Serre duality.
I will try to briefly explain why they aren't equal. Suppose that $\mathcal{E}$ and $\mathcal{F}$ are line bundles and that $\mathcal{F} = O\_X$ and $\mathcal{F}$ is a high power of an ample line bundle. 


Then
$$Ext^i(\mathcal{E, F}) = H^i(X, \mathcal{E}^{\vee}), \text{ and } Ext^{n-i}(\mathcal{F}, \mathcal{E} \otimes \omega\_X) = H^{n-i}(X, \omega\_X \otimes \mathcal{E})^\\*$$
In fact, if these two groups are equal in general, then it follows that $X$ is Cohen-Macaulay. 


First note that $H^{n-i}(X, \omega\_X \otimes \mathcal{E})^\\* = 0$ for all $i < n$ by Serre vanishing.


Now we work on the other side. Being Cohen-Macaulay is equivalent to requiring the dualizing complex $\omega\_X^{\bullet}$ to be a sheaf. Otherwise, the dualizing complex is a *complex* unsurprisingly. In general, $X$ is Cohen-Macaulay, if and only if $\omega\_X^{\bullet}$ only has cohomology in $-\dim X = -n$ (and has zero cohomology elsewhere). 


Now, or a line bundle $L$ that is a high power of an ample line bundle
$$H^i(X, L^{\vee}) = \mathbb{H}^{n-i}(X, L \otimes \omega\_X^{\bullet}[-n])$$ (this is a very special case of Grothendieck duality).
However, the right side is simply $H^0(X, L\otimes {\bf h}^{-i}(\omega\_X^{\bullet}))$ by analysis of the spectral sequence (here ${\bf h}^j$ simply means take the $j$th cohomology of the complex of sheaves). On the other hand, since $L$ is a high power of an ample line bundle, $H^0(X, L^{\vee} \otimes {\bf h}^{-i}(\omega\_X^{\bullet}) \neq 0$ as long as ${\bf h}^{-i}(\omega\_X^{\bullet}) \neq 0$. Thus if $\omega\_X^{\bullet}$ has cohomology not in $-\dim X$, then there will be some non-vanishing cohomology.


Let me know if this makes sense.


**On the plus side:** If you are willing to work with Grothendieck duality and not Serre duality, then there are probably statements like the one you want. 


Indeed, the point of this is that somehow the tensor functor you are writing there is not the correct functor (you may need the derived version for example). 


