Given $0<\alpha, \beta<1$, $a,b>0$, $a^2+b^2<1$.


I am trying to determine the asymptotic behaviour of  

$$F(a,b):=\int\_{\substack{a/2<x<2a\\\\b/\sqrt{2}<\sqrt{1-x^2}<\sqrt{2}b}}\frac{dx}{|x-a|^{\alpha}|\sqrt{1-x^2}-b|^{\beta}}$$
as $a^2+b^2\to 1$ and $a\rightarrow 0$ simultaneously.


Notation: Given $A,B>0$, as customary we use $ A\approx B$ to denote the fact that there exist two constants $c\_1, c\_2$ such that $c\_1 A\leq B \leq c\_2 A$.


We can estimate
$$|\sqrt{1-x^2}-b|=\frac{|x^2+b^2-1|}{\sqrt{1-x^2}+b}\approx b^{-1}|x^2+b^2-1|\approx |x^2+b^2-1|\qquad (1)$$
since $b\approx 1$ when $a$ is small enough and $a^2+b^2$ is close enogh to $1$.


Now, use the estimate $(1)$ then translate $x-a\to y$ to get


$$F(a,b)\approx \int\_{\substack{-a/2<y<a\\\\ -b^2 <y^2+2a-(1-(a^2+b^2))<b^2/2}}\frac{dy}{|y|^{\alpha}|y^2+2a-(1-(a^2+b^2))|^{\beta}}$$


Let us look then at
$$G(a,b):= \int\_{\substack{0<y<a\\\\ 0<y^2+2a-(1-(a^2+b^2))<b^2/2}}\frac{dy}{|y|^{\alpha}|y^2+2a-(1-(a^2+b^2))|^{\beta}}.$$
When $0<y<a$ we have $y^2+2a\approx a$. Indeed, when $0<y<a<1$ we have
$2a\leq y^2+2a \leq a^2+2a<3a$.


It is easy now to estimate $G$ both when $a>>1-(a^2+b^2)$ and when $a<<1-(a^2+b^2)$.


What I need help with is finding the asymptotic behaviour of $F$ (equivalently $G$)
when $a\approx 1-(a^2+b^2)$ which is the most interesting part of course.



The asymptotic behaviour of a singular integral
$\newcommand{\al}{\alpha}\newcommand{\be}{\beta} $As discussed in comments, given
\begin{equation\*}
a/2<x<2a \tag{00}\label{00}
\end{equation\*}
and $a\to0$ (so that $b\to1$), the restriction $b/\sqrt{2}<\sqrt{1-x^2}<\sqrt{2}b$ will automatically hold (eventually). So,
\begin{equation\*}
F(a,b)=\int\_{a/2}^{2a}\frac{dx}{|x-a|^\al|\sqrt{1-x^2}-b|^\be}. \tag{10}\label{10}
\end{equation\*}
Refining your formula (1) slightly, we get
\begin{equation\*}
 |\sqrt{1-x^2}-b|=\tfrac{1+o(1)}2\,|x^2-c^2|, \tag{20}\label{20}
\end{equation\*}
where
\begin{equation\*}
 c:=\sqrt{1-b^2}, \tag{30}\label{30}
\end{equation\*}
so that $0<a<c\to0$.


Suppose now that the limit
\begin{equation\*}
 r:=\lim\frac ca \tag{35}\label{35}
\end{equation\*}
exists. Then $r\in[1,\infty]$.


If $r=\infty$, then, by \eqref{20} and \eqref{00}, $|\sqrt{1-x^2}-b|\sim\tfrac12\,c^2$ and hence, by \eqref{10},  

\begin{equation\*}
F(a,b)\sim(2/c^2)^\be\int\_{a/2}^{2a}\frac{dx}{|x-a|^\al}
=(2/c^2)^\be\frac{2^{1-\al}+2^{\al-1}}{1-\al}\,a^{1-\al}. \tag{40}\label{40}
\end{equation\*}


If $r\in(1,\infty)$ (or $r=1$ and $\al+\be<1$), then, by \eqref{10} and \eqref{20},
\begin{equation\*}
F(a,b)\sim2^\be\int\_{a/2}^{2a}\frac{dx}{|x-a|^\al |x^2-r^2 a^2|^\be}
=2^\be a^{1-\al-2\be} C\_{\al,\be;r}, \tag{50}\label{50}
\end{equation\*}
where
\begin{equation}
 C\_{\al,\be;r}:=\int\_{1/2}^{2}\frac{du}{|u-1|^\al |u^2-r^2|^\be}. 
\end{equation}
The constant $C\_{\al,\be;r}$ admits a very complicated expression in terms of the gamma function and certain hypergeometric functions; this expression is hardly of any interest.


It is also clear from these considerations that $F(a,b)$ does not have any definite asymptotic if the limit $r$ in \eqref{35} does not exist. The asymptotic of $F(a,b)$ in the case when $r=1$ and $\al+\be\ge1$ (whenever such an asymptotic exists) will depend on the way $\frac ca$ goes to $1$.


