Can one get cancellation in exponential sums such as, say,
$$
\sum\_{n\sim N} e(\lfloor n^\theta\rfloor^\beta),
$$
for fixed positive $\theta,\beta\not\in\mathbb Z$? When $\theta < 1$, it seems possible without issue by grouping together values taken on by $\lfloor n^\theta\rfloor$ and then using exponent pairs, but for $\theta > 1$, this obviously doesn't work.



Exponential sum involving floor function
Let me try to provide a partial answer in the case when $0<\beta<2$ much in line with what Terry suggested in the comments. Perhaps it is possible to extend this method to all $\beta>2$ but the computations get more complicated.


Using Taylor expansion and $\lfloor n^\theta\rfloor=n^\theta-\{n^\theta\}$ we obtain
$$
\lfloor n^\theta\rfloor^\beta = n^{\theta\beta}+\mathrm{o}\_{n\to\infty}(1)
$$
when $0<\beta<1$ and
$$
\lfloor n^\theta\rfloor^\beta = n^{\theta\beta}-\beta \{n^\theta\}n^{\theta(\beta-1)}+\mathrm{o}\_{n\to\infty}(1)
$$
when $1<\beta<2$. In the first case, one has $\sum\_{1\leq n\leq N}e(\lfloor n^\theta\rfloor^\beta)\approx\sum\_{1\leq n\leq N}e(n^{\theta\beta})$ and hence cancellation occurs exactly when $\theta\beta$ is not an integer (one can use the Kusmin-Landau inequality/ van der Corput method to obtain good bounds on the cancellation in this case, see for example the book by Graham and Kolesnik: <https://doi.org/10.1017/CBO9780511661976>).


So lets focus on the second case when $1<\beta<2$, which seems more complicated. Using the notation from the comments, let
$$
H(\mathbb{R})=\begin{pmatrix} 1 & \mathbb{R} & \mathbb{R} \\ 0 & 1 & \mathbb{R} \\ 0 & 0 & 1 \end{pmatrix}~~~\text{and}~~~H(\mathbb{Z})=\begin{pmatrix} 1 & \mathbb{Z} & \mathbb{Z} \\ 0 & 1 & \mathbb{Z} \\ 0 & 0 & 1 \end{pmatrix}.
$$
Consider $G=\mathbb{R}\times H(\mathbb{R})$ and $\Gamma=\mathbb{Z}\times H(\mathbb{Z})$ and take their quotient to obtain the nilmanifold $G/\Gamma=\mathbb{R}/\mathbb{Z}\times H(\mathbb{R})/H(\mathbb{Z})$. Consider the function
$$
F\left(u,\begin{pmatrix} 1 & x & z \\ 0 & 1 & y \\ 0 & 0 & 1 \end{pmatrix}\right)= e(u-z+x\lfloor y\rfloor),
$$
which is a Riemann integrable and well defined function on $X$. (This is because $(\{x\},\{y\}, \{z-x\lfloor y\rfloor\})$ is a fundamental domain for the Heisenberg nilmanifold, see here for details: <https://en.wikipedia.org/wiki/Nilmanifold>). Now consider the sequence
$$
g(n)=\left(n^{\theta\beta},\begin{pmatrix} 1 & \beta n^{\theta(\beta-1)} & \beta n^{\theta\beta} \\ 0 & 1 & n^\theta \\ 0 & 0 & 1 \end{pmatrix}\right)
$$
Note that with this choice of $X$, $F$,and $g$ we have
$$
\frac{1}{N}\sum\_{1\leq n\leq N}e(\lfloor n^\theta\rfloor^\beta) \approx \frac{1}{N}\sum\_{1\leq n\leq N}e(n^{\theta\beta}-\beta \{n^\theta\}n^{\theta(\beta-1)})=\frac{1}{N}\sum\_{1\leq n\leq N} F(g(n)).
$$
Due to the main result in <https://arxiv.org/abs/2006.02028> we know that the sequence $g(n)$ is uniformly distributed in the nilmanifold $X$ if and only if $(n^{\theta\beta},\beta n^{\theta(\beta-1)},n^\theta)$ is uniformly distributed in $\mathbb{T}^3=\mathbb{R}^3/\mathbb{Z}^3$. So if neither $\theta\beta$ nor $\theta(\beta-1)$ are integers then the sequence $g(n)$ is uniformly distributed in $X$ and hence
$$
\frac{1}{N}\sum\_{1\leq n\leq N} F(g(n))\rightarrow \int F\,d\mu\_X,
$$
where $\mu\_X$ is the natural (and normalized) Haar measure on $X$. But since $\int F\,d\mu\_X=0$, it follows that
$$
\frac{1}{N}\sum\_{1\leq n\leq N}e(\lfloor n^\theta\rfloor^\beta)\rightarrow 0.
$$


Note that if $\theta(\beta-1)$ is an integer then the sequence $g(n)$ is not uniformly distributed on the entire nilmanifold $X$ but rather distributes uniformly on a sub-nilmanifold $Y\subset X$. But due to the simple nature of $F$, it is not so hard to see that for any such sub-nilmanifold $Y$ one has $\int F\,d\mu\_Y=0$, which in turn still implies cancellation. So the only condition seems to be that $\theta\beta$ is not an integer.


Please let me know if this is helpful or if you have any questions.


