A couple of years ago, I came up with the following question, to which I have no answer to this day. I have asked a few people about this, most of my teachers and some friends, but no one had ever heard of the question before, and no one knew the answer.


I hope this is an original question, but seeing how natural it is, I doubt this is the first time someone has asked it.


First, some motivation. Take $P$ any nonzero complex polynomial. It is an easy and classical exercise to show that the roots of its derivative $P'$ lie in the convex hull of its own roots (I know this as the Gauss-Lucas property). To show this, you simply write $P = a \cdot \prod\_{i=1}^{r}(X-\alpha\_i)^{m\_i}$ where the $\alpha\_i~(i=1,\dots,r)$ are the different roots of $P$, and $m\_i$ the corresponding multiplicities, and evaluate $\frac{P'}{P}=\sum\_i \frac{m\_i}{X-\alpha\_i}$ on a root $\beta$ of $P'$ which is not also a root of $P$. You'll end up with an expression of $\beta$ as a convex combination of $\alpha\_1,\dots,\alpha\_r$. It is worth mentioning that all the convex coefficients are $>0$, so the new root cannot lie on the edge of the convex hull of $P$'s roots.


Now fix $P$ a certain nonzero complex polynomial, and consider $\Pi$, its primitive (antiderivative) that vanishes at $0:~\Pi(0)=0$ and $\Pi'=P$. For each complex $\omega$, write $\Pi\_{\omega}=\Pi-\omega$, so that you get all the primitives of $P$. Also, define for any polynomial $Q$, $\mathrm{Conv}(Q)$, the convex hull of $Q$'s roots.


**MAIN QUESTION:** describe
$\mathrm{Hull}(P)=\bigcap\_{\omega\in\mathbb{C}}\mathrm{Conv}(\Pi\_{\omega})$.


By the property cited above, $\mathrm{Hull}(P)$ is a convex compact subset of the complex plane that contains $\mathrm{Conv}(P)$, but I strongly suspect that it is in general larger.


Here are some easy observations:


1. replacing $P$ (resp. $\Pi$) by $\lambda P$ (resp. $\lambda \Pi$) will not change the result, and considering $P(aX+b)$ will change $\mathrm{Hull}(P)$ accordingly. Hence we can suppose both $P$ and $\Pi$ to be monic. The fact that $\Pi$ is no longer a primitive of $P$ is of no consequence.
2. the intersection defining $\mathrm{Hull}(P)$ can be taken for $\omega$ ranging in a compact subset of $\mathbb{C}$: as $|\omega| \rightarrow \infty$, the roots of $\Pi\_{\omega}$ will tend to become close to the $(\deg (P)+1)$-th roots of $\omega$, so for large enough $\omega$, their convex hull will always contain, say, $\mathrm{Conv}(\Pi)$.
3. $\mathrm{Hull}(P)$ can be explicitly calculated in the following cases:
$P=X^n$, $P$ of degree $1$ or $2$. There are only 2 kinds of degree $2$ polynomials: two simple roots or a double root. Using $z\rightarrow az+b$, one only has to consider $P=X^2$ and $P=X(X-1)$. The first one yields {$0$}, which equals $\mathrm{Conv}(X^2)$, the second one gives $[0,1]=\mathrm{Conv}(X(X-1))$.


Also, if $\Pi$ is a real polynomial of odd degree $n+1$ that has all its roots real and simple, say $\lambda\_1 < \mu\_1 < \lambda\_2 < \dots < \mu\_n < \lambda\_{n+1}$, where I have also placed $P$'s roots $\mu\_1, \dots, \mu\_n$, and if you further assume that $\Pi(\mu\_{2j}) \leq \Pi(\mu\_n) \leq\Pi(\mu\_1) \leq\Pi(\mu\_{2j+1})$ for all suitable $j$ (a condition that is best understood with a picture), then $\mathrm{Hull}(P)=\mathrm{Conv}(P)=[\mu\_1,\mu\_n]$: just vary $\omega$ between $[\Pi(\mu\_n), \Pi(\mu\_1)]$; the resulting polynomial $\Pi\_{\omega}$ is always split over the real numbers and you get 


$$[\mu\_1,\mu\_n]=\mathrm{Conv}(P)\subset\mathrm{Hull}(P)\subset
\mathrm{Conv}(\Pi\_{\Pi(\mu\_1)})\cap
\mathrm{Conv}(\Pi\_{\Pi(\mu\_n)}) = \\= [\mu\_1,\dots]\cap
[\dots,\mu\_n]=[\mu\_1,\mu\_n]$$


4. The equation $\Pi\_{\omega}(z)=\Pi(z)-\omega=0$ defines a Riemann surface, but I
don't see how that could be of any use.


Computing $\mathrm{Hull}(P)$ for the next most simple
polynomial $P=X^3-1$ has proven a challenge, and I can only conjecture what
it might be.


Computing $\mathrm{Hull}(X^3-1)$ requires factorizing degree 4
polynomials, so one naturally tries to look for good values of $\omega$,
the $\omega$ that allow for easy factorization of $\Pi\_{\omega}=X^4-4X-\omega$---for instance, the $\omega$ that produces a double root. All that remains to be done
afterwards is to factor a quadratic polynomial. The problem is symmetric,
and you can focus on the case where 1 is the double root (i.e., $\omega=-3$).
Plugging in the result in the intersection, and rotating twice, you obtain the following superset of
$\mathrm{Hull}(X^3-1)$: a hexagon that is the intersection of three similar isoceles
triangles with their main vertex located on the three third roots of unity $1,j,j^2$


**QUESTION:** is this hexagon equal to $\mathrm{Hull}(X^3-1)$?


Here's why I think this might be.


Consider the question of how the convex hulls of the roots of $\Pi\_{\omega}$ vary as $\omega$ varies.
When $\omega\_0$ is such that all roots of $\Pi\_{\omega\_0}$ are simple, then the inverse function theorem shows that the roots of $\Pi\_{\omega}$
with $\omega$ in a small neighborhood of $\omega\_0$ vary holomorphically $\sim$
linearly in $\omega-\omega\_0$: $z(\omega)-z(\omega\_0)\sim \omega-\omega\_0$. If however $\omega\_0$ is such that $\Pi\_{\omega\_0}$
has a multiple root $z\_0$ of multiplicity $m>1$, then a small variation of $\omega$
about $\omega\_0$ will split the multiple root $z\_0$ into
$m$ distinct roots of $\Pi\_{\omega}$ that will spread out roughly as
$z\_0+c(\omega-\omega\_0)^{\frac{1}{m}}$, where $c$ is some nonzero coefficient. This
means that for small variations, these roots will move at much higher velocities
than the simple roots, and they will constitute the major contribution to the variation of
$\mathrm{Conv}(\Pi\_{\omega})$; also, they spread out evenly, and (at least if the
multiplicity is greater or equal to $3$) they will tend to increase the convex hull
around $z\_0$. Thus it seems not too unreasonable to conjecture that the convex hull
$\mathrm{Conv}(\Pi\_{\omega})$ has what one can only describe as
critical points at the $\omega\_0$ that produce roots with
multiplicities. I'm fairly certain there is a sort of calculus on convex sets that
would allow one to make this statement precise, but I don't know see what it could be.


Back to $X^3-1$: explicit calculations suggest that up to second order, the double root $1$ of $X^4-4X+3-h$ for $|h|<<1$ splits in half nicely (here $\omega=-3+h$), and the convex hull will continue to contain the aforementioned hexagon. 


**QUESTION (Conjecture):** is it true that
$\mathrm{Hull}(P)=\bigcap\_{\omega\in\mathrm{MR}}\mathrm{Conv}(\Pi\_{\omega})$, where
$\mathrm{MR}$ is the set of all $\omega\_0$ such that
$\Pi\_{\omega\_0}$ has a multiple root, i.e., the set of all $\Pi(\alpha\_i)$ where the
$\alpha\_i$ are the roots of $P$?


All previous examples of calculations agree with this, and I have tried as best I can to justify this guess heuristically.


Are you aware of a solution? Is this a classical problem? Is anybody brave enough to
make a computer program that would compute some intersections of convex hulls
obtained from the roots to see if my conjecture is valid?



Polynomial roots and convexity
First, a counterexample to your conjecture. Let $\Pi = x^4+x^3+4x^2+4x = x(x+1)(x^2+4)$, so $P = 4x^3+3x^2+8x+4$. The critical values of $\omega$ are $1.06638, 3.89455 + 2.87687i, 3.89455 - 2.87687i$, and by inspection (using Mathematica) we see that for each of these values of $\omega$, $\mbox{Conv}(\Pi\_\omega)$ contains a neighborhood of $0$.


Now for a calculus on convex sets. Every convex set is the intersection of a set of halfplanes. Call a halfplane in this collection essential if removing all of the halfplanes in an open set of halfplanes (in the halfplane topology) containing it from our set of halfplanes makes the intersection of the halfplanes in our set bigger. We wish to find a characterization of the essential halfplanes of $\mbox{Hull}(P)$.


First of all, I claim that any essential halfplane of $\mbox{Hull}(P)$ occurs as an essential halfplane of $\mbox{Conv}(\Pi\_\omega)$ for some $\omega$. This follows from continuity - for any open set around our essential halfplane there is some $\omega$, take the limit of a subsequence of these $\omega$s...


Now, suppose that the halfplane $\mbox{Re}(x) \le 0$ occurs as an essential halfplane of some $\mbox{Conv}(\Pi\_\omega)$, i.e. there are at least two roots of $\Pi\_\omega$ with real part $0$, and the rest of the roots have negative real part. If the number of roots on the line $\mbox{Re}(x) = 0$ (counted with multiplicity) is two, then by holomorphicity we can always find a direction to move $\omega$ so that either both roots move to the left, or both roots stay on the line $\mbox{Re}(x) = 0$ and move towards eachother. If we can ever make both roots move to the left, then clearly the halfplane $\mbox{Re}(x) \le 0$ is not an essential halfplane of $\mbox{Hull}(P)$, otherwise we keep pushing the roots towards eachother until either they run into eachother or until a third root hits the line $\mbox{Re}(x) = 0$. In any case, we see that if a halfplane is essential for $\mbox{Hull}(P)$, then there is some $\omega$ such that the halfplane is essential for $\mbox{Conv}(\Pi\_\omega)$ and such that at least three roots (counted with multiplicity) of $\Pi\_\omega$ are on the boundary of the halfplane, or two of the roots are equal and $\Pi\_\omega$ has no other roots.


So if we let $L$ be the set of $\omega$s such that three of the roots of $\Pi\_\omega$ lie on a line, we get that $\mbox{Hull}(P) = \cap\_{\omega \in L} \mbox{Conv}(\Pi\_\omega)$ if $\deg P \ge 2$.


Edit: Actually, I think there is a problem with this. It's conceivable that two roots are on the line $\mbox{Re}(x) = 0$ and have derivatives (with respect to $\omega$) pointed in opposite directions, such that we can't simply push them towards eachother. For instance, the map from one root to the other root could, locally, look like the fractional linear transform sending the left halfplane to a circle contained in the right halfplane and tangent to the line $\mbox{Re}(x) = 0$ at the other root. So, we may need to enlarge the set $L$ to contain also those $\omega$s for which the ratio of the derivatives of two of the roots (with respect to $\omega$) is a negative real number.


Edit 2: It turns out that this isn't an issue. Call the two roots on the line $\mbox{Re}(x) = 0$ $r\_1$ and $r\_2$. Suppose that locally, $r\_1(\epsilon) = \epsilon$, $r\_2(\epsilon) = i - m\epsilon + a\epsilon^k + O(\epsilon^{k+1})$, $a \ne 0$, $m > 0$. Note that if we had $r\_2(\epsilon) = i-m\epsilon$, then the intersection of the halfplanes corresponding to $r\_1(\epsilon), r\_2(\epsilon)$ and $r\_1(-\epsilon), r\_2(-\epsilon)$ would be contained in the halfplane $\mbox{Re}(x) \le 0$, and the intersection of their boundaries would be located at $i/(m+1)$. Now if $k$ is even, then the correction term shifts the intersection of the boundaries by $a\epsilon^k/(m+1) + O(\epsilon^{k+1})$, so if we choose $\epsilon$ small such that $a\epsilon^k$ is real and negative, then we see that the halfplane $\mbox{Re}(x) \le 0$ is not essential. If $k$ is odd, then if we choose $\epsilon$ small such that $\mbox{Re}(\epsilon) < 0$ and $a\epsilon^k$ is a positive real times $i$, then $r\_2(\epsilon)$ is shifted up and $r\_2(-\epsilon)$ is shifted down, so the intersection of the boundaries will be shifted to the left (draw a picture), so again the halfplane $\mbox{Re}(x) \le 0$ is not essential.


