Why is constant velocity drift not an accepted explanation of Hubble expansion? Edward A Milne at the Oxford University proposed that what Hubble was
  observing was simply a natural sorting of random galactic motions. If
  a group of galaxies formed together moving at various speeds, it was
  natural that the fastest ones would now be the most distant, while the
  slow-moving ones would remain nearby.


Source: https://aeon.co/essays/how-they-pinned-a-single-momentous-number-on-the-universe

Why is this incorrect?

I understand that gravity wouldn't normally allow expansion at constant velocity, but we have already conjectured dark energy to account for that.
The simplest answer is that Milne's explanation requires a non-uniform cosmos, with a formerly crowded "starting area" that expands into empty space.  Observations indicate that the cosmos is nearly uniform at very large scales.  With Milne's model of a non-expanding space and uniform cosmos, galaxies that started near to us with high velocities that ended up far away would be replaced by galaxies that started far away and ended up near, so there would be no observable relationship between distance and velocity.

There is a slew of other observational evidence against the naivete' of Milne's model, of course.  The cosmic microwave background shows that space use to be dense and hot; galaxies evolve, interact, and combine over time; there are intergalactic clouds that influence galaxies' trajectories.  Only when you can account for all that does incorporating elements like dark energy into your model make sense.