What is radiation pressure and how does it prevent a star from forming? This is a follow up to: Is there a theoretical maximum size limit for a star?

The answer there talks about the radiation pressure preventing a star from forming. 



What reaction is causing this radiation pressure and why does this potentially prevent a star from forming?
Radiation pressure is nothing but electromagnetic interaction.

Imagine a hydrogen atom hit by a stream of photons coming from the same direction. Although the atom as a whole is neutral, the electron and the proton are physically displaced, forming a dipole, i.e. a positive-negative charge couple. Some of the photons therefore scatter against the dipole transferring to it some momentum. So the atom start moving in the same direction as the photons. If the photons are in the ultraviolet, the electron can be exited to higher orbitals and possibly stripped from the atom. In this case the scattering is even more efficient.

Now imagine a star surrounded by a layer of hydrogen. Gravity attracts the layer towards the star. The photons emitted by the star try to push the hydrogen atoms away from it, through the electromagnetic force. 

Very massive stars are very luminous and hot, which means that they emit a lot of ultraviolet photons. When the pressure transferred from the photons to the layer is larger than the gravitational attraction, then the layer begins expanding, effectively stopping growth of the star.

In the figure posted by the OP there is also dust. I don't know the details of photons-dust-gas interactions (we need a stellar atmosphere expert, I guess), but the basic principle is nonetheless the same.