What is the YORP effect exactly? Is it just the non-central component of the Yarkovsky effect? This answer to Where have all the Vulcanoids gone? links to the aptly-titled The YORP Effect Can Efficiently Destroy 100 Kilometer Planetesimals At The Inner Edge Of The Solar System which says in part:

... The YORP effect destroys Vulcanoids by spinning them up so fast that the gravitational accelerations holding components of the body together are matched by centrifugal accelerations, this causes the body to rotationally fission. i.e break apart. We calculated the timescale of this fission process for a parent Vulcanoid and for each of their subsequent generational fragments. We show that objects with radii up to 100 kilometers in size are efficiently destroyed by the YORP effect doing so in a timescale that is much younger than the age of the Solar System...

Tha answer also links to the explanation of the YROP effect in The YORP Effect and Bennu
 which says:

The YORP effect is a similar phenomenon that affects the rotation rate and pole orientation of an asteroid. YORP is an acronym that combines four scientist’s names: Yarkovsky, O’Keefe, Radzievskii, and Paddack. Building on the work of Yarkovsky, V. V. Radzievskii showed in 1954 that variations in albedo across the surface of a small body in space could increase its rotation rate. This phenomenon is essentially the Crooke’s radiometer effect. Stephen Paddack and John O’Keefe, who were separately working on the origin of tektites and interplanetary dust, went on to show that an object’s shape strongly influences the change in rotation. David Rubincam synthesized these ideas in 1999, and showed that YORP creates a thermal torque akin to a “windmill” effect on asteroids. This torque can modify the rotation rate and obliquity of an asteroid, depending on the external geometry of the body.

This is frustrating because of course the Crooke's radiometer works by interacting with the molecules of the low pressure gas intentionally included inside the glass bulb, and not by radiation, but at least we get the idea that a net torque is produced due to a non-uniformity of some optical property.
Wikipedia's Yarkovsky–O'Keefe–Radzievskii–Paddack effect works hard to try to explain but the explanation is long and I am not confident it's actually accurate.
Answers to What is the difference between the Yarkovsky effect and YORP effect? touch on this but do not answer this question.
Question: What is the YORP effect exactly? Is it just the non-central component of the Yarkovsky effect?

Yarkovsky only considers the center of mass recoil from thermal radiation of a rotating body in sunlight, is YORP just the tangential component of this recoil?
Or does it require the body to be non-uniform in order to spin-up the body?
Would a uniform sphere in fact spin-down?
From What is the difference between the Yarkovsky effect and YORP effect?, both the Yarkovsky and YORP effects describe the changes in momentum to a body in orbit due to the re-radiation of photons.  The Yarkovsky effect only describes the changes to the orbital parameters of the body:

The Yarkovsky effect describes a small but significant force that
affects the orbital motion of meteoroids and asteroids

The YORP effect only describes the changes to the rotational momentum of the body:

This rotational variant has been coined the
Yarkovsky-O’Keefe-Radzievskii-Paddack (YORP) effect

Consider a perfect uniform sphere in orbit around the Sun.  For any point on the sphere, the average direction of re-radiated photons will be normal to the local tangent plane (or directly opposite the center of gravity). In this case, we expect a Yarkovsky effect (change in orbital parameters), but no YORP effect (change in rotational momentum). Graphic from this source.

In contrast, consider an irregular body with a huge flat face offset from the center of gravity.  When photons are re-radiated from the flat face, on average they will impart some angular momentum change (YORP effect) and some orbital parameter change (Yarkovsky effect).


Yarkovsky only considers the center of mass recoil from thermal
radiation of a rotating body in sunlight, is YORP just the tangential
component of this recoil?

The forces causing the YORP and Yarkovsky effects are not tangential in general. For a recoil for some impulse on a sphere, the rotational and linear forces imparted are tangential.  However, for an irregular shape like a rod, we wouldn't expect the two forces to be tangential. Here is a little drawing (extremely exaggerated for effect):

In the above drawing, the Yarkovsky vector represents purely translational momentum change and the YORP vector represents purely rotational momentum change. The sum of the vectors that represent forces that cause the YORP and Yarkovsky effects should sum to the mass recoil vector (which is the negative of the impulse):

They YORP and Yarkovsky forces don't necessarily need to be at right angles.  In three dimensions, sorting out the forces can get extremely complicated.    It can be quite complicated for a rod even if the reaction mass vector is at the tip. It's hard for me to imagine the recoil from a &quot;crescent moon&quot; shaped asteroid that has a center of mass exterior to the asteroid body.

Or does it require the body to be non-uniform in order to spin-up the
body?

The body must be irregular in shape to spin it up.  A perfect sphere with a non-uniform surface would not be subject to the YORP effect.  This is as mentioned above, since on average, photon emission at a specific point will be 90 degrees from the tangent plane, effectively canceling all YORP effects from individual emissions.

Would a uniform sphere in fact spin-down?

Again, a sphere would not have any YORP effects because the average re-radiation vector would be perfectly opposite the center of mass.
Note:
When an asteroid strikes a planetary sized body in an inelastic collision, momentum is also conserved.  Some of the asteroid's momentum goes into rotational momentum of the body and some of the momentum goes into the momentum of the center of mass (equivalent to the orbital parameters).  I like this analogy for thinking about the Yarkovsky and YORP effects.