Earth Rotation and Orbit Geometry Let Line A be the projection of the axis of Earth's rotation onto the plane of the eliptic.  Let Line B the line connecting the center of the Earth and center of the Sun.

At what times during the year is Line A perpendicular to Line B?
The axis points in pretty much the same direction all the time, though it does wobble slightly, ranging between 22.1 degrees and 24.5 degrees every 41,000 years.   (Source)

Perpendicular between two lines is a 2 dimensional property, so, while the axis is never "straight up" so to speak, there is a 90 degree angle between the axis and the line between the center of the sun and the center of the earth during  the spring and fall equinox.  Those are the only times.