Is there a star over my head? Say I'm standing up straight, and I draw a straight line from my core through the top of my head (perpendicular to the ground). What is the probability that that line intersects with a star?

EDIT: I'm not trying to exclude any stars. This should include stars which we've observed and stars which we haven't yet observed but can predict due to other things we've determined (like the overall star density of the universe). Also it should include all stars regardless of naked eye magnitude limit.
Summary

There's a 1 in 500 billion chance you're standing under a star outside the Milky Way, a 1 in 3.3 billion chance you're standing under a Milky Way star, and a 1 in 184 thousand chance you're standing under the Sun right now.

Big, fat, stinking, Warning! I did my best to keep my math straight, but this is all stuff I just came up with. I make no guarantees it's completely accurate, but the numbers seem to pass the sanity check so I think we're good.

Caveat the First: The numbers for stars other than the Sun are based on data with a great deal of uncertainty, such as the number of stars in the universe and the average size of a star. The numbers above could easily be off by a factor of 10 in either direction, and are merely intended to give a rough idea of how empty space is.

Caveat the Second: The numbers for the Sun and the Milky Way are based on the assumption that you are standing (or floating) at a random point on Earth. Anyone outside the tropics will never have the Sun over their head. People in the northern hemisphere are more likely to have Milky Way stars over their head, with the best odds being people near 36.8° N, because at that latitude straight up passes through the galactic center once a day.26

Note: You can mostly ignore everything in this answer and just look up the solid angle of the Sun to get the same result. All the other stars are really far away and very spread out. The difference in solid angle subtended is five thousandths of a percent more when we add the rest of the universe to the Sun.

Background

Let's try to get a somewhat realistic, hard number. To do that, we'll need some assumptions.

As pointed out in Michael Walsby's answer1, if the universe is infinite (and homogeneous2), there is only an infinitesimal chance of there not being a star overhead, which normal math treats as exactly zero chance. So let's presume the universe is finite.

Presumptions


Specifically, let's presume the universe only consists of the observable universe. (Look up the expansion of the universe3 for further information.)
Further, let's presume the contents of the observable universe are measured at their current (presumed) positions, not the position they appear to be. (If we see light from a star from 400 million years after the universe began, we would measure it as being about 13.5 billion light years away, but we calculate that it's likely closer to 45 billion light years away due to expansion.)
We'll take the number of stars in the observable universe to be $10^{24}$. A 2013 estimate4 was $10^{21}$, a 2014 estimate5 was $10^{23}$, and a 2017 estimate6 was $10^{24}$, with each article expecting the estimate to increase as we get better telescopes over time. So we'll take the highest value and use it.
We'll take the size of the observable universe7 to be $8.8\cdot10^{26} \text{m (diameter)}$, giving a surface area8 of $2.433\cdot10^{54} \text{m}^2$ 9, and a volume10 of $3.568\cdot10^{80} \text{m}^3$ 11.
We'll take the average size of a star to be the size of the Sun, $1.4\cdot10^{9}\text{m (diameter)}$ 12. (I can't find any sources for average star size, just that the Sun is an average star.)


Model

From here, we're going to cheat a bit. Realistically, we should model each galaxy separately. But we're just going to pretend the entire universe is perfectly uniform (this is true enough as we get farther away from Earth in the grand scheme of the cosmos). Further, we're going to start counting far enough out to ignore the Milky Way and Sun entirely, then add them back in later with different calculations.

Given the above presumptions, we can easily calculate the stellar density of the observable universe to be $\delta = \frac{10^{24}\text{stars}}{3.568\cdot10^{80} m^3} = 2.803\cdot10^{-57} \frac{\text{stars}}{\text{m}^3}$ 13.

Next, we need to calculate the solid angle14 subtended by a star. The solid angle of a sphere is given by $\Omega=2\pi\left(1-\frac{\sqrt{d^2-r^2}}{d}\right)\text{ sr}$ 15, where $\Omega$ is the solid angle in steradians16 (sr), $d$ is the distance to the sphere and $r$ is the radius of the sphere. Using $D$ as the diameter, that converts to $\Omega=2\pi\left(1-\frac{\sqrt{d^2-\left(\frac{D}{2}\right)^2}}{d}\right)\text{ sr}$. Given the average diameter presumed above ($1.4\cdot10^{9}\text{m}$), this gives an average solid angle of $\Omega=2\pi\left(1-\frac{\sqrt{d^2-4.9\cdot10^{17}\text{m}^2}}{d}\right)\text{ sr}$ 17.

At this point, we could set up a proper integral, but my calculus is rather rusty, and not very sharp to begin with. So I'm going to approximate the answer using a series of concentric shells, each having a thickness of $10^{22}\text{m}$ (about a million light years). We'll put our first shell $10^{22}\text{m}$ away, then work our way out from there.

We'll calculate the total solid angle of each shell, then add all the shells together to get the solid angle subtended by the entire observable universe.

The last problem to fix here is that of overlap. Some stars in the farther shells will overlap stars in the nearby shells, causing us to overestimate the total coverage. So we'll calculate the probability of any given star overlapping and modify the result from there.

We'll ignore any overlap within a given shell, modeling as if every star in a shell is at a fixed distance, evenly distributed throughout the shell.

Probability of Overlap

For a given star to overlap closer stars, it needs to be at a position already covered by the closer stars. For our purposes, we'll treat overlaps as binary: either the star is totally overlapped, or not overlapped at all.

The probability will be given by the amount of solid angle already subtended by previous shells divided by the total solid angle in the sky ($4\pi\text{ sr}$).

Let's call the probability a given star, $i$, is overlapped $P_i$, the solid angle subtended by that star $\Omega_i$, and the number of stars $n$. The amount of non-overlapped solid angle subtended by a given shell, $k$, is then $\Omega_{kT}=(1-P_1)\Omega_1+(1-P_2)\Omega_2+\ldots+(1-P_n)\Omega_n\frac{\text{ sr}}{star}$. Since we've said stars in a shell don't overlap each other, $P_i$ is the same for all $i$ in a given shell, allowing us to simplify the above equation to $\Omega_{kT}=(1-P_k)(\Omega_1+\Omega_2+\ldots+\Omega_n)\frac{\text{ sr}}{star}$, where $P_k$ is the probability of overlap for shell $k$. Since we're treating all the stars as having the same, average size, this simplifies even further to $\Omega_{kT}=(1-P_k)\Omega_k n\frac{\text{ sr}}{star}$, where $\Omega_k$ is the solid angle of a star in shell $k$.

Calculating Solid Angle

The number of stars in a shell is given by the volume of the shell times the stellar density of said shell. For far away shells, we can treat the volume of the shell as being its surface area times its thickness. $V_\text{shell}=4\pi d^2 t$, where $d$ is the distance to the shell and $t$ is its thickness. Using $\delta$ as the stellar density, the number of stars is simply $n=\delta V_\text{shell}=\delta 4\pi d^2 t$.

From here, we can use the calculation for solid angle of a shell (from Probability of Overlap, above) to get $\Omega_{kT}=(1-P_k)\Omega_k \delta 4\pi d^2 t\frac{\text{ sr}}{star}$.

Note that $P_k$ is given by the partial sum of solid angle for all previous shells divided by total solid angle. And $\Omega_k$ is given by $\Omega_k=2\pi\left(1-\frac{\sqrt{d_k^2-4.9\cdot10^{17}\text{m}^2}}{d_k}\right)\frac{\text{ sr}}{star}$ (from Model, above).

This gives us $\Omega_{kT}=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)2\pi\left(1-\frac{\sqrt{d_k^2-4.9\cdot10^{17}\text{m}^2}}{d_k}\right) \delta 4\pi d^2 t\text{ sr}$. Given that each shell is $10^{22}\text{m}$ away, we can substitute $d_k$ with $k 10^{22}\text{m}$. Likewise, $t$ can be substituted with $10^{22}\text{m}$. And we already calculated $\delta=2.803\cdot10^{-57} \frac{\text{stars}}{\text{m}^3}$ (from Model, above).

This gives us
$\Omega_{kT}=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)2\pi\left(1-\frac{\sqrt{(k 10^{22}\text{m})^2-4.9\cdot10^{17}\text{m}^2}}{k 10^{22}\text{m}}\right) 2.803\cdot10^{-57}\frac{\text{stars}}{\text{m}^3} 4\pi (k 10^{22}\text{m})^2 10^{22}\text{m}\frac{\text{ sr}}{star}$  

$=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)\left(1-\frac{\sqrt{k^2 10^{44}-4.9\cdot10^{17}}}{k 10^{22}}\right) 2.803\cdot10^{-57} 8\pi^2 k^2 10^{66}\text{ sr}$  

$=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)\,2.213\cdot 10^{11}\,k^2\left(1-\frac{\sqrt{k^2 10^{44}-4.9\cdot10^{17}}}{k 10^{22}}\right)\text{ sr}$

From here, we can just plug the numbers into a calculation program.

$\Omega_{T}=\sum_{k=1}^{k_\text{max}} \Omega_{kT}$

Where $k_\text{max}$ is just the radius of the observable universe divided by the thickness of a given shell. Thus $k_\text{max}$$=\frac{4.4\cdot 10^{26} \text{m}}{10^{22} \text{m}}$$=4.4\cdot 10^4$$=44000$

$\Omega_{T}=\sum_{k=1}^{44000} \Omega_{kT}$

Results

Because of the large numbers involved, it's difficult to just run this in a program. I resorted to writing a custom C++ program using the ttmath library18 for large numbers. The result was $2.386\cdot 10^{-11}\text{ sr}$, or $1.898\cdot 10^{-12}$ of the entire sky. Conversely, there's about a 1 in 500 billion chance you're standing under a star right now.

Note that we ignored the Milky Way and the Sun for this.

The C++ program can be found at PasteBin25. You'll have to get ttmath working properly. I added some instructions to the top of the C++ code to get you started if you care to make it work. It's not elegant or anything, just enough to function.

The Sun

WolframAlpha helpfully informed me the Sun has a solid angle of about $6.8\cdot 10^{-5}\text{ sr}$, or about 2.8 million times more than all the stars in the universe combined. The solid angle formula above gives the same answer18 if we provide the Sun's 150 gigameter distance and 0.7 gigameter radius.

The Milky Way

We could get an approximation for the Milky Way by taking its size and density and doing the same calculations as above, except on a smaller scale. However, the galaxy is very flat, so the odds greatly depend on whether you happen to be standing in the galactic plane or not. Also, we're off to one side, so there are far more stars towards the galactic center than away.

If we approximate the galaxy as a cylinder with a radius of $5\cdot 10^{20}\text{ m}$ (about 52000 light years) and a height of $2\cdot 10^{16}\text{ m}$ (about 2 light years), we get a volume of $1.571\cdot 10^{58}\text{ m}^3$ 20.

Current estimates of the galaxy's radius are closer to 100000 light years21 22, but I'm presuming the vast majority of stars are a lot closer than that.

There are estimated to be 100 to 400 billion stars in the Milky Way21. Let's pick 200 billion for our purposes. This puts the density of the Milky Way at $\delta = \frac{200\cdot10^{9}\text{stars}}{1.571\cdot 10^{58}\text{ m}^3} = 1.273\cdot10^{-47} \frac{\text{stars}}{\text{m}^3}$ 22, or about 4.5 billion times denser than the universe at large.

This time, we'll take shells $10^{17}\text{ m}$ thick (about 10 light years) and go out from there. But we need to re-organize the math into a spherical form, so we'll presume the galaxy has the same volume, but is a sphere. This gives it a radius of $1.554\cdot 10^{19}\text{ m}$ 24, or 155.4 shells. We'll round to 155 shells.

$\Omega_{T}=\sum_{k=1}^{155} \Omega_{kT}$

Using our formula from above (Calculating Solid Angle), we can start substituting numbers.

$\Omega_{kT}=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)2\pi\left(1-\frac{\sqrt{d_k^2-4.9\cdot10^{17}\text{m}^2}}{d_k}\right) \delta 4\pi d^2 t\frac{\text{sr}}{\text{star}}$

$=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)2\pi\left(1-\frac{\sqrt{(k\cdot 10^{17}\text{ m})^2-4.9\cdot10^{17}\text{ m}^2}}{k\cdot 10^{17}\text{ m}}\right) 1.273\cdot10^{-47} \frac{\text{stars}}{\text{m}^3} 4\pi (k\cdot 10^{17}\text{ m})^2 10^{17}\text{ m}\frac{\text{sr}}{\text{star}}$

$=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)\left(1-\frac{\sqrt{k^2\cdot 10^{34}\text{ m}^2-4.9\cdot10^{17}\text{ m}^2}}{k 10^{17}\text{ m}}\right) 1.273\cdot 10^{-47} \frac{\text{stars}}{\text{m}^3} 8\pi^2 k^2 10^{51}\text{ m}^3\frac{\text{sr}}{\text{star}}$

$=\left(1-\frac{\Omega_{(k-1)T}}{4\pi}\right)\cdot\,1.005\cdot 10^6\,k^2\,\left(1-\frac{\sqrt{k^2\cdot 10^{34}-4.9\cdot10^{17}}}{k 10^{17}}\right)\text{ sr}$

Plugging this into the program gives $3.816\cdot 10^{-9}\text{ sr}$, which is $3.037\cdot 10^{-10}$ of the total sky. The odds you're standing under a star in the Milky Way are about 1 in 3.3 billion.

Solid Angle Totals

Solid angle is:


Sun, $6.8\cdot 10^{-5}\text{ sr}$  
Milky Way, $3.816\cdot 10^{-9}\text{ sr}$  
Universe, $2.386\cdot 10^{-11}\text{ sr}$  
Total, $6.800384\cdot 10^{-5}\text{ sr}$ (the extra digits are basically meaningless, adding about five thousandths of a percent to the Sun's solid angle)  
Milky Way plus Universe, $3.840\cdot 10^{-9}\text{ sr}$ (about 0.6% more than just the Milky Way)


References

1 Michael Walsby's answer to this question, is there a star over my head?. https://astronomy.stackexchange.com/a/33294/10678
2 A Wikipedia article, Cosmological principle. https://en.wikipedia.org/wiki/Cosmological_principle
3 A Wikipedia article, Expansion of the universe. https://en.wikipedia.org/wiki/Expansion_of_the_universe
4 A UCSB ScienceLine quest, About how many stars are in space?, from 2013. https://scienceline.ucsb.edu/getkey.php?key=3775
5 A Sky and Telescope article, How Many Stars are There in the Universe?, from 2014. https://www.skyandtelescope.com/astronomy-resources/how-many-stars-are-there/
6 A Space.com article, How Many Stars Are In The Universe?, from 2017. https://www.space.com/26078-how-many-stars-are-there.html
7 A Wikipedia article, Observable universe. https://en.wikipedia.org/wiki/Observable_universe
8 A Wikipedia article, Sphere, section Enclosed volume. https://en.wikipedia.org/wiki/Sphere#Enclosed_volume
9 A WolframAlpha calculation, surface area of a sphere, diameter 8.8*10^26 m. https://www.wolframalpha.com/input/?i=surface+area+of+a+sphere%2C+diameter+8.8*10%5E26+m
10 A Wikipedia article, Sphere, section Surface area. https://en.wikipedia.org/wiki/Sphere#Surface_area
11 A WolframAlpha calculation, volume of a sphere, diameter 8.8*10^26 m. https://www.wolframalpha.com/input/?i=volume+of+a+sphere%2C+diameter+8.8*10%5E26+m
12 A nineplanets.org article, The Sun. https://nineplanets.org/sol.html
13 A WolframAlpha calculation, (10^24 stars) / (3.568⋅10^80 m^3). https://www.wolframalpha.com/input/?i=%2810%5E24+stars%29+%2F+%283.568%E2%8B%8510%5E80+m%5E3%29
14 A Wikipedia article, Solid angle. https://en.wikipedia.org/wiki/Solid_angle
15 Harish Chandra Rajpoot's answer to a geometry.se question, Calculating Solid angle for a sphere in space. https://math.stackexchange.com/a/1264753/265963
16 A Wikipedia article, Steradian. https://en.wikipedia.org/wiki/Steradian
17  A WolframAlpha calculation, 2*pi*(1-sqrt(d^2-(1.4*10^9 m/2)^2)/d). https://www.wolframalpha.com/input/?i=2*pi*%281-sqrt%28d%5E2-%281.4*10%5E9+m%2F2%29%5E2%29%2Fd%29
18 Website for ttmath. https://www.ttmath.org/
19 A WolframAlpha calculation, 2*pi*(1 - sqrt(d^2 - r^2)/d), where d = 150 billion, r=0.7 billion. https://www.wolframalpha.com/input/?i=2*pi*%281+-+sqrt%28d%5E2+-+r%5E2%29%2Fd%29%2C+where+d+%3D+150+billion%2C+r%3D0.7+billion
20 A WolframAlpha calculation, pi * (5*10^20 m)^2 * (2*10^16 m). https://www.wolframalpha.com/input/?i=pi+*+%285*10%5E20+m%29%5E2+*+%282*10%5E16+m%29
21 A Wikipedia article, Milky Way. https://en.wikipedia.org/wiki/Milky_Way
22 A Space.com article from 2018, It Would Take 200,000 Years at Light Speed to Cross the Milky Way. https://www.space.com/41047-milky-way-galaxy-size-bigger-than-thought.html
23 A WolframAlpha calculation, (200*10^9 stars) / (1.571*10^58 m^3). https://www.wolframalpha.com/input/?i=(200*10^9+stars)+%2F+(1.571*10^58+m^3)
24 A WolframAlpha calculation, solve for r:  (4/3)*pi*r^3 = 1.571*10^58 m^3. https://www.wolframalpha.com/input/?i=solve+for+r%3A++%284%2F3%29*pi*r%5E3+%3D+1.571*10%5E58+m%5E3
25 My C++ program code on PasteBin. https://pastebin.com/XZTzeRpG
26 A Physics Forums post, Orientation of the Earth, Sun and Solar System in the Milky Way. Specifically, Figure 1, showing angles of 60.2° for the Sun, and 23.4° less than that for Earth. https://www.physicsforums.com/threads/orientation-of-the-earth-sun-and-solar-system-in-the-milky-way.888643/