Is it possible to calculate how much a star's brightness dims when a planet transits in front of it from a viewer's perspective? Last night I got to thinking about what would happen if Jupiter and Venus suddenly switched places.

Since Venus comes the closest to Earth than any other planet and Jupiter is much larger, does that mean that Jupiter (now following the orbit of Venus) would appear large enough to eclipse the Sun from Earth's perspective?

Jupiter's angular diameter in this scenario is given by $$\delta = 2\arctan \left(\frac{d}{2D}\right)$$

where 


$\delta$ is its angular diameter, 
$d$ is its actual diameter, 
$D$ is the distance between both bodies. 


The closest Venus and Earth ever get to each other (in the near future) is $d = 39.5 \times 10^6$ km, and Jupiter's diameter is $D = 139,820$ km.

So if Jupiter replaced Venus, its maximum angular diameter would be $$\delta = 2\arctan \left(\frac{139,820}{2 \times 39.5 \times 10^6}\right)$$
$$\delta = 0.202812°, or \ 0° 12  ' 10.12 " $$

The Sun's angular diameter as seen from Earth is about 30 arcminutes, and so Jupiter in this scenario with only 12 arcminutes would not appear to be large enough to completely eclipse the Sun. 

However, assuming it is a total transit, it would still cover $≈16.27 \%$ of the Sun's disk as seen from Earth. Surely that would result in a noticeable dim in the brightness of the Sun as seen from Earth, no? 

Is there an equation to calculate how much a star's brightness dims when a planet transits in front of it from a viewer's perspective?
You answer your question basically yourself with the calculation - that's the right way to approach it, your math is correct. Stars and planets are big enough, that simple geometric optics work well to calculate the degree of occulatation when one body passes in front of another. This is the method used to estimate the size of exoplanets pretty accurately (we know stelar sizes from the mass-luminosity-radius relations).

Typical drop in luminosity from observed exoplanets is 1% or less - which happens to be exactly the square ratio of solar radius to jupiter radius. You correctly get bigger luminosity drop as you don't put both objects in infinity, thus don't directly compare radii but their apparant angular diameter.

The only not exactly straight-forward part is the influence of the atmosphere of the transiting body as it contributes to absorption depending on density. That is one of two reason that for any brightness curve of exoplanet transits you don't see an immediate drop in brightness (the other is the result of the limb darkening of the star). (See e.g. this light curve) This is used to infer exoplanet atmospheric properties. It can even be taken so far as to make atmospheric spectroscopy by comparing transition spectrum and normal spectrum of the star.

You, however, seem implicitly to ask (also) about the human factor, how we, as humans, perceive the brightness variation. That is an entirely different story as our eyes cover a MUCH larger brightness span as they work logarithmically - thus small changes in linear brightness (eclipsing part of the sun's cross-section is simply that) don't make a big impression. CCDs or CMOS work linearily, thus detect such changes much easier - but are bad when it comes to covering large brightness variation. 

Anyone who observed a solar eclipse can tell the story that you don't notice much of a brightness drop until the very last moments when the sun becomes a very tiny sickle just moments before totality. The most notable difference is behaviour of the shadows (they change shape) and the sharpness of shadows. It gets (slightly) colder, as less light arrives - but the remaining light source is as bright per unit area as the already eclipsed part is so that the perceived brightness remains nearly the same.
It's similar when you switch on two identical lamps - the room does not seem twice as bright compared to only one lamp.