Could a black hole pass quiescently through the Oort cloud? Imagine a stellar-mass black hole passing through the Oort cloud, perhaps a quarter  to half a light-year from the Sun. How black would the black hole be? That is, any matter that fell into the black hole would release energy on the way in, and we know there are comets out there, and presumably smaller objects down to microscopic dust. But we also know they are very few and far between. Has anyone done a serious analysis of whether there would be enough infalling matter for the resulting radiation to be detectable?

As two commenters already recognized, this really is about radiation from infalling matter, and not the broader question of how we could detect such a black hole. Answers to that broader question in the comments are interesting, though.
An approach would be to use the Bondi-Hoyle (B-H) accretion rate and then assume that some large fraction of the gravitational potential energy was radiated from the innermost stable circular orbit (ISCO) as the material spiralled into the black hole via an accretion disc.

To calculate the B-H accretion rate we need how fast the black hole traverses the Oort cloud and the density of matter there.
$$\dot{M} \simeq \lambda\frac{4\pi G^2 M^2 \rho}{v^3},$$
where $M$ is the black hole mass, and $\lambda$ is an accretion efficiency parameter that is judged to be $\sim 0.01$ on the basis of observed compact objects (see Fender et al. 2013).
The luminosity will be
$$ L \simeq \eta \frac{GM \dot{M}}{R_{\rm ISCO}},$$
where $R_{\rm ISCO}= 6GM/c^2$ for a Schwarzschild black hole. Thus
$$ L \sim \eta \lambda \frac{4\pi G^2 M^2 \rho c^2 }{6v^3}.$$

Getting all my information from wikipedia, I assume there are 5 Earth masses in the Oort cloud in a spherical shell from 20,000 to 50,000 au. This gives an average density $\rho \sim 2\times 10^{-23}$ kg/m$^3$.

This is actually lower by a factor of a thousand or so than the density of the interstellar medium. Thus accretion from the ISM is far more important on average. Putting the numbers in for a 1 solar mass black hole and $\lambda=0.01$, $\eta =1$:
$$L = 3\times 10^{23}\left(\frac{\rho}{10^{-20}\ {\rm kg m}^{-3}}\right)\left(\frac{v}{10\ {\rm km s}^{-1}}\right)^{-3}\ {\rm W.}$$

This is quite an appreciable luminosity and much would emerge at UV and X-ray wavelengths. It would be readily detectable. The flux at a distance of 50,000 au would be $5\times 10^{-10}$ W/m$^2$, which is 4 orders of magnitude above the faint flux limit of the ROSAT all sky X-ray survey.