Would a direct hit by a small comet be catastrophic? Would a direct hit by a short-period comet with a 3 mile diameter nucleus cause enough damage on Earth to threaten or destroy human life and the ability to produce crops?
Here, have some fun with this...

http://impact.ese.ic.ac.uk/ImpactEffects/

It doesn't go into long term effects, but the short term is pretty devastating for a significant portion of the planet.  'Nuclear winter' effects seem inevitable for those not atomised, scorched, crushed, or covered in ejecta.

Not good, nope.