 You're correct. The Ark was not necessary for atonement. It was the blood sacrifices that were necessary. And there's even a tradition that with the Second Temple, there was just a rock there in the Holy of Holies and blood would be poured out there because there was no Ark of the Covenant.