Could there be an "remove noise" function to remove noise from noisy_latents, given the noise and the timestep?
When using “epsilon” prediction mode I notice that the model predicts a pre-defined noise with normal distribution. Since the noise as well as thenoisy_latentsis too ambiguous for any sementic loss (like measuring the clip feature distance between generated image & text prompt) during training, I wonder if there’s a way to directly substract the noise from thenoisy_latents(instead of denoising step by step), ending up a pure latent without noise?I guess this is practicable becausenoisy_latentslooks like some weighted addition oflatentsandnoiseinsidescheduler.add_noisefunction, but my poor math cannot afford to invert this process. May some kind-hearted guy help me?