Why sigma_data = 0.5 in EDM?
Hi, in Karras‘s paper Elucidating the Design Space of Diffusion-Based Generative Models, they specifysigma_{data}as 0.5, where is number come from?github.comhuggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_consistency_models.py#L58This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the genericmethods the library implements for all schedulers such as loading and saving.Args:num_train_timesteps (`int`, defaults to 40):The number of diffusion steps to train the model.sigma_min (`float`, defaults to 0.002):Minimum noise magnitude in the sigma schedule. Defaults to 0.002 from the original implementation.sigma_max (`float`, defaults to 80.0):Maximum noise magnitude in the sigma schedule. Defaults to 80.0 from the original implementation.sigma_data (`float`, defaults to 0.5):The standard deviation of the data distribution from the EDM[paper](https://huggingface.co/papers/2206.00364). Defaults to 0.5 from the original implementation.s_noise (`float`, defaults to 1.0):The amount of additional noise to counteract loss of detail during sampling. A reasonable range is [1.000,1.011]. Defaults to 1.0 from the original implementation.rho (`float`, defaults to 7.0):The parameter for calculating the Karras sigma schedule from the EDM[paper](https://huggingface.co/papers/2206.00364). Defaults to 7.0 from the original implementation.clip_denoised (`bool`, defaults to `True`):Whether to clip the denoised outputs to `(-1, 1)`.
2024-01-12T07:24:40Z
That’s the standard deviation of the data. If you look at a more recent paper[2312.02696] Analyzing and Improving the Training Dynamics of Diffusion Models, that paper actually uses different data, so they could have used a different sigma_data; but instead they normalize the data to have standard deviation 0.5 so that they don’t have to change the other hyperparameters of EDM.