What does optimizer_update_8bit function in bitsandbytes.functional actually do with its state1 and state2 parameters?
Hi, I am having problems with the use of optimizer_update_8bit in one of my projects. I hope to use this function to perform quantized gradient updates, but I have not found any documentation on the meaning of its state1 and state2 parameters. Do these two parameters provide optimizer_update_8bit with the values of corresponding optimizer states, or do they only provide optimizer_update_8bit with the storage location of optimizer states, while the values of states will be calculated by optimizer_update_8bit and stored in them?I would be very grateful for any reply that gives a reasonable explanation or provides documentation on this function. Thanks!