Decision Transformer for Discrete action
I’m studying the decision transformer referring toTrain your first Decision Transformer.In the post, the example is for “halfcheetah” ( action space is continuous) andthe following model code is used.I’m trying to apply this to the discrete action space.I added the logit layer for the discrete action and changed the loss function as below.( red color: removed , blue color: added )Is this the right approach?image576×618 16.7 KB
2024-04-26T04:31:30Z
Hey,I’m also trying out something similar recently. Based on my implementation, your implementation looks almost similar to mine, except for the fact that I did not use an additional linear layer and directly used the outputs as the logit. Also, I had encode my actions into one-hot-encoding, so I had to do some reshaping with the action targets, but otherwise, I think this is pretty much spot on.I usually just print out the shapes and the intermediate variables at least once for a sanity check to make sure everything looks right and nothing it broadcasted incorrectly.