TimeSeriesTransformerModel dimensionality issue
Hi, fairly new to TimeSeriesTransformerModel here but I can’t figure out what’s going on with the model I’m trying to initialise:It seems like I have to force feature_size to be 73 to match mat1 (understand the math reasoning but I can’t find what feature_size is in the documentation)Error Code:image1350×972 46.4 KBHowever, if I force the feature size to be 73, the output looks empty…tensor(, size=(1, 0, 64), grad_fn=)Any help would be appreciated!Edit:Does this happen because the prediction len is 0? I’ve noticed the encoder_last_hidden_states is populated… This is all I actually need as I want to take the encoded representation and fit it to a separate fc head… Could someone confirm my thinking that the encoder_last_hidden_states should be the embeddings for the time_series data prior to any prediction?
2024-08-19T10:13:58Z
hi@ArgyleRIf you haven’t yet, please see the exampleTime Series Transformer.I’m checking your code.
2024-08-19T11:48:18Z
cc@kashif