Predictions near mean value during training

[ghost]

Hello Xieyyyy

I am using a similar model to your one with attention for both spatial and temporal and the dataset is just an updated version from 2020.

During training, I printed out the predictions after the final linear layer and all the predictions seem to be near the mean Value of the training data.

Is there a possible reason for this ?

I checked attention coefficients, position encoding and other parts of the code but having no luck.

Could you please give some pointers from your experience? My main question is that during training, the trend should be captured in the predictions or not?