Regarding to policy/model/weights

[zyzhang1130]

Do you mind to clarify is the policy/model/weights saved after each epoch/iteration? If not how should I make it happen? If yes where is it saved at? I see you just call the model 'args.model_path' in agent.py and save(self, path, filename) without specifically assigning a path or name.

Thank you for replying.

[zyzhang1130]

Hi, I found this piece of info quite relevant! Kaixhin/Rainbow#58
However, I am not proficient to incorporate it into the rainbow code of yours (judging based on the time it was added, your current version rainbow probably does not have this feature). Do you mind to take a look at it?

Thank you so much

[zyzhang1130]

Actually the contributor of Rainbow says these lines save the model weights:
https://github.com/Kaixhin/Rainbow/blob/d3afb5ad570137d675d6c7c903c050c8a19db084/main.py#L179-L181
How do you think I should incorporate it into cups-lr Rainbow code?

Thank you.

[beduffy]

Line 79 in rainbow/test.py which is called periodically from the main.py file is the code which saves the model:

# Save model parameters if improved
if avg_reward > best_avg_reward:
    best_avg_reward = avg_reward
    dqn.save(path='weights', filename='rainbow_{}.pt'.format(num_steps))

This is therefore saved in the rainbow directory from which is related to #17. Set your evaluation-interval lower. If it still doesn't save the model after args.evaluation_interval steps let me know.

[zyzhang1130]

Sorry, can I check does the current repo allow resume training feature? Thank you.