Add note on loading and resetting environments (DLR-RM#340)

* Update documentation and changelog * Fix docs * Update examples.rst Co-authored-by: Antonin RAFFIN <[email protected]>
squalidux · Mar 5, 2021 · f13de5b · f13de5b
1 parent 65100a4
commit f13de5b
Show file tree

Hide file tree

Showing 2 changed files with 13 additions and 3 deletions.
diff --git a/docs/guide/examples.rst b/docs/guide/examples.rst
@@ -480,13 +480,22 @@ By default, the replay buffer is not saved when calling ``model.save()``, in ord
 However, SB3 provides a ``save_replay_buffer()`` and ``load_replay_buffer()`` method to save it separately.
 
 
-.. image:: ../_static/img/colab-badge.svg
-   :target: https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/advanced_saving_loading.ipynb
-
 Stable-Baselines3 automatic creation of an environment for evaluation.
 For that, you only need to specify ``create_eval_env=True`` when passing the Gym ID of the environment while creating the agent.
 Behind the scene, SB3 uses an :ref:`EvalCallback <callbacks>`.
 
+
+.. note::
+
+	For training model after loading it, we recommend loading the replay buffer to ensure stable learning (for off-policy algorithms).
+	You also need to pass ``reset_num_timesteps=True`` to ``learn`` function which initializes the environment
+	and agent for training if a new environment was created since saving the model.
+
+
+.. image:: ../_static/img/colab-badge.svg
+   :target: https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/advanced_saving_loading.ipynb
+
+
 .. code-block:: python
 
   from stable_baselines3 import SAC

diff --git a/docs/misc/changelog.rst b/docs/misc/changelog.rst
@@ -14,6 +14,7 @@ Documentation:
 ^^^^^^^^^^^^^^
 - Fixed examples
 - Added new project using SB3: rl_reach (@PierreExeter)
+- Add a note on continual learning and resetting environment
 
 
 Pre-Release 0.11.1 (2021-02-27)