<linkrel="index"title="Index"href="../../genindex/"/><linkrel="search"title="Search"href="../../search/"/><linkrel="next"title="Env"href="../../api/env/"/><linkrel="prev"title="Compatibility with Gym"href="../gym_compatibility/"/>
<liclass="toctree-l1 has-children"><aclass="reference internal"href="../../environments/toy_text/">Toy Text</a><inputclass="toctree-checkbox"id="toctree-checkbox-6"name="toctree-checkbox-6"role="switch"type="checkbox"/><labelfor="toctree-checkbox-6"><divclass="visually-hidden">Toggle navigation of Toy Text</div><iclass="icon"><svg><usehref="#svg-arrow-right"></use></svg></i></label><ul>
<liclass="toctree-l2"><aclass="reference internal"href="../../tutorials/gymnasium_basics/environment_creation/">Make your own custom environment</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../../tutorials/gymnasium_basics/vector_envs_tutorial/">Training A2C with Vector Envs and Domain Randomization</a></li>
</ul>
</li>
<liclass="toctree-l1 has-children"><aclass="reference internal"href="../../tutorials/training_agents/">Training Agents</a><inputclass="toctree-checkbox"id="toctree-checkbox-9"name="toctree-checkbox-9"role="switch"type="checkbox"/><labelfor="toctree-checkbox-9"><divclass="visually-hidden">Toggle navigation of Training Agents</div><iclass="icon"><svg><usehref="#svg-arrow-right"></use></svg></i></label><ul>
<liclass="toctree-l2"><aclass="reference internal"href="../../tutorials/training_agents/reinforce_invpend_gym_v26/">Training using REINFORCE for Mujoco</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../../tutorials/training_agents/blackjack_tutorial/">Solving Blackjack with Q-Learning</a></li>
<liclass="toctree-l1"><aclass="reference external"href="https://github.com/Farama-Foundation/Gymnasium/blob/main/docs/README.md">Contribute to the Docs</a></li>
<p>Gymnasium is a fork of <aclass="reference external"href="https://github.com/openai/gym/releases/tag/0.26.2">OpenAI Gym v0.26</a>, which introduced a large breaking change from <aclass="reference external"href="https://github.com/openai/gym/releases/tag/v0.21.0">Gym v0.21</a>.In this guide, we briefly outline the API changes from Gym v0.21 - which a number of tutorials have been written for - to Gym v0.26 (and later, including 1.0.0). For environments still stuck in the v0.21 API, see the <aclass="reference external"href="/content/gym_compatibility">guide</a></p>
<spanclass="n">action</span><spanclass="o">=</span><spanclass="n">env</span><spanclass="o">.</span><spanclass="n">action_space</span><spanclass="o">.</span><spanclass="n">sample</span><spanclass="p">()</span><spanclass="c1"># agent policy that uses the observation and info</span>
<h2>Example code for v0.26 and later, including v1.0.0<aclass="headerlink"href="#example-code-for-v0-26-and-later-including-v1-0-0"title="Link to this heading">¶</a></h2>
<spanclass="n">action</span><spanclass="o">=</span><spanclass="n">env</span><spanclass="o">.</span><spanclass="n">action_space</span><spanclass="o">.</span><spanclass="n">sample</span><spanclass="p">()</span><spanclass="c1"># agent policy that uses the observation and info</span>
<h2>Seed and random number generator<aclass="headerlink"href="#seed-and-random-number-generator"title="Link to this heading">¶</a></h2>
<p>The <codeclass="docutils literal notranslate"><spanclass="pre">Env.seed()</span></code> has been removed from the Gym v0.26 environments in favour of <codeclass="docutils literal notranslate"><spanclass="pre">Env.reset(seed=seed)</span></code>. This allows seeding to only be changed on environment reset. The decision to remove <codeclass="docutils literal notranslate"><spanclass="pre">seed</span></code> was because some environments use emulators that cannot change random number generators within an episode and must be done at the beginning of a new episode. We are aware of cases where controlling the random number generator is important, in these cases, if the environment uses the built-in random number generator, users can set the seed manually with the attribute <aclass="reference internal"href="../../api/env/#gymnasium.Env.np_random"title="gymnasium.Env.np_random"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">np_random</span></code></a>.</p>
<p>Gymnasium v0.26 changed to using <codeclass="docutils literal notranslate"><spanclass="pre">numpy.random.Generator</span></code> instead of a custom random number generator. This means that several functions such as <codeclass="docutils literal notranslate"><spanclass="pre">randint</span></code> were removed in favour of <codeclass="docutils literal notranslate"><spanclass="pre">integers</span></code>. While some environments might use external random number generator, we recommend using the attribute <aclass="reference internal"href="../../api/env/#gymnasium.Env.np_random"title="gymnasium.Env.np_random"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">np_random</span></code></a> that wrappers and external users can access and utilise.</p>
</section>
<sectionid="environment-reset">
<h2>Environment Reset<aclass="headerlink"href="#environment-reset"title="Link to this heading">¶</a></h2>
<p>In v0.26+, <aclass="reference internal"href="../../api/env/#gymnasium.Env.reset"title="gymnasium.Env.reset"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code></a> takes two optional parameters and returns one value. This contrasts to v0.21 which takes no parameters and returns <codeclass="docutils literal notranslate"><spanclass="pre">None</span></code>. The two parameters are <codeclass="docutils literal notranslate"><spanclass="pre">seed</span></code> for setting the random number generator and <codeclass="docutils literal notranslate"><spanclass="pre">options</span></code> which allows additional data to be passed to the environment on reset. For example, in classic control, the <codeclass="docutils literal notranslate"><spanclass="pre">options</span></code> parameter now allows users to modify the range of the state bound. See the original <aclass="reference external"href="https://github.com/openai/gym/pull/2921">PR</a> for more details.</p>
<p><aclass="reference internal"href="../../api/env/#gymnasium.Env.reset"title="gymnasium.Env.reset"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code></a> further returns <codeclass="docutils literal notranslate"><spanclass="pre">info</span></code>, similar to the <codeclass="docutils literal notranslate"><spanclass="pre">info</span></code> returned by <aclass="reference internal"href="../../api/env/#gymnasium.Env.step"title="gymnasium.Env.step"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code></a>. This is important because <codeclass="docutils literal notranslate"><spanclass="pre">info</span></code> can include metrics or valid action mask that is used or saved in the next step.</p>
<p>To update older environments, we highly recommend that <codeclass="docutils literal notranslate"><spanclass="pre">super().reset(seed=seed)</span></code> is called on the first line of <aclass="reference internal"href="../../api/env/#gymnasium.Env.reset"title="gymnasium.Env.reset"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code></a>. This will automatically update the <aclass="reference internal"href="../../api/env/#gymnasium.Env.np_random"title="gymnasium.Env.np_random"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">np_random</span></code></a> with the seed value.</p>
</section>
<sectionid="environment-step">
<h2>Environment Step<aclass="headerlink"href="#environment-step"title="Link to this heading">¶</a></h2>
<p>In v0.21, the type definition of <aclass="reference internal"href="../../api/env/#gymnasium.Env.step"title="gymnasium.Env.step"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code></a> is <codeclass="docutils literal notranslate"><spanclass="pre">tuple[ObsType,</span><spanclass="pre">SupportsFloat,</span><spanclass="pre">bool,</span><spanclass="pre">dict[str,</span><spanclass="pre">Any]</span></code> representing the next observation, the reward from the step, if the episode is done and additional info from the step. Due to reproducibility issues that will be expanded on in a blog post soon, we have changed the type definition to <codeclass="docutils literal notranslate"><spanclass="pre">tuple[ObsType,</span><spanclass="pre">SupportsFloat,</span><spanclass="pre">bool,</span><spanclass="pre">bool,</span><spanclass="pre">dict[str,</span><spanclass="pre">Any]]</span></code> adding an extra boolean value. This extra bool corresponds to the older <cite>done</cite> now changed to <cite>terminated</cite> and <cite>truncated</cite>. These changes were introduced in Gym <aclass="reference external"href="https://github.com/openai/gym/releases/tag/0.26.0">v0.26</a> (turned off by default in <aclass="reference external"href="https://github.com/openai/gym/releases/tag/0.25.0">v25</a>).</p>
<p>For users wishing to update, in most cases, replacing <codeclass="docutils literal notranslate"><spanclass="pre">done</span></code> with <codeclass="docutils literal notranslate"><spanclass="pre">terminated</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">truncated=False</span></code> in <aclass="reference internal"href="../../api/env/#gymnasium.Env.step"title="gymnasium.Env.step"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code></a> should address most issues. However, environments that have reasons for episode truncation rather than termination should read through the associated <aclass="reference external"href="https://github.com/openai/gym/pull/2752">PR</a>. For users looping through an environment, they should modify <codeclass="docutils literal notranslate"><spanclass="pre">done</span><spanclass="pre">=</span><spanclass="pre">terminated</span><spanclass="pre">or</span><spanclass="pre">truncated</span></code> as is show in the example code. For training libraries, the primary difference is to change <codeclass="docutils literal notranslate"><spanclass="pre">done</span></code> to <codeclass="docutils literal notranslate"><spanclass="pre">terminated</span></code>, indicating whether bootstrapping should or shouldn’t happen.</p>
</section>
<sectionid="timelimit-wrapper">
<h2>TimeLimit Wrapper<aclass="headerlink"href="#timelimit-wrapper"title="Link to this heading">¶</a></h2>
<p>In v0.21, the <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">TimeLimit</span></code> wrapper added an extra key in the <codeclass="docutils literal notranslate"><spanclass="pre">info</span></code> dictionary <codeclass="docutils literal notranslate"><spanclass="pre">TimeLimit.truncated</span></code> whenever the agent reached the time limit without reaching a terminal state.</p>
<p>In v0.26+, this information is instead communicated through the <cite>truncated</cite> return value described in the previous section, which is <cite>True</cite> if the agent reaches the time limit, whether or not it reaches a terminal state. The old dictionary entry is equivalent to <codeclass="docutils literal notranslate"><spanclass="pre">truncated</span><spanclass="pre">and</span><spanclass="pre">not</span><spanclass="pre">terminated</span></code></p>
<p>In v0.26, a new render API was introduced such that the render mode is fixed at initialisation as some environments don’t allow on-the-fly render mode changes. Therefore, users should now specify the <aclass="reference internal"href="../../api/env/#gymnasium.Env.render_mode"title="gymnasium.Env.render_mode"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">render_mode</span></code></a> within <codeclass="docutils literal notranslate"><spanclass="pre">gym.make</span></code> as shown in the v0.26+ example code above.</p>
<p>For a more complete explanation of the changes, please refer to this <aclass="reference external"href="https://younis.dev/blog/render-api/">summary</a>.</p>
</section>
<sectionid="removed-code">
<h2>Removed code<aclass="headerlink"href="#removed-code"title="Link to this heading">¶</a></h2>
<ulclass="simple">
<li><p>GoalEnv - This was removed, users needing it should reimplement the environment or use Gymnasium Robotics which contains an implementation of this environment.</p></li>
<li><p><codeclass="docutils literal notranslate"><spanclass="pre">from</span><spanclass="pre">gym.envs.classic_control</span><spanclass="pre">import</span><spanclass="pre">rendering</span></code> - This was removed in favour of users implementing their own rendering systems. Gymnasium environments are coded using pygame.</p></li>
<li><p>Robotics environments - The robotics environments have been moved to the <aclass="reference external"href="https://robotics.farama.org/">Gymnasium Robotics</a> project.</p></li>
<li><p>Monitor wrapper - This wrapper was replaced with two separate wrapper, <aclass="reference internal"href="../../api/wrappers/misc_wrappers/#gymnasium.wrappers.RecordVideo"title="gymnasium.wrappers.RecordVideo"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">RecordVideo</span></code></a> and <aclass="reference internal"href="../../api/wrappers/misc_wrappers/#gymnasium.wrappers.RecordEpisodeStatistics"title="gymnasium.wrappers.RecordEpisodeStatistics"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">RecordEpisodeStatistics</span></code></a></p></li>
