<spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.Env.</span></span><spanclass="sig-name descname"><spanclass="pre">step</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">self</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">action</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">ActType</span></span></em><spanclass="sig-paren">)</span><spanclass="sig-return"><spanclass="sig-return-icon">→</span><spanclass="sig-return-typehint"><spanclass="pre">Tuple</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">ObsType</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="w"></span><spanclass="pre">float</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="w"></span><spanclass="pre">bool</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="w"></span><spanclass="pre">bool</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="w"></span><spanclass="pre">dict</span><spanclass="p"><spanclass="pre">]</span></span></span></span><aclass="headerlink"href="#gymnasium.Env.step"title="Permalink to this definition">#</a></dt>
<dd><p>Run one timestep of the environment’s dynamics.</p>
<p>When end of episode is reached, you are responsible for calling <aclass="reference internal"href="#gymnasium.Env.reset"title="gymnasium.Env.reset"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code></a> to reset this environment’s state.
Accepts an action and returns either a tuple <cite>(observation, reward, terminated, truncated, info)</cite>.</p>
<li><p><strong>observation</strong> (<em>object</em>) – this will be an element of the environment’s <aclass="reference internal"href="#gymnasium.Env.observation_space"title="gymnasium.Env.observation_space"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">observation_space</span></code></a>.
This may, for instance, be a numpy array containing the positions and velocities of certain objects.</p></li>
<li><p><strong>reward</strong> (<em>float</em>) – The amount of reward returned as a result of taking the action.</p></li>
<li><p><strong>terminated</strong> (<em>bool</em>) – whether a <cite>terminal state</cite> (as defined under the MDP of the task) is reached.
In this case further step() calls could return undefined results.</p></li>
<li><p><strong>truncated</strong> (<em>bool</em>) – whether a truncation condition outside the scope of the MDP is satisfied.
Typically a timelimit, but could also be used to indicate agent physically going out of bounds.
Can be used to end the episode prematurely before a <cite>terminal state</cite> is reached.</p></li>
<li><p><strong>info</strong> (<em>dictionary</em>) –<cite>info</cite> contains auxiliary diagnostic information (helpful for debugging, learning, and logging).
This might, for instance, contain: metrics that describe the agent’s performance state, variables that are
hidden from observations, or individual reward terms that are combined to produce the total reward.
It also can contain information that distinguishes truncation and termination, however this is deprecated in favour
of returning two booleans, and will be removed in a future version.</p></li>
<li><p><strong>done</strong> (<em>bool</em>) – (Deprecated) A boolean value for if the episode has ended, in which case further <aclass="reference internal"href="#gymnasium.Env.step"title="gymnasium.Env.step"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code></a> calls will
return undefined results.
A done signal may be emitted for different reasons: Maybe the task underlying the environment was solved successfully,
a certain timelimit was exceeded, or the physics simulation has entered an invalid state.</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.Env.</span></span><spanclass="sig-name descname"><spanclass="pre">reset</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">self</span></span></em>, <emclass="sig-param"><spanclass="o"><spanclass="pre">*</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">seed</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">Optional</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">int</span><spanclass="p"><spanclass="pre">]</span></span></span><spanclass="w"></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="w"></span><spanclass="default_value"><spanclass="pre">None</span></span></em>, <emclass="sig-param"><spanclass="n"><spanclass="pre">options</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">Optional</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">dict</span><spanclass="p"><spanclass="pre">]</span></span></span><spanclass="w"></span><spanclass="o"><spanclass="pre">=</span></span><spanclass="w"></span><spanclass="default_value"><spanclass="pre">None</span></span></em><spanclass="sig-paren">)</span><spanclass="sig-return"><spanclass="sig-return-icon">→</span><spanclass="sig-return-typehint"><spanclass="pre">Tuple</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">ObsType</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="w"></span><spanclass="pre">dict</span><spanclass="p"><spanclass="pre">]</span></span></span></span><aclass="headerlink"href="#gymnasium.Env.reset"title="Permalink to this definition">#</a></dt>
<dd><p>Resets the environment to an initial state and returns the initial observation.</p>
<p>This method can reset the environment’s random number generator(s) if <codeclass="docutils literal notranslate"><spanclass="pre">seed</span></code> is an integer or
if the environment has not yet initialized a random number generator.
If the environment already has a random number generator and <aclass="reference internal"href="#gymnasium.Env.reset"title="gymnasium.Env.reset"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code></a> is called with <codeclass="docutils literal notranslate"><spanclass="pre">seed=None</span></code>,
the RNG should not be reset. Moreover, <aclass="reference internal"href="#gymnasium.Env.reset"title="gymnasium.Env.reset"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code></a> should (in the typical use case) be called with an
integer seed right after initialization and then never again.</p>
<li><p><strong>seed</strong> (<em>optional int</em>) – The seed that is used to initialize the environment’s PRNG.
If the environment does not already have a PRNG and <codeclass="docutils literal notranslate"><spanclass="pre">seed=None</span></code> (the default option) is passed,
a seed will be chosen from some source of entropy (e.g. timestamp or /dev/urandom).
However, if the environment already has a PRNG and <codeclass="docutils literal notranslate"><spanclass="pre">seed=None</span></code> is passed, the PRNG will <em>not</em> be reset.
If you pass an integer, the PRNG will be reset even if it already exists.
Usually, you want to pass an integer <em>right after the environment has been initialized and then never again</em>.
Please refer to the minimal example above to see this paradigm in action.</p></li>
<li><p><strong>options</strong> (<em>optional dict</em>) – Additional information to specify how the environment is reset (optional,
<li><p><strong>observation</strong> (<em>object</em>) – Observation of the initial state. This will be an element of <aclass="reference internal"href="#gymnasium.Env.observation_space"title="gymnasium.Env.observation_space"><codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">observation_space</span></code></a>
(typically a numpy array) and is analogous to the observation returned by <aclass="reference internal"href="#gymnasium.Env.step"title="gymnasium.Env.step"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code></a>.</p></li>
<li><p><strong>info</strong> (<em>dictionary</em>) – This dictionary contains auxiliary information complementing <codeclass="docutils literal notranslate"><spanclass="pre">observation</span></code>. It should be analogous to
the <codeclass="docutils literal notranslate"><spanclass="pre">info</span></code> returned by <aclass="reference internal"href="#gymnasium.Env.step"title="gymnasium.Env.step"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code></a>.</p></li>
<spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.Env.</span></span><spanclass="sig-name descname"><spanclass="pre">render</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">self</span></span></em><spanclass="sig-paren">)</span><spanclass="sig-return"><spanclass="sig-return-icon">→</span><spanclass="sig-return-typehint"><spanclass="pre">Optional</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">Union</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">RenderFrame</span><spanclass="p"><spanclass="pre">,</span></span><spanclass="w"></span><spanclass="pre">List</span><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">RenderFrame</span><spanclass="p"><spanclass="pre">]</span></span><spanclass="p"><spanclass="pre">]</span></span><spanclass="p"><spanclass="pre">]</span></span></span></span><aclass="headerlink"href="#gymnasium.Env.render"title="Permalink to this definition">#</a></dt>
<dd><p>Compute the render frames as specified by render_mode attribute during initialization of the environment.</p>
<p>The set of supported modes varies per environment. (And some
third-party environments may not support rendering at all.)
By convention, if render_mode is:</p>
<ulclass="simple">
<li><p>None (default): no render is computed.</p></li>
<li><p>human: render return None.
The environment is continuously rendered in the current display or terminal. Usually for human consumption.</p></li>
<li><p>rgb_array: return a single frame representing the current state of the environment.
A frame is a numpy.ndarray with shape (x, y, 3) representing RGB values for an x-by-y pixel image.</p></li>
<li><p>rgb_array_list: return a list of frames representing the states of the environment since the last reset.
Each frame is a numpy.ndarray with shape (x, y, 3), as with <cite>rgb_array</cite>.</p></li>
<li><p>ansi: Return a strings (str) or StringIO.StringIO containing a
terminal-style text representation for each time step.
The text can include newlines and ANSI escape sequences (e.g. for colors).</p></li>
</ul>
<divclass="admonition note">
<pclass="admonition-title">Note</p>
<p>Make sure that your class’s metadata ‘render_modes’ key includes
the list of supported modes. It’s recommended to call super()
in implementations to use the functionality of this method.</p>
</div>
</dd></dl>
<sectionid="attributes">
<h3>Attributes<aclass="headerlink"href="#attributes"title="Permalink to this heading">#</a></h3>
<spanclass="sig-prename descclassname"><spanclass="pre">Env.</span></span><spanclass="sig-name descname"><spanclass="pre">action_space</span></span><emclass="property"><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><aclass="reference internal"href="../spaces/#gymnasium.spaces.Space"title="gymnasium.spaces.space.Space"><spanclass="pre">Space</span></a><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">ActType</span><spanclass="p"><spanclass="pre">]</span></span></em><aclass="headerlink"href="#gymnasium.Env.action_space"title="Permalink to this definition">#</a></dt>
<dd><p>This attribute gives the format of valid actions. It is of datatype <cite>Space</cite> provided by Gymnasium. For example, if the action space is of type <cite>Discrete</cite> and gives the value <cite>Discrete(2)</cite>, this means there are two valid discrete actions: 0 & 1.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">Env.</span></span><spanclass="sig-name descname"><spanclass="pre">observation_space</span></span><emclass="property"><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><aclass="reference internal"href="../spaces/#gymnasium.spaces.Space"title="gymnasium.spaces.space.Space"><spanclass="pre">Space</span></a><spanclass="p"><spanclass="pre">[</span></span><spanclass="pre">ObsType</span><spanclass="p"><spanclass="pre">]</span></span></em><aclass="headerlink"href="#gymnasium.Env.observation_space"title="Permalink to this definition">#</a></dt>
<dd><p>This attribute gives the format of valid observations. It is of datatype <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">Space</span></code> provided by Gymnasium. For example, if the observation space is of type <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">Box</span></code> and the shape of the object is <codeclass="docutils literal notranslate"><spanclass="pre">(4,)</span></code>, this denotes a valid observation will be an array of 4 numbers. We can check the box bounds as well with attributes.</p>
<spanclass="sig-prename descclassname"><spanclass="pre">Env.</span></span><spanclass="sig-name descname"><spanclass="pre">reward_range</span></span><emclass="property"><spanclass="w"></span><spanclass="p"><spanclass="pre">=</span></span><spanclass="w"></span><spanclass="pre">(-inf,</span><spanclass="pre">inf)</span></em><aclass="headerlink"href="#gymnasium.Env.reward_range"title="Permalink to this definition">#</a></dt>
<dd><p>This attribute is a tuple corresponding to min and max possible rewards. Default range is set to <codeclass="docutils literal notranslate"><spanclass="pre">(-inf,+inf)</span></code>. You can set it if you want a narrower range.</p>
</dd></dl>
</section>
<sectionid="additional-methods">
<h3>Additional Methods<aclass="headerlink"href="#additional-methods"title="Permalink to this heading">#</a></h3>
<spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.Env.</span></span><spanclass="sig-name descname"><spanclass="pre">close</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">self</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#gymnasium.Env.close"title="Permalink to this definition">#</a></dt>
<dd><p>Override close in your subclass to perform any necessary cleanup.</p>
<p>Environments will automatically <aclass="reference internal"href="#gymnasium.Env.close"title="gymnasium.Env.close"><codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">close()</span></code></a> themselves when
garbage collected or when the program exits.</p>
</dd></dl>
</section>
</section>
<sectionid="gymnasium-wrapper">
<h2>gymnasium.Wrapper<aclass="headerlink"href="#gymnasium-wrapper"title="Permalink to this heading">#</a></h2>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.</span></span><spanclass="sig-name descname"><spanclass="pre">Wrapper</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">env</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">Env</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#gymnasium.Wrapper"title="Permalink to this definition">#</a></dt>
<dd><p>Wraps an environment to allow a modular transformation of the <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code> and <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code> methods.</p>
<p>This class is the base class for all wrappers. The subclass could override
some methods to change the behavior of the original environment without touching the
original code.</p>
<divclass="admonition note">
<pclass="admonition-title">Note</p>
<p>Don’t forget to call <codeclass="docutils literal notranslate"><spanclass="pre">super().__init__(env)</span></code> if the subclass overrides <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">__init__()</span></code>.</p>
</div>
<p>Wraps an environment to allow a modular transformation of the <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code> and <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code> methods.</p>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.</span></span><spanclass="sig-name descname"><spanclass="pre">ObservationWrapper</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">env</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">Env</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#gymnasium.ObservationWrapper"title="Permalink to this definition">#</a></dt>
<dd><p>Superclass of wrappers that can modify observations using <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">observation()</span></code> for <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code> and <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code>.</p>
<p>If you would like to apply a function to the observation that is returned by the base environment before
passing it to learning code, you can simply inherit from <aclass="reference internal"href="#gymnasium.ObservationWrapper"title="gymnasium.ObservationWrapper"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">ObservationWrapper</span></code></a> and overwrite the method
<codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">observation()</span></code> to implement that transformation. The transformation defined in that method must be
defined on the base environment’s observation space. However, it may take values in a different space.
In that case, you need to specify the new observation space of the wrapper by setting <codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">self.observation_space</span></code>
in the <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">__init__()</span></code> method of your wrapper.</p>
<p>For example, you might have a 2D navigation task where the environment returns dictionaries as observations with
keys <codeclass="docutils literal notranslate"><spanclass="pre">"agent_position"</span></code> and <codeclass="docutils literal notranslate"><spanclass="pre">"target_position"</span></code>. A common thing to do might be to throw away some degrees of
freedom and only consider the position of the target relative to the agent, i.e.
<codeclass="docutils literal notranslate"><spanclass="pre">observation["target_position"]</span><spanclass="pre">-</span><spanclass="pre">observation["agent_position"]</span></code>. For this, you could implement an
<p>Among others, Gymnasium provides the observation wrapper <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">TimeAwareObservation</span></code>, which adds information about the
index of the timestep to the observation.</p>
<p>Wraps an environment to allow a modular transformation of the <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code> and <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code> methods.</p>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.</span></span><spanclass="sig-name descname"><spanclass="pre">RewardWrapper</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">env</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">Env</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#gymnasium.RewardWrapper"title="Permalink to this definition">#</a></dt>
<dd><p>Superclass of wrappers that can modify the returning reward from a step.</p>
<p>If you would like to apply a function to the reward that is returned by the base environment before
passing it to learning code, you can simply inherit from <aclass="reference internal"href="#gymnasium.RewardWrapper"title="gymnasium.RewardWrapper"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">RewardWrapper</span></code></a> and overwrite the method
<codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reward()</span></code> to implement that transformation.
This transformation might change the reward range; to specify the reward range of your wrapper,
you can simply define <codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">self.reward_range</span></code> in <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">__init__()</span></code>.</p>
<p>Let us look at an example: Sometimes (especially when we do not have control over the reward
because it is intrinsic), we want to clip the reward to a range to gain some numerical stability.
To do that, we could, for instance, implement the following wrapper:</p>
<p>Wraps an environment to allow a modular transformation of the <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code> and <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code> methods.</p>
<emclass="property"><spanclass="pre">class</span><spanclass="w"></span></em><spanclass="sig-prename descclassname"><spanclass="pre">gymnasium.</span></span><spanclass="sig-name descname"><spanclass="pre">ActionWrapper</span></span><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n"><spanclass="pre">env</span></span><spanclass="p"><spanclass="pre">:</span></span><spanclass="w"></span><spanclass="n"><spanclass="pre">Env</span></span></em><spanclass="sig-paren">)</span><aclass="headerlink"href="#gymnasium.ActionWrapper"title="Permalink to this definition">#</a></dt>
<dd><p>Superclass of wrappers that can modify the action before <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">env.step()</span></code>.</p>
<p>If you would like to apply a function to the action before passing it to the base environment,
you can simply inherit from <aclass="reference internal"href="#gymnasium.ActionWrapper"title="gymnasium.ActionWrapper"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">ActionWrapper</span></code></a> and overwrite the method <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">action()</span></code> to implement
that transformation. The transformation defined in that method must take values in the base environment’s
action space. However, its domain might differ from the original action space.
In that case, you need to specify the new action space of the wrapper by setting <codeclass="xref py py-attr docutils literal notranslate"><spanclass="pre">self.action_space</span></code> in
the <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">__init__()</span></code> method of your wrapper.</p>
<p>Let’s say you have an environment with action space of type <aclass="reference internal"href="../spaces/fundamental/#gymnasium.spaces.Box"title="gymnasium.spaces.Box"><codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">gymnasium.spaces.Box</span></code></a>, but you would only like
<p>Among others, Gymnasium provides the action wrappers <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">ClipAction</span></code> and <codeclass="xref py py-class docutils literal notranslate"><spanclass="pre">RescaleAction</span></code>.</p>
<p>Wraps an environment to allow a modular transformation of the <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">step()</span></code> and <codeclass="xref py py-meth docutils literal notranslate"><spanclass="pre">reset()</span></code> methods.</p>
