gaspersic/Gymnasium
1
0
Fork 0
mirror of https://github.com/Farama-Foundation/Gymnasium.git synced 2025-08-02 06:16:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update Writing in Docs (#84)

Browse Source
This commit is contained in:
Jet
2022-10-27 16:53:03 +01:00
committed by GitHub
parent cad6746ca6
commit 37125d4930
2 changed files with 44 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
docs/content/gym_compatibility.md
View File

@@ -5,17 +5,19 @@ title: Compatibility With Gym
# Compatibility with Gym
Gymnasium provides a number of compatibility methods for a range of Environment implementations.
Gymnasium provides a number of compatibility methods for a range of Environment implementations.
## Loading OpenAI Gym environments
```{eval-rst}
.. py:currentmodule:: gymnasium.wrappers
For environments that are registered solely in OpenAI Gym, it is still possible to import environments within Gymnasium however they will not appear in the gymnasium environment registry. Introduced in Gymnasium v0.26.3, using the special environment ``"GymV26Environment-v0"``, passing an ``env_name`` along with any other keyword will be passed to ``gym.make``. This environment, :class:`EnvCompatibility`, is also compatibility with passing gym environment instances with the ``env`` keyword.
For environments that are registered solely in OpenAI Gym and not in Gymnasium, Gymnasium v0.26.3 and above allows importing them through either a special environment or a wrapper.
The ``"GymV26Environment-v0"`` environment was introduced in Gymnasium v0.26.3, and allows importing of Gym environments through the ``env_name`` argument along with other relevant kwargs environment kwargs.
To perform conversion through a wrapper, the environment itself can be passed to the wrapper :class:`EnvCompatibility` through the ``env`` kwarg.
```
An example of this is atari 0.8.0 which does not have a gymnasium implementation.
An example of this is atari 0.8.0 which does not have a gymnasium implementation.
```python
import gymnasium as gym
env = gym.make("GymV26Environment-v0", env_id="ALE/Pong-v5")
@@ -26,7 +28,9 @@ env = gym.make("GymV26Environment-v0", env_id="ALE/Pong-v5")
```{eval-rst}
.. py:currentmodule:: gymnasium
A number of environments have not updated to the recent Gym changes, in particular since v0.21. Therefore, to increase backward compatibility, Gym and Gymnasium v0.26+ include an ``apply_api_compatibility`` in :meth:`make` parameter that applies a wrappers to convert v0.21 environment to the v0.26 API.
A number of environments have not updated to the recent Gym changes, in particular since v0.21.
This update is significant for the introduction of ``termination`` and ``truncation`` signatures in favour of the previously used ``done``.
To allow backward compatibility, Gym and Gymnasium v0.26+ include an ``apply_api_compatibility`` kwarg when calling :meth:`make` that automatically converts a v0.21 API compliant environment one that is compatible with v0.26+.
```
```python
@@ -34,8 +38,8 @@ import gym
env = gym.make("OldV21Env-v0", apply_api_compatibility=True)
```
## Step API Compatibility
## Step API Compatibility
```{eval-rst}
If environments implement the (old) done step API, Gymnasium provides both functions (:meth:`gymnasium.utils.step_api_compatibility.convert_to_terminated_truncated_step_api`) and wrappers (:class:`gymnasium.wrappers.StepAPICompatibility`) that will convert the step function to the (new) termination and truncation step API.
If environments implement the (old) done step API, Gymnasium provides both functions (:meth:`gymnasium.utils.step_api_compatibility.convert_to_terminated_truncated_step_api`) and wrappers (:class:`gymnasium.wrappers.StepAPICompatibility`) that will convert an environment with the old step API (using ``done``) to the new step API (using ``termination`` and ``truncation``).
```

47
docs/content/migration-guide.md
View File

@@ -8,7 +8,10 @@ title: Migration Guide
```{eval-rst}
.. py:currentmodule:: gymnasium.wrappers
Gymnasium is a fork of `OpenAI Gym v26 <https://github.com/openai/gym/releases/tag/0.26.2>`_, therefore, this requires environments and training libraries to update to the v26 API. In this guide, we briefly outline the changes to the `Gym v21 <https://github.com/openai/gym/releases/tag/v0.21.0>`_ API which a number of tutorials and environment have been written in. For environments that have not updated, users can use the :class:`EnvCompatibility` wrapper, for more information see the `guide </content/gym_compatibility>`_
Gymnasium is a fork of `OpenAI Gym v26 <https://github.com/openai/gym/releases/tag/0.26.2>`_, which introduced a large breaking change from `Gym v21 <https://github.com/openai/gym/releases/tag/v0.21.0>`_.
In this guide, we briefly outline the API changes from Gym v21 - which a number of tutorials have been written for - to Gym v26.
For environments still stuck in the v21 API, users can use the :class:`EnvCompatibility` wrapper to convert them to v26 compliant.
For more information, see the `guide </content/gym_compatibility>`_
```
### Example code for v21
@@ -22,7 +25,7 @@ done = False
while not done:
action = env.action_space.sample() # agent policy that uses the observation and info
observation, reward, done, info = env.step(action)
env.render(mode="human")
env.close()
@@ -49,33 +52,50 @@ env.close()
```{eval-rst}
.. py:currentmodule:: gymnasium.Env
The ``Env.seed()`` has been removed from the Gym v26 environments in favour of ``Env.reset(seed=seed)`` as a majority of the time, users call seed just before resetting the environment. This decision to remove ``seed`` was due to environment that use emulators often cannot change the random number generator within an episode and must be done at the beginning of a new episode. We are aware of cases where controlling the random number generator is important, in these cases, if the environment uses the built in random number generator, users can set :attr:`np_random`.
The ``Env.seed()`` has been removed from the Gym v26 environments in favour of ``Env.reset(seed=seed)``.
This allows seeding to only be changed on environment reset.
The decision to remove ``seed`` was because some environments use emulators that cannot change random number generators within an episode and must be done at the beginning of a new episode.
We are aware of cases where controlling the random number generator is important, in these cases, if the environment uses the built in random number generator, users can set the seed manually with the attribute :attr:`np_random`.
Gymnasium v26 changed to using the ``numpy.random.Generator`` instead of a custom random number generator. This means that several functions such as ``randint`` were removed in favour of ``integers``. While some environments might use external random number generator, we recommend using the :attr:`np_random` that wrappers and external users can access and utilise.
Gymnasium v26 changed to using ``numpy.random.Generator`` instead of a custom random number generator.
This means that several functions such as ``randint`` were removed in favour of ``integers``.
While some environments might use external random number generator, we recommend using the attribute :attr:`np_random` that wrappers and external users can access and utilise.
```
## Environment Reset
```{eval-rst}
In v26, :meth:`reset` has two new parameters compared to v21 along with an extra return value. The two parameters of ``reset`` are ``seed`` for setting the random number generator with the second parameter being ``options`` allowing additional data to be passed to the environment on reset. For example, in the classic control, the options parameter now allows users to modify the range of the state bound. See the original `PR <https://github.com/openai/gym/pull/2921>`_ for more details.
In v26, :meth:`reset` takes two optional parameters and returns one value.
This contrasts to v21 which takes no parameters and returns ``None``.
The two parameters are ``seed`` for setting the random number generator and ``options`` which allows additional data to be passed to the environment on reset.
For example, in classic control, the options parameter now allows users to modify the range of the state bound.
See the original `PR <https://github.com/openai/gym/pull/2921>`_ for more details.
For the new return value, ``info``, this is similar to the ``info`` returned by the :meth:`step`. This is important for ``info`` can include metrics or valid action mask that is used or saved in the next step.
:meth:`reset` further returns ``info``, similar to the ``info`` returned by :meth:`step`.
This is important because ``info`` can include metrics or valid action mask that is used or saved in the next step.
To update environments, we highly recommend that the first line of the environment :meth:`reset` function is ``super().reset(seed=seed)`` which will automatically update the :attr:`np_random` with the seed value.
To update older environments, we highly recommend that ``super().reset(seed=seed)`` is called on the first line of :meth:`reset`.
This will automatically update the :attr:`np_random` with the seed value.
```
## Environment Step
## Environment Step
```{eval-rst}
In v21, the type definition of :meth:`step` is ``tuple[ObsType, SupportsFloat, bool, dict[str, Any]`` representing the next observation, the reward from the step, if the episode is done and additional info from the step. Due to reproductibility issues that will be expanded on a blog post soon to be published, we have changed the type definition to ``tuple[ObsType, SupportsFloat, bool, bool, dict[str, Any]]`` adding an extra boolean value. This is as the ``done`` value has been replaced with two variables, `terminated` and `truncated`. These changes were introduced in Gym `v26 <https://github.com/openai/gym/releases/tag/0.26.0>`_ (turned off by default in `v25 <https://github.com/openai/gym/releases/tag/0.25.0>`_).
In v21, the type definition of :meth:`step` is ``tuple[ObsType, SupportsFloat, bool, dict[str, Any]`` representing the next observation, the reward from the step, if the episode is done and additional info from the step.
Due to reproductibility issues that will be expanded on in a blog post soon, we have changed the type definition to ``tuple[ObsType, SupportsFloat, bool, bool, dict[str, Any]]`` adding an extra boolean value.
This extra bool corresponds to the older `done` now changed to `terminated` and `truncated`.
These changes were introduced in Gym `v26 <https://github.com/openai/gym/releases/tag/0.26.0>`_ (turned off by default in `v25 <https://github.com/openai/gym/releases/tag/0.25.0>`_).
For users wishing to update, in most cases, replacing ``done`` with ``terminated`` and ``truncated=False`` in environment should address most environments. However, for environments that have custom reasons for an episode to truncate rather than terminate should read through the associated `PR <https://github.com/openai/gym/pull/2752>`_. For users looping through an environment, they should modify ``done = terminated or truncated`` as is show in the example code. For training libraries, primarily requires changing ``done`` to ``terminated``, indicating that bootstraping should or shouldn't happen.
For users wishing to update, in most cases, replacing ``done`` with ``terminated`` and ``truncated=False`` in :meth:`step` should address most issues.
However, environments that have reasons for episode truncation rather than termination should read through the associated `PR <https://github.com/openai/gym/pull/2752>`_.
For users looping through an environment, they should modify ``done = terminated or truncated`` as is show in the example code.
For training libraries, the primary difference is to change ``done`` to ``terminated``, indicating whether bootstraping should or shouldn't happen.
```
## Environment Render
```{eval-rst}
In v26, a new render API was introduced such that the render mode is fixed at initialisation as some environment don't allow the render mode to change on fly or allow the render mode to pre-compute data at initialisation. Therefore, users should now specify the render mode within ``gym.make`` as shown in the v26 example code above.
In v26, a new render API was introduced such that the render mode is fixed at initialisation as some environments don't allow on-the-fly render mode changes. Therefore, users should now specify the :attr:`render_mode` within ``gym.make`` as shown in the v26 example code above.
For a more complete explanation of the changes, please refer to this `summary <https://younis.dev/blog/render-api/>`_.
```
@@ -86,7 +106,8 @@ For a more complete explanation of the changes, please refer to this `summary <h
.. py:currentmodule:: gymnasium.wrappers
* GoalEnv - This was removed, users needing it should reimplement the environment or use Gymnasium Robotics which contains an implementation of this environment.
* ``from gym.envs.classic_control import rendering`` - This was removed in favour of users implementing their own rendering systems. Gymnasium environments are coded using pygame.
* Robotics environments - The robotics environments have been moved to the `Gymnasium Robotics <https://robotics.farama.org/>`_ project.
* ``from gym.envs.classic_control import rendering`` - This was removed in favour of users implementing their own rendering systems.
Gymnasium environments are coded using pygame.
* Robotics environments - The robotics environments have been moved to the `Gymnasium Robotics <https://robotics.farama.org/>`_ project.
* Monitor wrapper - This wrapper was replaced with two separate wrapper, :class:`RecordVideo` and :class:`RecordEpisodeStatistics`
```