Update and rerun pre-commit hooks for better code quality (#179)

.github/ISSUE_TEMPLATE/question.yml

@@ -6,13 +6,12 @@ body:
   - type: markdown
     attributes:
       value: >
-        If you're a beginner and have basic questions, please ask on 
-        [r/reinforcementlearning](https://www.reddit.com/r/reinforcementlearning/) or in the 
-        [RL Discord](https://discord.com/invite/xhfNqQv) (if you're new please use the beginners channel). 
-        Basic questions that are not bugs or feature requests will be closed without reply, because GitHub 
-        issues are not an appropriate venue for these. Advanced/nontrivial questions, especially in areas where 
+        If you're a beginner and have basic questions, please ask on
+        [r/reinforcementlearning](https://www.reddit.com/r/reinforcementlearning/) or in the
+        [RL Discord](https://discord.com/invite/xhfNqQv) (if you're new please use the beginners channel).
+        Basic questions that are not bugs or feature requests will be closed without reply, because GitHub
+        issues are not an appropriate venue for these. Advanced/nontrivial questions, especially in areas where
         documentation is lacking, are very much welcome.
-        
   - type: textarea
     id: question
     attributes:

.github/PULL_REQUEST_TEMPLATE.md

@@ -14,6 +14,7 @@ Please delete options that are not relevant.
 - [ ] This change requires a documentation update
 
 ### Screenshots
+
 Please attach before and after screenshots of the change if applicable.
 
 <!--

.github/stale.yml

@@ -59,4 +59,4 @@ only: issues
 
 # issues:
 #   exemptLabels:
-#     - confirmed
+#     - confirmed

.github/workflows/build-docs.yml

@@ -42,4 +42,4 @@ jobs:
         with:
           folder: _build
           target-folder: main
-          clean: false
+          clean: false

.github/workflows/docs-manual-versioning.yml

@@ -66,4 +66,4 @@ jobs:
           folder: _build
           clean-exclude: |
             *.*.*/
-            main
+            main

.github/workflows/docs-versioning.yml

@@ -56,4 +56,4 @@ jobs:
           folder: _build
           clean-exclude: |
             *.*.*/
-            main
+            main

.gitignore

@@ -4,13 +4,17 @@
 .DS_Store
 .cache
 .pytest_cache/
+__pycache__/
 
 # Setuptools distribution and build folders.
 /dist/
 /build
+/wheels
+/wheelhouse
 
 # Virtualenv
 /env
+/venv
 
 # Python egg metadata, regenerated from source files by setuptools.
 /*.egg-info

.pre-commit-config.yaml

@@ -1,17 +1,30 @@
----
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
 repos:
-  - repo: https://github.com/python/black
-    rev: 22.3.0
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
     hooks:
-      - id: black
+      - id: check-symlinks
+      - id: destroyed-symlinks
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-toml
+      - id: check-ast
+      - id: check-added-large-files
+      - id: check-merge-conflict
+      - id: check-executables-have-shebangs
+      - id: check-shebang-scripts-are-executable
+      - id: detect-private-key
+      - id: debug-statements
   - repo: https://github.com/codespell-project/codespell
-    rev: v2.1.0
+    rev: v2.2.2
     hooks:
       - id: codespell
         args:
           - --ignore-words-list=nd,reacher,thist,ths, ure, referenc,wile
   - repo: https://github.com/PyCQA/flake8
-    rev: 4.0.1
+    rev: 6.0.0
     hooks:
       - id: flake8
         args:
@@ -21,11 +34,19 @@ repos:
           - --max-line-length=456
           - --show-source
           - --statistics
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v3.3.0
+    hooks:
+      - id: pyupgrade
+        args: ["--py37-plus"]
   - repo: https://github.com/PyCQA/isort
     rev: 5.10.1
     hooks:
       - id: isort
-        args: ["--profile", "black"]
+  - repo: https://github.com/python/black
+    rev: 22.10.0
+    hooks:
+      - id: black
   - repo: https://github.com/pycqa/pydocstyle
     rev: 6.1.1
     hooks:
@@ -36,11 +57,6 @@ repos:
           - --explain
           - --convention=google
         additional_dependencies: ["toml"]
-  - repo: https://github.com/asottile/pyupgrade
-    rev: v2.32.0
-    hooks:
-      - id: pyupgrade
-        args: ["--py37-plus"]
   - repo: local
     hooks:
       - id: pyright

CODE_OF_CONDUCT.rst

@@ -65,4 +65,3 @@ Attribution
 -----------
 This Code of Conduct is adapted from `Python's Code of Conduct <https://www.python.org/psf/conduct/>`_, which is under a `Creative Commons License
 <https://creativecommons.org/licenses/by-sa/3.0/>`_.
-

CONTRIBUTING.md

@@ -12,33 +12,39 @@ Notably, we are not accepting these forms of contributions:
 - New environments
 
 # Development
+
 This section contains technical instructions & hints for the contributors.
 
 ## Type checking
-The project uses `pyright` to check types. 
-To type check locally, install `pyright` per official [instructions](https://github.com/microsoft/pyright#command-line). 
+
+The project uses `pyright` to check types.
+To type check locally, install `pyright` per official [instructions](https://github.com/microsoft/pyright#command-line).
 It's configuration lives within `pyproject.toml`. It includes list of included and excluded files currently supporting type checks.
 To run `pyright` for the project, run the pre-commit process (`pre-commit run --all-files`) or `pyright --project=pyproject.toml`
 Alternatively, pyright is a built-in feature of VSCode that will automatically provide type hinting.
 
 ### Adding typing to more modules and packages
-If you would like to add typing to a module in the project, 
-the list of included, excluded and strict files can be found in pyproject.toml (pyproject.toml -> [tool.pyright]). 
+
+If you would like to add typing to a module in the project,
+the list of included, excluded and strict files can be found in pyproject.toml (pyproject.toml -> [tool.pyright]).
 To run `pyright` for the project, run the pre-commit process (`pre-commit run --all-files`) or `pyright`
 
 ## Git hooks
+
 The CI will run several checks on the new code pushed to the Gymnasium repository. These checks can also be run locally without waiting for the CI by following the steps below:
+
 1. [install `pre-commit`](https://pre-commit.com/#install),
 2. Install the Git hooks by running `pre-commit install`.
 
-Once those two steps are done, the Git hooks will be run automatically at every new commit. 
-The Git hooks can also be run manually with `pre-commit run --all-files`, and if needed they can be skipped (not recommended) with `git commit --no-verify`. 
+Once those two steps are done, the Git hooks will be run automatically at every new commit.
+The Git hooks can also be run manually with `pre-commit run --all-files`, and if needed they can be skipped (not recommended) with `git commit --no-verify`.
 **Note:** you may have to run `pre-commit run --all-files` manually a couple of times to make it pass when you commit, as each formatting tool will first format the code and fail the first time but should pass the second time.
 
 Additionally, for pull requests, the project runs a number of tests for the whole project using [pytest](https://docs.pytest.org/en/latest/getting-started.html#install-pytest).
-These tests can be run locally with `pytest` in the root folder. 
+These tests can be run locally with `pytest` in the root folder.
 
 ## Docstrings
+
 Pydocstyle has been added to the pre-commit process such that all new functions follow the [google docstring style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html).
 All new functions require either a short docstring, a single line explaining the purpose of a function
 or a multiline docstring that documents each argument and the return type (if there is one) of the function.
@@ -46,17 +52,22 @@ In addition, new file and class require top docstrings that should outline the p
 For classes, code block examples can be provided in the top docstring and not the constructor arguments.
 
 To check your docstrings are correct, run `pre-commit run --all-files` or `pydocstyle --source --explain --convention=google`.
-If all docstrings that fail, the source and reason for the failure is provided. 
+If all docstrings that fail, the source and reason for the failure is provided.
 
 ## Building the docs
+
 Make sure that you have install the requirements:
+
 ```shell
 cd docs
 pip install -r requirements.txt
 ```
+
 Then run
-```
-python scripts/gen_mds.py 
+
+```shell
+python scripts/gen_mds.py
 make dirhtml
 ```
-Now, navigate to `_build/dirhtml` and open `index.html` in your browser.
+
+Now, navigate to `_build/dirhtml` and open `index.html` in your browser.

README.md

@@ -4,12 +4,10 @@
     <img src="https://raw.githubusercontent.com/Farama-Foundation/Gymnasium/main/gymnasium-text.png" width="500px"/>
 </p>
 
-
 Gymnasium is an open source Python library for developing and comparing reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API. This is a fork of OpenAI's [Gym](https://github.com/openai/gym) library by the maintainers (OpenAI handed over maintenance a few years ago to an outside team), and is where future maintenance will occur going forward
 
 The documentation website is at [gymnasium.farama.org](https://gymnasium.farama.org), and we have a public discord server (which we also use to coordinate development work) that you can join here: https://discord.gg/bnJ6kubTg6
 
-
 ## Environments
 
 Gymnasium includes the following families of environments along with a wide variety of third-party environments
@@ -20,7 +18,6 @@ Gymnasium includes the following families of environments along with a wide vari
 * [Atari](https://gymnasium.farama.org/environments/atari/) - A set of 57 Atari 2600 environments simulated through Stella and the Arcade Learning Environment that have a high range of complexity for agents to learn.
 * [Third-party](https://gymnasium.farama.org/environments/third_party_environments/) - A number of environments have been created that are compatible with the Gymnasium API. Be aware of the version that the software was created for and use the `apply_env_compatibility` in `gymnasium.make` if necessary.
 
-
 ## Installation
 
 To install the base Gymnasium library, use `pip install gymnasium`
@@ -59,8 +56,6 @@ Please note that this is an incomplete list, and just includes libraries that th
 
 Gymnasium keeps strict versioning for reproducibility reasons. All environments end in a suffix like "\_v0".  When changes are made to environments that might impact learning results, the number is increased by one to prevent potential confusion. These inherent from Gym.
 
-
 ## Development Roadmap
 
 We have a roadmap for future development work for Gymnasium available here: https://github.com/Farama-Foundation/Gymnasium/issues/12
-

docs/.gitignore

@@ -15,4 +15,4 @@ environments/box2d/*.md
 environments/classic_control/*.md
 environments/mujoco/*.md
 environments/third_party_environments/*.md
-environments/toy_text/*.md
+environments/toy_text/*.md

docs/_static/img/github_icon.svg

@@ -1 +1 @@
-<svg xmlns="http://www.w3.org/2000/svg" width="72" height="72" viewBox="0 0 24 24" fill="none" stroke="white" stroke-width="0.5" stroke-linecap="round" stroke-linejoin="round" class="feather feather-github"><path d="M9 19c-5 1.5-5-2.5-7-3m14 6v-3.87a3.37 3.37 0 0 0-.94-2.61c3.14-.35 6.44-1.54 6.44-7A5.44 5.44 0 0 0 20 4.77 5.07 5.07 0 0 0 19.91 1S18.73.65 16 2.48a13.38 13.38 0 0 0-7 0C6.27.65 5.09 1 5.09 1A5.07 5.07 0 0 0 5 4.77a5.44 5.44 0 0 0-1.5 3.78c0 5.42 3.3 6.61 6.44 7A3.37 3.37 0 0 0 9 18.13V22"></path></svg>
+<svg xmlns="http://www.w3.org/2000/svg" width="72" height="72" viewBox="0 0 24 24" fill="none" stroke="white" stroke-width="0.5" stroke-linecap="round" stroke-linejoin="round" class="feather feather-github"><path d="M9 19c-5 1.5-5-2.5-7-3m14 6v-3.87a3.37 3.37 0 0 0-.94-2.61c3.14-.35 6.44-1.54 6.44-7A5.44 5.44 0 0 0 20 4.77 5.07 5.07 0 0 0 19.91 1S18.73.65 16 2.48a13.38 13.38 0 0 0-7 0C6.27.65 5.09 1 5.09 1A5.07 5.07 0 0 0 5 4.77a5.44 5.44 0 0 0-1.5 3.78c0 5.42 3.3 6.61 6.44 7A3.37 3.37 0 0 0 9 18.13V22"></path></svg>

docs/api/env.md

@@ -24,7 +24,7 @@ title: Utils
 .. autoattribute:: gymnasium.Env.action_space
 
     The Space object corresponding to valid actions, all valid actions should be contained with the space. For example, if the action space is of type `Discrete` and gives the value `Discrete(2)`, this means there are two valid discrete actions: 0 & 1.
-    
+
     .. code::
 
         >>> env.action_space
@@ -33,9 +33,9 @@ title: Utils
         Box(-3.4028234663852886e+38, 3.4028234663852886e+38, (4,), float32)
 
 .. autoattribute:: gymnasium.Env.observation_space
-    
+
     The Space object corresponding to valid observations, all valid observations should be contained with the space. For example, if the observation space is of type :class:`Box` and the shape of the object is ``(4,)``, this denotes a valid observation will be an array of 4 numbers. We can check the box bounds as well with attributes.
-    
+
     .. code::
 
         >>> env.observation_space.high
@@ -46,19 +46,19 @@ title: Utils
 .. autoattribute:: gymnasium.Env.metadata
 
     The metadata of the environment containing rendering modes, rendering fps, etc
-    
+
 .. autoattribute:: gymnasium.Env.render_mode
 
     The render mode of the environment determined at initialisation
 
 .. autoattribute:: gymnasium.Env.reward_range
-    
+
     A tuple corresponding to the minimum and maximum possible rewards for an agent over an episode. The default reward range is set to :math:`(-\infty,+\infty)`.
 
 .. autoattribute:: gymnasium.Env.spec
 
     The ``EnvSpec`` of the environment normally set during :py:meth:`gymnasium.make`
-``` 
+```
 
 ### Additional Methods
 

docs/api/experimental.md

@@ -14,7 +14,7 @@ experimental/vector_wrappers
 
 ## Functional Environments
 
-The gymnasium ``Env`` provides high flexibility for the implementation of individual environments however this can complicate parallelism of environments. Therefore, we propose the :class:`gymnasium.experimental.FuncEnv` where each part of environment has its own function related to it. 
+The gymnasium ``Env`` provides high flexibility for the implementation of individual environments however this can complicate parallelism of environments. Therefore, we propose the :class:`gymnasium.experimental.FuncEnv` where each part of environment has its own function related to it.
 
 ## Wrappers
 
@@ -28,12 +28,13 @@ Gymnasium already contains a large collection of wrappers, but we believe that t
  * In v28, we aim to rewrite the VectorEnv to not inherit from Env, as a result new vectorised versions of the wrappers will be provided.
 
 ### Lambda Observation Wrappers
+
 ```{eval-rst}
 .. py:currentmodule:: gymnasium
 
-.. list-table:: 
+.. list-table::
     :header-rows: 1
-    
+
     * - Old name
       - New name
       - Vector version
@@ -71,11 +72,11 @@ Gymnasium already contains a large collection of wrappers, but we believe that t
       - VectorDtypeObservation (*)
       - Yes
     * - :class:`wrappers.PixelObservationWrapper`
-      - PixelObservation 
+      - PixelObservation
       - VectorPixelObservation
       - No
     * - :class:`NormalizeObservation`
-      - NormalizeObservation 
+      - NormalizeObservation
       - VectorNormalizeObservation
       - No
     * - :class:`TimeAwareObservation`
@@ -97,12 +98,13 @@ Gymnasium already contains a large collection of wrappers, but we believe that t
 ```
 
 ### Lambda Action Wrappers
+
 ```{eval-rst}
 .. py:currentmodule:: gymnasium
 
-.. list-table:: 
+.. list-table::
     :header-rows: 1
-    
+
     * - Old name
       - New name
       - Vector version
@@ -130,12 +132,13 @@ Gymnasium already contains a large collection of wrappers, but we believe that t
 ```
 
 ### Lambda Reward Wrappers
+
 ```{eval-rst}
 .. py:currentmodule:: gymnasium
 
-.. list-table:: 
+.. list-table::
     :header-rows: 1
-    
+
     * - Old name
       - New name
       - Vector version
@@ -154,12 +157,13 @@ Gymnasium already contains a large collection of wrappers, but we believe that t
 ```
 
 ### Common Wrappers
+
 ```{eval-rst}
 .. py:currentmodule:: gymnasium
 
-.. list-table:: 
+.. list-table::
     :header-rows: 1
-    
+
     * - Old name
       - New name
       - Vector version
@@ -171,7 +175,7 @@ Gymnasium already contains a large collection of wrappers, but we believe that t
       - VectorPassiveEnvChecker
     * - :class:`wrappers.OrderEnforcing`
       - OrderEnforcing
-      - VectorOrderEnforcing (*)  
+      - VectorOrderEnforcing (*)
     * - :class:`wrappers.EnvCompatibility`
       - Moved to `shimmy <https://github.com/Farama-Foundation/Shimmy/blob/main/shimmy/openai_gym_compatibility.py>`_
       - Not Implemented
@@ -193,12 +197,13 @@ Gymnasium already contains a large collection of wrappers, but we believe that t
 ```
 
 ### Vector Only Wrappers
+
 ```{eval-rst}
 .. py:currentmodule:: gymnasium
 
-.. list-table:: 
+.. list-table::
     :header-rows: 1
-    
+
     * - Old name
       - New name
     * - :class:`wrappers.VectorListInfo`

docs/api/experimental/functional.md

@@ -33,4 +33,4 @@ title: Functional
 
 ```{eval-rst}
 ... autoclass:: gymasnium.experimental.func2env.FunctionalJaxCompatibilityEnv
-```
+```

docs/api/experimental/wrappers.md

@@ -43,4 +43,4 @@
 
 ```{eval-rst}
 
-```
+```

docs/api/registry.md

@@ -5,7 +5,7 @@ title: Registry
 # Registry
 
 Gymnasium allows users to automatically load environments, pre-wrapped with several important wrappers.
-Environments can also be created through python imports. 
+Environments can also be created through python imports.
 
 ## Make
 
@@ -30,9 +30,8 @@ This will not include environments registered only in OpenAI Gym however can be
 .. autofunction:: gymnasium.spec
 ```
 
-
 ## Pretty print registry
 
 ```{eval-rst}
 .. autofunction:: gymnasium.pprint_registry
-```
+```

docs/api/spaces.md

@@ -2,7 +2,6 @@
 title: Spaces
 ---
 
-
 # Spaces
 
 ```{toctree}
@@ -18,6 +17,7 @@ spaces/vector_utils
 ```
 
 ## The Base Class
+
 ```{eval-rst}
 .. autoclass:: gymnasium.spaces.Space
 ```
@@ -42,7 +42,7 @@ Each space implements the following functions:
 .. autofunction:: gymnasium.spaces.space.Space.seed
 .. autofunction:: gymnasium.spaces.space.Space.to_jsonable
 .. autofunction:: gymnasium.spaces.space.Space.from_jsonable
-``` 
+```
 
 ## Fundamental Spaces
 
@@ -55,7 +55,7 @@ Gymnasium has a number of fundamental spaces that are used as building boxes for
 * :py:class:`Discrete` - Supports a single discrete number of values with an optional start for the values
 * :py:class:`MultiDiscrete` - Supports single or matrices of binary values, used for holding down a button or if an agent has an object
 * :py:class:`MultiBinary` - Supports multiple discrete values with multiple axes, used for controller actions
-* :py:class:`Text` - Supports strings, used for passing agent messages, mission details, etc 
+* :py:class:`Text` - Supports strings, used for passing agent messages, mission details, etc
 ```
 
 ## Composite Spaces
@@ -71,7 +71,7 @@ Often environment spaces require joining fundamental spaces together for vectori
 
 ## Utils
 
-Gymnasium contains a number of helpful utility functions for flattening and unflattening spaces. 
+Gymnasium contains a number of helpful utility functions for flattening and unflattening spaces.
 This can be important for passing information to neural networks.
 
 ```{eval-rst}

docs/api/spaces/composite.md

@@ -7,7 +7,7 @@
 
 .. automethod:: gymnasium.spaces.Dict.sample
 .. automethod:: gymnasium.spaces.Dict.seed
-``` 
+```
 
 ## Tuple
 
@@ -16,7 +16,7 @@
 
 .. automethod:: gymnasium.spaces.Tuple.sample
 .. automethod:: gymnasium.spaces.Tuple.seed
-``` 
+```
 
 ## Sequence
 
@@ -25,7 +25,7 @@
 
 .. automethod:: gymnasium.spaces.Sequence.sample
 .. automethod:: gymnasium.spaces.Sequence.seed
-``` 
+```
 
 ## Graph
 

docs/api/spaces/fundamental.md

@@ -12,7 +12,7 @@ title: Fundamental Spaces
 .. automethod:: gymnasium.spaces.Box.sample
 .. automethod:: gymnasium.spaces.Box.seed
 .. automethod:: gymnasium.spaces.Box.is_bounded
-``` 
+```
 
 ## Discrete
 
@@ -20,7 +20,7 @@ title: Fundamental Spaces
 .. autoclass:: gymnasium.spaces.Discrete
 .. automethod:: gymnasium.spaces.Discrete.sample
 .. automethod:: gymnasium.spaces.Discrete.seed
-``` 
+```
 
 ## MultiBinary
 
@@ -28,7 +28,7 @@ title: Fundamental Spaces
 .. autoclass:: gymnasium.spaces.MultiBinary
 .. automethod:: gymnasium.spaces.MultiBinary.sample
 .. automethod:: gymnasium.spaces.MultiBinary.seed
-``` 
+```
 
 ## MultiDiscrete
 
@@ -37,7 +37,7 @@ title: Fundamental Spaces
 
 .. automethod:: gymnasium.spaces.MultiDiscrete.sample
 .. automethod:: gymnasium.spaces.MultiDiscrete.seed
-``` 
+```
 
 ## Text
 
@@ -46,4 +46,4 @@ title: Fundamental Spaces
 
 .. automethod:: gymnasium.spaces.Text.sample
 .. automethod:: gymnasium.spaces.Text.seed
-``` 
+```

docs/api/spaces/utils.md

@@ -9,4 +9,4 @@ title: Utils
 .. autofunction:: gymnasium.spaces.utils.flatten
 .. autofunction:: gymnasium.spaces.utils.flatdim
 .. autofunction:: gymnasium.spaces.utils.unflatten
-``` 
+```

docs/api/spaces/vector_utils.md

@@ -8,7 +8,7 @@ title: Vector Utils
 .. autofunction:: gymnasium.vector.utils.batch_space
 .. autofunction:: gymnasium.vector.utils.concatenate
 .. autofunction:: gymnasium.vector.utils.iterate
-``` 
+```
 
 ## Shared Memory Utils
 

docs/api/utils.md

@@ -9,11 +9,11 @@ title: Utils
 ```{eval-rst}
 .. autofunction:: gymnasium.utils.play.play
 .. autoclass:: gymnasium.utils.play.PlayPlot
-    
+
     .. automethod:: callback
 
 .. autoclass:: gymnasium.utils.play.PlayableGame
-    
+
     .. automethod:: process_event
 ```
 
@@ -42,4 +42,4 @@ title: Utils
 
 ```{eval-rst}
 .. autofunction:: gymnasium.utils.env_checker.check_env
-``` 
+```

docs/api/wrappers.md

@@ -42,9 +42,9 @@ wrappers/reward_wrappers
 .. autoproperty:: gymnasium.Wrapper.np_random
 .. attribute:: gymnasium.Wrapper.env
 
-    The environment (one level underneath) this wrapper. 
-    
-    This may itself be a wrapped environment. 
+    The environment (one level underneath) this wrapper.
+
+    This may itself be a wrapped environment.
     To obtain the environment underneath all layers of wrappers, use :attr:`gymnasium.Wrapper.unwrapped`.
 
 .. autoproperty:: gymnasium.Wrapper.unwrapped
@@ -52,88 +52,88 @@ wrappers/reward_wrappers
 
 ## Gymnasium Wrappers
 
-Gymnasium provides a number of commonly used wrappers listed below. More information can be found on the particular 
+Gymnasium provides a number of commonly used wrappers listed below. More information can be found on the particular
 wrapper in the page on the wrapper type
 
 ```{eval-rst}
 .. py:currentmodule:: gymnasium.wrappers
 
-.. list-table:: 
+.. list-table::
     :header-rows: 1
-    
+
     * - Name
       - Type
       - Description
-    * - :class:`AtariPreprocessing`      
-      - Misc Wrapper            
-      - Implements the common preprocessing applied tp Atari environments                                                                                                                      
-    * - :class:`AutoResetWrapper`        
-      - Misc Wrapper            
-      - The wrapped environment will automatically reset when the terminated or truncated  state is reached.                                                                                   
-    * - :class:`ClipAction`              
-      - Action Wrapper      
-      - Clip the continuous action to the valid bound specified by the environment's `action_space`                                                                                            
-    * - :class:`EnvCompatibility`        
-      - Misc Wrapper            
-      - Provides compatibility for environments written in the OpenAI Gym v0.21 API to look like Gymnasium environments                                                                        
-    * - :class:`FilterObservation`       
-      - Observation Wrapper 
-      - Filters a dictionary observation spaces to only requested keys                                                                                                                          
-    * - :class:`FlattenObservation`      
-      - Observation Wrapper 
-      - An Observation wrapper that flattens the observation                                                                                                                                   
-    * - :class:`FrameStack`              
-      - Observation Wrapper 
-      - AnObservation wrapper that stacks the observations in a rolling manner.                                                                                                                
-    * - :class:`GrayScaleObservation`    
-      - Observation Wrapper 
-      - Convert the image observation from RGB to gray scale.                                                                                                                                  
-    * - :class:`HumanRendering`          
-      - Misc Wrapper            
-      - Allows human like rendering for environments that support "rgb_array" rendering                                                                                                        
-    * - :class:`NormalizeObservation`    
-      - Observation Wrapper            
-      - This wrapper will normalize observations s.t. each coordinate is centered with unit variance.                                                                                          
-    * - :class:`NormalizeReward`         
-      - Reward Wrapper            
-      - This wrapper will normalize immediate rewards s.t. their exponential moving average has a fixed variance.                                                                              
-    * - :class:`OrderEnforcing`          
-      - Misc Wrapper            
-      - This will produce an error if `step` or `render` is called before `reset`                                                                                                              
-    * - :class:`PixelObservationWrapper` 
-      - Observation Wrapper 
-      - Augment observations by pixel values obtained via `render` that can be added to or replaces the environments observation.                                                              
-    * - :class:`RecordEpisodeStatistics` 
-      - Misc Wrapper            
-      - This will keep track of cumulative rewards and episode lengths returning them at the end.                                                                                              
-    * - :class:`RecordVideo`             
-      - Misc Wrapper            
-      - This wrapper will record videos of rollouts.                                                                                                                                           
-    * - :class:`RenderCollection`        
-      - Misc Wrapper            
-      - Enable list versions of render modes, i.e. "rgb_array_list" for "rgb_array" such that the rendering for each step are saved in a list until `render` is called.                        
-    * - :class:`RescaleAction`           
-      - Action Wrapper      
-      - Rescales the continuous action space of the environment to a range \[`min_action`, `max_action`], where `min_action` and `max_action` are numpy arrays or floats.                      
-    * - :class:`ResizeObservation`       
-      - Observation Wrapper 
-      - This wrapper works on environments with image observations (or more generally observations of shape AxBxC) and resizes the observation to the shape given by the tuple `shape`.        
-    * - :class:`StepAPICompatibility`    
-      - Misc Wrapper            
-      - Modifies an environment step function from (old) done to the (new) termination / truncation API.                                                                                       
-    * - :class:`TimeAwareObservation`    
-      - Observation Wrapper 
-      - Augment the observation with current time step in the trajectory (by appending it to the observation).                                                                                 
-    * - :class:`TimeLimit`               
-      - Misc Wrapper            
-      - This wrapper will emit a truncated signal if the specified number of steps is exceeded in an episode.                                                                                  
-    * - :class:`TransformObservation`    
-      - Observation Wrapper 
-      - This wrapper will apply function to observations                                                                                                                                       
-    * - :class:`TransformReward`         
-      - Reward Wrapper      
-      - This wrapper will apply function to rewards                                                                                                                                            
-    * - :class:`VectorListInfo`          
-      - Misc Wrapper            
-      - This wrapper will convert the info of a vectorized environment from the `dict` format to a `list` of dictionaries where the i-th dictionary contains info of the i-th environment. 
-```
+    * - :class:`AtariPreprocessing`
+      - Misc Wrapper
+      - Implements the common preprocessing applied tp Atari environments
+    * - :class:`AutoResetWrapper`
+      - Misc Wrapper
+      - The wrapped environment will automatically reset when the terminated or truncated  state is reached.
+    * - :class:`ClipAction`
+      - Action Wrapper
+      - Clip the continuous action to the valid bound specified by the environment's `action_space`
+    * - :class:`EnvCompatibility`
+      - Misc Wrapper
+      - Provides compatibility for environments written in the OpenAI Gym v0.21 API to look like Gymnasium environments
+    * - :class:`FilterObservation`
+      - Observation Wrapper
+      - Filters a dictionary observation spaces to only requested keys
+    * - :class:`FlattenObservation`
+      - Observation Wrapper
+      - An Observation wrapper that flattens the observation
+    * - :class:`FrameStack`
+      - Observation Wrapper
+      - AnObservation wrapper that stacks the observations in a rolling manner.
+    * - :class:`GrayScaleObservation`
+      - Observation Wrapper
+      - Convert the image observation from RGB to gray scale.
+    * - :class:`HumanRendering`
+      - Misc Wrapper
+      - Allows human like rendering for environments that support "rgb_array" rendering
+    * - :class:`NormalizeObservation`
+      - Observation Wrapper
+      - This wrapper will normalize observations s.t. each coordinate is centered with unit variance.
+    * - :class:`NormalizeReward`
+      - Reward Wrapper
+      - This wrapper will normalize immediate rewards s.t. their exponential moving average has a fixed variance.
+    * - :class:`OrderEnforcing`
+      - Misc Wrapper
+      - This will produce an error if `step` or `render` is called before `reset`
+    * - :class:`PixelObservationWrapper`
+      - Observation Wrapper
+      - Augment observations by pixel values obtained via `render` that can be added to or replaces the environments observation.
+    * - :class:`RecordEpisodeStatistics`
+      - Misc Wrapper
+      - This will keep track of cumulative rewards and episode lengths returning them at the end.
+    * - :class:`RecordVideo`
+      - Misc Wrapper
+      - This wrapper will record videos of rollouts.
+    * - :class:`RenderCollection`
+      - Misc Wrapper
+      - Enable list versions of render modes, i.e. "rgb_array_list" for "rgb_array" such that the rendering for each step are saved in a list until `render` is called.
+    * - :class:`RescaleAction`
+      - Action Wrapper
+      - Rescales the continuous action space of the environment to a range \[`min_action`, `max_action`], where `min_action` and `max_action` are numpy arrays or floats.
+    * - :class:`ResizeObservation`
+      - Observation Wrapper
+      - This wrapper works on environments with image observations (or more generally observations of shape AxBxC) and resizes the observation to the shape given by the tuple `shape`.
+    * - :class:`StepAPICompatibility`
+      - Misc Wrapper
+      - Modifies an environment step function from (old) done to the (new) termination / truncation API.
+    * - :class:`TimeAwareObservation`
+      - Observation Wrapper
+      - Augment the observation with current time step in the trajectory (by appending it to the observation).
+    * - :class:`TimeLimit`
+      - Misc Wrapper
+      - This wrapper will emit a truncated signal if the specified number of steps is exceeded in an episode.
+    * - :class:`TransformObservation`
+      - Observation Wrapper
+      - This wrapper will apply function to observations
+    * - :class:`TransformReward`
+      - Reward Wrapper
+      - This wrapper will apply function to rewards
+    * - :class:`VectorListInfo`
+      - Misc Wrapper
+      - This wrapper will convert the info of a vectorized environment from the `dict` format to a `list` of dictionaries where the i-th dictionary contains info of the i-th environment.
+```

docs/api/wrappers/action_wrappers.md

@@ -9,8 +9,8 @@
 ```
 
 ## Available Action Wrappers
+
 ```{eval-rst}
 .. autoclass:: gymnasium.wrappers.ClipAction
 .. autoclass:: gymnasium.wrappers.RescaleAction
 ```
-

docs/api/wrappers/misc_wrappers.md

@@ -1,4 +1,5 @@
 # Misc Wrappers
+
 ```{eval-rst}
 .. autoclass:: gymnasium.wrappers.AtariPreprocessing
 .. autoclass:: gymnasium.wrappers.AutoResetWrapper

docs/conf.py

@@ -22,6 +22,7 @@ from furo import gen_tutorials
 
 import gymnasium
 
+
 project = "Gymnasium"
 copyright = "2022 Farama Foundation"
 author = "Farama Foundation"

docs/content/basic_usage.md

@@ -6,20 +6,20 @@ firstpage:
 
 # Basic Usage
 
-Gymnasium is a project that provide an API for all single agent reinforcement learning environments that include implementations of common environments: cartpole, pendulum, mountain-car, mujoco, atari, and more. 
+Gymnasium is a project that provide an API for all single agent reinforcement learning environments that include implementations of common environments: cartpole, pendulum, mountain-car, mujoco, atari, and more.
 
-The API contains four key functions: ``make``, ``reset``, ``step`` and ``render`` that this basic usage will introduce you to. At the core of Gymnasium is ``Env`` which is a high level python class representing a markov decision process from reinforcement learning theory (this is not a perfect reconstruction missing several components of MDPs). Within gymnasium, environments (MDPs) are implements as ``Env`` along with ``Wrappers`` that can change the results passed to the user. 
+The API contains four key functions: ``make``, ``reset``, ``step`` and ``render`` that this basic usage will introduce you to. At the core of Gymnasium is ``Env`` which is a high level python class representing a markov decision process from reinforcement learning theory (this is not a perfect reconstruction missing several components of MDPs). Within gymnasium, environments (MDPs) are implements as ``Env`` along with ``Wrappers`` that can change the results passed to the user.
 
 ## Initializing Environments
 
-Initializing environments is very easy in Gymnasium and can be done via the ``make`` function: 
+Initializing environments is very easy in Gymnasium and can be done via the ``make`` function:
 
 ```python
 import gymnasium as gym
 env = gym.make('CartPole-v1')
 ```
 
-This will return an ``Env`` for users to interact with. To see all environments you can create, use ``gymnasium.envs.registry.keys()``.``make`` includes a number of additional parameters to adding wrappers, specifying keywords to the environment and more. 
+This will return an ``Env`` for users to interact with. To see all environments you can create, use ``gymnasium.envs.registry.keys()``.``make`` includes a number of additional parameters to adding wrappers, specifying keywords to the environment and more.
 
 ## Interacting with the Environment
 
@@ -63,9 +63,9 @@ The output should look something like this:
 
 ### Explaining the code
 
-First, an environment is created using ``make`` with an additional keyword `"render_mode"` that specifies how the environment should be visualised. See ``render`` for details on the default meaning of different render modes. In this example, we use the ``"LunarLander"`` environment where the agent controls a spaceship that needs to land safely. 
+First, an environment is created using ``make`` with an additional keyword `"render_mode"` that specifies how the environment should be visualised. See ``render`` for details on the default meaning of different render modes. In this example, we use the ``"LunarLander"`` environment where the agent controls a spaceship that needs to land safely.
 
-After initializing the environment, we ``reset`` the environment to get the first observation of the environment. For initializing the environment with a particular random seed or options (see environment documentation for possible values) use the ``seed`` or ``options`` parameters with ``reset``. 
+After initializing the environment, we ``reset`` the environment to get the first observation of the environment. For initializing the environment with a particular random seed or options (see environment documentation for possible values) use the ``seed`` or ``options`` parameters with ``reset``.
 
 Next, the agent performs an action in the environment, ``step``, this can be imagined as moving a robot or pressing a button on a games' controller that causes a change within the environment. As a result, the agent receives a new observation from the updated environment along with a reward for taking the action. This reward could be for instance positive for destroying an enemy or a negative reward for moving into lava. One such action-observation exchange is referred to as a *timestep*.
 
@@ -79,7 +79,7 @@ In the example, we sampled random actions via ``env.action_space.sample()`` inst
 
 Every environment should have the attributes ``action_space`` and ``observation_space``, both of which should be instances of classes that inherit from ``Space``. Gymnasium has support for a major of possible spaces are users need:
 
-- ``Box``: describes an n-dimensional continuous space. It's a bounded space where we can define the upper and lower 
+- ``Box``: describes an n-dimensional continuous space. It's a bounded space where we can define the upper and lower
   limits which describe the valid values our observations can take.
 - ``Discrete``: describes a discrete space where {0, 1, ..., n-1} are the possible values our observation or action can take.
   Values can be shifted to {a, a+1, ..., a+n-1} using an optional argument.
@@ -88,7 +88,7 @@ Every environment should have the attributes ``action_space`` and ``observation_
 - ``MultiBinary``: creates an n-shape binary space. Argument n can be a number or a list of numbers.
 - ``MultiDiscrete``: consists of a series of ``Discrete`` action spaces with a different number of actions in each element.
 
-For example usage of spaces, see their [documentation](/api/spaces) along with [utility functions](/api/spaces/utils). There are a couple of more niche spaces ``Graph``, ``Sequence`` and ``Text``. 
+For example usage of spaces, see their [documentation](/api/spaces) along with [utility functions](/api/spaces/utils). There are a couple of more niche spaces ``Graph``, ``Sequence`` and ``Text``.
 
 ## Modifying the environment
 

docs/content/migration-guide.md

@@ -15,6 +15,7 @@ For more information, see the `guide </content/gym_compatibility>`_
 ```
 
 ### Example code for v21
+
 ```python
 import gym
 env = gym.make("LunarLander-v2", options={})
@@ -32,6 +33,7 @@ env.close()
 ```
 
 ### Example code for v26
+
 ```python
 import gym
 env = gym.make("LunarLander-v2", render_mode="human")

docs/environments/atari.md

@@ -121,16 +121,20 @@ the arguments `difficulty` and `mode` when constructing the environment. This do
 action spaces of default flavor choices.
 
 ## Observation Space
+
 The observation issued by an Atari environment may be:
+
 - the RGB image that is displayed to a human player,
 - a grayscale version of that image or
 - the state of the 128 Bytes of RAM of the console.
 
 ## Rewards
+
 The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can
 find these manuals on [AtariAge](https://atariage.com/).
 
 ## Stochasticity
+
 It was pointed out in [[1]](#1) that Atari games are entirely deterministic. Thus, agents could achieve
 state-of-the-art performance by simply memorizing an optimal sequence of actions while completely ignoring observations from the environment.
 To avoid this, ALE implements sticky actions: Instead of always simulating the action passed to the environment, there is a small
@@ -143,6 +147,7 @@ repeated `frameskip` many times. Otherwise, if `frameskip` is a tuple, the numbe
 random between `frameskip[0]` (inclusive) and `frameskip[1]` (exclusive) in each environment step.
 
 ## Common Arguments
+
 When initializing Atari environments via `gymnasium.make`, you may pass some additional arguments. These work for any
 Atari environment. However, legal values for `mode` and `difficulty` depend on the environment.
 
@@ -170,6 +175,7 @@ action space will be reduced to a subset.
 > This will guarantee proper scaling, audio support, and proper framerates
 
 ## Version History and Naming Schemes
+
 All Atari games are available in three versions. They differ in the default settings of the arguments above.
 The differences are listed in the following table:
 
@@ -204,6 +210,7 @@ are in the "ALE" namespace. The suffix "-ram" is still available. Thus, we get t
 | ALE/Amidar-ram-v5 | `"ram"`     | `5`          | `0.25`                       | `True`               |
 
 ## Flavors
+
 Some games allow the user to set a difficulty level and a game mode. Different modes/difficulties may have different
 game dynamics and (if a reduced action space is used) different action spaces. We follow the convention of [[2]](#2) and
 refer to the combination of difficulty level and game mode as a flavor of a game. The following table shows
@@ -290,4 +297,4 @@ Machado et al.
 "Revisiting the Arcade Learning Environment: Evaluation Protocols
 and Open Problems for General Agents"
 Journal of Artificial Intelligence Research (2018)
-URL: https://jair.org/index.php/jair/article/view/11182
+URL: https://jair.org/index.php/jair/article/view/11182

docs/environments/atari/adventure.md

@@ -5,7 +5,7 @@ firstpage:
 
 # Adventure
 
-```{figure} ../../_static/videos/atari/adventure.gif 
+```{figure} ../../_static/videos/atari/adventure.gif
 :width: 120px
 :name: adventure
 ```
@@ -18,25 +18,30 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (250, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/Adventure-v5")` | 
+| Import            | `gymnasium.make("ALE/Adventure-v5")` |
 
 ## Description
+
 You must find the enchanted chalice and return it to the golden castle. You can pick up various objects (keys, a sword,
 a bridge, or a magnet) and have to fight or outmanoeuvre dragons.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -50,27 +55,28 @@ via `gymnasium.make`.
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Adventure-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Adventure   | `[0, 1, 2]` | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Adventure-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
-* v0: Initial versions release (1.0.0)
+* v0: Initial versions release (1.0.0)

docs/environments/atari/air_raid.md

@@ -4,7 +4,7 @@ title: Air Raid
 
 # Air Raid
 
-```{figure} ../../_static/videos/atari/air_raid.gif 
+```{figure} ../../_static/videos/atari/air_raid.gif
 :width: 120px
 :name: air_raid
 ```
@@ -17,17 +17,19 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (250, 160, 3)                      |
 | Observation High  | 255                                |
 | Observation Low   | 0                                  |
-| Import            | `gymnasium.make("ALE/AirRaid-v5")` | 
+| Import            | `gymnasium.make("ALE/AirRaid-v5")` |
 
 ## Description
-You control a ship that can move sideways. You must protect two buildings (one on the right and one on the left side of the screen) from 
+
+You control a ship that can move sideways. You must protect two buildings (one on the right and one on the left side of the screen) from
 flying saucers that are trying to drop bombs on them.
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -40,12 +42,15 @@ flavor looks like this:
 | 5   | LEFTFIRE  |
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -56,29 +61,31 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/AirRaid-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
 | AirRaid     | `[1, ..., 8]` | `[0]`              | `1`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("AirRaid-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
-* v0: Initial versions release (1.0.0)
+* v0: Initial versions release (1.0.0)

docs/environments/atari/alien.md

@@ -4,7 +4,7 @@ title: Alien
 
 # Alien
 
-```{figure} ../../_static/videos/atari/alien.gif 
+```{figure} ../../_static/videos/atari/alien.gif
 :width: 120px
 :name: alien
 ```
@@ -17,26 +17,31 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                    |
 | Observation High  | 255                              |
 | Observation Low   | 0                                |
-| Import            | `gymnasium.make("ALE/Alien-v5")` | 
+| Import            | `gymnasium.make("ALE/Alien-v5")` |
 
 ## Description
+
 You are stuck in a maze-like space ship with three aliens. You goal is to destroy their eggs that are scattered
-all over the ship while simultaneously avoiding the aliens (they are trying to kill you). You have a flamethrower that can help you turn 
+all over the ship while simultaneously avoiding the aliens (they are trying to kill you). You have a flamethrower that can help you turn
 them away in tricky situations. Moreover, you can occasionally collect a power-up (pulsar) that gives you the temporary ability to kill aliens.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=815).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -48,34 +53,35 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points by destroying eggs, killing aliens, using pulsars, and collecting special prizes. When you are caught
 by an alien, you will lose one of your lives. The number of lives you have depends on the game flavor. For a
 table of scores corresponding to the different achievements, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=815).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Alien-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
 | Alien       | `[0, ..., 3]` | `[0, ..., 3]`      | `0`          |
 
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Alien-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/amidar.md

@@ -4,7 +4,7 @@ title: Amidar
 
 # Amidar
 
-```{figure} ../../_static/videos/atari/amidar.gif 
+```{figure} ../../_static/videos/atari/amidar.gif
 :width: 120px
 :name: amidar
 ```
@@ -17,21 +17,22 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                     |
 | Observation High  | 255                               |
 | Observation Low   | 0                                 |
-| Import            | `gymnasium.make("ALE/Amidar-v5")` | 
+| Import            | `gymnasium.make("ALE/Amidar-v5")` |
 
 ## Description
+
 This game is similar to Pac-Man: You are trying to visit all places on a 2-dimensional grid while simultaneously avoiding
 your enemies. You can turn the tables at one point in the game: Your enemies turn into chickens and you can
 catch them. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=817).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
-
 | Num | Action    |
 |-----|-----------|
 | 0   | NOOP      |
@@ -44,13 +45,17 @@ flavor looks like this:
 | 7   | RIGHTFIRE |
 | 8   | LEFTFIRE  |
 | 9   | DOWNFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -62,32 +67,34 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points by traversing new parts of the grid. Coloring an entire box in the maze or catching chickens gives extra points.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=817).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Amidar-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Amidar      | `[0]`       | `[0, 3]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Amidar-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/assault.md

@@ -1,9 +1,10 @@
 ---
 title: Assault
 ---
+
 # Assault
 
-```{figure} ../../_static/videos/atari/assault.gif 
+```{figure} ../../_static/videos/atari/assault.gif
 :width: 120px
 :name: assault
 ```
@@ -16,17 +17,19 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                      |
 | Observation High  | 255                                |
 | Observation Low   | 0                                  |
-| Import            | `gymnasium.make("ALE/Assault-v5")` | 
+| Import            | `gymnasium.make("ALE/Assault-v5")` |
 
 ## Description
+
 You control a vehicle that can move sideways. A big mother ship circles overhead and continually deploys smaller drones.
 You must destroy these enemies and dodge their attacks. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=827).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -38,13 +41,17 @@ flavor looks like this:
 | 4   | LEFT      |
 | 5   | RIGHTFIRE |
 | 6   | LEFTFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -55,9 +62,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Assault-v5")
 ```
 
@@ -67,15 +75,15 @@ The various ways to configure the environment are described in detail in the art
 |-------------|-------------|--------------------|--------------|
 | Assault     | `[0]`       | `[0]`              | `0`          |
 
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Assault-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/asterix.md

@@ -1,9 +1,10 @@
 ---
 title: Asterix
 ---
+
 # Asterix
 
-```{figure} ../../_static/videos/atari/asterix.gif 
+```{figure} ../../_static/videos/atari/asterix.gif
 :width: 120px
 :name: asterix
 ```
@@ -16,21 +17,23 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                      |
 | Observation High  | 255                                |
 | Observation Low   | 0                                  |
-| Import            | `gymnasium.make("ALE/Asterix-v5")` | 
+| Import            | `gymnasium.make("ALE/Asterix-v5")` |
 
 ## Description
+
 You are Asterix and can move horizontally (continuously) and vertically (discretely). Objects
-move horizontally across the screen: lyres and other (more useful) objects. Your goal is to guide 
+move horizontally across the screen: lyres and other (more useful) objects. Your goal is to guide
 Asterix in such a way as to avoid lyres and collect as many other objects as possible. You score points by collecting
 objects and lose a life whenever you collect a lyre. You have three lives available at the beginning. If you score sufficiently
 many points, you will be awarded additional points.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=3325).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -44,13 +47,17 @@ flavor looks like this:
 | 6   | UPLEFT    |
 | 7   | DOWNRIGHT |
 | 8   | DOWNLEFT  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -63,10 +70,12 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 A table of scores awarded for collecting the different objects is provided on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=3325).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Asterix-v5")
 ```
 
@@ -76,16 +85,16 @@ The various ways to configure the environment are described in detail in the art
 |-------------|-------------|--------------------|--------------|
 | Asterix     | `[0]`       | `[0]`              | `0`          |
 
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Asterix-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
-* v0: Initial versions release (1.0.0)
+* v0: Initial versions release (1.0.0)

docs/environments/atari/asteroids.md

@@ -1,9 +1,10 @@
 ---
 title: Asteroids
 ---
+
 # Asteroids
 
-```{figure} ../../_static/videos/atari/asteroids.gif 
+```{figure} ../../_static/videos/atari/asteroids.gif
 :width: 120px
 :name: asteroids
 ```
@@ -16,19 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/Asteroids-v5")` | 
+| Import            | `gymnasium.make("ALE/Asteroids-v5")` |
 
 ## Description
+
 This is a well-known arcade game: You control a spaceship in an asteroid field and must break up asteroids by shooting
-them. Once all asteroids are destroyed, you enter a new level and new asteroids will appear. You will occasionally 
-be attacked by a flying saucer. 
+them. Once all asteroids are destroyed, you enter a new level and new asteroids will appear. You will occasionally
+be attacked by a flying saucer.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=828&itemTypeID=HTMLMANUAL)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action      |
@@ -48,14 +51,16 @@ flavor looks like this:
 | 12  | UPRIGHTFIRE |
 | 13  | UPLEFTFIRE  |
 
-
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -68,31 +73,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You score points for destroying asteroids, satellites and UFOs. The smaller the asteroid, the more points you score
 for destroying it.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=828&itemTypeID=HTMLMANUAL).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Asteroids-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes         | Valid Difficulties | Default Mode |
 |-------------|---------------------|--------------------|--------------|
 | Asteroids   | `[0, ..., 31, 128]` | `[0, 3]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Asteroids-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/atlantis.md

@@ -4,7 +4,7 @@ title: Atlantis
 
 # Atlantis
 
-```{figure} ../../_static/videos/atari/atlantis.gif 
+```{figure} ../../_static/videos/atari/atlantis.gif
 :width: 120px
 :name: atlantis
 ```
@@ -17,9 +17,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                       |
 | Observation High  | 255                                 |
 | Observation Low   | 0                                   |
-| Import            | `gymnasium.make("ALE/Atlantis-v5")` | 
+| Import            | `gymnasium.make("ALE/Atlantis-v5")` |
 
 ## Description
+
 Your job is to defend the submerged city of Atlantis. Your enemies slowly descend towards the city and you must
 destroy them before they reach striking distance. To this end, you control three defense posts.
 You lose if your enemies manage to destroy all seven of Atlantis' installations. You may rebuild installations
@@ -27,10 +28,11 @@ after you have fought of a wave of enemies and scored a sufficient number of poi
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=835).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -39,13 +41,17 @@ flavor looks like this:
 | 1   | FIRE      |
 | 2   | RIGHTFIRE |
 | 3   | LEFTFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -57,33 +63,35 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points for destroying enemies, keeping installations protected during attack waves. You score more points
 if you manage to destroy your enemies with one of the outer defense posts.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=835).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Atlantis-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
 | Atlantis    | `[0, ..., 3]` | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Amidar-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/bank_heist.md

@@ -1,9 +1,10 @@
 ---
 title: Bank Heist
 ---
+
 # Bank Heist
 
-```{figure} ../../_static/videos/atari/bank_heist.gif 
+```{figure} ../../_static/videos/atari/bank_heist.gif
 :width: 120px
 :name: Bank Heist
 ```
@@ -16,27 +17,33 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/BankHeist-v5")` | 
+| Import            | `gymnasium.make("ALE/BankHeist-v5")` |
 
 ## Description
+
 You are a bank robber and (naturally) want to rob as many banks as possible. You control your getaway car and must
 navigate maze-like cities. The police chases you and will appear whenever you rob a bank. You may destroy police cars
 by dropping sticks of dynamite. You can fill up your gas tank by entering a new city.
-At the beginning of the game you have four lives. Lives are lost if you run out of gas, are caught by the police, 
+At the beginning of the game you have four lives. Lives are lost if you run out of gas, are caught by the police,
 or run over the dynamite you have previously dropped.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1008).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -48,33 +55,35 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points for robbing banks and destroying police cars. If you rob nine or more banks, and then leave the city,
 you will score extra points.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1008).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/BankHeist-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes                     | Valid Difficulties | Default Mode |
 |-------------|---------------------------------|--------------------|--------------|
 | BankHeist   | `[0, 4, 8, 12, 16, 20, 24, 28]` | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("BankHeist-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/battle_zone.md

@@ -1,10 +1,10 @@
 ---
 title: Battle Zone
 ---
-   
+
 # Battle Zone
 
-```{figure} ../../_static/videos/atari/battle_zone.gif 
+```{figure} ../../_static/videos/atari/battle_zone.gif
 :width: 120px
 :name: BattleZone
 ```
@@ -17,27 +17,31 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                         |
 | Observation High  | 255                                   |
 | Observation Low   | 0                                     |
-| Import            | `gymnasium.make("ALE/BattleZone-v5")` | 
+| Import            | `gymnasium.make("ALE/BattleZone-v5")` |
 
 ## Description
-You control a tank and must destroy enemy vehicles. This game is played in a first-person perspective and creates 
+
+You control a tank and must destroy enemy vehicles. This game is played in a first-person perspective and creates
 a 3D illusion. A radar screen shows enemies around you. You start with 5 lives and gain up to 2 extra lives if you reach
 a sufficient score.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=859&itemTypeID=HTMLMANUAL)
 
-
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -50,31 +54,33 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You receive points for destroying enemies.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=859&itemTypeID=HTMLMANUAL).
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/BattleZone-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | BattleZone  | `[1, 2, 3]` | `[0]`              | `1`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("BattleZone-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/beam_rider.md

@@ -8,9 +8,10 @@ grid:
    - Observation Low: 0
    - Import: <code>gymnasium.make("ALE/BeamRider-v5")</code>
 ---
+
 # Beam Rider
 
-```{figure} ../../_static/videos/atari/beam_rider.gif 
+```{figure} ../../_static/videos/atari/beam_rider.gif
 :width: 120px
 :name: BeamRider
 ```
@@ -23,18 +24,20 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/BeamRider-v5")` | 
+| Import            | `gymnasium.make("ALE/BeamRider-v5")` |
 
 ## Description
+
 You control a space-ship that travels forward at a constant speed. You can only steer it sideways between discrete
 positions. Your goal is to destroy enemy ships, avoid their attacks and dodge space debris.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SystemID=2600&SoftwareID=860&itemTypeID=MANUAL)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action      |
@@ -49,15 +52,16 @@ flavor looks like this:
 | 7   | RIGHTFIRE   |
 | 8   | LEFTIFIRE   |
 
-
-
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -70,30 +74,33 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You score points for destroying enemies.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SystemID=2600&SoftwareID=860&itemTypeID=MANUAL).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/BeamRider-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | BeamRider   | `[0]`       | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("BeamRider-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/berzerk.md

@@ -1,9 +1,10 @@
 ---
 title: Berzerk
 ---
+
 # Berzerk
 
-```{figure} ../../_static/videos/atari/berzerk.gif 
+```{figure} ../../_static/videos/atari/berzerk.gif
 :width: 120px
 :name: Berzerk
 ```
@@ -16,26 +17,30 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                      |
 | Observation High  | 255                                |
 | Observation Low   | 0                                  |
-| Import            | `gymnasium.make("ALE/Berzerk-v5")` | 
+| Import            | `gymnasium.make("ALE/Berzerk-v5")` |
 
 ## Description
+
 You are stuck in a maze with evil robots. You must destroy them and avoid touching the walls of the maze, as this will kill you. You may be awarded extra lives after scoring a sufficient number of points, depending on the game mode.
 You may also be chased by an undefeatable enemy, Evil Otto, that you must avoid. Evil Otto does not appear in the default mode.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=866&itemTypeID=HTMLMANUAL)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
-
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -48,30 +53,33 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You score points for destroying robots.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=866&itemTypeID=HTMLMANUAL).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Berzerk-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes               | Valid Difficulties | Default Mode |
 |-------------|---------------------------|--------------------|--------------|
 | Berzerk     | `[1, ..., 9, 16, 17, 18]` | `[0]`              | `1`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Berzerk-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/bowling.md

@@ -1,9 +1,10 @@
 ---
 title: Bowling
 ---
+
 # Bowling
 
-```{figure} ../../_static/videos/atari/bowling.gif 
+```{figure} ../../_static/videos/atari/bowling.gif
 :width: 120px
 :name: Bowling
 ```
@@ -16,19 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                      |
 | Observation High  | 255                                |
 | Observation Low   | 0                                  |
-| Import            | `gymnasium.make("ALE/Bowling-v5")` | 
+| Import            | `gymnasium.make("ALE/Bowling-v5")` |
 
 ## Description
+
 Your goal is to score as many points as possible in the game of Bowling. A game consists of 10 frames and you have two
 tries per frame. Knocking down all pins on the first try is called a "strike". Knocking down all pins on the second roll
 is called a "spar". Otherwise, the frame is called "open".
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=879)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action      |
@@ -40,14 +43,16 @@ flavor looks like this:
 | 4   | UPFIRE      |
 | 5   | DOWNFIRE    |
 
-
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -60,32 +65,35 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
-You receive points for knocking down pins. The exact score depends on whether you manage a "strike", "spare" or "open" 
+
+You receive points for knocking down pins. The exact score depends on whether you manage a "strike", "spare" or "open"
 frame. Moreover, the points you score for one frame may depend on following frames.
 You can score up to 300 points in one game (if you manage to do 12 strikes).
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=879).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Bowling-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Bowling     | `[0, 2, 4]` | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Bowling-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/boxing.md

@@ -1,9 +1,10 @@
 ---
 title: Boxing
 ---
+
 # Boxing
 
-```{figure} ../../_static/videos/atari/boxing.gif 
+```{figure} ../../_static/videos/atari/boxing.gif
 :width: 120px
 :name: Boxing
 ```
@@ -16,24 +17,30 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                     |
 | Observation High  | 255                               |
 | Observation Low   | 0                                 |
-| Import            | `gymnasium.make("ALE/Boxing-v5")` | 
+| Import            | `gymnasium.make("ALE/Boxing-v5")` |
 
 ## Description
+
 You fight an opponent in a boxing ring. You score points for hitting the opponent. If you score 100 points, your opponent is
 knocked out.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=882).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -45,32 +52,34 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points by landing punches.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=882).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Boxing-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Boxing      | `[0]`       | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Boxing-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/breakout.md

@@ -1,9 +1,10 @@
 ---
 title: Breakout
 ---
+
 # Breakout
 
-```{figure} ../../_static/videos/atari/breakout.gif 
+```{figure} ../../_static/videos/atari/breakout.gif
 :width: 120px
 :name: Breakout
 ```
@@ -16,19 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                       |
 | Observation High  | 255                                 |
 | Observation Low   | 0                                   |
-| Import            | `gymnasium.make("ALE/Breakout-v5")` | 
+| Import            | `gymnasium.make("ALE/Breakout-v5")` |
 
 ## Description
+
 Another famous Atari game. The dynamics are similar to pong: You move a paddle and hit the ball in a brick wall at the
 top of the screen. Your goal is to destroy the brick wall. You can try to break through the wall and let the ball
 wreak havoc on the other side, all on its own! You have five lives.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=889).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action |
@@ -37,13 +40,17 @@ flavor looks like this:
 | 1   | FIRE   |
 | 2   | RIGHT  |
 | 3   | LEFT   |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -55,32 +62,34 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points by destroying bricks in the wall. The reward for destroying a brick depends on the color of the brick.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=889).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Breakout-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes                                     | Valid Difficulties | Default Mode |
 |-------------|-------------------------------------------------|--------------------|--------------|
 | Breakout    | `[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44]` | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Breakout-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/carnival.md

@@ -1,9 +1,10 @@
 ---
 title: Carnival
 ---
+
 # Carnival
 
-```{figure} ../../_static/videos/atari/carnival.gif 
+```{figure} ../../_static/videos/atari/carnival.gif
 :width: 120px
 :name: Carnival
 ```
@@ -16,22 +17,23 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (214, 160, 3)                       |
 | Observation High  | 255                                 |
 | Observation Low   | 0                                   |
-| Import            | `gymnasium.make("ALE/Carnival-v5")` | 
+| Import            | `gymnasium.make("ALE/Carnival-v5")` |
 
 ## Description
+
 This is a "shoot 'em up" game. Targets move horizontally across the screen and you must shoot them. You are
 in control of a gun that can be moved horizontally. The supply of ammunition is limited and chickens may steal some bullets
 from you if you don't hit them in time.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=908).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
-
 | Num | Action    |
 |-----|-----------|
 | 0   | NOOP      |
@@ -40,13 +42,17 @@ flavor looks like this:
 | 3   | LEFT      |
 | 4   | RIGHTFIRE |
 | 5   | LEFTFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -58,33 +64,35 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points by destroying targets. Points (or bullets) may be subtracted if you hit the target when it shows a minus sign.
 You will score extra points if it shows a plus sign!
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=908).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Carnival-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Carnival    | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Carnival-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/centipede.md

@@ -1,9 +1,10 @@
 ---
 title: Centipede
 ---
+
 # Centipede
 
-```{figure} ../../_static/videos/atari/centipede.gif 
+```{figure} ../../_static/videos/atari/centipede.gif
 :width: 120px
 :name: Centipede
 ```
@@ -16,28 +17,32 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/Centipede-v5")` | 
+| Import            | `gymnasium.make("ALE/Centipede-v5")` |
 
 ## Description
-You are an elf and must use your magic wands to fend off spiders, fleas and centipedes. Your goal is to protect mushrooms in 
+
+You are an elf and must use your magic wands to fend off spiders, fleas and centipedes. Your goal is to protect mushrooms in
 an enchanted forest. If you are bitten by a spider, flea or centipede, you will be temporally paralyzed and you will
 lose a magic wand. The game ends once you have lost all wands. You may receive additional wands after scoring
 a sufficient number of points.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=911).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
-
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -50,32 +55,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You score points by hitting centipedes, scorpions, fleas and spiders. Additional points are awarded after every round
 (i.e. after you have lost a wand) for mushrooms that were not destroyed.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=911).
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Centipede-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Centipede   | `[22, 86]`  | `[0]`              | `22`         |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Centipede-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/chopper_command.md

@@ -1,9 +1,10 @@
 ---
 title: Chopper Command
 ---
+
 # Chopper Command
 
-```{figure} ../../_static/videos/atari/chopper_command.gif 
+```{figure} ../../_static/videos/atari/chopper_command.gif
 :width: 120px
 :name: ChopperCommand
 ```
@@ -16,25 +17,30 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                             |
 | Observation High  | 255                                       |
 | Observation Low   | 0                                         |
-| Import            | `gymnasium.make("ALE/ChopperCommand-v5")` | 
+| Import            | `gymnasium.make("ALE/ChopperCommand-v5")` |
 
 ## Description
+
 You control a helicopter and must protect truck convoys. To that end, you need to shoot down enemy aircraft.
 A mini-map is displayed at the bottom of the screen.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=921).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -47,32 +53,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You score points by destroying planes and other helicopters. You score extra points at the end of every wave, depending on the number
 of trucks that have survived.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=921).
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/ChopperCommand-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment    | Valid Modes | Valid Difficulties | Default Mode |
 |----------------|-------------|--------------------|--------------|
 | ChopperCommand | `[0, 2]`    | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("ChopperCommand-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/crazy_climber.md

@@ -1,9 +1,10 @@
 ---
 title: Crazy Climber
 ---
+
 # Crazy Climber
 
-```{figure} ../../_static/videos/atari/crazy_climber.gif 
+```{figure} ../../_static/videos/atari/crazy_climber.gif
 :width: 120px
 :name: CrazyClimber
 ```
@@ -16,23 +17,24 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (250, 160, 3)                           |
 | Observation High  | 255                                     |
 | Observation Low   | 0                                       |
-| Import            | `gymnasium.make("ALE/CrazyClimber-v5")` | 
+| Import            | `gymnasium.make("ALE/CrazyClimber-v5")` |
 
 ## Description
+
 You are a climber trying to reach the top of four builidings, while avoiding obstacles like closing
 windows and falling objects. When you receive damage (windows closing or objects) you will fall and
 lose one life; you have a total of 5 lives before the end games. At the top of each building, there's
 a helicopter which you need to catch to get to the next building. The goal is to climb as fast as
-possible while receiving the least amount of damage. 
-
+possible while receiving the least amount of damage.
 
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=113).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -41,18 +43,22 @@ flavor looks like this:
 | 1   | UP        |
 | 2   | RIGHT     |
 | 3   | LEFT      |
-| 4   | DOWN      |  
+| 4   | DOWN      |
 | 5   | UPRIGHT   |
 | 6   | UPLEFT    |
 | 7   | DOWNRIGHT |
 | 8   | DOWNLEFT  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -65,29 +71,32 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 A table of scores awarded for completing each row of a building is provided on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=113).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/CrazyClimber-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment  | Valid Modes   | Valid Difficulties | Default Mode |
 |--------------|---------------|--------------------|--------------|
 | CrazyClimber | `[0, ..., 3]` | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("CrazyClimber-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/defender.md

@@ -1,10 +1,10 @@
 ---
 title: Defender
 ---
-   
+
 # Defender
 
-```{figure} ../../_static/videos/atari/defender.gif 
+```{figure} ../../_static/videos/atari/defender.gif
 :width: 120px
 :name: Defender
 ```
@@ -17,28 +17,32 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                       |
 | Observation High  | 255                                 |
 | Observation Low   | 0                                   |
-| Import            | `gymnasium.make("ALE/Defender-v5")` | 
+| Import            | `gymnasium.make("ALE/Defender-v5")` |
 
 ## Description
+
 Aliens attack the earth. You control a spaceship and must defend humanity by destroying alien ships and rescuing humanoids.
 You have three lives and three smart bombs. You lose a live when you are shot down by an alien spaceship.
-Points are scored by destroying enemies and retrieving humans that are being abducted. You have an unlimited number of 
+Points are scored by destroying enemies and retrieving humans that are being abducted. You have an unlimited number of
 laser missiles.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=128)
 
-
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -51,31 +55,33 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You receive points for destroying enemies, rescuing abducted humans and keeping humans alive.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=128).
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Defender-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes       | Valid Difficulties | Default Mode |
 |-------------|-------------------|--------------------|--------------|
 | Defender    | `[1, ..., 9, 16]` | `[0, 1]`           | `1`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Defender-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/demon_attack.md

@@ -1,9 +1,10 @@
 ---
 title: Demon Attack
 ---
+
 # Demon Attack
 
-```{figure} ../../_static/videos/atari/demon_attack.gif 
+```{figure} ../../_static/videos/atari/demon_attack.gif
 :width: 120px
 :name: DemonAttack
 ```
@@ -16,9 +17,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                          |
 | Observation High  | 255                                    |
 | Observation Low   | 0                                      |
-| Import            | `gymnasium.make("ALE/DemonAttack-v5")` | 
+| Import            | `gymnasium.make("ALE/DemonAttack-v5")` |
 
 ## Description
+
 You are facing waves of demons in the ice planet of Krybor. Points are accumulated by destroying
 demons. You begin with 3 reserve bunkers, and can increase its number (up to 6) by avoiding enemy
 attacks. Each attack wave you survive without any hits, grants you a new bunker. Every time an enemy
@@ -28,10 +30,11 @@ the game ends.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=135).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -40,18 +43,22 @@ flavor looks like this:
 | 1   | UP        |
 | 2   | RIGHT     |
 | 3   | LEFT      |
-| 4   | DOWN      |  
+| 4   | DOWN      |
 | 5   | UPRIGHT   |
 | 6   | UPLEFT    |
 | 7   | DOWNRIGHT |
 | 8   | DOWNLEFT  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -64,31 +71,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 Each enemy you slay gives you points. The amount of points depends on the type of demon and which
 wave you are in. A detailed table of scores is provided on [the AtariAge
 page](https://atariage.com/manual_html_page.php?SoftwareLabelID=135).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/DemonAttack-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes    | Valid Difficulties | Default Mode |
 |-------------|----------------|--------------------|--------------|
 | DemonAttack | `[1, 3, 5, 7]` | `[0, 1]`           | `1`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("DemonAttack-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/double_dunk.md

@@ -1,9 +1,10 @@
 ---
 title: Double Dunk
 ---
+
 # Double Dunk
 
-```{figure} ../../_static/videos/atari/double_dunk.gif 
+```{figure} ../../_static/videos/atari/double_dunk.gif
 :width: 120px
 :name: DoubleDunk
 ```
@@ -16,9 +17,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (250, 160, 3)                         |
 | Observation High  | 255                                   |
 | Observation Low   | 0                                     |
-| Import            | `gymnasium.make("ALE/DoubleDunk-v5")` | 
+| Import            | `gymnasium.make("ALE/DoubleDunk-v5")` |
 
 ## Description
+
 You are playing a 2v2 game of basketball. At the start of each possession, you select between a set
 of different plays and then execute them to either score or prevent your rivals from scoring. The
 game lasts a set amount of time or until one of the teams reaches a certain score
@@ -26,10 +28,11 @@ game lasts a set amount of time or until one of the teams reaches a certain scor
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=153).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -38,18 +41,22 @@ flavor looks like this:
 | 1   | UP        |
 | 2   | RIGHT     |
 | 3   | LEFT      |
-| 4   | DOWN      |  
+| 4   | DOWN      |
 | 5   | UPRIGHT   |
 | 6   | UPLEFT    |
 | 7   | DOWNRIGHT |
 | 8   | DOWNLEFT  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -62,31 +69,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 Scores follow the rules of basketball. You can get either 3 points, 2 points foul line) depending
 from where you shoot. After a defensive foul, a successful shot from the foul line gives you 1
 point.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/DoubleDunk-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes    | Valid Difficulties | Default Mode |
 |-------------|----------------|--------------------|--------------|
 | DoubleDunk  | `[0, ..., 15]` | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("DoubleDunk-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/elevator_action.md

@@ -1,9 +1,10 @@
 ---
 title: Elevator Action
 ---
+
 # Elevator Action
 
-```{figure} ../../_static/videos/atari/elevator_action.gif 
+```{figure} ../../_static/videos/atari/elevator_action.gif
 :width: 120px
 :name: ElevatorAction
 ```
@@ -16,9 +17,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (250, 160, 3)                             |
 | Observation High  | 255                                       |
 | Observation Low   | 0                                         |
-| Import            | `gymnasium.make("ALE/ElevatorAction-v5")` | 
+| Import            | `gymnasium.make("ALE/ElevatorAction-v5")` |
 
 ## Description
+
 You are a secret agent that must retrieve some secret documents and reach the ground level of a
 building by going down an elevator/stairs. Once you reach the ground level, you are picked up and
 taken to the next level. You are equipped with a gun to defend yourself against enemy agents waiting
@@ -29,10 +31,11 @@ This is an unreleased prototype based on the arcade game. Limited documentation
 page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=1131).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -41,18 +44,22 @@ flavor looks like this:
 | 1   | UP        |
 | 2   | RIGHT     |
 | 3   | LEFT      |
-| 4   | DOWN      |  
+| 4   | DOWN      |
 | 5   | UPRIGHT   |
 | 6   | UPLEFT    |
 | 7   | DOWNRIGHT |
 | 8   | DOWNLEFT  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -65,31 +72,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You start with 4 lives and are awarded 100 points for each enemy shot, and 500 points for each
 secret document collected (visiting a red door). Each time you get shot you lose one life and the
 game ends when losing all lives.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/ElevatorAction-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment    | Valid Modes | Valid Difficulties | Default Mode |
 |----------------|-------------|--------------------|--------------|
 | ElevatorAction | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("ElevatorAction-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/enduro.md

@@ -1,9 +1,10 @@
 ---
 title: Enduro
 ---
+
 # Enduro
 
-```{figure} ../../_static/videos/atari/enduro.gif 
+```{figure} ../../_static/videos/atari/enduro.gif
 :width: 120px
 :name: Enduro
 ```
@@ -16,20 +17,22 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (250, 160, 3)                     |
 | Observation High  | 255                               |
 | Observation Low   | 0                                 |
-| Import            | `gymnasium.make("ALE/Enduro-v5")` | 
+| Import            | `gymnasium.make("ALE/Enduro-v5")` |
 
 ## Description
+
 You are a racer in the National Enduro, a long-distance endurance race. You must overtake a certain
 amount of cars each day to stay on the race. The first day you need to pass 200 cars, and 300 for
-each following day. The game ends if you do not meet your overtake quota for the day. 
+each following day. The game ends if you do not meet your overtake quota for the day.
 
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=163).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -38,18 +41,22 @@ flavor looks like this:
 | 1   | UP        |
 | 2   | RIGHT     |
 | 3   | LEFT      |
-| 4   | DOWN      |  
+| 4   | DOWN      |
 | 5   | UPRIGHT   |
 | 6   | UPLEFT    |
 | 7   | DOWNRIGHT |
 | 8   | DOWNLEFT  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -62,29 +69,32 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You get 1 point for each vehicle you overtake.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Enduro-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Enduro      | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Enduro-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/fishing_derby.md

@@ -1,9 +1,10 @@
 ---
 title: FishingDerby
 ---
+
 # FishingDerby
 
-```{figure} ../../_static/videos/atari/fishing_derby.gif 
+```{figure} ../../_static/videos/atari/fishing_derby.gif
 :width: 120px
 :name: FishingDerby
 ```
@@ -16,18 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                           |
 | Observation High  | 255                                     |
 | Observation Low   | 0                                       |
-| Import            | `gymnasium.make("ALE/FishingDerby-v5")` | 
+| Import            | `gymnasium.make("ALE/FishingDerby-v5")` |
 
 ## Description
+
 your objective is to catch more sunfish than your opponent. But it's not just between you and the other fisherman, as a big, black shark is lurking just below the surface, waiting to steal your catch! Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=182).
 
 ### Rewards
+
 The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can
 find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=182).
 
+Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).
 
-Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). 
 ### Action Space
+
 The action space a subset of the following discrete set of legal actions:
 
 | Num | Action        |
@@ -52,19 +56,23 @@ The action space a subset of the following discrete set of legal actions:
 | 17  | DOWNLEFTFIRE  |
 
 If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have
-any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full 
+any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full
 legal space by passing the keyword argument `full_action_space=True` to `make`.
 
-The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing 
+The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing
 the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the
-action spaces of default flavors. 
+action spaces of default flavors.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -75,9 +83,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/FishingDerby-v5")
 ```
 
@@ -87,15 +96,15 @@ The various ways to configure the environment are described in detail in the art
 |--------------|-------------|--------------------|--------------|
 | FishingDerby | `[0]`       | `[0, ..., 3]`      | `0`          |
 
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("FishingDerby-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 | Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` |
 |---------|--------------|------------------------------|----------------------|

docs/environments/atari/freeway.md

@@ -1,9 +1,10 @@
 ---
 title: Freeway
 ---
+
 # Freeway
 
-```{figure} ../../_static/videos/atari/freeway.gif 
+```{figure} ../../_static/videos/atari/freeway.gif
 :width: 120px
 :name: Freeway
 ```
@@ -16,18 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                      |
 | Observation High  | 255                                |
 | Observation Low   | 0                                  |
-| Import            | `gymnasium.make("ALE/Freeway-v5")` | 
+| Import            | `gymnasium.make("ALE/Freeway-v5")` |
 
 ## Description
+
 your objective is to guide your chicken across lane after lane of busy rush hour traffic. You receive a point for every chicken that makes it to the top of the screen after crossing all the lanes of traffic. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=192).
 
 ### Rewards
+
 The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can
 find these manuals on [AtariAge](https://atariage.com/manual_thumbs.php?SoftwareLabelID=192).
 
+Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).
 
-Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). 
 ### Action Space
+
 The action space a subset of the following discrete set of legal actions:
 
 | Num | Action        |
@@ -52,19 +56,23 @@ The action space a subset of the following discrete set of legal actions:
 | 17  | DOWNLEFTFIRE  |
 
 If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have
-any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full 
+any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full
 legal space by passing the keyword argument `full_action_space=True` to `make`.
 
-The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing 
+The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing
 the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the
-action spaces of default flavors. 
+action spaces of default flavors.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -75,9 +83,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Freeway-v5")
 ```
 
@@ -85,18 +94,17 @@ The various ways to configure the environment are described in detail in the art
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
-| Freeway     | `[0, ..., 7]` | `[0, 1]`           | `0`          |      
+| Freeway     | `[0, ..., 7]` | `[0, 1]`           | `0`          |
 
-
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Freeway-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 |Version|`frameskip=`|`repeat_action_probability=`|`full_action_space=`|
 | ----- | --------- | ------------------------- | ---------|

docs/environments/atari/frostbite.md

@@ -1,9 +1,10 @@
 ---
 title: Frostbite
 ---
+
 # Frostbite
 
-```{figure} ../../_static/videos/atari/frostbite.gif 
+```{figure} ../../_static/videos/atari/frostbite.gif
 :width: 120px
 :name: Frostbite
 ```
@@ -16,18 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/Frostbite-v5")` | 
+| Import            | `gymnasium.make("ALE/Frostbite-v5")` |
 
 ## Description
+
  In Frostbite, the player controls "Frostbite Bailey" who hops back and forth across across an Arctic river, changing the color of the ice blocks from white to blue. Each time he does so, a block is added to his igloo. [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=199).
 
 ### Rewards
+
 The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can
 find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=199).
 
+Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).
 
-Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). 
 ### Action Space
+
 The action space a subset of the following discrete set of legal actions:
 
 | Num | Action        |
@@ -52,19 +56,23 @@ The action space a subset of the following discrete set of legal actions:
 | 17  | DOWNLEFTFIRE  |
 
 If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have
-any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full 
+any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full
 legal space by passing the keyword argument `full_action_space=True` to `make`.
 
-The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing 
+The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing
 the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the
-action spaces of default flavors. 
+action spaces of default flavors.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -75,9 +83,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Frostbite-v5")
 ```
 
@@ -85,18 +94,17 @@ The various ways to configure the environment are described in detail in the art
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
-| Frostbite   | `[0, 2]`    | `[0]`              | `0`          |     
+| Frostbite   | `[0, 2]`    | `[0]`              | `0`          |
 
-
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Frostbite-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 | Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` |
 |---------|--------------|------------------------------|----------------------|

docs/environments/atari/gopher.md

@@ -1,9 +1,10 @@
 ---
 title: Gopher
 ---
+
 # Gopher
 
-```{figure} ../../_static/videos/atari/gopher.gif 
+```{figure} ../../_static/videos/atari/gopher.gif
 :width: 120px
 :name: Gopher
 ```
@@ -16,18 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                     |
 | Observation High  | 255                               |
 | Observation Low   | 0                                 |
-| Import            | `gymnasium.make("ALE/Gopher-v5")` | 
+| Import            | `gymnasium.make("ALE/Gopher-v5")` |
 
 ## Description
+
 The player controls a shovel-wielding farmer who protects a crop of three carrots from a gopher. [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=218).
 
 ### Rewards
+
 The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can
 find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=218).
 
+Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).
 
-Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). 
 ### Action Space
+
 The action space a subset of the following discrete set of legal actions:
 
 | Num | Action        |
@@ -52,19 +56,23 @@ The action space a subset of the following discrete set of legal actions:
 | 17  | DOWNLEFTFIRE  |
 
 If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have
-any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full 
+any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full
 legal space by passing the keyword argument `full_action_space=True` to `make`.
 
-The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing 
+The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing
 the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the
-action spaces of default flavors. 
+action spaces of default flavors.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -75,9 +83,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Gopher-v5")
 ```
 
@@ -85,18 +94,17 @@ The various ways to configure the environment are described in detail in the art
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
-| Gopher      | `[0, 2]`    | `[0, 1]`           | `0`          |    
+| Gopher      | `[0, 2]`    | `[0, 1]`           | `0`          |
 
-
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Gopher-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 | Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` |
 |---------|--------------|------------------------------|----------------------|

docs/environments/atari/gravitar.md

@@ -1,9 +1,10 @@
 ---
 title: Gravitar
 ---
+
 # Gravitar
 
-```{figure} ../../_static/videos/atari/gravitar.gif 
+```{figure} ../../_static/videos/atari/gravitar.gif
 :width: 120px
 :name: Gravitar
 ```
@@ -16,18 +17,21 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                       |
 | Observation High  | 255                                 |
 | Observation Low   | 0                                   |
-| Import            | `gymnasium.make("ALE/Gravitar-v5")` | 
+| Import            | `gymnasium.make("ALE/Gravitar-v5")` |
 
 ## Description
+
 The player controls a small blue spacecraft. The game starts in a fictional solar system with several planets to explore. If the player moves his ship into a planet, he will be taken to a side-view landscape. Player has to destroy red bunkers [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=223).
 
 ### Rewards
+
 The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can
 find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=223).
 
+Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).
 
-Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). 
 ### Action Space
+
 The action space a subset of the following discrete set of legal actions:
 
 | Num | Action        |
@@ -52,19 +56,23 @@ The action space a subset of the following discrete set of legal actions:
 | 17  | DOWNLEFTFIRE  |
 
 If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have
-any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full 
+any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full
 legal space by passing the keyword argument `full_action_space=True` to `make`.
 
-The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing 
+The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing
 the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the
-action spaces of default flavors. 
+action spaces of default flavors.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -75,9 +83,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Gravitar-v5")
 ```
 
@@ -85,18 +94,17 @@ The various ways to configure the environment are described in detail in the art
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
-| Gravitar    | `[0, ..., 4]` | `[0]`              | `0`          |    
+| Gravitar    | `[0, ..., 4]` | `[0]`              | `0`          |
 
-
-
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Gravitar-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 | Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` |
 |---------|--------------|------------------------------|----------------------|

docs/environments/atari/hero.md

@@ -1,10 +1,10 @@
 ---
 title: Hero
 ---
-   
+
 # Hero
 
-```{figure} ../../_static/videos/atari/hero.gif 
+```{figure} ../../_static/videos/atari/hero.gif
 :width: 120px
 :name: Hero
 ```
@@ -17,28 +17,32 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                   |
 | Observation High  | 255                             |
 | Observation Low   | 0                               |
-| Import            | `gymnasium.make("ALE/Hero-v5")` | 
+| Import            | `gymnasium.make("ALE/Hero-v5")` |
 
 ## Description
+
 You need to rescue miners that are stuck in a mine shaft. You have access to various tools: A propeller backpack that
 allows you to fly wherever you want, sticks of dynamite that can be used to blast through walls, a laser beam to kill
-vermin, and a raft to float across stretches of lava. 
+vermin, and a raft to float across stretches of lava.
 You have a limited amount of power. Once you run out, you lose a live.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=228)
 
-
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -51,32 +55,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You score points for shooting critters, rescuing miners, and dynamiting walls.
 Extra points are rewarded for any power remaining after rescuing a miner.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=228).
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Hero-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
 | Hero        | `[0, ..., 4]` | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Hero-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/ice_hockey.md

@@ -1,9 +1,10 @@
 ---
 title: IceHockey
 ---
+
 # IceHockey
 
-```{figure} ../../_static/videos/atari/ice_hockey.gif 
+```{figure} ../../_static/videos/atari/ice_hockey.gif
 :width: 120px
 :name: IceHockey
 ```
@@ -16,26 +17,31 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/IceHockey-v5")` | 
+| Import            | `gymnasium.make("ALE/IceHockey-v5")` |
 
 ## Description
+
 Your goal is to score as many points as possible in a standard game of Ice Hockey over a 3-minute time period. The ball is usually called "the puck".
 There are 32 shot angles ranging from the extreme left to the extreme right. The angles can only aim towards the opponent's goal.
 Just as in real hockey, you can pass the puck by shooting it off the sides of the rink. This can be really key when you're in position to score a goal.
  Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=241)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -47,33 +53,35 @@ instead. The respective observation spaces are
 respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You score points by shooting the puck into your opponent's goal. Your opponent scores in the same manner.
 There are no limits to how many points you can get per game, other than the time limit of 3-minute games.
 For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=241).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/IceHockey-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | IceHockey   | `[0, 2]`    | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Icehockey-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/jamesbond.md

@@ -1,9 +1,10 @@
 ---
 title: Jamesbond
 ---
+
 # Jamesbond
 
-```{figure} ../../_static/videos/atari/jamesbond.gif 
+```{figure} ../../_static/videos/atari/jamesbond.gif
 :width: 120px
 :name: Jamesbond
 ```
@@ -16,9 +17,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                        |
 | Observation High  | 255                                  |
 | Observation Low   | 0                                    |
-| Import            | `gymnasium.make("ALE/Jamesbond-v5")` | 
+| Import            | `gymnasium.make("ALE/Jamesbond-v5")` |
 
 ## Description
+
 Your mission is to control Mr. Bond's specially designed multipurpose craft to complete a variety of missions.
 The craft moves forward with a right motion and slightly back with a left motion.
 An up or down motion causes the craft to jump or dive.
@@ -26,17 +28,21 @@ You can also fire by either lobbing a bomb to the bottom of the screen or firing
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=250)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -48,33 +54,35 @@ instead. The respective observation spaces are
 respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 The game ends when you complete the last mission or when you lose the last craft. In either case, you'll receive your final score.
 There will be a rating based on your score. The highest rating in NOVICE is 006. The highest rating in AGENT is 007.
 For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=250).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Jamesbond-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Jamesbond   | `[0, 1]`    | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Jamesbond-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/journey_escape.md

@@ -1,9 +1,10 @@
 ---
 title: JourneyEscape
 ---
+
 # JourneyEscape
 
-```{figure} ../../_static/videos/atari/journey_escape.gif 
+```{figure} ../../_static/videos/atari/journey_escape.gif
 :width: 120px
 :name: JourneyEscape
 ```
@@ -16,18 +17,20 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                            |
 | Observation High  | 255                                      |
 | Observation Low   | 0                                        |
-| Import            | `gymnasium.make("ALE/JourneyEscape-v5")` | 
+| Import            | `gymnasium.make("ALE/JourneyEscape-v5")` |
 
 ## Description
+
 You must lead all 5 members of JOURNEY through waves of pesky characters and backstage obstacles to the Scarab Escape Vehicle before time runs out.
 You must also protect $50,000 in concert cash from grasping groupies, photographers, and promoters.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=252)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action        |
@@ -48,13 +51,17 @@ flavor looks like this:
 | 15  | UPLEFTFIRE    |
 | 16  | DOWNRIGHTFIRE |
 | 17  | DOWNLEFTFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -66,33 +73,35 @@ instead. The respective observation spaces are
 respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 At the start of the game, you will have $50,000 and 60 units of time.
 Your end game score with be dependent on how much time you have remaining and who you encounter along the way.
 For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=252).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/JourneyEscape-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 |      Environment | Valid Modes                                                                                                                                                                         | Valid Difficulties | Default Mode |
 |------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|--------------|
 |    JourneyEscape | `[0]`                                                                                                                                                                               |           `[0, 1]` | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("JourneyEscape-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/kangaroo.md

@@ -1,9 +1,10 @@
 ---
 title: Kangaroo
 ---
+
 # Kangaroo
 
-```{figure} ../../_static/videos/atari/kangaroo.gif 
+```{figure} ../../_static/videos/atari/kangaroo.gif
 :width: 120px
 :name: Kangaroo
 ```
@@ -16,25 +17,30 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                       |
 | Observation High  | 255                                 |
 | Observation Low   | 0                                   |
-| Import            | `gymnasium.make("ALE/Kangaroo-v5")` | 
+| Import            | `gymnasium.make("ALE/Kangaroo-v5")` |
 
 ## Description
+
 The object of the game is to score as many points as you can while controlling Mother Kangaroo to rescue her precious baby. You start the game with three lives.
 During this rescue mission, Mother Kangaroo encounters many obstacles. You need to help her climb ladders, pick bonus fruit, and throw punches at monkeys.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=923)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -46,33 +52,35 @@ instead. The respective observation spaces are
 respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 Your score will be shown at the top right corner of the game.
 Your end game score with be dependent on how much time you have remaining and who you encounter along the way.
 For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=923).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Kangaroo-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Kangaroo    | `[0, 1]`    | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Kangaroo-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/krull.md

@@ -1,9 +1,10 @@
 ---
 title: Krull
 ---
+
 # Krull
 
-```{figure} ../../_static/videos/atari/krull.gif 
+```{figure} ../../_static/videos/atari/krull.gif
 :width: 120px
 :name: Krull
 ```
@@ -16,25 +17,30 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                    |
 | Observation High  | 255                              |
 | Observation Low   | 0                                |
-| Import            | `gymnasium.make("ALE/Krull-v5")` | 
+| Import            | `gymnasium.make("ALE/Krull-v5")` |
 
 ## Description
+
 Your mission is to find and enter the Beast's Black Fortress, rescue Princess Lyssa, and destroy the Beast.
 The task is not an easy one, for the location of the Black Fortress changes with each sunrise on Krull.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=267)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -46,33 +52,35 @@ instead. The respective observation spaces are
 respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
+
 You will receive various scores for each monster you kill.
 You can play the game until you have lost all your lives.
 For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=267).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Krull-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Krull       | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Krull-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/kung_fu_master.md

@@ -1,9 +1,10 @@
 ---
 title: Kung Fu Master
 ---
+
 # Kung Fu Master
 
-```{figure} ../../_static/videos/atari/kung_fu_master.gif 
+```{figure} ../../_static/videos/atari/kung_fu_master.gif
 :width: 120px
 :name: KungFuMaster
 ```
@@ -16,12 +17,14 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation Space | (210, 160, 3)                           |
 | Observation High  | 255                                     |
 | Observation Low   | 0                                       |
-| Import            | `gymnasium.make("ALE/KungFuMaster-v5")` | 
+| Import            | `gymnasium.make("ALE/KungFuMaster-v5")` |
 
 ## Description
+
 You are a Kung-Fu Master fighting your way through the Evil Wizard's temple. Your goal is to rescue Princess Victoria, defeating various enemies along the way. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=268).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action        |
@@ -40,13 +43,17 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 11  | UPLEFTFIRE    |
 | 12  | DOWNRIGHTFIRE |
 | 13  | DOWNLEFTFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -57,9 +64,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/KungFuMaster-v5")
 ```
 
@@ -69,14 +77,15 @@ The various ways to configure the environment are described in detail in the art
 |--------------|-------------|--------------------|--------------|
 | KungFuMaster | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("KungFuMaster-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/montezuma_revenge.md

@@ -1,9 +1,10 @@
 ---
 title: Montezuma Revenge
 ---
+
 # Montezuma Revenge
 
-```{figure} ../../_static/videos/atari/montezuma_revenge.gif 
+```{figure} ../../_static/videos/atari/montezuma_revenge.gif
 :width: 120px
 :name: MontezumaRevenge
 ```
@@ -19,19 +20,25 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/MontezumaRevenge-v5")` |
 
 ## Description
+
 Your goal is to acquire Montezuma's treasure by making your way through a maze of chambers within the emperor's fortress. You must avoid deadly creatures while collecting valuables and tools which can help you escape with the treasure. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=310).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
-Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions 
+Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions
 will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -42,9 +49,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/MontezumaRevenge-v5")
 ```
 
@@ -54,14 +62,15 @@ The various ways to configure the environment are described in detail in the art
 |------------------|-------------|--------------------|--------------|
 | MontezumaRevenge | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("MontezumaRevenge-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/ms_pacman.md

@@ -1,9 +1,10 @@
 ---
 title: Ms Pacman
 ---
+
 # Ms Pacman
- 
-```{figure} ../../_static/videos/atari/ms_pacman.gif 
+
+```{figure} ../../_static/videos/atari/ms_pacman.gif
 :width: 120px
 :name: MsPacman
 ```
@@ -17,11 +18,13 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Observation High  | 255                                 |
 | Observation Low   | 0                                   |
 | Import            | `gymnasium.make("ALE/MsPacman-v5")` |
- 
+
 ## Description
+
 Your goal is to collect all of the pellets on the screen while avoiding the ghosts.
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action    |
@@ -35,13 +38,17 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 6   | UPLEFT    |
 | 7   | DOWNRIGHT |
 | 8   | DOWNLEFT  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -52,9 +59,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/MsPacman-v5")
 ```
 
@@ -64,16 +72,16 @@ The various ways to configure the environment are described in detail in the art
 |-------------|---------------|--------------------|--------------|
 | MsPacman    | `[0, ..., 3]` | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("MsPacman-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/name_this_game.md

@@ -1,9 +1,10 @@
 ---
 title: Name This Game
 ---
+
 # Name This Game
 
-```{figure} ../../_static/videos/atari/name_this_game.gif 
+```{figure} ../../_static/videos/atari/name_this_game.gif
 :width: 120px
 :name: NameThisGame
 ```
@@ -19,9 +20,11 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/NameThisGame-v5")` |
 
 ## Description
+
 Your goal is to defend the treasure that you have discovered. You must fight off a shark and an octopus while keeping an eye on your oxygen supply. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=323).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action    |
@@ -34,12 +37,15 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 5   | LEFTFIRE  |
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -50,9 +56,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/NameThisGame-v5")
 ```
 
@@ -62,14 +69,15 @@ The various ways to configure the environment are described in detail in the art
 |--------------|---------------|--------------------|--------------|
 | NameThisGame | `[8, 24, 40]` | `[0, 1]`           | `8`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("NameThisGame-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/phoenix.md

@@ -1,9 +1,10 @@
 ---
 title: Phoenix
 ---
+
 # Phoenix
 
-```{figure} ../../_static/videos/atari/phoenix.gif 
+```{figure} ../../_static/videos/atari/phoenix.gif
 :width: 120px
 :name: Phoenix
 ```
@@ -19,9 +20,11 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Phoenix-v5")` |
 
 ## Description
+
 Your goal is to reach and shoot the alien pilot. On your way there, you must eliminate waves of war birds while avoiding their bombs. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=355).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action    |
@@ -34,13 +37,17 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 5   | RIGHTFIRE |
 | 6   | LEFTFIRE  |
 | 7   | DOWNFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -51,9 +58,10 @@ instead. The respective observation spaces are
 
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Phoenix-v5")
 ```
 
@@ -63,14 +71,15 @@ The various ways to configure the environment are described in detail in the art
 |-------------|-------------|--------------------|--------------|
 | Phoenix     | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Phoenix-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/pitfall.md

@@ -1,9 +1,10 @@
 ---
 title: Pitfall
 ---
+
 # Pitfall
 
-```{figure} ../../_static/videos/atari/pitfall.gif 
+```{figure} ../../_static/videos/atari/pitfall.gif
 :width: 120px
 :name: Pitfall
 ```
@@ -19,19 +20,24 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Pitfall-v5")` |
 
 ## Description
+
 You control Pitfall Harry and are tasked with collecting all the treasures in a jungle within 20 minutes. You have three lives. The game is over if you collect all the treasures or if you die or if the time runs out.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=360)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -44,32 +50,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You get score points for collecting treasure, you lose points through some misfortunes like falling down a hole.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=360).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Pitfall-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Pitfall     | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Pitfall-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/pong.md

@@ -1,9 +1,10 @@
 ---
 title: Pong
 ---
+
 # Pong
 
-```{figure} ../../_static/videos/atari/pong.gif 
+```{figure} ../../_static/videos/atari/pong.gif
 :width: 120px
 :name: Pong
 ```
@@ -19,13 +20,15 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Pong-v5")` |
 
 ## Description
-You control the right paddle, you compete against the left paddle controlled by the computer. You each try to keep deflecting the ball away from your goal and into your opponent's goal. 
+
+You control the right paddle, you compete against the left paddle controlled by the computer. You each try to keep deflecting the ball away from your goal and into your opponent's goal.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=587)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
-number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -37,13 +40,15 @@ flavor looks like this:
 | 4   | RIGHTFIRE |
 | 5   | LEFTFIRE  |
 
-
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -56,32 +61,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 You get score points for getting the ball to pass the opponent's paddle. You lose points if the ball passes your paddle.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=587).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Pong-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Pong        | `[0, 1]`    | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Pong-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/pooyan.md

@@ -1,9 +1,10 @@
 ---
 title: Pooyan
 ---
+
 # Pooyan
 
-```{figure} ../../_static/videos/atari/pooyan.gif 
+```{figure} ../../_static/videos/atari/pooyan.gif
 :width: 120px
 :name: Pooyan
 ```
@@ -20,13 +21,14 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 
 ## Description
 
-You are a mother pig protecting her piglets (Pooyans) from wolves. In the first scene, you can move up and down a rope. Try to shoot the worker's balloons, while guarding yourself from attacks. If the wolves reach the ground safely they will get behind and try to eat you. In the second scene, the wolves try to float up. You have to try and stop them using arrows and bait. You die if a wolf eats you, or a stone or rock hits you. 
+You are a mother pig protecting her piglets (Pooyans) from wolves. In the first scene, you can move up and down a rope. Try to shoot the worker's balloons, while guarding yourself from attacks. If the wolves reach the ground safely they will get behind and try to eat you. In the second scene, the wolves try to float up. You have to try and stop them using arrows and bait. You die if a wolf eats you, or a stone or rock hits you.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=372)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
-number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action   |
@@ -37,12 +39,16 @@ flavor looks like this:
 | 3   | DOWN     |
 | 4   | UPFIRE   |
 | 5   | DOWNFIRE |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -55,32 +61,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
-If you hit a balloon, wolf or stone with an arrow you score points.  
+
+If you hit a balloon, wolf or stone with an arrow you score points.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=372).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Pooyan-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes        | Valid Difficulties | Default Mode |
 |-------------|--------------------|--------------------|--------------|
 | Pooyan      | `[10, 30, 50, 70]` | `[0]`              | `10`         |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Pooyan-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/private_eye.md

@@ -1,9 +1,10 @@
 ---
 title: PrivateEye
 ---
+
 # PrivateEye
 
-```{figure} ../../_static/videos/atari/private_eye.gif 
+```{figure} ../../_static/videos/atari/private_eye.gif
 :width: 120px
 :name: PrivateEye
 ```
@@ -19,19 +20,24 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/PrivateEye-v5")` |
 
 ## Description
-You control the French Private Eye Pierre Touche. Navigate the city streets, parks, secret passages, dead-ends and one-ways in search of the ringleader, Henri Le Fiend and his gang. You also need to find evidence and stolen goods that are scattered about. There are five cases, complete each case before its statute of limitations expires. 
+
+You control the French Private Eye Pierre Touche. Navigate the city streets, parks, secret passages, dead-ends and one-ways in search of the ringleader, Henri Le Fiend and his gang. You also need to find evidence and stolen goods that are scattered about. There are five cases, complete each case before its statute of limitations expires.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=376)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -44,32 +50,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
-You score points for completing your tasks like gathering evidence, nabbing questionable characters or closing cases etc. You lose points if you get hit or if your auto is on a pothole. 
+
+You score points for completing your tasks like gathering evidence, nabbing questionable characters or closing cases etc. You lose points if you get hit or if your auto is on a pothole.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=376).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/PrivateEye-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
 | PrivateEye  | `[0, ..., 4]` | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("PrivateEye-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/qbert.md

@@ -1,9 +1,10 @@
 ---
 title: Qbert
 ---
+
 # Qbert
 
-```{figure} ../../_static/videos/atari/qbert.gif 
+```{figure} ../../_static/videos/atari/qbert.gif
 :width: 120px
 :name: Qbert
 ```
@@ -19,13 +20,15 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Qbert-v5")` |
 
 ## Description
-You are Q*bert. Your goal is to change the color of all the cubes on the pyramid to the pyramid's 'destination' color. To do this, you must hop on each cube on the pyramid one at a time while avoiding nasty creatures that lurk there. 
+
+You are Q*bert. Your goal is to change the color of all the cubes on the pyramid to the pyramid's 'destination' color. To do this, you must hop on each cube on the pyramid one at a time while avoiding nasty creatures that lurk there.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=1224&itemTypeID=HTMLMANUAL)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
-number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action |
@@ -38,11 +41,14 @@ flavor looks like this:
 | 5   | DOWN   |
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -55,32 +61,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
-You score points for changing color of the cubes to their destination colors or by defeating enemies. You also gain points for completing a level. 
+
+You score points for changing color of the cubes to their destination colors or by defeating enemies. You also gain points for completing a level.
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=1224&itemTypeID=HTMLMANUAL).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Qbert-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Qbert       | `[0]`       | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Qbert-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/riverraid.md

@@ -1,9 +1,10 @@
 ---
 title: Riverraid
 ---
+
 # Riverraid
 
-```{figure} ../../_static/videos/atari/riverraid.gif 
+```{figure} ../../_static/videos/atari/riverraid.gif
 :width: 120px
 :name: Riverraid
 ```
@@ -19,25 +20,31 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Riverraid-v5")` |
 
 ## Description
-You control a jet that flies over a river: you can move it sideways and fire missiles to destroy enemy objects. Each time an enemy object is destroyed you score points (i.e. rewards). 
 
-You lose a jet when you run out of fuel: fly over a fuel depot when you begin to run low. 
+You control a jet that flies over a river: you can move it sideways and fire missiles to destroy enemy objects. Each time an enemy object is destroyed you score points (i.e. rewards).
 
-You lose a jet even when it collides with the river bank or one of the enemy objects (except fuel depots). 
+You lose a jet when you run out of fuel: fly over a fuel depot when you begin to run low.
+
+You lose a jet even when it collides with the river bank or one of the enemy objects (except fuel depots).
 
 The game begins with a squadron of three jets in reserve and you're given an additional jet (up to 9) for each 10,000 points you score.
 
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=409)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -50,6 +57,7 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 Score points are your only reward. You get score points each time you destroy an enemy object:
 
 | Enemy Object | Score Points |
@@ -58,31 +66,33 @@ Score points are your only reward. You get score points each time you destroy an
 | Helicopter   | 60           |
 | Fuel Depot   | 80           |
 | Jet          | 100          |
-| Bridge       | 500          | 
+| Bridge       | 500          |
 
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=409).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Riverraid-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Riverraid   | `[0]`       | `[0,1]`            | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Riverraid-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/road_runner.md

@@ -1,9 +1,10 @@
 ---
 title: Road Runner
 ---
+
 # Road Runner
 
-```{figure} ../../_static/videos/atari/road_runner.gif 
+```{figure} ../../_static/videos/atari/road_runner.gif
 :width: 120px
 :name: RoadRunner
 ```
@@ -19,28 +20,34 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/RoadRunner-v0")` |
 
 ## Description
+
 You control the Road Runner(TM) in a race; you can control the direction to run in and times to jumps.
-The goal is to outrun Wile E. Coyote(TM) while avoiding the hazards of the desert. 
+The goal is to outrun Wile E. Coyote(TM) while avoiding the hazards of the desert.
 
 The game begins with three lives.  You lose a life when the coyote
 catches you, picks you up in a rocket, or shoots you with a cannon.  You also
 lose a life when a truck hits you, you hit a land mine, you fall off a cliff,
 or you get hit by a falling rock.
- 
+
 You score points (i.e. rewards) by eating seeds along the road, eating steel shot, and
 destroying the coyote.
 
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=412)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -53,6 +60,7 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 Score points are your only reward. You get score points each time you:
 
 | actions                                               | points |
@@ -63,28 +71,30 @@ Score points are your only reward. You get score points each time you:
 | get the coyote hit by a truck                         | 1000   |
 
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=412).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/RoadRunner-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | RoadRunner  | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("RoadRunner-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/robotank.md

@@ -1,9 +1,10 @@
 ---
 title: Robot Tank
 ---
+
 # Robot Tank
 
-```{figure} ../../_static/videos/atari/robotank.gif 
+```{figure} ../../_static/videos/atari/robotank.gif
 :width: 120px
 :name: Robotank
 ```
@@ -19,30 +20,36 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Robotank-v0")` |
 
 ## Description
+
 You control your Robot Tanks to destroy enemies and avoid enemy fire.
 
-Game ends when all of your Robot Tanks are 
+Game ends when all of your Robot Tanks are
     destroyed or all 12 enemy squadrons are destroyed.
 
-The game begins with one active Robot Tank and three reserves. 
+The game begins with one active Robot Tank and three reserves.
 Your Robot Tank may get lost when it is hit by enemy
     rocket fire - your video scrambles with static interference when this
     happens - or just become damaged - sensors report the damage by flashing on your control panel (look at V/C/R/T squares).
-    
+
 You earn one bonus Robot Tank for every enemy squadron destroyed. The maximum
    number of bonus Robot Tanks allowed at any one time is 12.
 
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=416)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -55,35 +62,38 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 The number of enemies destroyed is the only reward.
 
 A small tank appears at the top of your screen for each enemy
-   you destroy.  A square with the number 12 appears each time a squadron of twelve enemies are 
+   you destroy.  A square with the number 12 appears each time a squadron of twelve enemies are
    destroyed.
-   
+
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=416).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Robotank-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Robotank    | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Robotank-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/seaquest.md

@@ -1,9 +1,10 @@
 ---
 title: Seaquest
 ---
+
 # Seaquest
 
-```{figure} ../../_static/videos/atari/seaquest.gif 
+```{figure} ../../_static/videos/atari/seaquest.gif
 :width: 120px
 :name: Seaquest
 ```
@@ -19,6 +20,7 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Seaquest-v0")` |
 
 ## Description
+
 You control a sub able to move in all directions and fire torpedoes.
 The goal is to retrieve as many divers as you
 can, while dodging and blasting enemy subs and killer sharks; points will be awarded accordingly.
@@ -38,14 +40,19 @@ to surface, with less than six divers, you lose one diver as well.
 Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=424)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -58,42 +65,45 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 Score points are your only reward.
 
 Blasting enemy sub and killer shark is worth
 20 points.  Every time you surface with six divers, the value of enemy subs
-and killer sharks increases by 10, up to a maximum of 90 points each. 
+and killer sharks increases by 10, up to a maximum of 90 points each.
 
 Rescued divers start at 50 points each.  Then, their point value increases by 50, every
-time you surface, up to a maximum of 1000 points each. 
+time you surface, up to a maximum of 1000 points each.
 
 You'll be further rewarded with bonus points for all the oxygen you have remaining the
 moment you surface.  The more oxygen you have left, the more bonus points
 you're given.
- 
+
 For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=424).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Seaquest-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Seaquest    | `[0]`       | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Seaquest-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/skiing.md

@@ -1,9 +1,10 @@
 ---
 title: Skiing
 ---
+
 # Skiings
 
-```{figure} ../../_static/videos/atari/skiing.gif 
+```{figure} ../../_static/videos/atari/skiing.gif
 :width: 120px
 :name: Skiing
 ```
@@ -19,21 +20,23 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Skiing-v0")` |
 
 ## Description
+
 You control a skier who can move sideways.
 
 The goal is to run through all gates (between the poles) in the fastest time.
 You are penalized five seconds for each gate you miss.
-           
-If you hit a gate or a tree, your skier will jump back up 
+
+If you hit a gate or a tree, your skier will jump back up
 and keep going.  But you do lose time, so be careful!
 
 Detailed documentation can be found on [the AtariAge page [SLALOM RACING section]](https://atariage.com/manual_html_page.php?SoftwareLabelID=434)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
 number of actions (those that are meaningful in this game) are available. The reduced action space may depend
-on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action |
@@ -41,13 +44,17 @@ flavor looks like this:
 | 0   | NOOP   |
 | 1   | RIGHT  |
 | 2   | LEFT   |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -60,31 +67,34 @@ respectively. The general article on Atari environments outlines different ways
 via `gymnasium.make`.
 
 ### Rewards
+
 Seconds are your only rewards - negative rewards and penalties (e.g. missing a gate) are assigned as additional seconds.
 
 For a more detailed documentation, see [the AtariAge page [SLALOM RACING section]](https://atariage.com/manual_html_page.php?SoftwareLabelID=434).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Skiing-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Skiing      | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Skiing-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/solaris.md

@@ -1,9 +1,10 @@
 ---
 title: Solaris
 ---
+
 # Solaris
 
-```{figure} ../../_static/videos/atari/solaris.gif 
+```{figure} ../../_static/videos/atari/solaris.gif
 :width: 120px
 :name: Solaris
 ```
@@ -19,17 +20,23 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Solaris-v5")` |
 
 ## Description
+
 You control a spaceship. Blast enemies before they can blast you. You can warp to different sectors. You have to defend Federation planets, and destroy Zylon forces. Keep track of your fuel, if you run out you lose a life. Warp to a Federation planet to refuel. The game ends if all your ships are destroyed or if you reach the Solaris planet. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=450)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -44,30 +51,31 @@ via `gymnasium.make`.
 ### Rewards
 
 You gain points for destroying enemies, rescuing cadets, making it through a corridor, destroying enemy planets etc. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=450).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Solaris-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Solaris     | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Solaris-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/space_invaders.md

@@ -1,9 +1,10 @@
 ---
 title: SpaceInvaders
 ---
+
 # SpaceInvaders
 
-```{figure} ../../_static/videos/atari/space_invaders.gif 
+```{figure} ../../_static/videos/atari/space_invaders.gif
 :width: 120px
 :name: SpaceInvaders
 ```
@@ -23,9 +24,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 Your objective is to destroy the space invaders by shooting your laser cannon at them before they reach the Earth. The game ends when all your lives are lost after taking enemy fire, or when they reach the earth. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=460)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
-number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -38,11 +40,14 @@ flavor looks like this:
 | 5   | LEFTFIRE  |
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -57,30 +62,31 @@ via `gymnasium.make`.
 ### Rewards
 
 You gain points for destroying space invaders. The invaders in the back rows are worth more points. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=460).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/SpaceInvaders-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment   | Valid Modes    | Valid Difficulties | Default Mode |
 |---------------|----------------|--------------------|--------------|
 | SpaceInvaders | `[0, ..., 15]` | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("SpaceInvaders-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/star_gunner.md

@@ -1,9 +1,10 @@
 ---
 title: StarGunner
 ---
+
 # StarGunner
 
-```{figure} ../../_static/videos/atari/star_gunner.gif 
+```{figure} ../../_static/videos/atari/star_gunner.gif
 :width: 120px
 :name: StarGunner
 ```
@@ -23,9 +24,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 Stop the alien invasion by shooting down alien saucers and creatures while avoiding bombs. More details can be found on [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-stargunner_16921.html)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
-number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action |
@@ -38,11 +40,14 @@ flavor looks like this:
 | 5   | DOWN   |
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -60,28 +65,28 @@ You score points for destroying enemies. You get bonus points for clearing a wav
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/StarGunner-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes   | Valid Difficulties | Default Mode |
 |-------------|---------------|--------------------|--------------|
 | StarGunner  | `[0, ..., 3]` | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("StarGunner-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/tennis.md

@@ -1,9 +1,10 @@
 ---
 title: Tennis
 ---
+
 # Tennis
 
-```{figure} ../../_static/videos/atari/tennis.gif 
+```{figure} ../../_static/videos/atari/tennis.gif
 :width: 120px
 :name: Tennis
 ```
@@ -20,19 +21,23 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 
 ## Description
 
-You control the orange player playing against a computer-controlled blue player. The game follows the rules of tennis. 
+You control the orange player playing against a computer-controlled blue player. The game follows the rules of tennis.
 The first player to win at least 6 games with a margin of at least two games wins the match. If the score is tied at 6-6, the first player to go 2 games up wins the match. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=555)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor.
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -47,30 +52,31 @@ via `gymnasium.make`.
 ### Rewards
 
 The scoring is as per the sport of tennis, played till one set. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=555).
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Tennis-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Tennis      | `[0, 2]`    | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Tennis-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/time_pilot.md

@@ -1,9 +1,10 @@
 ---
 title: TimePilot
 ---
+
 # TimePilot
 
-```{figure} ../../_static/videos/atari/time_pilot.gif 
+```{figure} ../../_static/videos/atari/time_pilot.gif
 :width: 120px
 :name: TimePilot
 ```
@@ -23,9 +24,10 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 You control an aircraft. Use it to destroy your enemies. As you progress in the game, you encounter enemies with technology that is increasingly from the future. More details can be found on [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-time-pilot_8038.html)
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment.
 However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced
-number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default 
+number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default
 flavor looks like this:
 
 | Num | Action    |
@@ -37,16 +39,19 @@ flavor looks like this:
 | 4   | LEFT      |
 | 5   | DOWN      |
 | 6   | UPFIRE    |
-| 7   | RIGHTFIRE | 
-| 8   | LEFTFIRE  | 
+| 7   | RIGHTFIRE |
+| 8   | LEFTFIRE  |
 | 9   | DOWNFIRE  |
 
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -58,34 +63,33 @@ instead. The respective observation spaces are
 respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
 
-
 ### Rewards
 
 You score points for destroying enemies, gaining more points for difficult enemies. For a more detailed documentation, see [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-time-pilot_8038.html).
 
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/TimePilot-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | TimePilot   | `[0]`       | `[0, 1, 2]`        | `0`          |
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("TimePilot-v0")`.
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed
 * v0: Initial versions release (1.0.0)
-

docs/environments/atari/tutankham.md

@@ -1,9 +1,10 @@
 ---
-title: Tutankham 
+title: Tutankham
 ---
+
 # Tutankham
 
-```{figure} ../../_static/videos/atari/tutankham.gif 
+```{figure} ../../_static/videos/atari/tutankham.gif
 :width: 120px
 :name: Tutankham
 ```
@@ -19,9 +20,11 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Tutankham-v5")` |
 
 ## Description
+
 Your goal is to rack up points by finding treasures in the mazes of the tomb while eliminating its guardians. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=572).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action    |
@@ -34,13 +37,17 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 5   | UPFIRE    |
 | 6   | RIGHTFIRE |
 | 7   | LEFTFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -51,9 +58,10 @@ instead. The respective observation spaces are
 
 The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Tutankham-v5")
 ```
 
@@ -63,14 +71,15 @@ The various ways to configure the environment are described in detail in the art
 |-------------|-------------|--------------------|--------------|
 | Tutankham   | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Tutankham-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/up_n_down.md

@@ -1,9 +1,10 @@
 ---
-title: Up n' Down 
+title: Up n' Down
 ---
-# Up n' Down 
 
-```{figure} ../../_static/videos/atari/up_n_down.gif 
+# Up n' Down
+
+```{figure} ../../_static/videos/atari/up_n_down.gif
 :width: 120px
 :name: UpNDown
 ```
@@ -19,9 +20,11 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/UpNDown-v5")` |
 
 ## Description
-Your goal is to steer your baja bugger to collect prizes and eliminate opponents. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=574). 
+
+Your goal is to steer your baja bugger to collect prizes and eliminate opponents. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=574).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action   |
@@ -32,13 +35,17 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 3   | DOWN     |
 | 4   | UPFIRE   |
 | 5   | DOWNFIRE |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -49,27 +56,29 @@ instead. The respective observation spaces are
 
 The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/UpNDown-v5")
 ```
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | UpNDown     | `[0]`       | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("UpNDown-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/venture.md

@@ -1,9 +1,10 @@
 ---
 title: Venture
 ---
+
 # Venture
 
-```{figure} ../../_static/videos/atari/venture.gif 
+```{figure} ../../_static/videos/atari/venture.gif
 :width: 120px
 :name: Venture
 ```
@@ -19,17 +20,23 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Venture-v5")` |
 
 ## Description
+
 Your goal is to capture the treasure in every chamber of the dungeon while eliminating the monsters. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=576).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. Even if you use v0 or v4 or specify full_action_space=False during initialization, all actions will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -40,28 +47,30 @@ instead. The respective observation spaces are
 
 The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Venture-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment | Valid Modes | Valid Difficulties | Default Mode |
 |-------------|-------------|--------------------|--------------|
 | Venture     | `[0]`       | `[0, ..., 3]`      | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Venture-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/video_pinball.md

@@ -1,9 +1,10 @@
 ---
 title: Video Pinball
 ---
+
 # Video Pinball
 
-```{figure} ../../_static/videos/atari/video_pinball.gif 
+```{figure} ../../_static/videos/atari/video_pinball.gif
 :width: 120px
 :name: VideoPinball
 ```
@@ -19,9 +20,11 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/VideoPinball-v5")` |
 
 ## Description
+
 Your goal is to keep the ball in play as long as possible and to score as many points as possible. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=588).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action    |
@@ -35,13 +38,17 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 6   | UPFIRE    |
 | 7   | RIGHTFIRE |
 | 8   | LEFTFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -52,28 +59,30 @@ instead. The respective observation spaces are
 
 The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/'VideoPinball-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment  | Valid Modes   | Valid Difficulties | Default Mode |
 |--------------|---------------|--------------------|--------------|
 | VideoPinball | `[0, ..., 2]` | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("VideoPinball-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/wizard_of_wor.md

@@ -1,9 +1,10 @@
 ---
 title: Wizard of Wor
 ---
+
 # Wizard of Wor
 
-```{figure} ../../_static/videos/atari/wizard_of_wor.gif 
+```{figure} ../../_static/videos/atari/wizard_of_wor.gif
 :width: 120px
 :name: WizardOfWor
 ```
@@ -19,9 +20,11 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/WizardOfWor-v5")` |
 
 ## Description
+
 Your goal is to beat the Wizard using your laser and radar scanner. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=598).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this:
 
 | Num | Action    |
@@ -36,13 +39,17 @@ By default, all actions that can be performed on an Atari 2600 are available in
 | 7   | RIGHTFIRE |
 | 8   | LEFTFIRE  |
 | 9   | DOWNFIRE  |
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -53,28 +60,30 @@ instead. The respective observation spaces are
 
 The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/WizardOfWor-v5")
 ```
 
 The various ways to configure the environment are described in detail in the article on Atari environments.
-It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. 
+It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`.
 A flavor is a combination of a game mode and a difficulty setting.
 
 | Environment  | Valid Modes | Valid Difficulties | Default Mode |
 |--------------|-------------|--------------------|--------------|
 | VideoPinball | `[0]`       | `[0, 1]`           | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("WizardOfWor-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/atari/zaxxon.md

@@ -2,9 +2,10 @@
 title: Zaxxon
 lastpage:
 ---
+
 # Zaxxon
 
-```{figure} ../../_static/videos/atari/zaxxon.gif 
+```{figure} ../../_static/videos/atari/zaxxon.gif
 :width: 120px
 :name: Zaxxon
 ```
@@ -20,17 +21,23 @@ This environment is part of the <a href='..'>Atari environments</a>. Please read
 | Import            | `gymnasium.make("ALE/Zaxxon-v5")` |
 
 ## Description
+
 Your goal is to stop the evil robot Zaxxon and its armies from enslaving the galaxy by piloting your fighter and shooting enemies. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=606).
 
 ## Actions
+
 By default, all actions that can be performed on an Atari 2600 are available in this environment. Even if you use v0 or v4 or specify full_action_space=False during initialization, all actions will be available in the default flavor.
+
 ## Observations
+
 By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is
 possible to observe
+
 - The 128 Bytes of RAM of the console
 - A grayscale image
 
 instead. The respective observation spaces are
+
 - `Box([0 ... 0], [255 ... 255], (128,), uint8)`
 - `Box([[0 ... 0]
  ...
@@ -41,9 +48,10 @@ instead. The respective observation spaces are
 
 The general article on Atari environments outlines different ways to instantiate corresponding environments
 via `gymnasium.make`.
+
 ## Arguments
 
-```
+```python
 env = gymnasium.make("ALE/Zaxxon-v5")
 ```
 
@@ -53,14 +61,15 @@ The various ways to configure the environment are described in detail in the art
 |-------------|-------------|--------------------|--------------|
 | Zaxxon      | `[0]`       | `[0]`              | `0`          |
 
-You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" 
-are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in 
+You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip"
+are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in
 the general article on Atari environments.
 The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Zaxxon-v0")`
 
 ## Version History
+
 A thorough discussion of the intricate differences between the versions and configurations can be found in the
-general article on Atari environments. 
+general article on Atari environments.
 
 * v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace.
 * v4: Stickiness of actions was removed

docs/environments/mujoco.md

@@ -35,7 +35,7 @@ pip install gymnasium[mujoco]
 
 These environments also require that the MuJoCo engine be installed. As of October 2021 DeepMind has acquired MuJoCo and is open-sourcing it in 2022, making it free for everyone. Instructions on installing the MuJoCo engine can be found on their [website](https://mujoco.org) and [GitHub repository](https://github.com/deepmind/mujoco). Using MuJoCo with Gymnasium also requires that the framework `mujoco` be installed (this dependency is installed with the above command).
 
-For MuJoCo V3 environments and older the `mujoco-py` framework is required (`pip install mujoco-py`) which can be found in the [GitHub repository](https://github.com/openai/mujoco-py/tree/master/mujoco_py) 
+For MuJoCo V3 environments and older the `mujoco-py` framework is required (`pip install mujoco-py`) which can be found in the [GitHub repository](https://github.com/openai/mujoco-py/tree/master/mujoco_py)
 
 There are ten Mujoco environments: Ant, HalfCheetah, Hopper, Humanoid, HumanoidStandup, IvertedDoublePendulum, InvertedPendulum, Reacher, Swimmer, and Walker. All of these environments are stochastic in terms of their initial state, with a Gaussian noise added to a fixed initial state in order to add stochasticity. The state spaces for MuJoCo environments in Gymnasium consist of two parts that are flattened and concatenated together: a position of a body part ('*mujoco-py.mjsim.qpos*') or joint and its corresponding velocity ('*mujoco-py.mjsim.qvel*'). Often, some of the first positional elements are omitted from the state space since the reward is calculated based on their values, leaving it up to the algorithm to infer those hidden values indirectly.
 

docs/environments/third_party_environments.md

@@ -9,134 +9,175 @@ There are a number of Reinforcement Learning environments built by authors not i
 ## Video Game environments
 
 ### [stable-retro: Classic retro games, a maintained version of OpenAI Retro](https://github.com/MatPoliquin/stable-retro)
+
 Supported fork of gym-retro with additional games, states, scenarios, etc. Open to PRs of additional games, features, and platforms since gym-retro is no longer maintained
 
 ### [flappy-bird-gym: A Flappy Bird environment for Gym](https://github.com/Talendar/flappy-bird-gym)
+
 A simple environment for single-agent reinforcement learning algorithms on a clone of [Flappy Bird](https://en.wikipedia.org/wiki/Flappy_Bird), the hugely popular arcade-style mobile game. Both state and pixel observation environments are available.
 
 ### [gym-derk: GPU accelerated MOBA environment](https://gym.derkgame.com/)
+
 This is a 3v3 MOBA environment where you train creatures to fight each other. It runs entirely on the GPU so you can easily have hundreds of instances running in parallel. There are around 15 items for the creatures, 60 "senses", 5 actions, and roughly 23 tweakable rewards. It's also possible to benchmark an agent against other agents online. It's available for free for training for personal use, and otherwise costs money; see licensing details on the website
 
 ### [SlimeVolleyGym: A simple environment for single and multi-agent reinforcement learning](https://github.com/hardmaru/slimevolleygym)
+
 A simple environment for benchmarking single and multi-agent reinforcement learning algorithms on a clone of Slime Volleyball game. The only dependencies are gym and NumPy. Both state and pixel observation environments are available. The motivation of this environment is to easily enable trained agents to play against each other, and also facilitate the training of agents directly in a multi-agent setting, thus adding an extra dimension for evaluating an agent's performance.
 
 ### [Unity ML Agents: Environments for Unity game engine](https://github.com/Unity-Technologies/ml-agents)
+
 Gym wrappers for arbitrary and premade environments with the Unity game engine.
 
 ### [PGE: Parallel Game Engine](https://github.com/222464/PGE)
+
 PGE is a FOSS 3D engine for AI simulations and can interoperate with the Gym. Contains environments with modern 3D graphics, and uses Bullet for physics.
 
 ## Robotics environments
 
 ### [gym-jiminy: Training Robots in Jiminy](https://github.com/duburcqa/jiminy)
+
 gym-jiminy presents an extension of the initial Gym for robotics using Jiminy, an extremely fast and light-weight simulator for poly-articulated systems using Pinocchio for physics evaluation and Meshcat for web-based 3D rendering.
 
 ### [gym-pybullet-drones: Environments for quadcopter control](https://github.com/JacopoPan/gym-pybullet-drones)
+
 A simple environment using [PyBullet](https://github.com/bulletphysics/bullet3) to simulate the dynamics of a [Bitcraze Crazyflie 2.x](https://www.bitcraze.io/documentation/hardware/crazyflie_2_1/crazyflie_2_1-datasheet.pdf) nanoquadrotor.
 
 ### [MarsExplorer: Environments for controlling robot on Mars](https://github.com/dimikout3/MarsExplorer)
+
 Mars Explorer is a Gym compatible environment designed and developed as an initial endeavor to bridge the gap between powerful Deep Reinforcement Learning methodologies and the problem of exploration/coverage of an unknown terrain.
 
 ### [panda-gym: Robotics environments using the PyBullet physics engine](https://github.com/qgallouedec/panda-gym/)
+
 PyBullet based simulations of a robotic arm moving objects.
 
 ### [robo-gym: Real-world and simulation robotics](https://github.com/jr-robotics/robo-gym)
+
 Robo-gym provides a collection of reinforcement learning environments involving robotic tasks applicable in both simulation and real-world robotics.
 
 ### [Offworld-gym](https://github.com/offworld-projects/offworld-gym)
+
 Gym environments that let you control physics robotics in a laboratory via the internet.
 
 ### [safe-control-gym](https://github.com/utiasDSL/safe-control-gym)
+
 PyBullet based CartPole and Quadrotor environments—with [CasADi](https://web.casadi.org) (symbolic) *a priori* dynamics and constraints—for learning-based control and model-based reinforcement learning.
 
 ### [gym-softrobot: Soft-robotics environments](https://github.com/skim0119/gym-softrobot/)
-A large-scale benchmark for co-optimizing the design and control of soft robots. 
+
+A large-scale benchmark for co-optimizing the design and control of soft robots.
 
 ### [iGibson](https://github.com/StanfordVL/iGibson/)
+
 A Simulation Environment to train Robots in Large Realistic Interactive Scenes
 
 ### [DexterousHands: dual dexterous hand manipulation tasks](https://github.com/PKU-MARL/DexterousHands/)
+
 This is a library that provides dual dexterous hand manipulation tasks through Isaac Gym
 
 ### [OmniIsaacGymEnvs](https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs/)
+
 Reinforcement Learning Environments for Omniverse Isaac Gym
 
 ## Autonomous Driving environments
 
 ### [sumo-rl](https://github.com/LucasAlegre/sumo-rl)
+
 Gym wrapper for various environments in the Sumo traffic simulator
 
 ### [gym-duckietown](https://github.com/duckietown/gym-duckietown)
+
 A lane-following simulator built for the [Duckietown](http://duckietown.org/) project (small-scale self-driving car course).
 
 ### [gym-electric-motor](https://github.com/upb-lea/gym-electric-motor)
+
 An environment for simulating a wide variety of electric drives taking into account different types of electric motors and converters. Control schemes can be continuous, yielding a voltage duty cycle, or discrete, determining converter switching states directly.
 
 ### [highway-env](https://github.com/eleurent/highway-env)
+
 An environment for behavioral planning in autonomous driving, with an emphasis on high-level perception and decision rather than low-level sensing and control. The difficulty of the task lies in understanding the social interactions with other drivers, whose behaviors are uncertain. Several scenes are proposed, such as highway, merge, intersection and roundabout.
 
 ### [CommonRoad-RL](https://commonroad.in.tum.de/tools/commonroad-rl)
+
 A Gym for solving motion planning problems for various traffic scenarios compatible with [CommonRoad benchmarks](https://commonroad.in.tum.de/scenarios), which provides configurable rewards, action spaces, and observation spaces.
 
 ### [tmrl: TrackMania 2020 through RL](https://github.com/trackmania-rl/tmrl/)
+
 tmrl is a distributed framework for training Deep Reinforcement Learning AIs in real-time applications. It is demonstrated on the TrackMania 2020 video game.
 
 ### [racing_dreamer](https://github.com/CPS-TUWien/racing_dreamer/)
+
 Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing
 
 ### [l2r: Multimodal control environment where agents learn how to race](https://github.com/learn-to-race/l2r/)
+
 An open-source reinforcement learning environment for autonomous racing.
 
 ### [racecar_gym](https://github.com/axelbr/racecar_gym/)
+
 A gym environment for a miniature racecar using the pybullet physics engine.
 
-## Other environments 
+## Other environments
 
 ### [CompilerGym: Optimise compiler tasks](https://github.com/facebookresearch/CompilerGym)
+
 Reinforcement learning environments for compiler optimization tasks, such as LLVM phase ordering, GCC flag tuning, and CUDA loop nest code generation.
 
 ### [CARL: context adaptive RL](https://github.com/automl/CARL)
+
 Configurable reinforcement learning environments for testing generalization, e.g. CartPole with variable pole lengths or Brax robots with different ground frictions.
 
 ### [matrix-mdp: Easily create discrete MDPs](https://github.com/Paul-543NA/matrix-mdp-gym)
+
 An environment to easily implement discrete MDPs as gym environments. Turn a set of matrices (`P_0(s)`, `P(s'| s, a)` and `R(s', s, a)`) into a gym environment that represents the discrete MDP ruled by these dynamics.
 
 ### [mo-gym: Multi-objective Reinforcement Learning environments](https://github.com/LucasAlegre/mo-gym)
+
 Multi-objective RL (MORL) gym environments, where the reward is a NumPy array of different (possibly conflicting) objectives.
 
 ### [gym-cellular-automata: Cellular Automata environments](https://github.com/elbecerrasoto/gym-cellular-automata)
+
 Environments where the agent interacts with _Cellular Automata_ by changing its cell states.
 
 ### [gym-sokoban: 2D Transportation Puzzles](https://github.com/mpSchrader/gym-sokoban)
+
 The environment consists of transportation puzzles in which the player's goal is to push all boxes to the warehouse's storage locations. The advantage of the environment is that it generates a new random level every time it is initialized or reset, which prevents overfitting to predefined levels.
 
 ### [DACBench: Benchmark Library for Dynamic Algorithm configuration](https://github.com/automl/DACBench)
-Environments for hyperparameter configuration using RL. Includes cheap surrogate benchmarks as well as real-world algorithms from e.g. AI Planning, Evolutionary Computation and Deep Learning. 
+
+Environments for hyperparameter configuration using RL. Includes cheap surrogate benchmarks as well as real-world algorithms from e.g. AI Planning, Evolutionary Computation and Deep Learning.
 
 ### [NLPGym: A toolkit to develop RL agents to solve NLP tasks](https://github.com/rajcscw/nlp-gym)
+
 [NLPGym](https://arxiv.org/pdf/2011.08272v1.pdf) provides interactive environments for standard NLP tasks such as sequence tagging, question answering, and sequence classification. Users can easily customize the tasks with their datasets, observations, features and reward functions.
 
 ### [gym-saturation: Environments used to prove theorems](https://github.com/inpefess/gym-saturation)
+
 An environment for guiding automated theorem provers based on saturation algorithms (e.g. [Vampire](https://github.com/vprover/vampire)).
 
 ### [ShinRL: Environments for evaluating RL algorithms](https://github.com/omron-sinicx/ShinRL/)
+
 ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectives (Deep RL Workshop 2021)
 
 ### [gymnax: Hardware Accelerated RL Environments](https://github.com/RobertTLange/gymnax/)
+
 RL Environments in JAX which allows for highly vectorised environments with support for a number of environments, Gym, MinAtari, bsuite and more.
 
 ### [gym-anytrading: Financial trading environments for FOREX and STOCKS](https://github.com/AminHP/gym-anytrading)
+
 AnyTrading is a collection of Gym environments for reinforcement learning-based trading algorithms with a great focus on simplicity, flexibility, and comprehensiveness.
 
 ### [gym-mtsim: Financial trading for MetaTrader 5 platform](https://github.com/AminHP/gym-mtsim)
+
 MtSim is a simulator for the MetaTrader 5 trading platform for reinforcement learning-based trading algorithms. MetaTrader 5 is a multi-asset platform that allows trading Forex, Stocks, Crypto, and Futures.
 
 ### [openmodelica-microgrid-gym: Environments for controlling power electronic converters in microgrids](https://github.com/upb-lea/openmodelica-microgrid-gym)
+
 The OpenModelica Microgrid Gym (OMG) package is a software toolbox for the simulation and control optimization of microgrids based on energy conversion by power electronic converters.
 
 ### [mobile-env: Environments for coordination of wireless mobile networks](https://github.com/stefanbschneider/mobile-env/)
+
 An open, minimalist Gym environment for autonomous coordination in wireless mobile networks.
 
 ### [GymFC: A flight control tuning and training framework](https://github.com/wil3/gymfc/)
-GymFC is a modular framework for synthesizing neuro-flight controllers. The architecture integrates digital twinning concepts to provide a seamless transfer of trained policies to hardware. The environment has been used to generate policies for the world's first open-source neural network flight control firmware [Neuroflight](https://github.com/wil3/neuroflight).
 
+GymFC is a modular framework for synthesizing neuro-flight controllers. The architecture integrates digital twinning concepts to provide a seamless transfer of trained policies to hardware. The environment has been used to generate policies for the world's first open-source neural network flight control firmware [Neuroflight](https://github.com/wil3/neuroflight).

docs/index.md

@@ -6,7 +6,6 @@ lastpage:
 
 # Gymnasium is a standard API for reinforcement learning, and a diverse collection of reference environments
 
-
 ```{figure} _static/videos/box2d/lunar_lander_continuous.gif
    :alt: Lunar Lander
    :width: 500

docs/requirements.txt

@@ -4,4 +4,4 @@ myst-parser
 sphinx_gallery
 git+https://github.com/Farama-Foundation/Celshast#egg=furo
 moviepy
-pygame
+pygame

docs/scripts/gen_envs_display.py

@@ -1,6 +1,7 @@
 import os
 import sys
 
+
 all_envs = [
     {
         "id": "mujoco",

docs/scripts/gen_gifs.py

@@ -6,9 +6,10 @@ import re
 
 from PIL import Image
 from tqdm import tqdm
-from utils import kill_strs
 
 import gymnasium
+from utils import kill_strs
+
 
 # snake to camel case: https://stackoverflow.com/questions/1175208/elegant-python-function-to-convert-camelcase-to-snake-case # noqa: E501
 pattern = re.compile(r"(?<!^)(?=[A-Z])")

docs/scripts/gen_mds.py

@@ -7,9 +7,10 @@ from functools import reduce
 
 import numpy as np
 from tqdm import tqdm
-from utils import kill_strs, trim
 
 import gymnasium as gym
+from utils import kill_strs, trim
+
 
 LAYOUT = "env"
 

docs/scripts/move_404.py

@@ -1,5 +1,6 @@
 import sys
 
+
 if __name__ == "__main__":
     if len(sys.argv) < 2:
         print("Provide a path")