gaspersic/Gymnasium
1
0
Fork 0
mirror of https://github.com/Farama-Foundation/Gymnasium.git synced 2025-07-31 22:04:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add missing commits in PR#29 (#34)

Browse Source
This commit is contained in:
Manuel Goulão
2022-10-03 19:01:14 +01:00
committed by GitHub
parent f0fd69d7f2
commit bd0f9cf023
21 changed files with 164 additions and 1254 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
.github/workflows/build-docs.yml vendored
View File

@@ -8,22 +8,22 @@ jobs:
docs: docs:
name: Generate Website name: Generate Website
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- uses: actions/setup-python@v4 - uses: actions/setup-python@v4
with: with:
python-version: '3.9' python-version: '3.9'
- name: Install dependencies - name: Install dependencies
run: pip install -r docs/requirements.txt run: pip install -r docs/requirements.txt
- name: Install Gymnasium - name: Install Gymnasium
run: pip install mujoco && pip install .[atari,accept-rom-license,box2d] run: pip install mujoco && pip install .[atari,accept-rom-license,box2d]
- name: Build Envs Docs - name: Build Envs Docs
run: python docs/scripts/gen_mds.py run: python docs/scripts/gen_mds.py && python docs/scripts/gen_envs_display.py
- name: Build - name: Build
run: sphinx-build -b dirhtml -v docs _build run: sphinx-build -b dirhtml -v docs _build

11
docs/.gitignore vendored
View File

@@ -2,4 +2,13 @@
__pycache__ __pycache__
.vscode/ .vscode/
build/ build/
_build/ _build/
environments/**/list.html
environments/**/complete_list.html
environments/box2d/*.md
environments/classic_control/*.md
environments/mujoco/*.md
environments/third_party_environments/*.md
environments/toy_text/*.md

166
docs/environments/atari/index.md → docs/environments/atari.md
View File

@@ -9,75 +9,75 @@ A set of Atari 2600 environment simulated through Stella and the Arcade Learning
```{toctree} ```{toctree}
:hidden: :hidden:
adventure atari/adventure
air_raid atari/air_raid
alien atari/alien
amidar atari/amidar
assault atari/assault
asterix atari/asterix
asteroids atari/asteroids
atlantis atari/atlantis
bank_heist atari/bank_heist
battle_zone atari/battle_zone
beam_rider atari/beam_rider
berzerk atari/berzerk
bowling atari/bowling
boxing atari/boxing
breakout atari/breakout
carnival atari/carnival
centipede atari/centipede
chopper_command atari/chopper_command
crazy_climber atari/crazy_climber
defender atari/defender
demon_attack atari/demon_attack
double_dunk atari/double_dunk
elevator_action atari/elevator_action
enduro atari/enduro
fishing_derby atari/fishing_derby
freeway atari/freeway
frostbite atari/frostbite
gopher atari/gopher
gravitar atari/gravitar
hero atari/hero
ice_hockey atari/ice_hockey
jamesbond atari/jamesbond
journey_escape atari/journey_escape
kangaroo atari/kangaroo
krull atari/krull
kung_fu_master atari/kung_fu_master
montezuma_revenge atari/montezuma_revenge
ms_pacman atari/ms_pacman
name_this_game atari/name_this_game
phoenix atari/phoenix
pitfall atari/pitfall
pong atari/pong
pooyan atari/pooyan
private_eye atari/private_eye
qbert atari/qbert
riverraid atari/riverraid
road_runner atari/road_runner
robotank atari/robotank
seaquest atari/seaquest
skiing atari/skiing
solaris atari/solaris
space_invaders atari/space_invaders
star_gunner atari/star_gunner
tennis atari/tennis
time_pilot atari/time_pilot
tutankham atari/tutankham
up_n_down atari/up_n_down
venture atari/venture
video_pinball atari/video_pinball
wizard_of_wor atari/wizard_of_wor
yars_revenge atari/yars_revenge
zaxxon atari/zaxxon
``` ```
```{raw} html ```{raw} html
:file: index.html :file: atari/list.html
``` ```
Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).
### AutoROM (installing the ROMs) ### AutoROM (installing the ROMs)
@@ -113,12 +113,12 @@ The action space is a subset of the following discrete set of legal actions:
| 17 | DOWNLEFTFIRE | | 17 | DOWNLEFTFIRE |
If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have
any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full
legal space by passing the keyword argument `full_action_space=True` to `make`. legal space by passing the keyword argument `full_action_space=True` to `make`.
The reduced action space of an Atari environment may depend on the "flavor" of the game. You can specify the flavor by providing The reduced action space of an Atari environment may depend on the "flavor" of the game. You can specify the flavor by providing
the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the
action spaces of default flavor choices. action spaces of default flavor choices.
### Observation Space ### Observation Space
The observation issued by an Atari environment may be: The observation issued by an Atari environment may be:
@@ -131,26 +131,26 @@ The exact reward dynamics depend on the environment and are usually documented i
find these manuals on [AtariAge](https://atariage.com/). find these manuals on [AtariAge](https://atariage.com/).
### Stochasticity ### Stochasticity
It was pointed out in [[1]](#1) that Atari games are entirely deterministic. Thus, agents could achieve It was pointed out in [[1]](#1) that Atari games are entirely deterministic. Thus, agents could achieve
state of the art performance by simply memorizing an optimal sequence of actions while completely ignoring observations from the environment. state of the art performance by simply memorizing an optimal sequence of actions while completely ignoring observations from the environment.
To avoid this, ALE implements sticky actions: Instead of always simulating the action passed to the environment, there is a small To avoid this, ALE implements sticky actions: Instead of always simulating the action passed to the environment, there is a small
probability that the previously executed action is used instead. probability that the previously executed action is used instead.
On top of this, Gymnasium implements stochastic frame skipping: In each environment step, the action is repeated for a random On top of this, Gymnasium implements stochastic frame skipping: In each environment step, the action is repeated for a random
number of frames. This behavior may be altered by setting the keyword argument `frameskip` to either a positive integer or number of frames. This behavior may be altered by setting the keyword argument `frameskip` to either a positive integer or
a tuple of two positive integers. If `frameskip` is an integer, frame skipping is deterministic, and in each step the action is a tuple of two positive integers. If `frameskip` is an integer, frame skipping is deterministic, and in each step the action is
repeated `frameskip` many times. Otherwise, if `frameskip` is a tuple, the number of skipped frames is chosen uniformly at repeated `frameskip` many times. Otherwise, if `frameskip` is a tuple, the number of skipped frames is chosen uniformly at
random between `frameskip[0]` (inclusive) and `frameskip[1]` (exclusive) in each environment step. random between `frameskip[0]` (inclusive) and `frameskip[1]` (exclusive) in each environment step.
### Common Arguments ### Common Arguments
When initializing Atari environments via `gymnasium.make`, you may pass some additional arguments. These work for any When initializing Atari environments via `gymnasium.make`, you may pass some additional arguments. These work for any
Atari environment. However, legal values for `mode` and `difficulty` depend on the environment. Atari environment. However, legal values for `mode` and `difficulty` depend on the environment.
- **mode**: `int`. Game mode, see [[2]](#2). Legal values depend on the environment and are listed in the table above. - **mode**: `int`. Game mode, see [[2]](#2). Legal values depend on the environment and are listed in the table above.
- **difficulty**: `int`. Difficulty of the game, see [[2]](#2). Legal values depend on the environment and are listed in - **difficulty**: `int`. Difficulty of the game, see [[2]](#2). Legal values depend on the environment and are listed in
the table above. Together with `mode`, this determines the "flavor" of the game. the table above. Together with `mode`, this determines the "flavor" of the game.
- **obs_type**: `str`. This argument determines what observations are returned by the environment. Its values are: - **obs_type**: `str`. This argument determines what observations are returned by the environment. Its values are:
@@ -168,7 +168,7 @@ action space will be reduced to a subset.
- **render_mode**: `str`. Specifies the rendering mode. Its values are: - **render_mode**: `str`. Specifies the rendering mode. Its values are:
- human: We'll interactively display the screen and enable game sounds. This will lock emulation to the ROMs specified FPS - human: We'll interactively display the screen and enable game sounds. This will lock emulation to the ROMs specified FPS
- rgb_array: we'll return the `rgb` key in step metadata with the current environment RGB frame. - rgb_array: we'll return the `rgb` key in step metadata with the current environment RGB frame.
> It is highly recommended to specify `render_mode` during construction instead of calling `env.render()`. > It is highly recommended to specify `render_mode` during construction instead of calling `env.render()`.
> This will guarantee proper scaling, audio support, and proper framerates > This will guarantee proper scaling, audio support, and proper framerates
@@ -282,15 +282,15 @@ the available modes and difficulty levels for different Atari games:
### References ### References
(#1)= (#1)=
<a id="1">[1]</a> <a id="1">[1]</a>
MG Bellemare, Y Naddaf, J Veness, and M Bowling. MG Bellemare, Y Naddaf, J Veness, and M Bowling.
"The arcade learning environment: An evaluation platform for general agents." "The arcade learning environment: An evaluation platform for general agents."
Journal of Artificial Intelligence Research (2012). Journal of Artificial Intelligence Research (2012).
(#2)= (#2)=
<a id="2">[2]</a> <a id="2">[2]</a>
Machado et al. Machado et al.
"Revisiting the Arcade Learning Environment: Evaluation Protocols "Revisiting the Arcade Learning Environment: Evaluation Protocols
and Open Problems for General Agents" and Open Problems for General Agents"
Journal of Artificial Intelligence Research (2018) Journal of Artificial Intelligence Research (2018)
URL: https://jair.org/index.php/jair/article/view/11182 URL: https://jair.org/index.php/jair/article/view/11182

749
docs/environments/atari/complete_list.html
View File

@@ -1,749 +0,0 @@
<div class="env-grid">
<a href="../adventure">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/adventure.gif">
</div>
<div class="cell__title">
<span>Adventure</span>
</div>
</div>
</a>
<a href="../air_raid">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/air_raid.gif">
</div>
<div class="cell__title">
<span>Air Raid</span>
</div>
</div>
</a>
<a href="../alien">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/alien.gif">
</div>
<div class="cell__title">
<span>Alien</span>
</div>
</div>
</a>
<a href="../amidar">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/amidar.gif">
</div>
<div class="cell__title">
<span>Amidar</span>
</div>
</div>
</a>
<a href="../assault">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/assault.gif">
</div>
<div class="cell__title">
<span>Assault</span>
</div>
</div>
</a>
<a href="../asterix">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/asterix.gif">
</div>
<div class="cell__title">
<span>Asterix</span>
</div>
</div>
</a>
<a href="../asteroids">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/asteroids.gif">
</div>
<div class="cell__title">
<span>Asteroids</span>
</div>
</div>
</a>
<a href="../atlantis">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/atlantis.gif">
</div>
<div class="cell__title">
<span>Atlantis</span>
</div>
</div>
</a>
<a href="../bank_heist">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/bank_heist.gif">
</div>
<div class="cell__title">
<span>Bank Heist</span>
</div>
</div>
</a>
<a href="../battle_zone">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/battle_zone.gif">
</div>
<div class="cell__title">
<span>Battle Zone</span>
</div>
</div>
</a>
<a href="../beam_rider">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/beam_rider.gif">
</div>
<div class="cell__title">
<span>Beam Rider</span>
</div>
</div>
</a>
<a href="../berzerk">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/berzerk.gif">
</div>
<div class="cell__title">
<span>Berzerk</span>
</div>
</div>
</a>
<a href="../bowling">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/bowling.gif">
</div>
<div class="cell__title">
<span>Bowling</span>
</div>
</div>
</a>
<a href="../boxing">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/boxing.gif">
</div>
<div class="cell__title">
<span>Boxing</span>
</div>
</div>
</a>
<a href="../breakout">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/breakout.gif">
</div>
<div class="cell__title">
<span>Breakout</span>
</div>
</div>
</a>
<a href="../carnival">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/carnival.gif">
</div>
<div class="cell__title">
<span>Carnival</span>
</div>
</div>
</a>
<a href="../centipede">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/centipede.gif">
</div>
<div class="cell__title">
<span>Centipede</span>
</div>
</div>
</a>
<a href="../chopper_command">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/chopper_command.gif">
</div>
<div class="cell__title">
<span>Chopper Command</span>
</div>
</div>
</a>
<a href="../crazy_climber">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/crazy_climber.gif">
</div>
<div class="cell__title">
<span>Crazy Climber</span>
</div>
</div>
</a>
<a href="../defender">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/defender.gif">
</div>
<div class="cell__title">
<span>Defender</span>
</div>
</div>
</a>
<a href="../demon_attack">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/demon_attack.gif">
</div>
<div class="cell__title">
<span>Demon Attack</span>
</div>
</div>
</a>
<a href="../double_dunk">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/double_dunk.gif">
</div>
<div class="cell__title">
<span>Double Dunk</span>
</div>
</div>
</a>
<a href="../elevator_action">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/elevator_action.gif">
</div>
<div class="cell__title">
<span>Elevator Action</span>
</div>
</div>
</a>
<a href="../enduro">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/enduro.gif">
</div>
<div class="cell__title">
<span>Enduro</span>
</div>
</div>
</a>
<a href="../fishing_derby">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/fishing_derby.gif">
</div>
<div class="cell__title">
<span>Fishing Derby</span>
</div>
</div>
</a>
<a href="../freeway">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/freeway.gif">
</div>
<div class="cell__title">
<span>Freeway</span>
</div>
</div>
</a>
<a href="../frostbite">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/frostbite.gif">
</div>
<div class="cell__title">
<span>Frostbite</span>
</div>
</div>
</a>
<a href="../gopher">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/gopher.gif">
</div>
<div class="cell__title">
<span>Gopher</span>
</div>
</div>
</a>
<a href="../gravitar">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/gravitar.gif">
</div>
<div class="cell__title">
<span>Gravitar</span>
</div>
</div>
</a>
<a href="../hero">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/hero.gif">
</div>
<div class="cell__title">
<span>Hero</span>
</div>
</div>
</a>
<a href="../ice_hockey">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/ice_hockey.gif">
</div>
<div class="cell__title">
<span>Ice Hockey</span>
</div>
</div>
</a>
<a href="../jamesbond">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/jamesbond.gif">
</div>
<div class="cell__title">
<span>Jamesbond</span>
</div>
</div>
</a>
<a href="../journey_escape">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/journey_escape.gif">
</div>
<div class="cell__title">
<span>Journey Escape</span>
</div>
</div>
</a>
<a href="../kangaroo">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/kangaroo.gif">
</div>
<div class="cell__title">
<span>Kangaroo</span>
</div>
</div>
</a>
<a href="../krull">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/krull.gif">
</div>
<div class="cell__title">
<span>Krull</span>
</div>
</div>
</a>
<a href="../kung_fu_master">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/kung_fu_master.gif">
</div>
<div class="cell__title">
<span>Kung Fu Master</span>
</div>
</div>
</a>
<a href="../montezuma_revenge">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/montezuma_revenge.gif">
</div>
<div class="cell__title">
<span>Montezuma Revenge</span>
</div>
</div>
</a>
<a href="../ms_pacman">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/ms_pacman.gif">
</div>
<div class="cell__title">
<span>Ms Pacman</span>
</div>
</div>
</a>
<a href="../name_this_game">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/name_this_game.gif">
</div>
<div class="cell__title">
<span>Name This Game</span>
</div>
</div>
</a>
<a href="../phoenix">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/phoenix.gif">
</div>
<div class="cell__title">
<span>Phoenix</span>
</div>
</div>
</a>
<a href="../pitfall">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/pitfall.gif">
</div>
<div class="cell__title">
<span>Pitfall</span>
</div>
</div>
</a>
<a href="../pong">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/pong.gif">
</div>
<div class="cell__title">
<span>Pong</span>
</div>
</div>
</a>
<a href="../pooyan">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/pooyan.gif">
</div>
<div class="cell__title">
<span>Pooyan</span>
</div>
</div>
</a>
<a href="../private_eye">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/private_eye.gif">
</div>
<div class="cell__title">
<span>Private Eye</span>
</div>
</div>
</a>
<a href="../qbert">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/qbert.gif">
</div>
<div class="cell__title">
<span>Qbert</span>
</div>
</div>
</a>
<a href="../riverraid">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/riverraid.gif">
</div>
<div class="cell__title">
<span>Riverraid</span>
</div>
</div>
</a>
<a href="../road_runner">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/road_runner.gif">
</div>
<div class="cell__title">
<span>Road Runner</span>
</div>
</div>
</a>
<a href="../robotank">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/robotank.gif">
</div>
<div class="cell__title">
<span>Robotank</span>
</div>
</div>
</a>
<a href="../seaquest">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/seaquest.gif">
</div>
<div class="cell__title">
<span>Seaquest</span>
</div>
</div>
</a>
<a href="../skiing">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/skiing.gif">
</div>
<div class="cell__title">
<span>Skiing</span>
</div>
</div>
</a>
<a href="../solaris">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/solaris.gif">
</div>
<div class="cell__title">
<span>Solaris</span>
</div>
</div>
</a>
<a href="../space_invaders">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/space_invaders.gif">
</div>
<div class="cell__title">
<span>Space Invaders</span>
</div>
</div>
</a>
<a href="../star_gunner">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/star_gunner.gif">
</div>
<div class="cell__title">
<span>Star Gunner</span>
</div>
</div>
</a>
<a href="../tennis">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/tennis.gif">
</div>
<div class="cell__title">
<span>Tennis</span>
</div>
</div>
</a>
<a href="../time_pilot">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/time_pilot.gif">
</div>
<div class="cell__title">
<span>Time Pilot</span>
</div>
</div>
</a>
<a href="../tutankham">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/tutankham.gif">
</div>
<div class="cell__title">
<span>Tutankham</span>
</div>
</div>
</a>
<a href="../up_n_down">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/up_n_down.gif">
</div>
<div class="cell__title">
<span>Up N Down</span>
</div>
</div>
</a>
<a href="../venture">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/venture.gif">
</div>
<div class="cell__title">
<span>Venture</span>
</div>
</div>
</a>
<a href="../video_pinball">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/video_pinball.gif">
</div>
<div class="cell__title">
<span>Video Pinball</span>
</div>
</div>
</a>
<a href="../wizard_of_wor">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/wizard_of_wor.gif">
</div>
<div class="cell__title">
<span>Wizard Of Wor</span>
</div>
</div>
</a>
<a href="../yars_revenge">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/yars_revenge.gif">
</div>
<div class="cell__title">
<span>Yars Revenge</span>
</div>
</div>
</a>
<a href="../zaxxon">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/zaxxon.gif">
</div>
<div class="cell__title">
<span>Zaxxon</span>
</div>
</div>
</a>
</div>

113
docs/environments/atari/index.html
View File

@@ -1,113 +0,0 @@
<div class="env-grid">
<a href="adventure">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/adventure.gif">
</div>
<div class="cell__title">
<span>Adventure</span>
</div>
</div>
</a>
<a href="air_raid">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/air_raid.gif">
</div>
<div class="cell__title">
<span>Air Raid</span>
</div>
</div>
</a>
<a href="alien">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/alien.gif">
</div>
<div class="cell__title">
<span>Alien</span>
</div>
</div>
</a>
<a href="amidar">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/amidar.gif">
</div>
<div class="cell__title">
<span>Amidar</span>
</div>
</div>
</a>
<a href="assault">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/assault.gif">
</div>
<div class="cell__title">
<span>Assault</span>
</div>
</div>
</a>
<a href="asterix">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/asterix.gif">
</div>
<div class="cell__title">
<span>Asterix</span>
</div>
</div>
</a>
<a href="asteroids">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/asteroids.gif">
</div>
<div class="cell__title">
<span>Asteroids</span>
</div>
</div>
</a>
<a href="atlantis">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/atlantis.gif">
</div>
<div class="cell__title">
<span>Atlantis</span>
</div>
</div>
</a>
<a href="bank_heist">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/atari/bank_heist.gif">
</div>
<div class="cell__title">
<span>Bank Heist</span>
</div>
</div>
</a>
</div>
<a href="./complete_list"><button class="more-btn">See More Environments</button></a>

18
docs/environments/box2d/index.md → docs/environments/box2d.md
View File

@@ -8,17 +8,17 @@ lastpage:
```{toctree} ```{toctree}
:hidden: :hidden:
bipedal_walker box2d/bipedal_walker
car_racing box2d/car_racing
lunar_lander box2d/lunar_lander
```
```{raw} html
:file: index.html
``` ```
```{raw} html
:file: box2d/list.html
```
These environments all involve toy games based around physics control, using [box2d](https://box2d.org/) based physics and PyGame based rendering. These environments were contributed back in the early days of Gymnasium by Oleg Klimov, and have become popular toy benchmarks ever since. All environments are highly configurable via arguments specified in each environment's documentation. These environments all involve toy games based around physics control, using [box2d](https://box2d.org/) based physics and PyGame based rendering. These environments were contributed back in the early days of Gymnasium by Oleg Klimov, and have become popular toy benchmarks ever since. All environments are highly configurable via arguments specified in each environment's documentation.
The unique dependencies for this set of environments can be installed via: The unique dependencies for this set of environments can be installed via:
````bash ````bash

0
docs/environments/box2d/.gitkeep Normal file
View File

41
docs/environments/box2d/index.html
View File

@@ -1,41 +0,0 @@
<div class="env-grid">
<a href="bipedal_walker">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/box2d/bipedal_walker.gif">
</div>
<div class="cell__title">
<span>Bipedal Walker</span>
</div>
</div>
</a>
<a href="car_racing">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/box2d/car_racing.gif">
</div>
<div class="cell__title">
<span>Car Racing</span>
</div>
</div>
</a>
<a href="lunar_lander">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/box2d/lunar_lander.gif">
</div>
<div class="cell__title">
<span>Lunar Lander</span>
</div>
</div>
</a>
</div>

14
docs/environments/classic_control/index.md → docs/environments/classic_control.md
View File

@@ -8,15 +8,15 @@ lastpage:
```{toctree} ```{toctree}
:hidden: :hidden:
acrobot classic_control/acrobot
cart_pole classic_control/cart_pole
mountain_car_continuous classic_control/mountain_car_continuous
mountain_car classic_control/mountain_car
pendulum classic_control/pendulum
``` ```
```{raw} html ```{raw} html
:file: index.html :file: classic_control/list.html
``` ```
The unique dependencies for this set of environments can be installed via: The unique dependencies for this set of environments can be installed via:

0
docs/environments/classic_control/.gitkeep Normal file
View File

65
docs/environments/classic_control/index.html
View File

@@ -1,65 +0,0 @@
<div class="env-grid">
<a href="acrobot">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/classic_control/acrobot.gif">
</div>
<div class="cell__title">
<span>Acrobot</span>
</div>
</div>
</a>
<a href="cart_pole">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/classic_control/cart_pole.gif">
</div>
<div class="cell__title">
<span>Cart Pole</span>
</div>
</div>
</a>
<a href="mountain_car_continuous">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/classic_control/mountain_car_continuous.gif">
</div>
<div class="cell__title">
<span>Mountain Car Continuous</span>
</div>
</div>
</a>
<a href="mountain_car">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/classic_control/mountain_car.gif">
</div>
<div class="cell__title">
<span>Mountain Car</span>
</div>
</div>
</a>
<a href="pendulum">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/classic_control/pendulum.gif">
</div>
<div class="cell__title">
<span>Pendulum</span>
</div>
</div>
</a>
</div>

2
docs/environments/mujoco/index.md → docs/environments/mujoco.md
View File

@@ -21,7 +21,7 @@ walker2d
``` ```
```{raw} html ```{raw} html
:file: index.html :file: mujoco/list.html
``` ```
MuJoCo stands for Multi-Joint dynamics with Contact. It is a physics engine for faciliatating research and development in robotics, biomechanics, graphics and animation, and other areas where fast and accurate simulation is needed. MuJoCo stands for Multi-Joint dynamics with Contact. It is a physics engine for faciliatating research and development in robotics, biomechanics, graphics and animation, and other areas where fast and accurate simulation is needed.

0
docs/environments/mujoco/.gitkeep Normal file
View File

125
docs/environments/mujoco/index.html
View File

@@ -1,125 +0,0 @@
<div class="env-grid">
<a href="ant">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/ant.gif">
</div>
<div class="cell__title">
<span>Ant</span>
</div>
</div>
</a>
<a href="half_cheetah">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/half_cheetah.gif">
</div>
<div class="cell__title">
<span>Half Cheetah</span>
</div>
</div>
</a>
<a href="hopper">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/hopper.gif">
</div>
<div class="cell__title">
<span>Hopper</span>
</div>
</div>
</a>
<a href="humanoid_standup">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/humanoid_standup.gif">
</div>
<div class="cell__title">
<span>Humanoid Standup</span>
</div>
</div>
</a>
<a href="humanoid">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/humanoid.gif">
</div>
<div class="cell__title">
<span>Humanoid</span>
</div>
</div>
</a>
<a href="inverted_double_pendulum">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/inverted_double_pendulum.gif">
</div>
<div class="cell__title">
<span>Inverted Double Pendulum</span>
</div>
</div>
</a>
<a href="inverted_pendulum">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/inverted_pendulum.gif">
</div>
<div class="cell__title">
<span>Inverted Pendulum</span>
</div>
</div>
</a>
<a href="reacher">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/reacher.gif">
</div>
<div class="cell__title">
<span>Reacher</span>
</div>
</div>
</a>
<a href="swimmer">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/swimmer.gif">
</div>
<div class="cell__title">
<span>Swimmer</span>
</div>
</div>
</a>
<a href="walker2d">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/mujoco/walker2d.gif">
</div>
<div class="cell__title">
<span>Walker2D</span>
</div>
</div>
</a>
</div>

0
docs/environments/third_party_environments/index.md → docs/environments/third_party_environments.md
View File

14
docs/environments/toy_text/index.md → docs/environments/toy_text.md
View File

@@ -8,18 +8,18 @@ lastpage:
```{toctree} ```{toctree}
:hidden: :hidden:
blackjack.md toy_text/blackjack.md
taxi.md toy_text/taxi.md
cliff_walking.md toy_text/cliff_walking.md
frozen_lake.md toy_text/frozen_lake.md
``` ```
```{raw} html ```{raw} html
:file: index.html :file: toy_text/list.html
``` ```
All toy text environments were created by us using native Python libraries such as StringIO. All toy text environments were created by us using native Python libraries such as StringIO.
These environments are designed to be extremely simple, with small discrete state and action spaces, and hence easy to learn. As a result, they are suitable for debugging implementations of reinforcement learning algorithms. These environments are designed to be extremely simple, with small discrete state and action spaces, and hence easy to learn. As a result, they are suitable for debugging implementations of reinforcement learning algorithms.
All environments are configurable via arguments specified in each environment's documentation. All environments are configurable via arguments specified in each environment's documentation.

0
docs/environments/toy_text/.gitkeep Normal file
View File

29
docs/environments/toy_text/index.html
View File

@@ -1,29 +0,0 @@
<div class="env-grid">
<a href="blackjack">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/toy_text/blackjack.gif">
</div>
<div class="cell__title">
<span>Blackjack</span>
</div>
</div>
</a>
<a href="frozen_lake">
<div class="env-grid__cell">
<div class="cell__image-container">
<img src="/_static/videos/toy_text/frozen_lake.gif">
</div>
<div class="cell__title">
<span>Frozen Lake</span>
</div>
</div>
</a>
</div>

14
docs/index.md
View File

@@ -26,7 +26,7 @@ for _ in range(1000):
if terminated or truncated: if terminated or truncated:
observation, info = env.reset() observation, info = env.reset()
env.close() env.close()
``` ```
```{toctree} ```{toctree}
:hidden: :hidden:
@@ -50,12 +50,12 @@ api/utils
:hidden: :hidden:
:caption: Environments :caption: Environments
environments/atari/index environments/atari
environments/mujoco/index environments/mujoco
environments/toy_text/index environments/toy_text
environments/classic_control/index environments/classic_control
environments/box2d/index environments/box2d
environments/third_party_environments/index environments/third_party_environments
``` ```
```{toctree} ```{toctree}

43
docs/scripts/gen_envs_display.py
View File

@@ -1,3 +1,4 @@
import os
import sys import sys
all_envs = [ all_envs = [
@@ -16,7 +17,7 @@ all_envs = [
"walker2d", "walker2d",
], ],
}, },
{"id": "toy_text", "list": ["blackjack", "frozen_lake"]}, {"id": "toy_text", "list": ["blackjack", "cliff_walking", "frozen_lake", "taxi"]},
{"id": "box2d", "list": ["bipedal_walker", "car_racing", "lunar_lander"]}, {"id": "box2d", "list": ["bipedal_walker", "car_racing", "lunar_lander"]},
{ {
"id": "classic_control", "id": "classic_control",
@@ -124,11 +125,13 @@ def generate_page(env, limit=-1, base_path=""):
cells = "\n".join(cells[:limit]) cells = "\n".join(cells[:limit])
more_btn = ( more_btn = (
"""<a href="./complete_list"> """
<button class="more-btn"> <a href="./complete_list">
See More Environments <button class="more-btn">
</button> See More Environments
</a>""" </button>
</a>
"""
if not non_limited_page if not non_limited_page
else "" else ""
) )
@@ -160,16 +163,32 @@ if __name__ == "__main__":
envs_path = f"../environments/{type_id}" envs_path = f"../environments/{type_id}"
if len(type_dict["list"]) > 20: if len(type_dict["list"]) > 20:
page = generate_page(type_dict, limit=9) page = generate_page(type_dict, limit=9)
fp = open(f"{envs_path}/index.html", "w+", encoding="utf-8") fp = open(
os.path.join(os.path.dirname(__file__), envs_path, "list.html"),
"w",
encoding="utf-8",
)
fp.write(page) fp.write(page)
fp.close() fp.close()
page = generate_page(type_dict, base_path="../") page = generate_page(type_dict, base_path="../")
fp = open(f"{envs_path}/complete_list.html", "w+", encoding="utf-8") fp = open(
os.path.join(
os.path.dirname(__file__), envs_path, "complete_list.html"
),
"w",
encoding="utf-8",
)
fp.write(page) fp.write(page)
fp.close() fp.close()
fp = open(f"{envs_path}/complete_list.md", "w+", encoding="utf-8") fp = open(
os.path.join(
os.path.dirname(__file__), envs_path, "complete_list.html"
),
"w",
encoding="utf-8",
)
env_name = " ".join(type_id.split("_")).title() env_name = " ".join(type_id.split("_")).title()
fp.write( fp.write(
f"# Complete List - {env_name}\n" f"# Complete List - {env_name}\n"
@@ -178,6 +197,10 @@ if __name__ == "__main__":
fp.close() fp.close()
else: else:
page = generate_page(type_dict) page = generate_page(type_dict)
fp = open(f"{envs_path}/index.html", "w+", encoding="utf-8") fp = open(
os.path.join(os.path.dirname(__file__), envs_path, "list.html"),
"w",
encoding="utf-8",
)
fp.write(page) fp.write(page)
fp.close() fp.close()

4
gymnasium/envs/toy_text/cliffwalking.py
View File

@@ -24,8 +24,7 @@ class CliffWalkingEnv(Env):
by Sutton and Barto](http://incompleteideas.net/book/bookdraft2018jan1.pdf). by Sutton and Barto](http://incompleteideas.net/book/bookdraft2018jan1.pdf).
With inspiration from: With inspiration from:
[https://github.com/dennybritz/reinforcement-learning/blob/master/lib/envs/cliff_walking.py] [https://github.com/dennybritz/reinforcement-learning/blob/master/lib/envs/cliff_walking.py](https://github.com/dennybritz/reinforcement-learning/blob/master/lib/envs/cliff_walking.py)
(https://github.com/dennybritz/reinforcement-learning/blob/master/lib/envs/cliff_walking.py)
### Description ### Description
The board is a 4x12 matrix, with (using NumPy matrix indexing): The board is a 4x12 matrix, with (using NumPy matrix indexing):
@@ -287,5 +286,6 @@ class CliffWalkingEnv(Env):
with closing(outfile): with closing(outfile):
return outfile.getvalue() return outfile.getvalue()
# Elf and stool from https://franuka.itch.io/rpg-snow-tileset # Elf and stool from https://franuka.itch.io/rpg-snow-tileset
# All other assets by ____ # All other assets by ____