diff --git a/docs/environments/atari.md b/docs/environments/atari.md index 1f640a656..9260e8282 100644 --- a/docs/environments/atari.md +++ b/docs/environments/atari.md @@ -5,7 +5,7 @@ lastpage: # Atari -A set of Atari 2600 environments simulated through Stella and the Arcade Learning Environment. +A set of Atari 2600 environments simulated through [Stella](https://github.com/stella-emu/stella) and the [Arcade Learning Environment](https://github.com/mgbellemare/Arcade-Learning-Environment). ```{toctree} :hidden: @@ -77,19 +77,19 @@ atari/zaxxon :file: atari/list.html ``` -Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). +Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1) through the Stella emulator. ## AutoROM (installing the ROMs) ALE-py doesn't include the atari ROMs (`pip install gymnasium[atari]`) which are necessary to make any of the atari environments. To install the atari ROM, use `pip install gymnasium[accept-rom-license]` which will install AutoROM and download the ROMs, install them in the default location. -In doing so, you agree to TODO +In doing so, you agree to own a license to these Atari 2600 ROMs and agree to not distribution these ROMS. -It is possible to install the ROMs in an alternative location, [repo](https://github.com/Farama-Foundation/AutoROM) has more information. +It is possible to install the ROMs in an alternative location, [AutoROM](https://github.com/Farama-Foundation/AutoROM) has more information. ## Action Space -The action space is a subset of the following discrete set of legal actions: +Each environment will use a sub-set of the full action space listed below: | Num | Action | |-----|---------------| @@ -112,21 +112,15 @@ The action space is a subset of the following discrete set of legal actions: | 16 | DOWNRIGHTFIRE | | 17 | DOWNLEFTFIRE | -If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have -any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full -legal space by passing the keyword argument `full_action_space=True` to `make`. - -The reduced action space of an Atari environment may depend on the "flavor" of the game. You can specify the flavor by providing -the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the -action spaces of default flavor choices. +By default, most environments use a smaller subset of the legal actions excluding any actions that don't have an effect in the game. +If users are interested in using all possible actions, pass the keyword argument `full_action_space=True` to `gymnasium.make`. ## Observation Space -The observation issued by an Atari environment may be: - -- the RGB image that is displayed to a human player, -- a grayscale version of that image or -- the state of the 128 Bytes of RAM of the console. +The Atari environments observation can be +1. The RGB image that is displayed to a human player using `obs_type="rgb"` with observation space `Box(0, 255, (210, 160, 3), np.uint8)` +2. The grayscale version of the RGB image using `obs_type="grayscale"` with observation space `Box(0, 255, (210, 160), np.uint8)` +3. The RAM state (128 bytes) from the console using `obs_type="ram"` with observation space `Box(0, 255, (128), np.uint8)` ## Rewards @@ -135,13 +129,17 @@ find these manuals on [AtariAge](https://atariage.com/). ## Stochasticity -It was pointed out in [[1]](#1) that Atari games are entirely deterministic. Thus, agents could achieve +As the Atari games are entirely deterministic, agents could achieve state-of-the-art performance by simply memorizing an optimal sequence of actions while completely ignoring observations from the environment. -To avoid this, ALE implements sticky actions: Instead of always simulating the action passed to the environment, there is a small -probability that the previously executed action is used instead. -On top of this, Gymnasium implements stochastic frame skipping: In each environment step, the action is repeated for a random -number of frames. This behavior may be altered by setting the keyword argument `frameskip` to either a positive integer or +To avoid this, there are several methods to avoid this. + +1. Sticky actions: Instead of always simulating the action passed to the environment, there is a small +probability that the previously executed action is used instead. In the v0 and v5 environments, the probability of +repeating an action is `25%` while in v4 environments, the probability is `0%`. Users can specify the repeat action +probability using `repeat_action_probability` to `make`. +2. Frameskipping: On each environment step, the action can be repeated for a random number of frames. This behavior +may be altered by setting the keyword argument `frameskip` to either a positive integer or a tuple of two positive integers. If `frameskip` is an integer, frame skipping is deterministic, and in each step the action is repeated `frameskip` many times. Otherwise, if `frameskip` is a tuple, the number of skipped frames is chosen uniformly at random between `frameskip[0]` (inclusive) and `frameskip[1]` (exclusive) in each environment step. @@ -149,7 +147,7 @@ random between `frameskip[0]` (inclusive) and `frameskip[1]` (exclusive) in each ## Common Arguments When initializing Atari environments via `gymnasium.make`, you may pass some additional arguments. These work for any -Atari environment. However, legal values for `mode` and `difficulty` depend on the environment. +Atari environment. - **mode**: `int`. Game mode, see [[2]](#2). Legal values depend on the environment and are listed in the table above. @@ -157,22 +155,20 @@ Atari environment. However, legal values for `mode` and `difficulty` depend on t the table above. Together with `mode`, this determines the "flavor" of the game. - **obs_type**: `str`. This argument determines what observations are returned by the environment. Its values are: - - ram: The 128 Bytes of RAM are returned - - rgb: An RGB rendering of the game is returned - - grayscale: A grayscale rendering is returned + - "ram": The 128 Bytes of RAM are returned + - "rgb": An RGB rendering of the game is returned + - "grayscale": A grayscale rendering is returned - **frameskip**: `int` or a tuple of two `int`s. This argument controls stochastic frame skipping, as described in the section on stochasticity. -- **repeat_action_probability**: `float`. The probability that an action sticks, as described in the section on stochasticity. +- **repeat_action_probability**: `float`. The probability that an action is repeated, also called "sticky actions", as described in the section on stochasticity. - **full_action_space**: `bool`. If set to `True`, the action space consists of all legal actions on the console. Otherwise, the action space will be reduced to a subset. - **render_mode**: `str`. Specifies the rendering mode. Its values are: - - human: We'll interactively display the screen and enable game sounds. This will lock emulation to the ROMs specified FPS - - rgb_array: we'll return the `rgb` key in step metadata with the current environment RGB frame. -> It is highly recommended to specify `render_mode` during construction instead of calling `env.render()`. -> This will guarantee proper scaling, audio support, and proper framerates + - human: Display the screen and enable game sounds. This will lock emulation to the ROMs specified FPS + - rgb_array: Returns the current environment RGB frame of the environment. ## Version History and Naming Schemes @@ -183,31 +179,31 @@ The differences are listed in the following table: |---------|--------------|------------------------------|----------------------| | v0 | `(2, 5,)` | `0.25` | `False` | | v4 | `(2, 5,)` | `0.0` | `False` | -| v5 | `5` | `0.25` | `True` | +| v5 | `5` | `0.25` | `False` | > Version v5 follows the best practices outlined in [[2]](#2). Thus, it is recommended to transition to v5 and -> customize the environment using the arguments above, if necessary. +customize the environment using the arguments above, if necessary. For each Atari game, several different configurations are registered in Gymnasium. The naming schemes are analogous for v0 and v4. Let us take a look at all variations of Amidar-v0 that are registered with gymnasium: -| Name | `obs_type=` | `frameskip=` | `repeat_action_probability=` | `full_action_space=` | -|----------------------------|-------------|--------------|------------------------------|----------------------| -| Amidar-v0 | `"rgb"` | `(2, 5,)` | `0.25` | `False` | -| AmidarDeterministic-v0 | `"rgb"` | `4` | `0.0` | `False` | -| AmidarNoframeskip-v0 | `"rgb"` | `1` | `0.25` | `False` | -| Amidar-ram-v0 | `"ram"` | `(2, 5,)` | `0.25` | `False` | -| Amidar-ramDeterministic-v0 | `"ram"` | `4` | `0.0` | `False` | -| Amidar-ramNoframeskip-v0 | `"ram"` | `1` | `0.25` | `False` | +| Name | `obs_type=` | `frameskip=` | `repeat_action_probability=` | +|----------------------------|-------------|--------------|------------------------------| +| Amidar-v0 | `"rgb"` | `(2, 5,)` | `0.25` | +| AmidarDeterministic-v0 | `"rgb"` | `4` | `0.0` | +| AmidarNoframeskip-v0 | `"rgb"` | `1` | `0.25` | +| Amidar-ram-v0 | `"ram"` | `(2, 5,)` | `0.25` | +| Amidar-ramDeterministic-v0 | `"ram"` | `4` | `0.0` | +| Amidar-ramNoframeskip-v0 | `"ram"` | `1` | `0.25` | Things change in v5: The suffixes "Deterministic" and "NoFrameskip" are no longer available. Instead, you must specify the environment configuration via arguments passed to `gymnasium.make`. Moreover, the v5 environments are in the "ALE" namespace. The suffix "-ram" is still available. Thus, we get the following table: -| Name | `obs_type=` | `frameskip=` | `repeat_action_probability=` | `full_action_space=` | -|-------------------|-------------|--------------|------------------------------|----------------------| -| ALE/Amidar-v5 | `"rgb"` | `5` | `0.25` | `True` | -| ALE/Amidar-ram-v5 | `"ram"` | `5` | `0.25` | `True` | +| Name | `obs_type=` | `frameskip=` | `repeat_action_probability=` | +|-------------------|-------------|--------------|------------------------------| +| ALE/Amidar-v5 | `"rgb"` | `5` | `0.25` | +| ALE/Amidar-ram-v5 | `"ram"` | `5` | `0.25` | ## Flavors @@ -216,72 +212,109 @@ game dynamics and (if a reduced action space is used) different action spaces. W refer to the combination of difficulty level and game mode as a flavor of a game. The following table shows the available modes and difficulty levels for different Atari games: -| Environment | Valid Modes | Default Mode | -|------------------|-------------------------------------------------|----------------| -| Adventure | `[0, 1, 2]` | `0` | -| AirRaid | `[1, ..., 8]` | `1` | -| Alien | `[0, ..., 3]` | `0` | -| Amidar | `[0]` | `0` | -| Assault | `[0]` | `0` | -| Asterix | `[0]` | `0` | -| Asteroids | `[0, ..., 31, 128]` | `0` | -| Atlantis | `[0, ..., 3]` | `0` | -| BankHeist | `[0, 4, 8, 12, 16, 20, 24, 28]` | `0` | -| BattleZone | `[1, 2, 3]` | `1` | -| BeamRider | `[0]` | `0` | -| Berzerk | `[1, 2, 3, 4, 5, 6, 7, 8, 9, 16, 17, 18]` | `1` | -| Bowling | `[0, 2, 4]` | `0` | -| Boxing | `[0]` | `0` | -| Breakout | `[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44]` | `0` | -| Carnival | `[0]` | `0` | -| Centipede | `[22, 86]` | `22` | -| ChopperCommand | `[0, 2]` | `0` | -| CrazyClimber | `[0, ..., 3]` | `0` | -| Defender | `[1, ..., 9, 16]` | `1` | -| DemonAttack | `[1, 3, 5, 7]` | `1` | -| DoubleDunk | `[0, ..., 15]` | `0` | -| ElevatorAction | `[0]` | `0` | -| Enduro | `[0]` | `0` | -| FishingDerby | `[0]` | `0` | -| Freeway | `[0, ..., 7]` | `0` | -| Frostbite | `[0, 2]` | `0` | -| Gopher | `[0, 2]` | `0` | -| Gravitar | `[0, ..., 4]` | `0` | -| Hero | `[0, ..., 4]` | `0` | -| IceHockey | `[0, 2]` | `0` | -| Jamesbond | `[0, 1]` | `0` | -| JourneyEscape | `[0]` | `0` | -| Kangaroo | `[0, 1]` | `0` | -| Krull | `[0]` | `0` | -| KungFuMaster | `[0]` | `0` | -| MontezumaRevenge | `[0]` | `0` | -| MsPacman | `[0, ..., 3]` | `0` | -| NameThisGame | `[8, 24, 40]` | `8` | -| Phoenix | `[0]` | `0` | -| Pitfall | `[0]` | `0` | -| Pong | `[0, 1]` | `0` | -| Pooyan | `[10, 30, 50, 70]` | `10` | -| PrivateEye | `[0, ..., 4]` | `0` | -| Qbert | `[0]` | `0` | -| Riverraid | `[0]` | `0` | -| RoadRunner | `[0]` | `0` | -| Robotank | `[0]` | `0` | -| Seaquest | `[0]` | `0` | -| Skiing | `[0]` | `0` | -| Solaris | `[0]` | `0` | -| SpaceInvaders | `[0, ..., 15]` | `0` | -| StarGunner | `[0, ..., 3]` | `0` | -| Tennis | `[0, 2]` | `0` | -| TimePilot | `[0]` | `0` | -| Tutankham | `[0, 4, 8, 12]` | `0` | -| UpNDown | `[0]` | `0` | -| Venture | `[0]` | `0` | -| VideoPinball | `[0, 2]` | `0` | -| WizardOfWor | `[0]` | `0` | -| YarsRevenge | `[0, 32, 64, 96]` | `0` | -| Zaxxon | `[0, 8, 16, 24]` | `0` | - -> Each game also has a valid difficulty for the opposing AI, which has a different range depending on the game. These values can have a range of 0 - n, where n can be found at [the ALE documentation](https://github.com/mgbellemare/Arcade-Learning-Environment/blob/master/docs/games.md) +| Environment | Possible Modes | Default Mode | Possible Difficulties | Default Difficulty | +|------------------|-----------------------------------------------|----------------|-------------------------|----------------------| +| Adventure | [0, 1, 2] | 0 | [0, 1, 2, 3] | 0 | +| AirRaid | [1, ..., 8] | 1 | [0] | 0 | +| Alien | [0, 1, 2, 3] | 0 | [0, 1, 2, 3] | 0 | +| Amidar | [0] | 0 | [0, 3] | 0 | +| Assault | [0] | 0 | [0] | 0 | +| Asterix | [0] | 0 | [0] | 0 | +| Asteroids | [0, ..., 31, 128] | 0 | [0, 3] | 0 | +| Atlantis | [0, 1, 2, 3] | 0 | [0] | 0 | +| Atlantis2 | [0] | 0 | [0] | 0 | +| Backgammon | [0] | 0 | [3] | 0 | +| BankHeist | [0, 4, 8, 12, 16, 20, 24, 28] | 0 | [0, 1, 2, 3] | 0 | +| BasicMath | [5, 6, 7, 8] | 5 | [0, 2, 3] | 0 | +| BattleZone | [1, 2, 3] | 1 | [0] | 0 | +| BeamRider | [0] | 0 | [0, 1] | 0 | +| Berzerk | [1, ..., 9, 16, 17, 18] | 1 | [0] | 0 | +| Blackjack | [0] | 0 | [0, 1, 2, 3] | 0 | +| Bowling | [0, 2, 4] | 0 | [0, 1] | 0 | +| Boxing | [0] | 0 | [0, 1, 2, 3] | 0 | +| Breakout | [0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44] | 0 | [0, 1] | 0 | +| Carnival | [0] | 0 | [0] | 0 | +| Casino | [0, 2, 3] | 0 | [0, 1, 2, 3] | 0 | +| Centipede | [22, 86] | 22 | [0] | 0 | +| ChopperCommand | [0, 2] | 0 | [0, 1] | 0 | +| CrazyClimber | [0, 1, 2, 3] | 0 | [0, 1] | 0 | +| Crossbow | [0, 2, 4, 6] | 0 | [0, 1] | 0 | +| Darkchambers | [0] | 0 | [0] | 0 | +| Defender | [1, ..., 9, 16] | 1 | [0, 1] | 0 | +| DemonAttack | [1, 3, 5, 7] | 1 | [0, 1] | 0 | +| DonkeyKong | [0] | 0 | [0] | 0 | +| DoubleDunk | [0, ..., 15] | 0 | [0] | 0 | +| Earthworld | [0] | 0 | [0] | 0 | +| ElevatorAction | [0] | 0 | [0] | 0 | +| Enduro | [0] | 0 | [0] | 0 | +| Entombed | [0] | 0 | [0, 2] | 0 | +| Et | [0, 1, 2] | 0 | [0, 1, 2, 3] | 0 | +| FishingDerby | [0] | 0 | [0, 1, 2, 3] | 0 | +| FlagCapture | [8, 9, 10] | 8 | [0] | 0 | +| Freeway | [0, ..., 7] | 0 | [0, 1] | 0 | +| Frogger | [0, 1, 2] | 0 | [0, 1] | 0 | +| Frostbite | [0, 2] | 0 | [0] | 0 | +| Galaxian | [1, ..., 9] | 1 | [0, 1] | 0 | +| Gopher | [0, 2] | 0 | [0, 1] | 0 | +| Gravitar | [0, 1, 2, 3, 4] | 0 | [0] | 0 | +| Hangman | [0, 1, 2, 3] | 0 | [0, 1] | 0 | +| HauntedHouse | [0, ..., 8] | 0 | [0, 1] | 0 | +| Hero | [0, 1, 2, 3, 4] | 0 | [0] | 0 | +| HumanCannonball | [0, ..., 7] | 0 | [0, 1] | 0 | +| IceHockey | [0, 2] | 0 | [0, 1, 2, 3] | 0 | +| Jamesbond | [0, 1] | 0 | [0] | 0 | +| JourneyEscape | [0] | 0 | [0, 1] | 0 | +| Kaboom | [0] | 0 | [0] | 0 | +| Kangaroo | [0, 1] | 0 | [0] | 0 | +| KeystoneKapers | [0] | 0 | [0] | 0 | +| KingKong | [0, 1, 2, 3] | 0 | [0] | 0 | +| Klax | [0, 1, 2] | 0 | [0] | 0 | +| Koolaid | [0] | 0 | [0] | 0 | +| Krull | [0] | 0 | [0] | 0 | +| KungFuMaster | [0] | 0 | [0] | 0 | +| LaserGates | [0] | 0 | [0] | 0 | +| LostLuggage | [0, 1] | 0 | [0, 1] | 0 | +| MarioBros | [0, 2, 4, 6] | 0 | [0] | 0 | +| MiniatureGolf | [0] | 0 | [0, 1] | 0 | +| MontezumaRevenge | [0] | 0 | [0] | 0 | +| MrDo | [0, 1, 2, 3] | 0 | [0] | 0 | +| MsPacman | [0, 1, 2, 3] | 0 | [0] | 0 | +| NameThisGame | [8, 24, 40] | 8 | [0, 1] | 0 | +| Othello | [0, 1, 2] | 0 | [0, 2] | 0 | +| Pacman | [0, ..., 7] | 0 | [0, 1] | 0 | +| Phoenix | [0] | 0 | [0] | 0 | +| Pitfall | [0] | 0 | [0] | 0 | +| Pitfall2 | [0] | 0 | [0] | 0 | +| Pong | [0, 1] | 0 | [0, 1, 2, 3] | 0 | +| Pooyan | [10, 30, 50, 70] | 10 | [0] | 0 | +| PrivateEye | [0, 1, 2, 3, 4] | 0 | [0, 1, 2, 3] | 0 | +| Qbert | [0] | 0 | [0, 1] | 0 | +| Riverraid | [0] | 0 | [0, 1] | 0 | +| RoadRunner | [0] | 0 | [0] | 0 | +| Robotank | [0] | 0 | [0] | 0 | +| Seaquest | [0] | 0 | [0, 1] | 0 | +| SirLancelot | [0] | 0 | [0] | 0 | +| Skiing | [0] | 0 | [0] | 0 | +| Solaris | [0] | 0 | [0] | 0 | +| SpaceInvaders | [0, ..., 15] | 0 | [0, 1] | 0 | +| SpaceWar | [6, ..., 17] | 6 | [0] | 0 | +| StarGunner | [0, 1, 2, 3] | 0 | [0] | 0 | +| Superman | [0] | 0 | [0, 1, 2, 3] | 0 | +| Surround | [0, 2] | 0 | [0, 1, 2, 3] | 0 | +| Tennis | [0, 2] | 0 | [0, 1, 2, 3] | 0 | +| Tetris | [0] | 0 | [0] | 0 | +| TimePilot | [0] | 0 | [0, 1, 2] | 0 | +| Trondead | [0] | 0 | [0, 1] | 0 | +| Turmoil | [0, ..., 8] | 0 | [0] | 0 | +| Tutankham | [0, 4, 8, 12] | 0 | [0] | 0 | +| UpNDown | [0] | 0 | [0, 1, 2, 3] | 0 | +| Venture | [0] | 0 | [0, 1, 2, 3] | 0 | +| VideoCheckers | [1, ..., 9, 11, ..., 19] | 1 | [0] | 0 | +| VideoPinball | [0, 2] | 0 | [0, 1] | 0 | +| WizardOfWor | [0] | 0 | [0, 1] | 0 | +| WordZapper | [0, ..., 23] | 0 | [0, 1, 2, 3] | 0 | +| YarsRevenge | [0, 32, 64, 96] | 0 | [0, 1] | 0 | +| Zaxxon | [0, 8, 16, 24] | 0 | [0] | 0 | ## References diff --git a/docs/environments/atari/adventure.md b/docs/environments/atari/adventure.md index 3490462e1..866168f2b 100644 --- a/docs/environments/atari/adventure.md +++ b/docs/environments/atari/adventure.md @@ -1,82 +1,94 @@ --- title: Adventure -firstpage: --- # Adventure ```{figure} ../../_static/videos/atari/adventure.gif :width: 120px -:name: adventure +:name: Adventure ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (250, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Adventure-v5")` | - ## Description -You must find the enchanted chalice and return it to the golden castle. You can pick up various objects (keys, a sword, -a bridge, or a magnet) and have to fight or outmanoeuvre dragons. -For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1). +You must find the enchanted chalice and return it to the golden castle. You can pick up various objects (keys, a sword,a bridge, or a magnet) and have to fight or outmanoeuvre dragons. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Adventure has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Adventure uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +Adventure has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| Adventure-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Adventure-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Adventure-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Adventure-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AdventureDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AdventureNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Adventure-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Adventure-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Adventure-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Adventure-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AdventureDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AdventureNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Adventure-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Adventure-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/Adventure-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Adventure | `[0, 1, 2]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Adventure-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/air_raid.md b/docs/environments/atari/air_raid.md index 9af48f0e1..0d214ef65 100644 --- a/docs/environments/atari/air_raid.md +++ b/docs/environments/atari/air_raid.md @@ -1,91 +1,82 @@ --- -title: Air Raid +title: AirRaid --- -# Air Raid +# AirRaid ```{figure} ../../_static/videos/atari/air_raid.gif :width: 120px -:name: air_raid +:name: AirRaid ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (250, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/AirRaid-v5")` | - ## Description -You control a ship that can move sideways. You must protect two buildings (one on the right and one on the left side of the screen) from -flying saucers that are trying to drop bombs on them. +You control a ship that can move sideways. You must protect two buildings (one on the right and one on the left side of the screen) from flying saucers that are trying to drop bombs on them. ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +AirRaid has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As AirRaid uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | RIGHTFIRE | -| 5 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `RIGHTFIRE` | +| `5` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +AirRaid has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| AirRaid-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| AirRaid-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| AirRaid-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| AirRaid-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AirRaidDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AirRaidNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| AirRaid-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| AirRaid-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| AirRaid-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| AirRaid-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AirRaidDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AirRaidNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/AirRaid-v5 | `"rgb"` | `4` | `0.25` | +| ALE/AirRaid-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/AirRaid-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| AirRaid | `[1, ..., 8]` | `[0]` | `1` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("AirRaid-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[1, ..., 8]` | `1` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/alien.md b/docs/environments/atari/alien.md index af113e04f..64a5b3703 100644 --- a/docs/environments/atari/alien.md +++ b/docs/environments/atari/alien.md @@ -6,83 +6,94 @@ title: Alien ```{figure} ../../_static/videos/atari/alien.gif :width: 120px -:name: alien +:name: Alien ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Alien-v5")` | - ## Description -You are stuck in a maze-like space ship with three aliens. You goal is to destroy their eggs that are scattered -all over the ship while simultaneously avoiding the aliens (they are trying to kill you). You have a flamethrower that can help you turn -them away in tricky situations. Moreover, you can occasionally collect a power-up (pulsar) that gives you the temporary ability to kill aliens. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=815). +You are stuck in a maze-like space ship with three aliens. You goal is to destroy their eggs that are scattered all over the ship while simultaneously avoiding the aliens (they are trying to kill you). You have a flamethrower that can help you turn them away in tricky situations. Moreover, you can occasionally collect a power-up (pulsar) that gives you the temporary ability to kill aliens. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=815) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Alien has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Alien uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by destroying eggs, killing aliens, using pulsars, and collecting special prizes. When you are caught by an alien, you will lose one of your lives. The number of lives you have depends on the game flavor. For a table of scores corresponding to the different achievements, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=815). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Alien-v5") -``` +Alien has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|---------------------------|-------------|--------------|------------------------------| +| Alien-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Alien-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Alien-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Alien-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AlienDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AlienNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Alien-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Alien-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Alien-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Alien-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AlienDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AlienNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Alien-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Alien-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| Alien | `[0, ..., 3]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Alien-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/amidar.md b/docs/environments/atari/amidar.md index fe05822db..59aa04fec 100644 --- a/docs/environments/atari/amidar.md +++ b/docs/environments/atari/amidar.md @@ -6,96 +6,87 @@ title: Amidar ```{figure} ../../_static/videos/atari/amidar.gif :width: 120px -:name: amidar +:name: Amidar ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Amidar-v5")` | - ## Description -This game is similar to Pac-Man: You are trying to visit all places on a 2-dimensional grid while simultaneously avoiding -your enemies. You can turn the tables at one point in the game: Your enemies turn into chickens and you can -catch them. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=817). +This game is similar to Pac-Man: You are trying to visit all places on a 2-dimensional grid while simultaneously avoiding your enemies. You can turn the tables at one point in the game: Your enemies turn into chickens and you can catch them. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=817) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Amidar has the action space `Discrete(10)` with the table below lists the meaning of each action's meanings. +As Amidar uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPFIRE | -| 7 | RIGHTFIRE | -| 8 | LEFTFIRE | -| 9 | DOWNFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPFIRE` | +| `7` | `RIGHTFIRE` | +| `8` | `LEFTFIRE` | +| `9` | `DOWNFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by traversing new parts of the grid. Coloring an entire box in the maze or catching chickens gives extra points. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=817). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Amidar-v5") -``` +Amidar has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Amidar-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Amidar-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Amidar-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Amidar-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AmidarDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AmidarNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Amidar-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Amidar-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Amidar-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Amidar-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AmidarDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AmidarNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Amidar-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Amidar-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Amidar | `[0]` | `[0, 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Amidar-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/assault.md b/docs/environments/atari/assault.md index d581ca3c5..e6ae997fc 100644 --- a/docs/environments/atari/assault.md +++ b/docs/environments/atari/assault.md @@ -6,85 +6,80 @@ title: Assault ```{figure} ../../_static/videos/atari/assault.gif :width: 120px -:name: assault +:name: Assault ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Assault-v5")` | - ## Description -You control a vehicle that can move sideways. A big mother ship circles overhead and continually deploys smaller drones. -You must destroy these enemies and dodge their attacks. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=827). +You control a vehicle that can move sideways. A big mother ship circles overhead and continually deploys smaller drones.You must destroy these enemies and dodge their attacks. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=827) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Assault has the action space `Discrete(7)` with the table below lists the meaning of each action's meanings. +As Assault uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | RIGHTFIRE | -| 6 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `RIGHTFIRE` | +| `6` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +Assault has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Assault-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Assault-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Assault-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Assault-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AssaultDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AssaultNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Assault-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Assault-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Assault-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Assault-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AssaultDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AssaultNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Assault-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Assault-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/Assault-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Assault | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Assault-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/asterix.md b/docs/environments/atari/asterix.md index e088a5910..39af78641 100644 --- a/docs/environments/atari/asterix.md +++ b/docs/environments/atari/asterix.md @@ -6,95 +6,85 @@ title: Asterix ```{figure} ../../_static/videos/atari/asterix.gif :width: 120px -:name: asterix +:name: Asterix ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Asterix-v5")` | - ## Description -You are Asterix and can move horizontally (continuously) and vertically (discretely). Objects -move horizontally across the screen: lyres and other (more useful) objects. Your goal is to guide -Asterix in such a way as to avoid lyres and collect as many other objects as possible. You score points by collecting -objects and lose a life whenever you collect a lyre. You have three lives available at the beginning. If you score sufficiently -many points, you will be awarded additional points. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=3325). +You are Asterix and can move horizontally (continuously) and vertically (discretely). Objects move horizontally across the screen: lyres and other (more useful) objects. Your goal is to guideAsterix in such a way as to avoid lyres and collect as many other objects as possible. You score points by collecting objects and lose a life whenever you collect a lyre. You have three lives available at the beginning. If you score sufficiently many points, you will be awarded additional points. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=3325) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Asterix has the action space `Discrete(9)` with the table below lists the meaning of each action's meanings. +As Asterix uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | DOWNRIGHT | -| 8 | DOWNLEFT | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `UP` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `UPRIGHT` | +| `6` | `UPLEFT` | +| `7` | `DOWNRIGHT` | +| `8` | `DOWNLEFT` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards A table of scores awarded for collecting the different objects is provided on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=3325). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Asterix-v5") -``` +Asterix has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -The various ways to configure the environment are described in detail in the article on Atari environments. +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Asterix-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Asterix-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Asterix-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Asterix-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AsterixDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AsterixNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Asterix-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Asterix-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Asterix-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Asterix-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AsterixDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AsterixNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Asterix-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Asterix-ram-v5 | `"ram"` | `4` | `0.25` | -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Asterix | `[0]` | `[0]` | `0` | +## Difficulty and modes -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Asterix-v0")`. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. + +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/asteroids.md b/docs/environments/atari/asteroids.md index 0e0e6b2a9..9750113bb 100644 --- a/docs/environments/atari/asteroids.md +++ b/docs/environments/atari/asteroids.md @@ -6,102 +6,92 @@ title: Asteroids ```{figure} ../../_static/videos/atari/asteroids.gif :width: 120px -:name: asteroids +:name: Asteroids ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Asteroids-v5")` | - ## Description -This is a well-known arcade game: You control a spaceship in an asteroid field and must break up asteroids by shooting -them. Once all asteroids are destroyed, you enter a new level and new asteroids will appear. You will occasionally -be attacked by a flying saucer. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=828&itemTypeID=HTMLMANUAL) +This is a well-known arcade game: You control a spaceship in an asteroid field and must break up asteroids by shooting them. Once all asteroids are destroyed, you enter a new level and new asteroids will appear. You will occasionally be attacked by a flying saucer. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=828) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Asteroids has the action space `Discrete(14)` with the table below lists the meaning of each action's meanings. +As Asteroids uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPRIGHT | -| 7 | UPLEFT | -| 8 | UPFIRE | -| 9 | RIGHTFIRE | -| 10 | LEFTFIRE | -| 11 | DOWNFIRE | -| 12 | UPRIGHTFIRE | -| 13 | UPLEFTFIRE | +| Value | Meaning | +|---------|---------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `UPFIRE` | +| `9` | `RIGHTFIRE` | +| `10` | `LEFTFIRE` | +| `11` | `DOWNFIRE` | +| `12` | `UPRIGHTFIRE` | +| `13` | `UPLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for destroying asteroids, satellites and UFOs. The smaller the asteroid, the more points you score for destroying it. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=828&itemTypeID=HTMLMANUAL). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Asteroids-v5") -``` +Asteroids has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| Asteroids-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Asteroids-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Asteroids-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Asteroids-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AsteroidsDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AsteroidsNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Asteroids-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Asteroids-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Asteroids-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Asteroids-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AsteroidsDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AsteroidsNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Asteroids-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Asteroids-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------------|--------------------|--------------| -| Asteroids | `[0, ..., 31, 128]` | `[0, 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Asteroids-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|---------------------|----------------|--------------------------|----------------------| +| `[0, ..., 31, 128]` | `0` | `[0, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/atlantis.md b/docs/environments/atari/atlantis.md index e9ec10827..3764bb356 100644 --- a/docs/environments/atari/atlantis.md +++ b/docs/environments/atari/atlantis.md @@ -6,93 +6,84 @@ title: Atlantis ```{figure} ../../_static/videos/atari/atlantis.gif :width: 120px -:name: atlantis +:name: Atlantis ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Atlantis-v5")` | - ## Description -Your job is to defend the submerged city of Atlantis. Your enemies slowly descend towards the city and you must -destroy them before they reach striking distance. To this end, you control three defense posts. -You lose if your enemies manage to destroy all seven of Atlantis' installations. You may rebuild installations -after you have fought of a wave of enemies and scored a sufficient number of points. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=835). +Your job is to defend the submerged city of Atlantis. Your enemies slowly descend towards the city and you must destroy them before they reach striking distance. To this end, you control three defense posts.You lose if your enemies manage to destroy all seven of Atlantis' installations. You may rebuild installations after you have fought of a wave of enemies and scored a sufficient number of points. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=835) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Atlantis has the action space `Discrete(4)` with the table below lists the meaning of each action's meanings. +As Atlantis uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHTFIRE | -| 3 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHTFIRE` | +| `3` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for destroying enemies, keeping installations protected during attack waves. You score more points if you manage to destroy your enemies with one of the outer defense posts. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=835). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Atlantis-v5") -``` +Atlantis has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Atlantis-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Atlantis-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Atlantis-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Atlantis-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| AtlantisDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| AtlantisNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Atlantis-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Atlantis-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Atlantis-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Atlantis-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| AtlantisDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| AtlantisNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Atlantis-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Atlantis-ram-v5 | `"ram"` | `4` | `0.25` | +| ALE/Atlantis2-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Atlantis2-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| Atlantis | `[0, ..., 3]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Amidar-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/bank_heist.md b/docs/environments/atari/bank_heist.md index 277f21ef8..e7a64b649 100644 --- a/docs/environments/atari/bank_heist.md +++ b/docs/environments/atari/bank_heist.md @@ -1,90 +1,99 @@ --- -title: Bank Heist +title: BankHeist --- -# Bank Heist +# BankHeist ```{figure} ../../_static/videos/atari/bank_heist.gif :width: 120px -:name: Bank Heist +:name: BankHeist ``` This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/BankHeist-v5")` | - ## Description -You are a bank robber and (naturally) want to rob as many banks as possible. You control your getaway car and must -navigate maze-like cities. The police chases you and will appear whenever you rob a bank. You may destroy police cars -by dropping sticks of dynamite. You can fill up your gas tank by entering a new city. -At the beginning of the game you have four lives. Lives are lost if you run out of gas, are caught by the police, -or run over the dynamite you have previously dropped. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1008). +You are a bank robber and (naturally) want to rob as many banks as possible. You control your getaway car and must navigate maze-like cities. The police chases you and will appear whenever you rob a bank. You may destroy police cars by dropping sticks of dynamite. You can fill up your gas tank by entering a new city.At the beginning of the game you have four lives. Lives are lost if you run out of gas, are caught by the police,or run over the dynamite you have previously dropped. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1008) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +BankHeist has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As BankHeist uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for robbing banks and destroying police cars. If you rob nine or more banks, and then leave the city, you will score extra points. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1008). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/BankHeist-v5") -``` +BankHeist has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| BankHeist-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| BankHeist-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| BankHeist-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| BankHeist-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| BankHeistDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| BankHeistNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| BankHeist-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| BankHeist-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| BankHeist-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| BankHeist-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| BankHeistDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| BankHeistNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/BankHeist-v5 | `"rgb"` | `4` | `0.25` | +| ALE/BankHeist-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------------------------|--------------------|--------------| -| BankHeist | `[0, 4, 8, 12, 16, 20, 24, 28]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("BankHeist-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|---------------------------------|----------------|--------------------------|----------------------| +| `[0, 4, 8, 12, 16, 20, 24, 28]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/battle_zone.md b/docs/environments/atari/battle_zone.md index b628a60dc..84cec3e46 100644 --- a/docs/environments/atari/battle_zone.md +++ b/docs/environments/atari/battle_zone.md @@ -1,8 +1,8 @@ --- -title: Battle Zone +title: BattleZone --- -# Battle Zone +# BattleZone ```{figure} ../../_static/videos/atari/battle_zone.gif :width: 120px @@ -11,77 +11,88 @@ title: Battle Zone This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/BattleZone-v5")` | - ## Description -You control a tank and must destroy enemy vehicles. This game is played in a first-person perspective and creates -a 3D illusion. A radar screen shows enemies around you. You start with 5 lives and gain up to 2 extra lives if you reach -a sufficient score. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=859&itemTypeID=HTMLMANUAL) +You control a tank and must destroy enemy vehicles. This game is played in a first-person perspective and creates a 3D illusion. A radar screen shows enemies around you. You start with 5 lives and gain up to 2 extra lives if you reach a sufficient score. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=859) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +BattleZone has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As BattleZone uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You receive points for destroying enemies. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=859&itemTypeID=HTMLMANUAL). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/BattleZone-v5") -``` +BattleZone has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------------|-------------|--------------|------------------------------| +| BattleZone-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| BattleZone-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| BattleZone-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| BattleZone-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| BattleZoneDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| BattleZoneNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| BattleZone-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| BattleZone-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| BattleZone-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| BattleZone-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| BattleZoneDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| BattleZoneNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/BattleZone-v5 | `"rgb"` | `4` | `0.25` | +| ALE/BattleZone-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| BattleZone | `[1, 2, 3]` | `[0]` | `1` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("BattleZone-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[1, 2, 3]` | `1` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/beam_rider.md b/docs/environments/atari/beam_rider.md index 478bcc946..02f708903 100644 --- a/docs/environments/atari/beam_rider.md +++ b/docs/environments/atari/beam_rider.md @@ -1,15 +1,8 @@ --- -layout: env -title: Beam Rider -grid: - - Action Space: Discrete(18) - - Observation Shape: (210, 160, 3) - - Observation High: 255 - - Observation Low: 0 - - Import: gymnasium.make("ALE/BeamRider-v5") +title: BeamRider --- -# Beam Rider +# BeamRider ```{figure} ../../_static/videos/atari/beam_rider.gif :width: 120px @@ -18,90 +11,81 @@ grid: This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/BeamRider-v5")` | - ## Description -You control a space-ship that travels forward at a constant speed. You can only steer it sideways between discrete -positions. Your goal is to destroy enemy ships, avoid their attacks and dodge space debris. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SystemID=2600&SoftwareID=860&itemTypeID=MANUAL) +You control a space-ship that travels forward at a constant speed. You can only steer it sideways between discrete positions. Your goal is to destroy enemy ships, avoid their attacks and dodge space debris. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareID=860) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +BeamRider has the action space `Discrete(9)` with the table below lists the meaning of each action's meanings. +As BeamRider uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | RIGHTFIRE | -| 8 | LEFTIFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `UPRIGHT` | +| `6` | `UPLEFT` | +| `7` | `RIGHTFIRE` | +| `8` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for destroying enemies. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SystemID=2600&SoftwareID=860&itemTypeID=MANUAL). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/BeamRider-v5") -``` +BeamRider has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| BeamRider-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| BeamRider-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| BeamRider-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| BeamRider-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| BeamRiderDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| BeamRiderNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| BeamRider-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| BeamRider-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| BeamRider-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| BeamRider-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| BeamRiderDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| BeamRiderNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/BeamRider-v5 | `"rgb"` | `4` | `0.25` | +| ALE/BeamRider-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| BeamRider | `[0]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("BeamRider-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/berzerk.md b/docs/environments/atari/berzerk.md index 3ada8ac0f..28d88be76 100644 --- a/docs/environments/atari/berzerk.md +++ b/docs/environments/atari/berzerk.md @@ -11,76 +11,88 @@ title: Berzerk This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Berzerk-v5")` | - ## Description -You are stuck in a maze with evil robots. You must destroy them and avoid touching the walls of the maze, as this will kill you. You may be awarded extra lives after scoring a sufficient number of points, depending on the game mode. -You may also be chased by an undefeatable enemy, Evil Otto, that you must avoid. Evil Otto does not appear in the default mode. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=866&itemTypeID=HTMLMANUAL) +You are stuck in a maze with evil robots. You must destroy them and avoid touching the walls of the maze, as this will kill you. You may be awarded extra lives after scoring a sufficient number of points, depending on the game mode.You may also be chased by an undefeatable enemy, Evil Otto, that you must avoid. Evil Otto does not appear in the default mode. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=866) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Berzerk has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Berzerk uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for destroying robots. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=866&itemTypeID=HTMLMANUAL). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Berzerk-v5") -``` +Berzerk has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Berzerk-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Berzerk-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Berzerk-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Berzerk-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| BerzerkDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| BerzerkNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Berzerk-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Berzerk-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Berzerk-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Berzerk-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| BerzerkDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| BerzerkNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Berzerk-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Berzerk-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------------------|--------------------|--------------| -| Berzerk | `[1, ..., 9, 16, 17, 18]` | `[0]` | `1` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Berzerk-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|---------------------------|----------------|--------------------------|----------------------| +| `[1, ..., 9, 16, 17, 18]` | `1` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/bowling.md b/docs/environments/atari/bowling.md index 94ab1a42a..8adaad830 100644 --- a/docs/environments/atari/bowling.md +++ b/docs/environments/atari/bowling.md @@ -11,59 +11,37 @@ title: Bowling This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Bowling-v5")` | - ## Description -Your goal is to score as many points as possible in the game of Bowling. A game consists of 10 frames and you have two -tries per frame. Knocking down all pins on the first try is called a "strike". Knocking down all pins on the second roll -is called a "spar". Otherwise, the frame is called "open". -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=879) +Your goal is to score as many points as possible in the game of Bowling. A game consists of 10 frames and you have two tries per frame. Knocking down all pins on the first try is called a "strike". Knocking down all pins on the second roll is called a "spar". Otherwise, the frame is called "open". + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=879) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Bowling has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As Bowling uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | DOWN | -| 4 | UPFIRE | -| 5 | DOWNFIRE | +| Value | Meaning | +|---------|------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `DOWN` | +| `4` | `UPFIRE` | +| `5` | `DOWNFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You receive points for knocking down pins. The exact score depends on whether you manage a "strike", "spare" or "open" @@ -71,30 +49,42 @@ frame. Moreover, the points you score for one frame may depend on following fram You can score up to 300 points in one game (if you manage to do 12 strikes). For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=879). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Bowling-v5") -``` +Bowling has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Bowling-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Bowling-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Bowling-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Bowling-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| BowlingDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| BowlingNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Bowling-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Bowling-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Bowling-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Bowling-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| BowlingDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| BowlingNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Bowling-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Bowling-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Bowling | `[0, 2, 4]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Bowling-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 2, 4]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/boxing.md b/docs/environments/atari/boxing.md index eb29769c4..a288cec38 100644 --- a/docs/environments/atari/boxing.md +++ b/docs/environments/atari/boxing.md @@ -11,76 +11,88 @@ title: Boxing This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Boxing-v5")` | - ## Description -You fight an opponent in a boxing ring. You score points for hitting the opponent. If you score 100 points, your opponent is -knocked out. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=882). +You fight an opponent in a boxing ring. You score points for hitting the opponent. If you score 100 points, your opponent is knocked out. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=882) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Boxing has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Boxing uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by landing punches. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=882). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Boxing-v5") -``` +Boxing has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Boxing-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Boxing-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Boxing-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Boxing-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| BoxingDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| BoxingNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Boxing-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Boxing-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Boxing-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Boxing-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| BoxingDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| BoxingNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Boxing-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Boxing-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Boxing | `[0]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Boxing-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/breakout.md b/docs/environments/atari/breakout.md index f3f31d3c2..26486a209 100644 --- a/docs/environments/atari/breakout.md +++ b/docs/environments/atari/breakout.md @@ -11,86 +11,76 @@ title: Breakout This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Breakout-v5")` | - ## Description -Another famous Atari game. The dynamics are similar to pong: You move a paddle and hit the ball in a brick wall at the -top of the screen. Your goal is to destroy the brick wall. You can try to break through the wall and let the ball -wreak havoc on the other side, all on its own! You have five lives. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=889). +Another famous Atari game. The dynamics are similar to pong: You move a paddle and hit the ball in a brick wall at the top of the screen. Your goal is to destroy the brick wall. You can try to break through the wall and let the ball wreak havoc on the other side, all on its own! You have five lives. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=889) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Breakout has the action space `Discrete(4)` with the table below lists the meaning of each action's meanings. +As Breakout uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|--------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHT | -| 3 | LEFT | +| Value | Meaning | +|---------|-----------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by destroying bricks in the wall. The reward for destroying a brick depends on the color of the brick. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=889). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Breakout-v5") -``` +Breakout has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Breakout-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Breakout-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Breakout-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Breakout-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| BreakoutDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| BreakoutNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Breakout-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Breakout-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Breakout-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Breakout-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| BreakoutDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| BreakoutNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Breakout-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Breakout-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------------------------------------------|--------------------|--------------| -| Breakout | `[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Breakout-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------------------------------------|----------------|--------------------------|----------------------| +| `[0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/carnival.md b/docs/environments/atari/carnival.md index 009115b46..932fab417 100644 --- a/docs/environments/atari/carnival.md +++ b/docs/environments/atari/carnival.md @@ -11,89 +11,79 @@ title: Carnival This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (214, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Carnival-v5")` | - ## Description -This is a "shoot 'em up" game. Targets move horizontally across the screen and you must shoot them. You are -in control of a gun that can be moved horizontally. The supply of ammunition is limited and chickens may steal some bullets -from you if you don't hit them in time. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=908). +This is a "shoot 'em up" game. Targets move horizontally across the screen and you must shoot them. You are in control of a gun that can be moved horizontally. The supply of ammunition is limited and chickens may steal some bullets from you if you don't hit them in time. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=908) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Carnival has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As Carnival uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | RIGHTFIRE | -| 5 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `RIGHTFIRE` | +| `5` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by destroying targets. Points (or bullets) may be subtracted if you hit the target when it shows a minus sign. You will score extra points if it shows a plus sign! For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=908). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Carnival-v5") -``` +Carnival has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Carnival-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Carnival-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Carnival-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Carnival-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| CarnivalDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| CarnivalNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Carnival-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Carnival-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Carnival-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Carnival-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| CarnivalDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| CarnivalNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Carnival-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Carnival-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Carnival | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Carnival-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/centipede.md b/docs/environments/atari/centipede.md index 74ddb0b41..c761ef38d 100644 --- a/docs/environments/atari/centipede.md +++ b/docs/environments/atari/centipede.md @@ -11,79 +11,89 @@ title: Centipede This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Centipede-v5")` | - ## Description -You are an elf and must use your magic wands to fend off spiders, fleas and centipedes. Your goal is to protect mushrooms in -an enchanted forest. If you are bitten by a spider, flea or centipede, you will be temporally paralyzed and you will -lose a magic wand. The game ends once you have lost all wands. You may receive additional wands after scoring -a sufficient number of points. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=911). +You are an elf and must use your magic wands to fend off spiders, fleas and centipedes. Your goal is to protect mushrooms in an enchanted forest. If you are bitten by a spider, flea or centipede, you will be temporally paralyzed and you will lose a magic wand. The game ends once you have lost all wands. You may receive additional wands after scoring a sufficient number of points. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=911) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Centipede has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Centipede uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (210, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by hitting centipedes, scorpions, fleas and spiders. Additional points are awarded after every round (i.e. after you have lost a wand) for mushrooms that were not destroyed. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=911). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Centipede-v5") -``` +Centipede has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| Centipede-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Centipede-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Centipede-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Centipede-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| CentipedeDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| CentipedeNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Centipede-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Centipede-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Centipede-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Centipede-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| CentipedeDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| CentipedeNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Centipede-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Centipede-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Centipede | `[22, 86]` | `[0]` | `22` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Centipede-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[22, 86]` | `22` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/chopper_command.md b/docs/environments/atari/chopper_command.md index 8b3c53844..9396c4393 100644 --- a/docs/environments/atari/chopper_command.md +++ b/docs/environments/atari/chopper_command.md @@ -1,8 +1,8 @@ --- -title: Chopper Command +title: ChopperCommand --- -# Chopper Command +# ChopperCommand ```{figure} ../../_static/videos/atari/chopper_command.gif :width: 120px @@ -11,77 +11,89 @@ title: Chopper Command This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/ChopperCommand-v5")` | - ## Description -You control a helicopter and must protect truck convoys. To that end, you need to shoot down enemy aircraft. -A mini-map is displayed at the bottom of the screen. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=921). +You control a helicopter and must protect truck convoys. To that end, you need to shoot down enemy aircraft.A mini-map is displayed at the bottom of the screen. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=921) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +ChopperCommand has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As ChopperCommand uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (210, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by destroying planes and other helicopters. You score extra points at the end of every wave, depending on the number of trucks that have survived. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=921). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/ChopperCommand-v5") -``` +ChopperCommand has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------------|-------------|--------------|------------------------------| +| ChopperCommand-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| ChopperCommand-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| ChopperCommand-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| ChopperCommand-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| ChopperCommandDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| ChopperCommandNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| ChopperCommand-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| ChopperCommand-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| ChopperCommand-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| ChopperCommand-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| ChopperCommandDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| ChopperCommandNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/ChopperCommand-v5 | `"rgb"` | `4` | `0.25` | +| ALE/ChopperCommand-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|----------------|-------------|--------------------|--------------| -| ChopperCommand | `[0, 2]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("ChopperCommand-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 2]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/crazy_climber.md b/docs/environments/atari/crazy_climber.md index 2917345c6..51eecb482 100644 --- a/docs/environments/atari/crazy_climber.md +++ b/docs/environments/atari/crazy_climber.md @@ -1,8 +1,8 @@ --- -title: Crazy Climber +title: CrazyClimber --- -# Crazy Climber +# CrazyClimber ```{figure} ../../_static/videos/atari/crazy_climber.gif :width: 120px @@ -11,93 +11,80 @@ title: Crazy Climber This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (250, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/CrazyClimber-v5")` | - ## Description -You are a climber trying to reach the top of four builidings, while avoiding obstacles like closing -windows and falling objects. When you receive damage (windows closing or objects) you will fall and -lose one life; you have a total of 5 lives before the end games. At the top of each building, there's -a helicopter which you need to catch to get to the next building. The goal is to climb as fast as -possible while receiving the least amount of damage. +You are a climber trying to reach the top of four buildings, while avoiding obstacles like closing windows and falling objects. When you receive damage (windows closing or objects) you will fall and lose one life; you have a total of 5 lives before the end games. At the top of each building, there's a helicopter which you need to catch to get to the next building. The goal is to climb as fast as possible while receiving the least amount of damage. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=113). +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=113) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +CrazyClimber has the action space `Discrete(9)` with the table below lists the meaning of each action's meanings. +As CrazyClimber uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | DOWNRIGHT | -| 8 | DOWNLEFT | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `UP` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `UPRIGHT` | +| `6` | `UPLEFT` | +| `7` | `DOWNRIGHT` | +| `8` | `DOWNLEFT` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards A table of scores awarded for completing each row of a building is provided on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=113). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/CrazyClimber-v5") -``` +CrazyClimber has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------------|-------------|--------------|------------------------------| +| CrazyClimber-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| CrazyClimber-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| CrazyClimber-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| CrazyClimber-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| CrazyClimberDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| CrazyClimberNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| CrazyClimber-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| CrazyClimber-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| CrazyClimber-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| CrazyClimber-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| CrazyClimberDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| CrazyClimberNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/CrazyClimber-v5 | `"rgb"` | `4` | `0.25` | +| ALE/CrazyClimber-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|--------------|---------------|--------------------|--------------| -| CrazyClimber | `[0, ..., 3]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("CrazyClimber-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/defender.md b/docs/environments/atari/defender.md index 7d0d08eae..cf0e2d836 100644 --- a/docs/environments/atari/defender.md +++ b/docs/environments/atari/defender.md @@ -11,78 +11,88 @@ title: Defender This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Defender-v5")` | - ## Description -Aliens attack the earth. You control a spaceship and must defend humanity by destroying alien ships and rescuing humanoids. -You have three lives and three smart bombs. You lose a live when you are shot down by an alien spaceship. -Points are scored by destroying enemies and retrieving humans that are being abducted. You have an unlimited number of -laser missiles. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=128) +Aliens attack the earth. You control a spaceship and must defend humanity by destroying alien ships and rescuing humanoids.You have three lives and three smart bombs. You lose a live when you are shot down by an alien spaceship.Points are scored by destroying enemies and retrieving humans that are being abducted. You have an unlimited number of laser missiles. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=128) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Defender has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Defender uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You receive points for destroying enemies, rescuing abducted humans and keeping humans alive. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=128). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Defender-v5") -``` +Defender has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Defender-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Defender-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Defender-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Defender-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| DefenderDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| DefenderNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Defender-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Defender-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Defender-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Defender-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| DefenderDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| DefenderNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Defender-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Defender-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------------|--------------------|--------------| -| Defender | `[1, ..., 9, 16]` | `[0, 1]` | `1` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Defender-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[1, ..., 9, 16]` | `1` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/demon_attack.md b/docs/environments/atari/demon_attack.md index 6d8f18048..5ae63c9c7 100644 --- a/docs/environments/atari/demon_attack.md +++ b/docs/environments/atari/demon_attack.md @@ -1,8 +1,8 @@ --- -title: Demon Attack +title: DemonAttack --- -# Demon Attack +# DemonAttack ```{figure} ../../_static/videos/atari/demon_attack.gif :width: 120px @@ -11,95 +11,79 @@ title: Demon Attack This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|----------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/DemonAttack-v5")` | - ## Description -You are facing waves of demons in the ice planet of Krybor. Points are accumulated by destroying -demons. You begin with 3 reserve bunkers, and can increase its number (up to 6) by avoiding enemy -attacks. Each attack wave you survive without any hits, grants you a new bunker. Every time an enemy -hits you, a bunker is destroyed. When the last bunker falls, the next enemy hit will destroy you and -the game ends. +You are facing waves of demons in the ice planet of Krybor. Points are accumulated by destroying demons. You begin with 3 reserve bunkers, and can increase its number (up to 6) by avoiding enemy attacks. Each attack wave you survive without any hits, grants you a new bunker. Every time an enemy hits you, a bunker is destroyed. When the last bunker falls, the next enemy hit will destroy you and the game ends. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=135). +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=135) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +DemonAttack has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As DemonAttack uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | DOWNRIGHT | -| 8 | DOWNLEFT | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `RIGHTFIRE` | +| `5` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards Each enemy you slay gives you points. The amount of points depends on the type of demon and which wave you are in. A detailed table of scores is provided on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=135). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/DemonAttack-v5") -``` +DemonAttack has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|---------------------------------|-------------|--------------|------------------------------| +| DemonAttack-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| DemonAttack-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| DemonAttack-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| DemonAttack-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| DemonAttackDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| DemonAttackNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| DemonAttack-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| DemonAttack-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| DemonAttack-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| DemonAttack-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| DemonAttackDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| DemonAttackNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/DemonAttack-v5 | `"rgb"` | `4` | `0.25` | +| ALE/DemonAttack-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|----------------|--------------------|--------------| -| DemonAttack | `[1, 3, 5, 7]` | `[0, 1]` | `1` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("DemonAttack-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[1, 3, 5, 7]` | `1` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/double_dunk.md b/docs/environments/atari/double_dunk.md index 4109486d3..3336b1673 100644 --- a/docs/environments/atari/double_dunk.md +++ b/docs/environments/atari/double_dunk.md @@ -1,8 +1,8 @@ --- -title: Double Dunk +title: DoubleDunk --- -# Double Dunk +# DoubleDunk ```{figure} ../../_static/videos/atari/double_dunk.gif :width: 120px @@ -11,93 +11,89 @@ title: Double Dunk This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (250, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/DoubleDunk-v5")` | - ## Description -You are playing a 2v2 game of basketball. At the start of each possession, you select between a set -of different plays and then execute them to either score or prevent your rivals from scoring. The -game lasts a set amount of time or until one of the teams reaches a certain score +You are playing a 2v2 game of basketball. At the start of each possession, you select between a set of different plays and then execute them to either score or prevent your rivals from scoring. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=153). +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=153) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +DoubleDunk has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As DoubleDunk uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | DOWNRIGHT | -| 8 | DOWNLEFT | +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards Scores follow the rules of basketball. You can get either 3 points, 2 points foul line) depending from where you shoot. After a defensive foul, a successful shot from the foul line gives you 1 point. -## Arguments +## Variants -```python -env = gymnasium.make("ALE/DoubleDunk-v5") -``` +DoubleDunk has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------------|-------------|--------------|------------------------------| +| DoubleDunk-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| DoubleDunk-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| DoubleDunk-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| DoubleDunk-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| DoubleDunkDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| DoubleDunkNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| DoubleDunk-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| DoubleDunk-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| DoubleDunk-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| DoubleDunk-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| DoubleDunkDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| DoubleDunkNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/DoubleDunk-v5 | `"rgb"` | `4` | `0.25` | +| ALE/DoubleDunk-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|----------------|--------------------|--------------| -| DoubleDunk | `[0, ..., 15]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("DoubleDunk-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, ..., 15]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/elevator_action.md b/docs/environments/atari/elevator_action.md index 236261dfa..8456866b6 100644 --- a/docs/environments/atari/elevator_action.md +++ b/docs/environments/atari/elevator_action.md @@ -1,8 +1,8 @@ --- -title: Elevator Action +title: ElevatorAction --- -# Elevator Action +# ElevatorAction ```{figure} ../../_static/videos/atari/elevator_action.gif :width: 120px @@ -11,96 +11,89 @@ title: Elevator Action This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (250, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/ElevatorAction-v5")` | - ## Description -You are a secret agent that must retrieve some secret documents and reach the ground level of a -building by going down an elevator/stairs. Once you reach the ground level, you are picked up and -taken to the next level. You are equipped with a gun to defend yourself against enemy agents waiting -for you in each floor. You gather points by shooting down enemy agents and visiting apartments -marked with a red door, which contain the secret documents. +You are a secret agent that must retrieve some secret documents and reach the ground level of a building by going down an elevator/stairs. Once you reach the ground level, you are picked up and taken to the next level. You are equipped with a gun to defend yourself against enemy agents waiting for you in each floor. You gather points by shooting down enemy agents and visiting apartments marked with a red door, which contain the secret documents.This is an unreleased prototype based on the arcade game. -This is an unreleased prototype based on the arcade game. Limited documentation can be found on [the AtariAge -page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=1131). +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=1131) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +ElevatorAction has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As ElevatorAction uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | DOWNRIGHT | -| 8 | DOWNLEFT | +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You start with 4 lives and are awarded 100 points for each enemy shot, and 500 points for each secret document collected (visiting a red door). Each time you get shot you lose one life and the game ends when losing all lives. -## Arguments +## Variants -```python -env = gymnasium.make("ALE/ElevatorAction-v5") -``` +ElevatorAction has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------------|-------------|--------------|------------------------------| +| ElevatorAction-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| ElevatorAction-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| ElevatorAction-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| ElevatorAction-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| ElevatorActionDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| ElevatorActionNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| ElevatorAction-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| ElevatorAction-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| ElevatorAction-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| ElevatorAction-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| ElevatorActionDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| ElevatorActionNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/ElevatorAction-v5 | `"rgb"` | `4` | `0.25` | +| ALE/ElevatorAction-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|----------------|-------------|--------------------|--------------| -| ElevatorAction | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("ElevatorAction-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/enduro.md b/docs/environments/atari/enduro.md index ecbc89411..b763b4c51 100644 --- a/docs/environments/atari/enduro.md +++ b/docs/environments/atari/enduro.md @@ -11,91 +11,80 @@ title: Enduro This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (250, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Enduro-v5")` | - ## Description -You are a racer in the National Enduro, a long-distance endurance race. You must overtake a certain -amount of cars each day to stay on the race. The first day you need to pass 200 cars, and 300 for -each following day. The game ends if you do not meet your overtake quota for the day. +You are a racer in the National Enduro, a long-distance endurance race. You must overtake a certain amount of cars each day to stay on the race. The first day you need to pass 200 cars, and 300 foreach following day. The game ends if you do not meet your overtake quota for the day. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=163). +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=163) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Enduro has the action space `Discrete(9)` with the table below lists the meaning of each action's meanings. +As Enduro uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | DOWNRIGHT | -| 8 | DOWNLEFT | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `DOWNRIGHT` | +| `6` | `DOWNLEFT` | +| `7` | `RIGHTFIRE` | +| `8` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You get 1 point for each vehicle you overtake. -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Enduro-v5") -``` +Enduro has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Enduro-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Enduro-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Enduro-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Enduro-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| EnduroDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| EnduroNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Enduro-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Enduro-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Enduro-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Enduro-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| EnduroDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| EnduroNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Enduro-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Enduro-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Enduro | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Enduro-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/fishing_derby.md b/docs/environments/atari/fishing_derby.md index 75728f99d..2ab86e0c9 100644 --- a/docs/environments/atari/fishing_derby.md +++ b/docs/environments/atari/fishing_derby.md @@ -11,18 +11,47 @@ title: FishingDerby This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/FishingDerby-v5")` | - ## Description -your objective is to catch more sunfish than your opponent. But it's not just between you and the other fisherman, as a big, black shark is lurking just below the surface, waiting to steal your catch! Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=182). +your objective is to catch more sunfish than your opponent. +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=182) + +## Actions + +FishingDerby has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As FishingDerby uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | + +## Observations + +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. + +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` + +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can @@ -30,87 +59,42 @@ find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?Softw Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). -### Action Space +## Variants -The action space a subset of the following discrete set of legal actions: +FishingDerby has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -| Num | Action | -|-----|---------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPRIGHT | -| 7 | UPLEFT | -| 8 | DOWNRIGHT | -| 9 | DOWNLEFT | -| 10 | UPFIRE | -| 11 | RIGHTFIRE | -| 12 | LEFTFIRE | -| 13 | DOWNFIRE | -| 14 | UPRIGHTFIRE | -| 15 | UPLEFTFIRE | -| 16 | DOWNRIGHTFIRE | -| 17 | DOWNLEFTFIRE | +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------------|-------------|--------------|------------------------------| +| FishingDerby-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| FishingDerby-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| FishingDerby-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| FishingDerby-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| FishingDerbyDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| FishingDerbyNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| FishingDerby-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| FishingDerby-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| FishingDerby-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| FishingDerby-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| FishingDerbyDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| FishingDerbyNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/FishingDerby-v5 | `"rgb"` | `4` | `0.25` | +| ALE/FishingDerby-ram-v5 | `"ram"` | `4` | `0.25` | -If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have -any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full -legal space by passing the keyword argument `full_action_space=True` to `make`. +## Difficulty and modes -The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing -the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the -action spaces of default flavors. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -## Observations - -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe - -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. - -## Arguments - -```python -env = gymnasium.make("ALE/FishingDerby-v5") -``` - -The various ways to configure the environment are described in detail in the article on Atari environments. - -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|--------------|-------------|--------------------|--------------| -| FishingDerby | `[0]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("FishingDerby-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -| Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` | -|---------|--------------|------------------------------|----------------------| -| v0 | `(2, 5,)` | `0.25` | `False` | -| v4 | `(2, 5,)` | `0.0` | `False` | -| v5 | `5` | `0.25` | `True` | - -> Version v5 follows the best practices outlined in [[2]](#2). Thus, it is recommended to transition to v5 and -> customize the environment using the arguments above, if necessary. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. +* v4: Stickiness of actions was removed +* v0: Initial versions release diff --git a/docs/environments/atari/freeway.md b/docs/environments/atari/freeway.md index 069862ac1..79ec16d29 100644 --- a/docs/environments/atari/freeway.md +++ b/docs/environments/atari/freeway.md @@ -11,18 +11,34 @@ title: Freeway This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Freeway-v5")` | - ## Description -your objective is to guide your chicken across lane after lane of busy rush hour traffic. You receive a point for every chicken that makes it to the top of the screen after crossing all the lanes of traffic. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=192). +your objective is to guide your chicken across lane after lane of busy rush hour traffic. You receive a point for every chicken that makes it to the top of the screen after crossing all the lanes of traffic. +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=192) + +## Actions + +Freeway has the action space `Discrete(3)` with the table below lists the meaning of each action's meanings. +As Freeway uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------| +| `0` | `NOOP` | +| `1` | `UP` | +| `2` | `DOWN` | + +## Observations + +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. + +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` + +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can @@ -30,87 +46,42 @@ find these manuals on [AtariAge](https://atariage.com/manual_thumbs.php?Software Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). -### Action Space +## Variants -The action space a subset of the following discrete set of legal actions: +Freeway has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -| Num | Action | -|-----|---------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPRIGHT | -| 7 | UPLEFT | -| 8 | DOWNRIGHT | -| 9 | DOWNLEFT | -| 10 | UPFIRE | -| 11 | RIGHTFIRE | -| 12 | LEFTFIRE | -| 13 | DOWNFIRE | -| 14 | UPRIGHTFIRE | -| 15 | UPLEFTFIRE | -| 16 | DOWNRIGHTFIRE | -| 17 | DOWNLEFTFIRE | +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Freeway-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Freeway-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Freeway-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Freeway-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| FreewayDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| FreewayNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Freeway-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Freeway-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Freeway-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Freeway-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| FreewayDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| FreewayNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Freeway-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Freeway-ram-v5 | `"ram"` | `4` | `0.25` | -If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have -any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full -legal space by passing the keyword argument `full_action_space=True` to `make`. +## Difficulty and modes -The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing -the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the -action spaces of default flavors. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -## Observations - -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe - -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. - -## Arguments - -```python -env = gymnasium.make("ALE/Freeway-v5") -``` - -The various ways to configure the environment are described in detail in the article on Atari environments. - -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| Freeway | `[0, ..., 7]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Freeway-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, ..., 7]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -|Version|`frameskip=`|`repeat_action_probability=`|`full_action_space=`| -| ----- | --------- | ------------------------- | ---------| -|v0 |`(2, 5,)` |`0.25` |`False` | -|v4 |`(2, 5,)` |`0.0` |`False` | -|v5 |`5` |`0.25` |`True` | - -> Version v5 follows the best practices outlined in [[2]](#2). Thus, it is recommended to transition to v5 and -> customize the environment using the arguments above, if necessary. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. +* v4: Stickiness of actions was removed +* v0: Initial versions release diff --git a/docs/environments/atari/frostbite.md b/docs/environments/atari/frostbite.md index 066686d75..098c8052b 100644 --- a/docs/environments/atari/frostbite.md +++ b/docs/environments/atari/frostbite.md @@ -11,18 +11,47 @@ title: Frostbite This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Frostbite-v5")` | - ## Description - In Frostbite, the player controls "Frostbite Bailey" who hops back and forth across across an Arctic river, changing the color of the ice blocks from white to blue. Each time he does so, a block is added to his igloo. [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=199). +In Frostbite, the player controls "Frostbite Bailey" who hops back and forth across across an Arctic river, changing the color of the ice blocks from white to blue. Each time he does so, a block is added to his igloo. +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=199) + +## Actions + +Frostbite has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Frostbite uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | + +## Observations + +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. + +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` + +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can @@ -30,87 +59,42 @@ find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?Softw Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). -### Action Space +## Variants -The action space a subset of the following discrete set of legal actions: +Frostbite has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -| Num | Action | -|-----|---------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPRIGHT | -| 7 | UPLEFT | -| 8 | DOWNRIGHT | -| 9 | DOWNLEFT | -| 10 | UPFIRE | -| 11 | RIGHTFIRE | -| 12 | LEFTFIRE | -| 13 | DOWNFIRE | -| 14 | UPRIGHTFIRE | -| 15 | UPLEFTFIRE | -| 16 | DOWNRIGHTFIRE | -| 17 | DOWNLEFTFIRE | +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| Frostbite-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Frostbite-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Frostbite-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Frostbite-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| FrostbiteDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| FrostbiteNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Frostbite-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Frostbite-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Frostbite-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Frostbite-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| FrostbiteDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| FrostbiteNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Frostbite-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Frostbite-ram-v5 | `"ram"` | `4` | `0.25` | -If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have -any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full -legal space by passing the keyword argument `full_action_space=True` to `make`. +## Difficulty and modes -The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing -the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the -action spaces of default flavors. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -## Observations - -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe - -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. - -## Arguments - -```python -env = gymnasium.make("ALE/Frostbite-v5") -``` - -The various ways to configure the environment are described in detail in the article on Atari environments. - -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Frostbite | `[0, 2]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Frostbite-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 2]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -| Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` | -|---------|--------------|------------------------------|----------------------| -| v0 | `(2, 5,)` | `0.25` | `False` | -| v4 | `(2, 5,)` | `0.0` | `False` | -| v5 | `5` | `0.25` | `True` | - -> Version v5 follows the best practices outlined in [[2]](#2). Thus, it is recommended to transition to v5 and -> customize the environment using the arguments above, if necessary. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. +* v4: Stickiness of actions was removed +* v0: Initial versions release diff --git a/docs/environments/atari/gopher.md b/docs/environments/atari/gopher.md index c52223837..de0e296e7 100644 --- a/docs/environments/atari/gopher.md +++ b/docs/environments/atari/gopher.md @@ -11,18 +11,39 @@ title: Gopher This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Gopher-v5")` | - ## Description -The player controls a shovel-wielding farmer who protects a crop of three carrots from a gopher. [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=218). +The player controls a shovel-wielding farmer who protects a crop of three carrots from a gopher. +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=218) + +## Actions + +Gopher has the action space `Discrete(8)` with the table below lists the meaning of each action's meanings. +As Gopher uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. + +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `UPFIRE` | +| `6` | `RIGHTFIRE` | +| `7` | `LEFTFIRE` | + +## Observations + +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. + +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` + +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can @@ -30,87 +51,42 @@ find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?Softw Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). -### Action Space +## Variants -The action space a subset of the following discrete set of legal actions: +Gopher has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -| Num | Action | -|-----|---------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPRIGHT | -| 7 | UPLEFT | -| 8 | DOWNRIGHT | -| 9 | DOWNLEFT | -| 10 | UPFIRE | -| 11 | RIGHTFIRE | -| 12 | LEFTFIRE | -| 13 | DOWNFIRE | -| 14 | UPRIGHTFIRE | -| 15 | UPLEFTFIRE | -| 16 | DOWNRIGHTFIRE | -| 17 | DOWNLEFTFIRE | +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Gopher-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Gopher-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Gopher-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Gopher-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| GopherDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| GopherNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Gopher-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Gopher-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Gopher-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Gopher-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| GopherDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| GopherNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Gopher-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Gopher-ram-v5 | `"ram"` | `4` | `0.25` | -If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have -any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full -legal space by passing the keyword argument `full_action_space=True` to `make`. +## Difficulty and modes -The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing -the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the -action spaces of default flavors. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -## Observations - -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe - -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. - -## Arguments - -```python -env = gymnasium.make("ALE/Gopher-v5") -``` - -The various ways to configure the environment are described in detail in the article on Atari environments. - -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Gopher | `[0, 2]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Gopher-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 2]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -| Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` | -|---------|--------------|------------------------------|----------------------| -| v0 | `(2, 5,)` | `0.25` | `False` | -| v4 | `(2, 5,)` | `0.0` | `False` | -| v5 | `5` | `0.25` | `True` | - -> Version v5 follows the best practices outlined in [[2]](#2). Thus, it is recommended to transition to v5 and -> customize the environment using the arguments above, if necessary. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. +* v4: Stickiness of actions was removed +* v0: Initial versions release diff --git a/docs/environments/atari/gravitar.md b/docs/environments/atari/gravitar.md index 929f5eefb..9c308b66f 100644 --- a/docs/environments/atari/gravitar.md +++ b/docs/environments/atari/gravitar.md @@ -11,18 +11,47 @@ title: Gravitar This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Gravitar-v5")` | - ## Description -The player controls a small blue spacecraft. The game starts in a fictional solar system with several planets to explore. If the player moves his ship into a planet, he will be taken to a side-view landscape. Player has to destroy red bunkers [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=223). +The player controls a small blue spacecraft. The game starts in a fictional solar system with several planets to explore. If the player moves his ship into a planet, he will be taken to a side-view landscape. +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=223) + +## Actions + +Gravitar has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Gravitar uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | + +## Observations + +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. + +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` + +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The exact reward dynamics depend on the environment and are usually documented in the game's manual. You can @@ -30,87 +59,42 @@ find these manuals on [AtariAge](https://atariage.com/manual_html_page.php?Softw Atari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1). -### Action Space +## Variants -The action space a subset of the following discrete set of legal actions: +Gravitar has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -| Num | Action | -|-----|---------------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPRIGHT | -| 7 | UPLEFT | -| 8 | DOWNRIGHT | -| 9 | DOWNLEFT | -| 10 | UPFIRE | -| 11 | RIGHTFIRE | -| 12 | LEFTFIRE | -| 13 | DOWNFIRE | -| 14 | UPRIGHTFIRE | -| 15 | UPLEFTFIRE | -| 16 | DOWNRIGHTFIRE | -| 17 | DOWNLEFTFIRE | +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Gravitar-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Gravitar-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Gravitar-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Gravitar-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| GravitarDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| GravitarNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Gravitar-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Gravitar-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Gravitar-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Gravitar-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| GravitarDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| GravitarNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Gravitar-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Gravitar-ram-v5 | `"ram"` | `4` | `0.25` | -If you use v0 or v4 and the environment is initialized via `make`, the action space will usually be much smaller since most legal actions don't have -any effect. Thus, the enumeration of the actions will differ. The action space can be expanded to the full -legal space by passing the keyword argument `full_action_space=True` to `make`. +## Difficulty and modes -The reduced action space of an Atari environment may depend on the flavor of the game. You can specify the flavor by providing -the arguments `difficulty` and `mode` when constructing the environment. This documentation only provides details on the -action spaces of default flavors. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -## Observations - -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe - -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. - -## Arguments - -```python -env = gymnasium.make("ALE/Gravitar-v5") -``` - -The various ways to configure the environment are described in detail in the article on Atari environments. - -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| Gravitar | `[0, ..., 4]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Gravitar-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3, 4]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -| Version | `frameskip=` | `repeat_action_probability=` | `full_action_space=` | -|---------|--------------|------------------------------|----------------------| -| v0 | `(2, 5,)` | `0.25` | `False` | -| v4 | `(2, 5,)` | `0.0` | `False` | -| v5 | `5` | `0.25` | `True` | - -> Version v5 follows the best practices outlined in [[2]](#2). Thus, it is recommended to transition to v5 and -> customize the environment using the arguments above, if necessary. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. +* v4: Stickiness of actions was removed +* v0: Initial versions release diff --git a/docs/environments/atari/hero.md b/docs/environments/atari/hero.md index d20123191..ee0bd641c 100644 --- a/docs/environments/atari/hero.md +++ b/docs/environments/atari/hero.md @@ -11,79 +11,89 @@ title: Hero This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Hero-v5")` | - ## Description -You need to rescue miners that are stuck in a mine shaft. You have access to various tools: A propeller backpack that -allows you to fly wherever you want, sticks of dynamite that can be used to blast through walls, a laser beam to kill -vermin, and a raft to float across stretches of lava. -You have a limited amount of power. Once you run out, you lose a live. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=228) +You need to rescue miners that are stuck in a mine shaft. You have access to various tools: A propeller backpack that allows you to fly wherever you want, sticks of dynamite that can be used to blast through walls, a laser beam to kill vermin, and a raft to float across stretches of lava.You have a limited amount of power. Once you run out, you lose a live. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=228) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Hero has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Hero uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for shooting critters, rescuing miners, and dynamiting walls. Extra points are rewarded for any power remaining after rescuing a miner. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=228). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Hero-v5") -``` +Hero has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------|-------------|--------------|------------------------------| +| Hero-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Hero-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Hero-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Hero-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| HeroDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| HeroNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Hero-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Hero-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Hero-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Hero-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| HeroDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| HeroNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Hero-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Hero-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| Hero | `[0, ..., 4]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Hero-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3, 4]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/ice_hockey.md b/docs/environments/atari/ice_hockey.md index 103b2f629..153a436f5 100644 --- a/docs/environments/atari/ice_hockey.md +++ b/docs/environments/atari/ice_hockey.md @@ -11,78 +11,89 @@ title: IceHockey This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/IceHockey-v5")` | - ## Description -Your goal is to score as many points as possible in a standard game of Ice Hockey over a 3-minute time period. The ball is usually called "the puck". -There are 32 shot angles ranging from the extreme left to the extreme right. The angles can only aim towards the opponent's goal. -Just as in real hockey, you can pass the puck by shooting it off the sides of the rink. This can be really key when you're in position to score a goal. - Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=241) +Your goal is to score as many points as possible in a standard game of Ice Hockey over a 3-minute time period. The ball is usually called "the puck".There are 32 shot angles ranging from the extreme left to the extreme right. The angles can only aim towards the opponent's goal.Just as in real hockey, you can pass the puck by shooting it off the sides of the rink. This can be really key when you're in position to score a goal. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=241) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +IceHockey has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As IceHockey uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points by shooting the puck into your opponent's goal. Your opponent scores in the same manner. There are no limits to how many points you can get per game, other than the time limit of 3-minute games. For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=241). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/IceHockey-v5") -``` +IceHockey has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| IceHockey-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| IceHockey-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| IceHockey-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| IceHockey-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| IceHockeyDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| IceHockeyNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| IceHockey-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| IceHockey-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| IceHockey-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| IceHockey-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| IceHockeyDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| IceHockeyNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/IceHockey-v5 | `"rgb"` | `4` | `0.25` | +| ALE/IceHockey-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| IceHockey | `[0, 2]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Icehockey-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 2]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/jamesbond.md b/docs/environments/atari/jamesbond.md index ef3e0e2aa..fc701ba76 100644 --- a/docs/environments/atari/jamesbond.md +++ b/docs/environments/atari/jamesbond.md @@ -11,79 +11,89 @@ title: Jamesbond This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Jamesbond-v5")` | - ## Description -Your mission is to control Mr. Bond's specially designed multipurpose craft to complete a variety of missions. -The craft moves forward with a right motion and slightly back with a left motion. -An up or down motion causes the craft to jump or dive. -You can also fire by either lobbing a bomb to the bottom of the screen or firing a fixed angle shot to the top of the screen. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=250) +Your mission is to control Mr. Bond's specially designed multipurpose craft to complete a variety of missions.The craft moves forward with a right motion and slightly back with a left motion.An up or down motion causes the craft to jump or dive.You can also fire by either lobbing a bomb to the bottom of the screen or firing a fixed angle shot to the top of the screen. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=250) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Jamesbond has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Jamesbond uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The game ends when you complete the last mission or when you lose the last craft. In either case, you'll receive your final score. There will be a rating based on your score. The highest rating in NOVICE is 006. The highest rating in AGENT is 007. For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=250). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Jamesbond-v5") -``` +Jamesbond has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| Jamesbond-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Jamesbond-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Jamesbond-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Jamesbond-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| JamesbondDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| JamesbondNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Jamesbond-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Jamesbond-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Jamesbond-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Jamesbond-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| JamesbondDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| JamesbondNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Jamesbond-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Jamesbond-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Jamesbond | `[0, 1]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Jamesbond-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/journey_escape.md b/docs/environments/atari/journey_escape.md index 7f328c001..8f34a1068 100644 --- a/docs/environments/atari/journey_escape.md +++ b/docs/environments/atari/journey_escape.md @@ -11,98 +11,89 @@ title: JourneyEscape This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/JourneyEscape-v5")` | - ## Description -You must lead all 5 members of JOURNEY through waves of pesky characters and backstage obstacles to the Scarab Escape Vehicle before time runs out. -You must also protect $50,000 in concert cash from grasping groupies, photographers, and promoters. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=252) +You must lead all 5 members of JOURNEY through waves of pesky characters and backstage obstacles to the Scarab Escape Vehicle before time runs out.You must also protect $50,000 in concert cash from grasping groupies, photographers, and promoters. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=252) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +JourneyEscape has the action space `Discrete(16)` with the table below lists the meaning of each action's meanings. +As JourneyEscape uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|---------------| -| 0 | NOOP | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPRIGHT | -| 7 | UPLEFT | -| 8 | DOWNRIGHT | -| 9 | DOWNLEFT | -| 11 | RIGHTFIRE | -| 12 | LEFTFIRE | -| 13 | DOWNFIRE | -| 14 | UPRIGHTFIRE | -| 15 | UPLEFTFIRE | -| 16 | DOWNRIGHTFIRE | -| 17 | DOWNLEFTFIRE | +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `UP` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `UPRIGHT` | +| `6` | `UPLEFT` | +| `7` | `DOWNRIGHT` | +| `8` | `DOWNLEFT` | +| `9` | `RIGHTFIRE` | +| `10` | `LEFTFIRE` | +| `11` | `DOWNFIRE` | +| `12` | `UPRIGHTFIRE` | +| `13` | `UPLEFTFIRE` | +| `14` | `DOWNRIGHTFIRE` | +| `15` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards At the start of the game, you will have $50,000 and 60 units of time. Your end game score with be dependent on how much time you have remaining and who you encounter along the way. For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=252). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/JourneyEscape-v5") -``` +JourneyEscape has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------------|-------------|--------------|------------------------------| +| JourneyEscape-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| JourneyEscape-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| JourneyEscape-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| JourneyEscape-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| JourneyEscapeDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| JourneyEscapeNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| JourneyEscape-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| JourneyEscape-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| JourneyEscape-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| JourneyEscape-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| JourneyEscapeDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| JourneyEscapeNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/JourneyEscape-v5 | `"rgb"` | `4` | `0.25` | +| ALE/JourneyEscape-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|--------------| -| JourneyEscape | `[0]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("JourneyEscape-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/kangaroo.md b/docs/environments/atari/kangaroo.md index 2965268f7..801f30dbb 100644 --- a/docs/environments/atari/kangaroo.md +++ b/docs/environments/atari/kangaroo.md @@ -11,77 +11,89 @@ title: Kangaroo This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Kangaroo-v5")` | - ## Description -The object of the game is to score as many points as you can while controlling Mother Kangaroo to rescue her precious baby. You start the game with three lives. -During this rescue mission, Mother Kangaroo encounters many obstacles. You need to help her climb ladders, pick bonus fruit, and throw punches at monkeys. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=923) +The object of the game is to score as many points as you can while controlling Mother Kangaroo to rescue her precious baby. You start the game with three lives.During this rescue mission, Mother Kangaroo encounters many obstacles. You need to help her climb ladders, pick bonus fruit, and throw punches at monkeys. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=923) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Kangaroo has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Kangaroo uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards Your score will be shown at the top right corner of the game. Your end game score with be dependent on how much time you have remaining and who you encounter along the way. For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=923). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Kangaroo-v5") -``` +Kangaroo has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Kangaroo-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Kangaroo-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Kangaroo-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Kangaroo-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| KangarooDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| KangarooNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Kangaroo-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Kangaroo-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Kangaroo-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Kangaroo-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| KangarooDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| KangarooNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Kangaroo-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Kangaroo-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Kangaroo | `[0, 1]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Kangaroo-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/krull.md b/docs/environments/atari/krull.md index eccd9d932..f7794883c 100644 --- a/docs/environments/atari/krull.md +++ b/docs/environments/atari/krull.md @@ -11,77 +11,89 @@ title: Krull This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Krull-v5")` | - ## Description -Your mission is to find and enter the Beast's Black Fortress, rescue Princess Lyssa, and destroy the Beast. -The task is not an easy one, for the location of the Black Fortress changes with each sunrise on Krull. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=267) +Your mission is to find and enter the Beast's Black Fortress, rescue Princess Lyssa, and destroy the Beast.The task is not an easy one, for the location of the Black Fortress changes with each sunrise on Krull. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=267) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +Krull has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Krull uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. [The general article on Atari environments](https://brosa.ca/blog/ale-release-v0.7) outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You will receive various scores for each monster you kill. You can play the game until you have lost all your lives. For a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=267). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Krull-v5") -``` +Krull has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|---------------------------|-------------|--------------|------------------------------| +| Krull-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Krull-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Krull-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Krull-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| KrullDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| KrullNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Krull-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Krull-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Krull-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Krull-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| KrullDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| KrullNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Krull-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Krull-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Krull | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Krull-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/kung_fu_master.md b/docs/environments/atari/kung_fu_master.md index 5df1baf71..866153466 100644 --- a/docs/environments/atari/kung_fu_master.md +++ b/docs/environments/atari/kung_fu_master.md @@ -1,8 +1,8 @@ --- -title: Kung Fu Master +title: KungFuMaster --- -# Kung Fu Master +# KungFuMaster ```{figure} ../../_static/videos/atari/kung_fu_master.gif :width: 120px @@ -11,82 +11,82 @@ title: Kung Fu Master This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/KungFuMaster-v5")` | - ## Description -You are a Kung-Fu Master fighting your way through the Evil Wizard's temple. Your goal is to rescue Princess Victoria, defeating various enemies along the way. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=268). +You are a Kung-Fu Master fighting your way through the Evil Wizard's temple. Your goal is to rescue Princess Victoria, defeating various enemies along the way. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=268) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +KungFuMaster has the action space `Discrete(14)` with the table below lists the meaning of each action's meanings. +As KungFuMaster uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|---------------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | DOWNRIGHT | -| 6 | DOWNLEFT | -| 7 | RIGHTFIRE | -| 8 | LEFTFIRE | -| 9 | DOWNFIRE | -| 10 | UPRIGHTFIRE | -| 11 | UPLEFTFIRE | -| 12 | DOWNRIGHTFIRE | -| 13 | DOWNLEFTFIRE | +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `UP` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `DOWNRIGHT` | +| `6` | `DOWNLEFT` | +| `7` | `RIGHTFIRE` | +| `8` | `LEFTFIRE` | +| `9` | `DOWNFIRE` | +| `10` | `UPRIGHTFIRE` | +| `11` | `UPLEFTFIRE` | +| `12` | `DOWNRIGHTFIRE` | +| `13` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +KungFuMaster has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------------|-------------|--------------|------------------------------| +| KungFuMaster-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| KungFuMaster-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| KungFuMaster-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| KungFuMaster-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| KungFuMasterDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| KungFuMasterNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| KungFuMaster-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| KungFuMaster-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| KungFuMaster-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| KungFuMaster-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| KungFuMasterDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| KungFuMasterNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/KungFuMaster-v5 | `"rgb"` | `4` | `0.25` | +| ALE/KungFuMaster-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/KungFuMaster-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|--------------|-------------|--------------------|--------------| -| KungFuMaster | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("KungFuMaster-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/montezuma_revenge.md b/docs/environments/atari/montezuma_revenge.md index 4301a700c..a3dd57cb6 100644 --- a/docs/environments/atari/montezuma_revenge.md +++ b/docs/environments/atari/montezuma_revenge.md @@ -1,8 +1,8 @@ --- -title: Montezuma Revenge +title: MontezumaRevenge --- -# Montezuma Revenge +# MontezumaRevenge ```{figure} ../../_static/videos/atari/montezuma_revenge.gif :width: 120px @@ -11,67 +11,84 @@ title: Montezuma Revenge This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/MontezumaRevenge-v5")` | - ## Description -Your goal is to acquire Montezuma's treasure by making your way through a maze of chambers within the emperor's fortress. You must avoid deadly creatures while collecting valuables and tools which can help you escape with the treasure. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=310). +Your goal is to acquire Montezuma's treasure by making your way through a maze of chambers within the emperor's fortress. You must avoid deadly creatures while collecting valuables and tools which can help you escape with the treasure. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=310) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions -will be available in the default flavor. +MontezumaRevenge has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As MontezumaRevenge uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +MontezumaRevenge has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------------------|-------------|--------------|------------------------------| +| MontezumaRevenge-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| MontezumaRevenge-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| MontezumaRevenge-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| MontezumaRevenge-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| MontezumaRevengeDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| MontezumaRevengeNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| MontezumaRevenge-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| MontezumaRevenge-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| MontezumaRevenge-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| MontezumaRevenge-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| MontezumaRevengeDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| MontezumaRevengeNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/MontezumaRevenge-v5 | `"rgb"` | `4` | `0.25` | +| ALE/MontezumaRevenge-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/MontezumaRevenge-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|------------------|-------------|--------------------|--------------| -| MontezumaRevenge | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("MontezumaRevenge-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/ms_pacman.md b/docs/environments/atari/ms_pacman.md index f21ffe1e6..6815d7890 100644 --- a/docs/environments/atari/ms_pacman.md +++ b/docs/environments/atari/ms_pacman.md @@ -1,8 +1,8 @@ --- -title: Ms Pacman +title: MsPacman --- -# Ms Pacman +# MsPacman ```{figure} ../../_static/videos/atari/ms_pacman.gif :width: 120px @@ -11,77 +11,77 @@ title: Ms Pacman This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/MsPacman-v5")` | - ## Description Your goal is to collect all of the pellets on the screen while avoiding the ghosts. +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_page.php?SoftwareLabelID=924) + ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +MsPacman has the action space `Discrete(9)` with the table below lists the meaning of each action's meanings. +As MsPacman uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPRIGHT | -| 6 | UPLEFT | -| 7 | DOWNRIGHT | -| 8 | DOWNLEFT | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `UP` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `UPRIGHT` | +| `6` | `UPLEFT` | +| `7` | `DOWNRIGHT` | +| `8` | `DOWNLEFT` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +MsPacman has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| MsPacman-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| MsPacman-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| MsPacman-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| MsPacman-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| MsPacmanDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| MsPacmanNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| MsPacman-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| MsPacman-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| MsPacman-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| MsPacman-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| MsPacmanDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| MsPacmanNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/MsPacman-v5 | `"rgb"` | `4` | `0.25` | +| ALE/MsPacman-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/MsPacman-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| MsPacman | `[0, ..., 3]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("MsPacman-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/name_this_game.md b/docs/environments/atari/name_this_game.md index 595dd2e5a..4be0779c6 100644 --- a/docs/environments/atari/name_this_game.md +++ b/docs/environments/atari/name_this_game.md @@ -1,8 +1,8 @@ --- -title: Name This Game +title: NameThisGame --- -# Name This Game +# NameThisGame ```{figure} ../../_static/videos/atari/name_this_game.gif :width: 120px @@ -11,74 +11,74 @@ title: Name This Game This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/NameThisGame-v5")` | - ## Description -Your goal is to defend the treasure that you have discovered. You must fight off a shark and an octopus while keeping an eye on your oxygen supply. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=323). +Your goal is to defend the treasure that you have discovered. You must fight off a shark and an octopus while keeping an eye on your oxygen supply. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=323) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +NameThisGame has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As NameThisGame uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | RIGHTFIRE | -| 5 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `RIGHTFIRE` | +| `5` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +NameThisGame has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------------|-------------|--------------|------------------------------| +| NameThisGame-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| NameThisGame-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| NameThisGame-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| NameThisGame-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| NameThisGameDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| NameThisGameNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| NameThisGame-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| NameThisGame-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| NameThisGame-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| NameThisGame-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| NameThisGameDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| NameThisGameNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/NameThisGame-v5 | `"rgb"` | `4` | `0.25` | +| ALE/NameThisGame-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/NameThisGame-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|--------------|---------------|--------------------|--------------| -| NameThisGame | `[8, 24, 40]` | `[0, 1]` | `8` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("NameThisGame-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[8, 24, 40]` | `8` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/phoenix.md b/docs/environments/atari/phoenix.md index 5dc65724f..ba9695904 100644 --- a/docs/environments/atari/phoenix.md +++ b/docs/environments/atari/phoenix.md @@ -11,76 +11,76 @@ title: Phoenix This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Phoenix-v5")` | - ## Description -Your goal is to reach and shoot the alien pilot. On your way there, you must eliminate waves of war birds while avoiding their bombs. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=355). +Your goal is to reach and shoot the alien pilot. On your way there, you must eliminate waves of war birds while avoiding their bombs. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=355) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +Phoenix has the action space `Discrete(8)` with the table below lists the meaning of each action's meanings. +As Phoenix uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | RIGHTFIRE | -| 6 | LEFTFIRE | -| 7 | DOWNFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `RIGHTFIRE` | +| `6` | `LEFTFIRE` | +| `7` | `DOWNFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +Phoenix has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Phoenix-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Phoenix-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Phoenix-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Phoenix-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| PhoenixDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| PhoenixNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Phoenix-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Phoenix-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Phoenix-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Phoenix-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| PhoenixDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| PhoenixNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Phoenix-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Phoenix-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/Phoenix-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Phoenix | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Phoenix-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/pitfall.md b/docs/environments/atari/pitfall.md index 34d5e3b37..b2d8e6bd5 100644 --- a/docs/environments/atari/pitfall.md +++ b/docs/environments/atari/pitfall.md @@ -11,73 +11,90 @@ title: Pitfall This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Pitfall-v5")` | - ## Description You control Pitfall Harry and are tasked with collecting all the treasures in a jungle within 20 minutes. You have three lives. The game is over if you collect all the treasures or if you die or if the time runs out. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=360) + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=360) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +Pitfall has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Pitfall uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You get score points for collecting treasure, you lose points through some misfortunes like falling down a hole. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=360). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Pitfall-v5") -``` +Pitfall has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Pitfall-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Pitfall-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Pitfall-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Pitfall-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| PitfallDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| PitfallNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Pitfall-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Pitfall-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Pitfall-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Pitfall-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| PitfallDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| PitfallNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Pitfall-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Pitfall-ram-v5 | `"ram"` | `4` | `0.25` | +| ALE/Pitfall2-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Pitfall2-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Pitfall | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Pitfall-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/pong.md b/docs/environments/atari/pong.md index 296e0362d..6e7cee930 100644 --- a/docs/environments/atari/pong.md +++ b/docs/environments/atari/pong.md @@ -11,84 +11,78 @@ title: Pong This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Pong-v5")` | - ## Description You control the right paddle, you compete against the left paddle controlled by the computer. You each try to keep deflecting the ball away from your goal and into your opponent's goal. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=587) + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=587) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Pong has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As Pong uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | RIGHTFIRE | -| 5 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `RIGHTFIRE` | +| `5` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You get score points for getting the ball to pass the opponent's paddle. You lose points if the ball passes your paddle. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=587). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Pong-v5") -``` +Pong has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------|-------------|--------------|------------------------------| +| Pong-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Pong-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Pong-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Pong-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| PongDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| PongNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Pong-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Pong-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Pong-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Pong-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| PongDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| PongNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Pong-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Pong-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Pong | `[0, 1]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Pong-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/pooyan.md b/docs/environments/atari/pooyan.md index 7457e5b13..29fe4c46b 100644 --- a/docs/environments/atari/pooyan.md +++ b/docs/environments/atari/pooyan.md @@ -11,84 +11,78 @@ title: Pooyan This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Pooyan-v5")` | - ## Description You are a mother pig protecting her piglets (Pooyans) from wolves. In the first scene, you can move up and down a rope. Try to shoot the worker's balloons, while guarding yourself from attacks. If the wolves reach the ground safely they will get behind and try to eat you. In the second scene, the wolves try to float up. You have to try and stop them using arrows and bait. You die if a wolf eats you, or a stone or rock hits you. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=372) + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=372) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Pooyan has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As Pooyan uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | DOWN | -| 4 | UPFIRE | -| 5 | DOWNFIRE | +| Value | Meaning | +|---------|------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `DOWN` | +| `4` | `UPFIRE` | +| `5` | `DOWNFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards If you hit a balloon, wolf or stone with an arrow you score points. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=372). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Pooyan-v5") -``` +Pooyan has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Pooyan-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Pooyan-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Pooyan-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Pooyan-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| PooyanDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| PooyanNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Pooyan-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Pooyan-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Pooyan-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Pooyan-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| PooyanDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| PooyanNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Pooyan-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Pooyan-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|--------------------|--------------------|--------------| -| Pooyan | `[10, 30, 50, 70]` | `[0]` | `10` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Pooyan-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|--------------------|----------------|--------------------------|----------------------| +| `[10, 30, 50, 70]` | `10` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/private_eye.md b/docs/environments/atari/private_eye.md index 6255fcc92..f4c5d2314 100644 --- a/docs/environments/atari/private_eye.md +++ b/docs/environments/atari/private_eye.md @@ -11,73 +11,88 @@ title: PrivateEye This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/PrivateEye-v5")` | - ## Description You control the French Private Eye Pierre Touche. Navigate the city streets, parks, secret passages, dead-ends and one-ways in search of the ringleader, Henri Le Fiend and his gang. You also need to find evidence and stolen goods that are scattered about. There are five cases, complete each case before its statute of limitations expires. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=376) + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=376) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +PrivateEye has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As PrivateEye uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for completing your tasks like gathering evidence, nabbing questionable characters or closing cases etc. You lose points if you get hit or if your auto is on a pothole. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=376). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/PrivateEye-v5") -``` +PrivateEye has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------------|-------------|--------------|------------------------------| +| PrivateEye-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| PrivateEye-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| PrivateEye-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| PrivateEye-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| PrivateEyeDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| PrivateEyeNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| PrivateEye-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| PrivateEye-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| PrivateEye-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| PrivateEye-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| PrivateEyeDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| PrivateEyeNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/PrivateEye-v5 | `"rgb"` | `4` | `0.25` | +| ALE/PrivateEye-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| PrivateEye | `[0, ..., 4]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("PrivateEye-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3, 4]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/qbert.md b/docs/environments/atari/qbert.md index 3d48058d9..ab3b78414 100644 --- a/docs/environments/atari/qbert.md +++ b/docs/environments/atari/qbert.md @@ -11,84 +11,78 @@ title: Qbert This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Qbert-v5")` | - ## Description You are Q*bert. Your goal is to change the color of all the cubes on the pyramid to the pyramid's 'destination' color. To do this, you must hop on each cube on the pyramid one at a time while avoiding nasty creatures that lurk there. -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=1224&itemTypeID=HTMLMANUAL) + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=1224) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Qbert has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As Qbert uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|--------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | +| Value | Meaning | +|---------|-----------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for changing color of the cubes to their destination colors or by defeating enemies. You also gain points for completing a level. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=1224&itemTypeID=HTMLMANUAL). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Qbert-v5") -``` +Qbert has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|---------------------------|-------------|--------------|------------------------------| +| Qbert-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Qbert-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Qbert-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Qbert-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| QbertDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| QbertNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Qbert-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Qbert-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Qbert-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Qbert-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| QbertDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| QbertNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Qbert-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Qbert-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Qbert | `[0]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Qbert-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/riverraid.md b/docs/environments/atari/riverraid.md index 9b3d0d428..e02628ce8 100644 --- a/docs/environments/atari/riverraid.md +++ b/docs/environments/atari/riverraid.md @@ -11,51 +11,47 @@ title: Riverraid This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Riverraid-v5")` | - ## Description -You control a jet that flies over a river: you can move it sideways and fire missiles to destroy enemy objects. Each time an enemy object is destroyed you score points (i.e. rewards). +You control a jet that flies over a river: you can move it sideways and fire missiles to destroy enemy objects. Each time an enemy object is destroyed you score points (i.e. rewards).You lose a jet when you run out of fuel: fly over a fuel depot when you begin to run low.You lose a jet even when it collides with the river bank or one of the enemy objects (except fuel depots).The game begins with a squadron of three jets in reserve and you're given an additional jet (up to 9) for each 10,000 points you score. -You lose a jet when you run out of fuel: fly over a fuel depot when you begin to run low. - -You lose a jet even when it collides with the river bank or one of the enemy objects (except fuel depots). - -The game begins with a squadron of three jets in reserve and you're given an additional jet (up to 9) for each 10,000 points you score. - -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=409) +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=409) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +Riverraid has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Riverraid uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards Score points are your only reward. You get score points each time you destroy an enemy object: @@ -70,30 +66,42 @@ Score points are your only reward. You get score points each time you destroy an For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=409). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Riverraid-v5") -``` +Riverraid has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| Riverraid-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Riverraid-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Riverraid-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Riverraid-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| RiverraidDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| RiverraidNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Riverraid-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Riverraid-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Riverraid-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Riverraid-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| RiverraidDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| RiverraidNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Riverraid-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Riverraid-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Riverraid | `[0]` | `[0,1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Riverraid-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/road_runner.md b/docs/environments/atari/road_runner.md index 9be51119c..39af4ce13 100644 --- a/docs/environments/atari/road_runner.md +++ b/docs/environments/atari/road_runner.md @@ -1,8 +1,8 @@ --- -title: Road Runner +title: RoadRunner --- -# Road Runner +# RoadRunner ```{figure} ../../_static/videos/atari/road_runner.gif :width: 120px @@ -11,54 +11,47 @@ title: Road Runner This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/RoadRunner-v0")` | - ## Description -You control the Road Runner(TM) in a race; you can control the direction to run in and times to jumps. -The goal is to outrun Wile E. Coyote(TM) while avoiding the hazards of the desert. +You control the Road Runner(TM) in a race; you can control the direction to run in and times to jumps.The goal is to outrun Wile E. Coyote(TM) while avoiding the hazards of the desert.The game begins with three lives. You lose a life when the coyote catches you, picks you up in a rocket, or shoots you with a cannon. You also lose a life when a truck hits you, you hit a land mine, you fall off a cliff,or you get hit by a falling rock.You score points (i.e. rewards) by eating seeds along the road, eating steel shot, and destroying the coyote. -The game begins with three lives. You lose a life when the coyote -catches you, picks you up in a rocket, or shoots you with a cannon. You also -lose a life when a truck hits you, you hit a land mine, you fall off a cliff, -or you get hit by a falling rock. - -You score points (i.e. rewards) by eating seeds along the road, eating steel shot, and -destroying the coyote. - -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=412) +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=412) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +RoadRunner has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As RoadRunner uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards Score points are your only reward. You get score points each time you: @@ -72,30 +65,42 @@ Score points are your only reward. You get score points each time you: For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=412). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/RoadRunner-v5") -``` +RoadRunner has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------------|-------------|--------------|------------------------------| +| RoadRunner-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| RoadRunner-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| RoadRunner-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| RoadRunner-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| RoadRunnerDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| RoadRunnerNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| RoadRunner-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| RoadRunner-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| RoadRunner-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| RoadRunner-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| RoadRunnerDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| RoadRunnerNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/RoadRunner-v5 | `"rgb"` | `4` | `0.25` | +| ALE/RoadRunner-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| RoadRunner | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("RoadRunner-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/robotank.md b/docs/environments/atari/robotank.md index 85b952c66..13ebe4f9f 100644 --- a/docs/environments/atari/robotank.md +++ b/docs/environments/atari/robotank.md @@ -1,8 +1,8 @@ --- -title: Robot Tank +title: Robotank --- -# Robot Tank +# Robotank ```{figure} ../../_static/videos/atari/robotank.gif :width: 120px @@ -11,56 +11,47 @@ title: Robot Tank This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Robotank-v0")` | - ## Description -You control your Robot Tanks to destroy enemies and avoid enemy fire. +You control your Robot Tanks to destroy enemies and avoid enemy fire.Game ends when all of your Robot Tanks are destroyed or all 12 enemy squadrons are destroyed.The game begins with one active Robot Tank and three reserves.Your Robot Tank may get lost when it is hit by enemy rocket fire - your video scrambles with static interference when this happens - or just become damaged - sensors report the damage by flashing on your control panel (look at V/C/R/T squares).You earn one bonus Robot Tank for every enemy squadron destroyed. The maximum number of bonus Robot Tanks allowed at any one time is 12. -Game ends when all of your Robot Tanks are - destroyed or all 12 enemy squadrons are destroyed. - -The game begins with one active Robot Tank and three reserves. -Your Robot Tank may get lost when it is hit by enemy - rocket fire - your video scrambles with static interference when this - happens - or just become damaged - sensors report the damage by flashing on your control panel (look at V/C/R/T squares). - -You earn one bonus Robot Tank for every enemy squadron destroyed. The maximum - number of bonus Robot Tanks allowed at any one time is 12. - -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=416) +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=416) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +Robotank has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Robotank uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The number of enemies destroyed is the only reward. @@ -71,30 +62,42 @@ A small tank appears at the top of your screen for each enemy For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=416). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Robotank-v5") -``` +Robotank has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Robotank-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Robotank-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Robotank-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Robotank-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| RobotankDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| RobotankNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Robotank-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Robotank-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Robotank-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Robotank-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| RobotankDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| RobotankNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Robotank-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Robotank-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Robotank | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Robotank-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/seaquest.md b/docs/environments/atari/seaquest.md index 561805ee2..cc800abdd 100644 --- a/docs/environments/atari/seaquest.md +++ b/docs/environments/atari/seaquest.md @@ -11,59 +11,47 @@ title: Seaquest This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Seaquest-v0")` | - ## Description -You control a sub able to move in all directions and fire torpedoes. -The goal is to retrieve as many divers as you -can, while dodging and blasting enemy subs and killer sharks; points will be awarded accordingly. +You control a sub able to move in all directions and fire torpedoes.The goal is to retrieve as many divers as you can, while dodging and blasting enemy subs and killer sharks; points will be awarded accordingly.The game begins with one sub and three waiting on the horizon. Each time you increase your score by 10,000 points, an extra sub will be delivered to yourbase. You can only have six reserve subs on the screen at one time.Your sub will explode if it collides with anything except your own divers.The sub has a limited amount of oxygen that decreases at a constant rate during the game. When the oxygen tank is almost empty, you need to surface and if you don't do it in time, your sub will blow up and you'll lose one diver. Each time you're forced to surface, with less than six divers, you lose one diver as well. -The game begins with one sub and three waiting on the horizon. Each time you -increase your score by 10,000 points, an extra sub will be delivered to your -base. You can only have six reserve subs on the screen at one time. - -Your sub will explode if it collides with anything -except your own divers. - -The sub has a limited amount of oxygen that decreases at a constant rate during the game. When the oxygen -tank is almost empty, you need to surface and if you don't do it in -time, yoursub will blow up and you'll lose one diver. Each time you're forced -to surface, with less than six divers, you lose one diver as well. - -Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=424) +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=424) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment.Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +Seaquest has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Seaquest uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards Score points are your only reward. @@ -81,30 +69,42 @@ you're given. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=424). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Seaquest-v5") -``` +Seaquest has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|------------------------------|-------------|--------------|------------------------------| +| Seaquest-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Seaquest-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Seaquest-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Seaquest-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| SeaquestDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| SeaquestNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Seaquest-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Seaquest-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Seaquest-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Seaquest-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| SeaquestDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| SeaquestNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Seaquest-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Seaquest-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Seaquest | `[0]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Seaquest-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/skiing.md b/docs/environments/atari/skiing.md index 3205e0d66..1c43a6b01 100644 --- a/docs/environments/atari/skiing.md +++ b/docs/environments/atari/skiing.md @@ -2,7 +2,7 @@ title: Skiing --- -# Skiings +# Skiing ```{figure} ../../_static/videos/atari/skiing.gif :width: 120px @@ -11,91 +11,76 @@ title: Skiing This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Skiing-v0")` | - ## Description -You control a skier who can move sideways. +You control a skier who can move sideways.The goal is to run through all gates (between the poles) in the fastest time.You are penalized five seconds for each gate you miss.If you hit a gate or a tree, your skier will jump back up and keep going. -The goal is to run through all gates (between the poles) in the fastest time. -You are penalized five seconds for each gate you miss. - -If you hit a gate or a tree, your skier will jump back up -and keep going. But you do lose time, so be careful! - -Detailed documentation can be found on [the AtariAge page [SLALOM RACING section]](https://atariage.com/manual_html_page.php?SoftwareLabelID=434) +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=434) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend -on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +Skiing has the action space `Discrete(3)` with the table below lists the meaning of each action's meanings. +As Skiing uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|--------| -| 0 | NOOP | -| 1 | RIGHT | -| 2 | LEFT | +| Value | Meaning | +|---------|-----------| +| `0` | `NOOP` | +| `1` | `RIGHT` | +| `2` | `LEFT` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards Seconds are your only rewards - negative rewards and penalties (e.g. missing a gate) are assigned as additional seconds. For a more detailed documentation, see [the AtariAge page [SLALOM RACING section]](https://atariage.com/manual_html_page.php?SoftwareLabelID=434). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Skiing-v5") -``` +Skiing has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Skiing-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Skiing-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Skiing-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Skiing-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| SkiingDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| SkiingNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Skiing-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Skiing-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Skiing-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Skiing-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| SkiingDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| SkiingNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Skiing-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Skiing-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Skiing | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Skiing-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/solaris.md b/docs/environments/atari/solaris.md index 39f828721..331e10b03 100644 --- a/docs/environments/atari/solaris.md +++ b/docs/environments/atari/solaris.md @@ -11,71 +11,87 @@ title: Solaris This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Solaris-v5")` | - ## Description -You control a spaceship. Blast enemies before they can blast you. You can warp to different sectors. You have to defend Federation planets, and destroy Zylon forces. Keep track of your fuel, if you run out you lose a life. Warp to a Federation planet to refuel. The game ends if all your ships are destroyed or if you reach the Solaris planet. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=450) +You control a spaceship. Blast enemies before they can blast you. You can warp to different sectors. You have to defend Federation planets, and destroy Zylon forces. Keep track of your fuel, if you run out you lose a life. Warp to a Federation planet to refuel. The game ends if all your ships are destroyed or if you reach the Solaris planet. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=450) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +Solaris has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Solaris uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You gain points for destroying enemies, rescuing cadets, making it through a corridor, destroying enemy planets etc. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=450). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Solaris-v5") -``` +Solaris has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Solaris-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Solaris-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Solaris-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Solaris-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| SolarisDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| SolarisNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Solaris-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Solaris-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Solaris-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Solaris-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| SolarisDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| SolarisNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Solaris-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Solaris-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Solaris | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Solaris-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/space_invaders.md b/docs/environments/atari/space_invaders.md index b6e8fb519..a22f2d72c 100644 --- a/docs/environments/atari/space_invaders.md +++ b/docs/environments/atari/space_invaders.md @@ -11,82 +11,77 @@ title: SpaceInvaders This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/SpaceInvaders-v5")` | - ## Description -Your objective is to destroy the space invaders by shooting your laser cannon at them before they reach the Earth. The game ends when all your lives are lost after taking enemy fire, or when they reach the earth. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=460) +Your objective is to destroy the space invaders by shooting your laser cannon at them before they reach the Earth. The game ends when all your lives are lost after taking enemy fire, or when they reach the earth. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=460) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +SpaceInvaders has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As SpaceInvaders uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | RIGHTFIRE | -| 5 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `RIGHTFIRE` | +| `5` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You gain points for destroying space invaders. The invaders in the back rows are worth more points. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=460). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/SpaceInvaders-v5") -``` +SpaceInvaders has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------------|-------------|--------------|------------------------------| +| SpaceInvaders-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| SpaceInvaders-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| SpaceInvaders-ramDeterministic-v0 | `"ram"` | `3` | `0.25` | +| SpaceInvaders-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| SpaceInvadersDeterministic-v0 | `"rgb"` | `3` | `0.25` | +| SpaceInvadersNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| SpaceInvaders-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| SpaceInvaders-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| SpaceInvaders-ramDeterministic-v4 | `"ram"` | `3` | `0.0` | +| SpaceInvaders-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| SpaceInvadersDeterministic-v4 | `"rgb"` | `3` | `0.0` | +| SpaceInvadersNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/SpaceInvaders-v5 | `"rgb"` | `4` | `0.25` | +| ALE/SpaceInvaders-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|---------------|----------------|--------------------|--------------| -| SpaceInvaders | `[0, ..., 15]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("SpaceInvaders-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, ..., 15]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/star_gunner.md b/docs/environments/atari/star_gunner.md index f6bdb585e..8f7235a1e 100644 --- a/docs/environments/atari/star_gunner.md +++ b/docs/environments/atari/star_gunner.md @@ -11,82 +11,87 @@ title: StarGunner This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|---------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/StarGunner-v5")` | - ## Description -Stop the alien invasion by shooting down alien saucers and creatures while avoiding bombs. More details can be found on [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-stargunner_16921.html) +Stop the alien invasion by shooting down alien saucers and creatures while avoiding bombs. + +For a more detailed documentation, see [the AtariAge page](http://www.atarimania.com/game-atari-2600-vcs-stargunner_16921.html) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +StarGunner has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As StarGunner uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. -| Num | Action | -|-----|--------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for destroying enemies. You get bonus points for clearing a wave and a level. For a more detailed documentation, see [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-stargunner_16921.html). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/StarGunner-v5") -``` +StarGunner has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|--------------------------------|-------------|--------------|------------------------------| +| StarGunner-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| StarGunner-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| StarGunner-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| StarGunner-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| StarGunnerDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| StarGunnerNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| StarGunner-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| StarGunner-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| StarGunner-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| StarGunner-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| StarGunnerDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| StarGunnerNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/StarGunner-v5 | `"rgb"` | `4` | `0.25` | +| ALE/StarGunner-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|---------------|--------------------|--------------| -| StarGunner | `[0, ..., 3]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("StarGunner-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 1, 2, 3]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/tennis.md b/docs/environments/atari/tennis.md index ef45f81e7..1e0a36b27 100644 --- a/docs/environments/atari/tennis.md +++ b/docs/environments/atari/tennis.md @@ -11,72 +11,87 @@ title: Tennis This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Tennis-v5")` | - ## Description -You control the orange player playing against a computer-controlled blue player. The game follows the rules of tennis. -The first player to win at least 6 games with a margin of at least two games wins the match. If the score is tied at 6-6, the first player to go 2 games up wins the match. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=555) +You control the orange player playing against a computer-controlled blue player. The game follows the rules of tennis.The first player to win at least 6 games with a margin of at least two games wins the match. If the score is tied at 6-6, the first player to go 2 games up wins the match. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=555) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -Even if you use v0 or v4 or specify `full_action_space=False` during initialization, all actions will be available in the default flavor. +Tennis has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Tennis uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards The scoring is as per the sport of tennis, played till one set. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=555). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/Tennis-v5") -``` +Tennis has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Tennis-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Tennis-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Tennis-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Tennis-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| TennisDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| TennisNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Tennis-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Tennis-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Tennis-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Tennis-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| TennisDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| TennisNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Tennis-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Tennis-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Tennis | `[0, 2]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Tennis-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 2]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/time_pilot.md b/docs/environments/atari/time_pilot.md index 069f53d9c..3de1c018d 100644 --- a/docs/environments/atari/time_pilot.md +++ b/docs/environments/atari/time_pilot.md @@ -11,85 +11,81 @@ title: TimePilot This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/TimePilot-v5")` | - ## Description -You control an aircraft. Use it to destroy your enemies. As you progress in the game, you encounter enemies with technology that is increasingly from the future. More details can be found on [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-time-pilot_8038.html) +You control an aircraft. Use it to destroy your enemies. As you progress in the game, you encounter enemies with technology that is increasingly from the future. + +For a more detailed documentation, see [the AtariAge page](http://www.atarimania.com/game-atari-2600-vcs-time-pilot_8038.html) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. -However, if you use v0 or v4 or specify `full_action_space=False` during initialization, only a reduced -number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of `mode` and `difficulty`). The reduced action space for the default -flavor looks like this: +TimePilot has the action space `Discrete(10)` with the table below lists the meaning of each action's meanings. +As TimePilot uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPFIRE | -| 7 | RIGHTFIRE | -| 8 | LEFTFIRE | -| 9 | DOWNFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPFIRE` | +| `7` | `RIGHTFIRE` | +| `8` | `LEFTFIRE` | +| `9` | `DOWNFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image - -instead. The respective observation spaces are - -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` - -respectively. The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` ### Rewards You score points for destroying enemies, gaining more points for difficult enemies. For a more detailed documentation, see [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-time-pilot_8038.html). -## Arguments +## Variants -```python -env = gymnasium.make("ALE/TimePilot-v5") -``` +TimePilot has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| TimePilot-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| TimePilot-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| TimePilot-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| TimePilot-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| TimePilotDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| TimePilotNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| TimePilot-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| TimePilot-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| TimePilot-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| TimePilot-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| TimePilotDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| TimePilotNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/TimePilot-v5 | `"rgb"` | `4` | `0.25` | +| ALE/TimePilot-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| TimePilot | `[0]` | `[0, 1, 2]` | `0` | -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "NoFrameskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("TimePilot-v0")`. +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1, 2]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/tutankham.md b/docs/environments/atari/tutankham.md index 87ac82d95..05e7cbffe 100644 --- a/docs/environments/atari/tutankham.md +++ b/docs/environments/atari/tutankham.md @@ -11,76 +11,76 @@ title: Tutankham This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|--------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Tutankham-v5")` | - ## Description -Your goal is to rack up points by finding treasures in the mazes of the tomb while eliminating its guardians. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=572). +Your goal is to rack up points by finding treasures in the mazes of the tomb while eliminating its guardians. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SoftwareLabelID=572) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +Tutankham has the action space `Discrete(8)` with the table below lists the meaning of each action's meanings. +As Tutankham uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | UP | -| 2 | RIGHT | -| 3 | LEFT | -| 4 | DOWN | -| 5 | UPFIRE | -| 6 | RIGHTFIRE | -| 7 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `UP` | +| `2` | `RIGHT` | +| `3` | `LEFT` | +| `4` | `DOWN` | +| `5` | `UPFIRE` | +| `6` | `RIGHTFIRE` | +| `7` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +Tutankham has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-------------------------------|-------------|--------------|------------------------------| +| Tutankham-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Tutankham-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Tutankham-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Tutankham-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| TutankhamDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| TutankhamNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Tutankham-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Tutankham-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Tutankham-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Tutankham-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| TutankhamDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| TutankhamNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Tutankham-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Tutankham-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/Tutankham-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Tutankham | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Tutankham-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 4, 8, 12]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/up_n_down.md b/docs/environments/atari/up_n_down.md index d18737c9e..d5fd1def1 100644 --- a/docs/environments/atari/up_n_down.md +++ b/docs/environments/atari/up_n_down.md @@ -1,8 +1,8 @@ --- -title: Up n' Down +title: UpNDown --- -# Up n' Down +# UpNDown ```{figure} ../../_static/videos/atari/up_n_down.gif :width: 120px @@ -11,75 +11,74 @@ title: Up n' Down This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/UpNDown-v5")` | - ## Description -Your goal is to steer your baja bugger to collect prizes and eliminate opponents. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=574). +Your goal is to steer your baja bugger to collect prizes and eliminate opponents. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=574) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +UpNDown has the action space `Discrete(6)` with the table below lists the meaning of each action's meanings. +As UpNDown uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | DOWN | -| 4 | UPFIRE | -| 5 | DOWNFIRE | +| Value | Meaning | +|---------|------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `DOWN` | +| `4` | `UPFIRE` | +| `5` | `DOWNFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +UpNDown has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| UpNDown-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| UpNDown-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| UpNDown-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| UpNDown-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| UpNDownDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| UpNDownNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| UpNDown-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| UpNDown-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| UpNDown-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| UpNDown-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| UpNDownDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| UpNDownNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/UpNDown-v5 | `"rgb"` | `4` | `0.25` | +| ALE/UpNDown-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes -```python -env = gymnasium.make("ALE/UpNDown-v5") -``` -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| UpNDown | `[0]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("UpNDown-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/venture.md b/docs/environments/atari/venture.md index 135b3cdb7..f1a4dd66c 100644 --- a/docs/environments/atari/venture.md +++ b/docs/environments/atari/venture.md @@ -11,67 +11,84 @@ title: Venture This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Venture-v5")` | - ## Description -Your goal is to capture the treasure in every chamber of the dungeon while eliminating the monsters. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=576). +Your goal is to capture the treasure in every chamber of the dungeon while eliminating the monsters. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=576) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. Even if you use v0 or v4 or specify full_action_space=False during initialization, all actions will be available in the default flavor. +Venture has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Venture uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +Venture has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|-----------------------------|-------------|--------------|------------------------------| +| Venture-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Venture-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Venture-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Venture-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| VentureDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| VentureNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Venture-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Venture-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Venture-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Venture-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| VentureDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| VentureNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Venture-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Venture-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/Venture-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Venture | `[0]` | `[0, ..., 3]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Venture-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1, 2, 3]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/video_pinball.md b/docs/environments/atari/video_pinball.md index 6ad02ad70..ab4559898 100644 --- a/docs/environments/atari/video_pinball.md +++ b/docs/environments/atari/video_pinball.md @@ -1,8 +1,8 @@ --- -title: Video Pinball +title: VideoPinball --- -# Video Pinball +# VideoPinball ```{figure} ../../_static/videos/atari/video_pinball.gif :width: 120px @@ -11,79 +11,77 @@ title: Video Pinball This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/VideoPinball-v5")` | - ## Description -Your goal is to keep the ball in play as long as possible and to score as many points as possible. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=588). +Your goal is to keep the ball in play as long as possible and to score as many points as possible. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=588) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +VideoPinball has the action space `Discrete(9)` with the table below lists the meaning of each action's meanings. +As VideoPinball uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPFIRE | -| 7 | RIGHTFIRE | -| 8 | LEFTFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPFIRE` | +| `7` | `RIGHTFIRE` | +| `8` | `LEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +VideoPinball has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------------|-------------|--------------|------------------------------| +| VideoPinball-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| VideoPinball-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| VideoPinball-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| VideoPinball-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| VideoPinballDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| VideoPinballNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| VideoPinball-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| VideoPinball-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| VideoPinball-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| VideoPinball-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| VideoPinballDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| VideoPinballNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/VideoPinball-v5 | `"rgb"` | `4` | `0.25` | +| ALE/VideoPinball-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/'VideoPinball-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|--------------|---------------|--------------------|--------------| -| VideoPinball | `[0, ..., 2]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("VideoPinball-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 2]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/wizard_of_wor.md b/docs/environments/atari/wizard_of_wor.md index a79f18403..0ffb1d54b 100644 --- a/docs/environments/atari/wizard_of_wor.md +++ b/docs/environments/atari/wizard_of_wor.md @@ -1,8 +1,8 @@ --- -title: Wizard of Wor +title: WizardOfWor --- -# Wizard of Wor +# WizardOfWor ```{figure} ../../_static/videos/atari/wizard_of_wor.gif :width: 120px @@ -11,80 +11,78 @@ title: Wizard of Wor This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|----------------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/WizardOfWor-v5")` | - ## Description -Your goal is to beat the Wizard using your laser and radar scanner. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=598). +Your goal is to beat the Wizard using your laser and radar scanner. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=598) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. However, if you use v0 or v4 or specify full_action_space=False during initialization, only a reduced number of actions (those that are meaningful in this game) are available. The reduced action space may depend on the flavor of the environment (the combination of mode and difficulty). The reduced action space for the default flavor looks like this: +WizardOfWor has the action space `Discrete(10)` with the table below lists the meaning of each action's meanings. +As WizardOfWor uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`. -| Num | Action | -|-----|-----------| -| 0 | NOOP | -| 1 | FIRE | -| 2 | UP | -| 3 | RIGHT | -| 4 | LEFT | -| 5 | DOWN | -| 6 | UPFIRE | -| 7 | RIGHTFIRE | -| 8 | LEFTFIRE | -| 9 | DOWNFIRE | +| Value | Meaning | +|---------|-------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPFIRE` | +| `7` | `RIGHTFIRE` | +| `8` | `LEFTFIRE` | +| `9` | `DOWNFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +WizardOfWor has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|---------------------------------|-------------|--------------|------------------------------| +| WizardOfWor-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| WizardOfWor-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| WizardOfWor-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| WizardOfWor-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| WizardOfWorDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| WizardOfWorNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| WizardOfWor-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| WizardOfWor-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| WizardOfWor-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| WizardOfWor-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| WizardOfWorDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| WizardOfWorNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/WizardOfWor-v5 | `"rgb"` | `4` | `0.25` | +| ALE/WizardOfWor-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/WizardOfWor-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. -A flavor is a combination of a game mode and a difficulty setting. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|--------------|-------------|--------------------|--------------| -| VideoPinball | `[0]` | `[0, 1]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("WizardOfWor-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0]` | `0` | `[0, 1]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/environments/atari/yars_revenge.md b/docs/environments/atari/yars_revenge.md new file mode 100644 index 000000000..6192d80f4 --- /dev/null +++ b/docs/environments/atari/yars_revenge.md @@ -0,0 +1,94 @@ +--- +title: YarsRevenge +--- + +# YarsRevenge + +```{figure} ../../_static/videos/atari/yars_revenge.gif +:width: 120px +:name: YarsRevenge +``` + +This environment is part of the Atari environments. Please read that page first for general information. + +## Description + +The objective is to break a path through the shield and destroy the Qotile with a blast from the Zorlon Cannon. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_page.php?SoftwareLabelID=603¤tPage=1&maxPages=12) + +## Actions + +YarsRevenge has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As YarsRevenge uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | + +## Observations + +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. + +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` + +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` + +## Variants + +YarsRevenge has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|---------------------------------|-------------|--------------|------------------------------| +| YarsRevenge-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| YarsRevenge-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| YarsRevenge-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| YarsRevenge-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| YarsRevengeDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| YarsRevengeNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| YarsRevenge-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| YarsRevenge-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| YarsRevenge-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| YarsRevenge-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| YarsRevengeDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| YarsRevengeNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/YarsRevenge-v5 | `"rgb"` | `4` | `0.25` | +| ALE/YarsRevenge-ram-v5 | `"ram"` | `4` | `0.25` | + +## Difficulty and modes + +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. + +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 32, 64, 96]` | `0` | `[0, 1]` | `0` | + +## Version History + +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. + +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. +* v4: Stickiness of actions was removed +* v0: Initial versions release diff --git a/docs/environments/atari/zaxxon.md b/docs/environments/atari/zaxxon.md index ed85c50a9..d209fbcea 100644 --- a/docs/environments/atari/zaxxon.md +++ b/docs/environments/atari/zaxxon.md @@ -1,6 +1,5 @@ --- title: Zaxxon -lastpage: --- # Zaxxon @@ -12,65 +11,84 @@ lastpage: This environment is part of the Atari environments. Please read that page first for general information. -| | | -|-------------------|-----------------------------------| -| Action Space | Discrete(18) | -| Observation Space | (210, 160, 3) | -| Observation High | 255 | -| Observation Low | 0 | -| Import | `gymnasium.make("ALE/Zaxxon-v5")` | - ## Description -Your goal is to stop the evil robot Zaxxon and its armies from enslaving the galaxy by piloting your fighter and shooting enemies. Detailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=606). +Your goal is to stop the evil robot Zaxxon and its armies from enslaving the galaxy by piloting your fighter and shooting enemies. + +For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=606) ## Actions -By default, all actions that can be performed on an Atari 2600 are available in this environment. Even if you use v0 or v4 or specify full_action_space=False during initialization, all actions will be available in the default flavor. +Zaxxon has the action space `Discrete(18)` with the table below lists the meaning of each action's meanings. +As Zaxxon uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`. + +| Value | Meaning | +|---------|-----------------| +| `0` | `NOOP` | +| `1` | `FIRE` | +| `2` | `UP` | +| `3` | `RIGHT` | +| `4` | `LEFT` | +| `5` | `DOWN` | +| `6` | `UPRIGHT` | +| `7` | `UPLEFT` | +| `8` | `DOWNRIGHT` | +| `9` | `DOWNLEFT` | +| `10` | `UPFIRE` | +| `11` | `RIGHTFIRE` | +| `12` | `LEFTFIRE` | +| `13` | `DOWNFIRE` | +| `14` | `UPRIGHTFIRE` | +| `15` | `UPLEFTFIRE` | +| `16` | `DOWNRIGHTFIRE` | +| `17` | `DOWNLEFTFIRE` | ## Observations -By default, the environment returns the RGB image that is displayed to human players as an observation. However, it is -possible to observe +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. -- The 128 Bytes of RAM of the console -- A grayscale image +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` -instead. The respective observation spaces are +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` -- `Box([0 ... 0], [255 ... 255], (128,), uint8)` -- `Box([[0 ... 0] - ... - [0 ... 0]], [[255 ... 255] - ... - [255 ... 255]], (250, 160), uint8) -` +## Variants -The general article on Atari environments outlines different ways to instantiate corresponding environments -via `gymnasium.make`. +Zaxxon has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. -## Arguments +| Env-id | obs_type= | frameskip= | repeat_action_probability= | +|----------------------------|-------------|--------------|------------------------------| +| Zaxxon-v0 | `"rgb"` | `(2, 5)` | `0.25` | +| Zaxxon-ram-v0 | `"ram"` | `(2, 5)` | `0.25` | +| Zaxxon-ramDeterministic-v0 | `"ram"` | `4` | `0.25` | +| Zaxxon-ramNoFrameskip-v0 | `"ram"` | `1` | `0.25` | +| ZaxxonDeterministic-v0 | `"rgb"` | `4` | `0.25` | +| ZaxxonNoFrameskip-v0 | `"rgb"` | `1` | `0.25` | +| Zaxxon-v4 | `"rgb"` | `(2, 5)` | `0.0` | +| Zaxxon-ram-v4 | `"ram"` | `(2, 5)` | `0.0` | +| Zaxxon-ramDeterministic-v4 | `"ram"` | `4` | `0.0` | +| Zaxxon-ramNoFrameskip-v4 | `"ram"` | `1` | `0.0` | +| ZaxxonDeterministic-v4 | `"rgb"` | `4` | `0.0` | +| ZaxxonNoFrameskip-v4 | `"rgb"` | `1` | `0.0` | +| ALE/Zaxxon-v5 | `"rgb"` | `4` | `0.25` | +| ALE/Zaxxon-ram-v5 | `"ram"` | `4` | `0.25` | -```python -env = gymnasium.make("ALE/Zaxxon-v5") -``` +## Difficulty and modes -The various ways to configure the environment are described in detail in the article on Atari environments. +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. -| Environment | Valid Modes | Valid Difficulties | Default Mode | -|-------------|-------------|--------------------|--------------| -| Zaxxon | `[0]` | `[0]` | `0` | - -You may use the suffix "-ram" to switch to the RAM observation space. In v0 and v4, the suffixes "Deterministic" and "Noframeskip" -are available. These are no longer supported in v5. In order to obtain equivalent behavior, pass keyword arguments to `gymnasium.make` as outlined in -the general article on Atari environments. -The versions v0 and v4 are not contained in the "ALE" namespace. I.e. they are instantiated via `gymnasium.make("Zaxxon-v0")` +| Available Modes | Default Mode | Available Difficulties | Default Difficulty | +|-------------------|----------------|--------------------------|----------------------| +| `[0, 8, 16, 24]` | `0` | `[0]` | `0` | ## Version History -A thorough discussion of the intricate differences between the versions and configurations can be found in the -general article on Atari environments. +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. -* v5: Stickiness was added back and stochastic frameskipping was removed. The entire action space is used by default. The environments are now in the "ALE" namespace. +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. * v4: Stickiness of actions was removed -* v0: Initial versions release (1.0.0) +* v0: Initial versions release diff --git a/docs/scripts/atari-docs.json b/docs/scripts/atari-docs.json new file mode 100644 index 000000000..3a893f7b0 --- /dev/null +++ b/docs/scripts/atari-docs.json @@ -0,0 +1,312 @@ +{ + "adventure": { + "env_description": "You must find the enchanted chalice and return it to the golden castle. You can pick up various objects (keys, a sword,a bridge, or a magnet) and have to fight or outmanoeuvre dragons.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=1", + "reward_description": "" + }, + "air_raid": { + "env_description": "You control a ship that can move sideways. You must protect two buildings (one on the right and one on the left side of the screen) from flying saucers that are trying to drop bombs on them.", + "atariage_url": "", + "reward_description": "" + }, + "alien": { + "env_description": "You are stuck in a maze-like space ship with three aliens. You goal is to destroy their eggs that are scattered all over the ship while simultaneously avoiding the aliens (they are trying to kill you). You have a flamethrower that can help you turn them away in tricky situations. Moreover, you can occasionally collect a power-up (pulsar) that gives you the temporary ability to kill aliens.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=815", + "reward_description": "### Rewards\n\nYou score points by destroying eggs, killing aliens, using pulsars, and collecting special prizes. When you are caught\nby an alien, you will lose one of your lives. The number of lives you have depends on the game flavor. For a\ntable of scores corresponding to the different achievements, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=815).\n" + }, + "amidar": { + "env_description": "This game is similar to Pac-Man: You are trying to visit all places on a 2-dimensional grid while simultaneously avoiding your enemies. You can turn the tables at one point in the game: Your enemies turn into chickens and you can catch them.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=817", + "reward_description": "### Rewards\n\nYou score points by traversing new parts of the grid. Coloring an entire box in the maze or catching chickens gives extra points.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=817).\n" + }, + "assault": { + "env_description": "You control a vehicle that can move sideways. A big mother ship circles overhead and continually deploys smaller drones.You must destroy these enemies and dodge their attacks.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=827", + "reward_description": "" + }, + "asterix": { + "env_description": "You are Asterix and can move horizontally (continuously) and vertically (discretely). Objects move horizontally across the screen: lyres and other (more useful) objects. Your goal is to guideAsterix in such a way as to avoid lyres and collect as many other objects as possible. You score points by collecting objects and lose a life whenever you collect a lyre. You have three lives available at the beginning. If you score sufficiently many points, you will be awarded additional points.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=3325", + "reward_description": "### Rewards\n\nA table of scores awarded for collecting the different objects is provided on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=3325).\n" + }, + "asteroids": { + "env_description": "This is a well-known arcade game: You control a spaceship in an asteroid field and must break up asteroids by shooting them. Once all asteroids are destroyed, you enter a new level and new asteroids will appear. You will occasionally be attacked by a flying saucer.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=828", + "reward_description": "### Rewards\n\nYou score points for destroying asteroids, satellites and UFOs. The smaller the asteroid, the more points you score\nfor destroying it.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=828&itemTypeID=HTMLMANUAL).\n" + }, + "atlantis": { + "env_description": "Your job is to defend the submerged city of Atlantis. Your enemies slowly descend towards the city and you must destroy them before they reach striking distance. To this end, you control three defense posts.You lose if your enemies manage to destroy all seven of Atlantis' installations. You may rebuild installations after you have fought of a wave of enemies and scored a sufficient number of points.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=835", + "reward_description": "### Rewards\n\nYou score points for destroying enemies, keeping installations protected during attack waves. You score more points\nif you manage to destroy your enemies with one of the outer defense posts.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=835).\n" + }, + "bank_heist": { + "env_description": "You are a bank robber and (naturally) want to rob as many banks as possible. You control your getaway car and must navigate maze-like cities. The police chases you and will appear whenever you rob a bank. You may destroy police cars by dropping sticks of dynamite. You can fill up your gas tank by entering a new city.At the beginning of the game you have four lives. Lives are lost if you run out of gas, are caught by the police,or run over the dynamite you have previously dropped.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=1008", + "reward_description": "### Rewards\n\nYou score points for robbing banks and destroying police cars. If you rob nine or more banks, and then leave the city,\nyou will score extra points.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=1008).\n" + }, + "battle_zone": { + "env_description": "You control a tank and must destroy enemy vehicles. This game is played in a first-person perspective and creates a 3D illusion. A radar screen shows enemies around you. You start with 5 lives and gain up to 2 extra lives if you reach a sufficient score.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=859", + "reward_description": "### Rewards\n\nYou receive points for destroying enemies.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=859&itemTypeID=HTMLMANUAL).\n" + }, + "beam_rider": { + "env_description": "You control a space-ship that travels forward at a constant speed. You can only steer it sideways between discrete positions. Your goal is to destroy enemy ships, avoid their attacks and dodge space debris.", + "atariage_url": "https://atariage.com/manual_thumbs.php?SoftwareID=860", + "reward_description": "### Rewards\n\nYou score points for destroying enemies.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_thumbs.php?SystemID=2600&SoftwareID=860&itemTypeID=MANUAL).\n" + }, + "berzerk": { + "env_description": "You are stuck in a maze with evil robots. You must destroy them and avoid touching the walls of the maze, as this will kill you. You may be awarded extra lives after scoring a sufficient number of points, depending on the game mode.You may also be chased by an undefeatable enemy, Evil Otto, that you must avoid. Evil Otto does not appear in the default mode.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=866", + "reward_description": "### Rewards\n\nYou score points for destroying robots.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=866&itemTypeID=HTMLMANUAL).\n" + }, + "bowling": { + "env_description": "Your goal is to score as many points as possible in the game of Bowling. A game consists of 10 frames and you have two tries per frame. Knocking down all pins on the first try is called a \"strike\". Knocking down all pins on the second roll is called a \"spar\". Otherwise, the frame is called \"open\".", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=879", + "reward_description": "### Rewards\n\nYou receive points for knocking down pins. The exact score depends on whether you manage a \"strike\", \"spare\" or \"open\"\nframe. Moreover, the points you score for one frame may depend on following frames.\nYou can score up to 300 points in one game (if you manage to do 12 strikes).\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=879).\n" + }, + "boxing": { + "env_description": "You fight an opponent in a boxing ring. You score points for hitting the opponent. If you score 100 points, your opponent is knocked out.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=882", + "reward_description": "### Rewards\n\nYou score points by landing punches.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=882).\n" + }, + "breakout": { + "env_description": "Another famous Atari game. The dynamics are similar to pong: You move a paddle and hit the ball in a brick wall at the top of the screen. Your goal is to destroy the brick wall. You can try to break through the wall and let the ball wreak havoc on the other side, all on its own! You have five lives.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=889", + "reward_description": "### Rewards\n\nYou score points by destroying bricks in the wall. The reward for destroying a brick depends on the color of the brick.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=889).\n" + }, + "carnival": { + "env_description": "This is a \"shoot 'em up\" game. Targets move horizontally across the screen and you must shoot them. You are in control of a gun that can be moved horizontally. The supply of ammunition is limited and chickens may steal some bullets from you if you don't hit them in time.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=908", + "reward_description": "### Rewards\n\nYou score points by destroying targets. Points (or bullets) may be subtracted if you hit the target when it shows a minus sign.\nYou will score extra points if it shows a plus sign!\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=908).\n" + }, + "centipede": { + "env_description": "You are an elf and must use your magic wands to fend off spiders, fleas and centipedes. Your goal is to protect mushrooms in an enchanted forest. If you are bitten by a spider, flea or centipede, you will be temporally paralyzed and you will lose a magic wand. The game ends once you have lost all wands. You may receive additional wands after scoring a sufficient number of points.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=911", + "reward_description": "### Rewards\n\nYou score points by hitting centipedes, scorpions, fleas and spiders. Additional points are awarded after every round\n(i.e. after you have lost a wand) for mushrooms that were not destroyed.\nDetailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=911).\n" + }, + "chopper_command": { + "env_description": "You control a helicopter and must protect truck convoys. To that end, you need to shoot down enemy aircraft.A mini-map is displayed at the bottom of the screen.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=921", + "reward_description": "### Rewards\n\nYou score points by destroying planes and other helicopters. You score extra points at the end of every wave, depending on the number\nof trucks that have survived.\nDetailed documentation can be found on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareID=921).\n" + }, + "crazy_climber": { + "env_description": "You are a climber trying to reach the top of four buildings, while avoiding obstacles like closing windows and falling objects. When you receive damage (windows closing or objects) you will fall and lose one life; you have a total of 5 lives before the end games. At the top of each building, there's a helicopter which you need to catch to get to the next building. The goal is to climb as fast as possible while receiving the least amount of damage.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=113", + "reward_description": "### Rewards\n\nA table of scores awarded for completing each row of a building is provided on [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=113).\n" + }, + "defender": { + "env_description": "Aliens attack the earth. You control a spaceship and must defend humanity by destroying alien ships and rescuing humanoids.You have three lives and three smart bombs. You lose a live when you are shot down by an alien spaceship.Points are scored by destroying enemies and retrieving humans that are being abducted. You have an unlimited number of laser missiles.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=128", + "reward_description": "### Rewards\n\nYou receive points for destroying enemies, rescuing abducted humans and keeping humans alive.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=128).\n" + }, + "demon_attack": { + "env_description": "You are facing waves of demons in the ice planet of Krybor. Points are accumulated by destroying demons. You begin with 3 reserve bunkers, and can increase its number (up to 6) by avoiding enemy attacks. Each attack wave you survive without any hits, grants you a new bunker. Every time an enemy hits you, a bunker is destroyed. When the last bunker falls, the next enemy hit will destroy you and the game ends.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=135", + "reward_description": "### Rewards\n\nEach enemy you slay gives you points. The amount of points depends on the type of demon and which\nwave you are in. A detailed table of scores is provided on [the AtariAge\npage](https://atariage.com/manual_html_page.php?SoftwareLabelID=135).\n" + }, + "double_dunk": { + "env_description": "You are playing a 2v2 game of basketball. At the start of each possession, you select between a set of different plays and then execute them to either score or prevent your rivals from scoring.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=153", + "reward_description": "### Rewards\n\nScores follow the rules of basketball. You can get either 3 points, 2 points foul line) depending\nfrom where you shoot. After a defensive foul, a successful shot from the foul line gives you 1\npoint.\n" + }, + "elevator_action": { + "env_description": "You are a secret agent that must retrieve some secret documents and reach the ground level of a building by going down an elevator/stairs. Once you reach the ground level, you are picked up and taken to the next level. You are equipped with a gun to defend yourself against enemy agents waiting for you in each floor. You gather points by shooting down enemy agents and visiting apartments marked with a red door, which contain the secret documents.This is an unreleased prototype based on the arcade game.", + "atariage_url": "https://atariage.com/manual_thumbs.php?SoftwareLabelID=1131", + "reward_description": "### Rewards\n\nYou start with 4 lives and are awarded 100 points for each enemy shot, and 500 points for each\nsecret document collected (visiting a red door). Each time you get shot you lose one life and the\ngame ends when losing all lives.\n" + }, + "enduro": { + "env_description": "You are a racer in the National Enduro, a long-distance endurance race. You must overtake a certain amount of cars each day to stay on the race. The first day you need to pass 200 cars, and 300 foreach following day. The game ends if you do not meet your overtake quota for the day.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=163", + "reward_description": "### Rewards\n\nYou get 1 point for each vehicle you overtake.\n" + }, + "fishing_derby": { + "env_description": "your objective is to catch more sunfish than your opponent.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=182", + "reward_description": "### Rewards\n\nThe exact reward dynamics depend on the environment and are usually documented in the game's manual. You can\nfind these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=182).\n\nAtari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).\n" + }, + "freeway": { + "env_description": "your objective is to guide your chicken across lane after lane of busy rush hour traffic. You receive a point for every chicken that makes it to the top of the screen after crossing all the lanes of traffic.", + "atariage_url": "https://atariage.com/manual_thumbs.php?SoftwareLabelID=192", + "reward_description": "### Rewards\n\nThe exact reward dynamics depend on the environment and are usually documented in the game's manual. You can\nfind these manuals on [AtariAge](https://atariage.com/manual_thumbs.php?SoftwareLabelID=192).\n\nAtari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).\n" + }, + "frostbite": { + "env_description": "In Frostbite, the player controls \"Frostbite Bailey\" who hops back and forth across across an Arctic river, changing the color of the ice blocks from white to blue. Each time he does so, a block is added to his igloo.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=199", + "reward_description": "### Rewards\n\nThe exact reward dynamics depend on the environment and are usually documented in the game's manual. You can\nfind these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=199).\n\nAtari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).\n" + }, + "gopher": { + "env_description": "The player controls a shovel-wielding farmer who protects a crop of three carrots from a gopher.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=218", + "reward_description": "### Rewards\n\nThe exact reward dynamics depend on the environment and are usually documented in the game's manual. You can\nfind these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=218).\n\nAtari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).\n" + }, + "gravitar": { + "env_description": "The player controls a small blue spacecraft. The game starts in a fictional solar system with several planets to explore. If the player moves his ship into a planet, he will be taken to a side-view landscape.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=223", + "reward_description": "### Rewards\n\nThe exact reward dynamics depend on the environment and are usually documented in the game's manual. You can\nfind these manuals on [AtariAge](https://atariage.com/manual_html_page.php?SoftwareLabelID=223).\n\nAtari environments are simulated via the Arcade Learning Environment (ALE) [[1]](#1).\n" + }, + "hero": { + "env_description": "You need to rescue miners that are stuck in a mine shaft. You have access to various tools: A propeller backpack that allows you to fly wherever you want, sticks of dynamite that can be used to blast through walls, a laser beam to kill vermin, and a raft to float across stretches of lava.You have a limited amount of power. Once you run out, you lose a live.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=228", + "reward_description": "### Rewards\n\nYou score points for shooting critters, rescuing miners, and dynamiting walls.\nExtra points are rewarded for any power remaining after rescuing a miner.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=228).\n" + }, + "ice_hockey": { + "env_description": "Your goal is to score as many points as possible in a standard game of Ice Hockey over a 3-minute time period. The ball is usually called \"the puck\".There are 32 shot angles ranging from the extreme left to the extreme right. The angles can only aim towards the opponent's goal.Just as in real hockey, you can pass the puck by shooting it off the sides of the rink. This can be really key when you're in position to score a goal.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=241", + "reward_description": "### Rewards\n\nYou score points by shooting the puck into your opponent's goal. Your opponent scores in the same manner.\nThere are no limits to how many points you can get per game, other than the time limit of 3-minute games.\nFor a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=241).\n" + }, + "jamesbond": { + "env_description": "Your mission is to control Mr. Bond's specially designed multipurpose craft to complete a variety of missions.The craft moves forward with a right motion and slightly back with a left motion.An up or down motion causes the craft to jump or dive.You can also fire by either lobbing a bomb to the bottom of the screen or firing a fixed angle shot to the top of the screen.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=250", + "reward_description": "### Rewards\n\nThe game ends when you complete the last mission or when you lose the last craft. In either case, you'll receive your final score.\nThere will be a rating based on your score. The highest rating in NOVICE is 006. The highest rating in AGENT is 007.\nFor a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=250).\n" + }, + "journey_escape": { + "env_description": "You must lead all 5 members of JOURNEY through waves of pesky characters and backstage obstacles to the Scarab Escape Vehicle before time runs out.You must also protect $50,000 in concert cash from grasping groupies, photographers, and promoters.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=252", + "reward_description": "### Rewards\n\nAt the start of the game, you will have $50,000 and 60 units of time.\nYour end game score with be dependent on how much time you have remaining and who you encounter along the way.\nFor a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=252).\n" + }, + "kangaroo": { + "env_description": "The object of the game is to score as many points as you can while controlling Mother Kangaroo to rescue her precious baby. You start the game with three lives.During this rescue mission, Mother Kangaroo encounters many obstacles. You need to help her climb ladders, pick bonus fruit, and throw punches at monkeys.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=923", + "reward_description": "### Rewards\n\nYour score will be shown at the top right corner of the game.\nYour end game score with be dependent on how much time you have remaining and who you encounter along the way.\nFor a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=923).\n" + }, + "krull": { + "env_description": "Your mission is to find and enter the Beast's Black Fortress, rescue Princess Lyssa, and destroy the Beast.The task is not an easy one, for the location of the Black Fortress changes with each sunrise on Krull.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=267", + "reward_description": "### Rewards\n\nYou will receive various scores for each monster you kill.\nYou can play the game until you have lost all your lives.\nFor a more detailed documentation, consult [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=267).\n" + }, + "kung_fu_master": { + "env_description": "You are a Kung-Fu Master fighting your way through the Evil Wizard's temple. Your goal is to rescue Princess Victoria, defeating various enemies along the way.", + "atariage_url": "https://atariage.com/manual_thumbs.php?SoftwareLabelID=268", + "reward_description": "" + }, + "montezuma_revenge": { + "env_description": "Your goal is to acquire Montezuma's treasure by making your way through a maze of chambers within the emperor's fortress. You must avoid deadly creatures while collecting valuables and tools which can help you escape with the treasure.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=310", + "reward_description": "" + }, + "ms_pacman": { + "env_description": "Your goal is to collect all of the pellets on the screen while avoiding the ghosts.", + "atariage_url": "https://atariage.com/manual_page.php?SoftwareLabelID=924", + "reward_description": "" + }, + "name_this_game": { + "env_description": "Your goal is to defend the treasure that you have discovered. You must fight off a shark and an octopus while keeping an eye on your oxygen supply.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=323", + "reward_description": "" + }, + "phoenix": { + "env_description": "Your goal is to reach and shoot the alien pilot. On your way there, you must eliminate waves of war birds while avoiding their bombs.", + "atariage_url": "https://atariage.com/manual_thumbs.php?SoftwareLabelID=355", + "reward_description": "" + }, + "pitfall": { + "env_description": "You control Pitfall Harry and are tasked with collecting all the treasures in a jungle within 20 minutes. You have three lives. The game is over if you collect all the treasures or if you die or if the time runs out.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=360", + "reward_description": "### Rewards\n\nYou get score points for collecting treasure, you lose points through some misfortunes like falling down a hole.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=360).\n" + }, + "pong": { + "env_description": "You control the right paddle, you compete against the left paddle controlled by the computer. You each try to keep deflecting the ball away from your goal and into your opponent's goal.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=587", + "reward_description": "### Rewards\n\nYou get score points for getting the ball to pass the opponent's paddle. You lose points if the ball passes your paddle.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=587).\n" + }, + "pooyan": { + "env_description": "You are a mother pig protecting her piglets (Pooyans) from wolves. In the first scene, you can move up and down a rope. Try to shoot the worker's balloons, while guarding yourself from attacks. If the wolves reach the ground safely they will get behind and try to eat you. In the second scene, the wolves try to float up. You have to try and stop them using arrows and bait. You die if a wolf eats you, or a stone or rock hits you.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=372", + "reward_description": "### Rewards\n\nIf you hit a balloon, wolf or stone with an arrow you score points.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=372).\n" + }, + "private_eye": { + "env_description": "You control the French Private Eye Pierre Touche. Navigate the city streets, parks, secret passages, dead-ends and one-ways in search of the ringleader, Henri Le Fiend and his gang. You also need to find evidence and stolen goods that are scattered about. There are five cases, complete each case before its statute of limitations expires.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=376", + "reward_description": "### Rewards\n\nYou score points for completing your tasks like gathering evidence, nabbing questionable characters or closing cases etc. You lose points if you get hit or if your auto is on a pothole.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=376).\n" + }, + "qbert": { + "env_description": "You are Q*bert. Your goal is to change the color of all the cubes on the pyramid to the pyramid's 'destination' color. To do this, you must hop on each cube on the pyramid one at a time while avoiding nasty creatures that lurk there.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareID=1224", + "reward_description": "### Rewards\n\nYou score points for changing color of the cubes to their destination colors or by defeating enemies. You also gain points for completing a level.\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SystemID=2600&SoftwareID=1224&itemTypeID=HTMLMANUAL).\n" + }, + "riverraid": { + "env_description": "You control a jet that flies over a river: you can move it sideways and fire missiles to destroy enemy objects. Each time an enemy object is destroyed you score points (i.e. rewards).You lose a jet when you run out of fuel: fly over a fuel depot when you begin to run low.You lose a jet even when it collides with the river bank or one of the enemy objects (except fuel depots).The game begins with a squadron of three jets in reserve and you're given an additional jet (up to 9) for each 10,000 points you score.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=409", + "reward_description": "### Rewards\n\nScore points are your only reward. You get score points each time you destroy an enemy object:\n\n| Enemy Object | Score Points |\n|--------------|--------------|\n| Tanker | 30 |\n| Helicopter | 60 |\n| Fuel Depot | 80 |\n| Jet | 100 |\n| Bridge | 500 |\n\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=409).\n" + }, + "road_runner": { + "env_description": "You control the Road Runner(TM) in a race; you can control the direction to run in and times to jumps.The goal is to outrun Wile E. Coyote(TM) while avoiding the hazards of the desert.The game begins with three lives. You lose a life when the coyote catches you, picks you up in a rocket, or shoots you with a cannon. You also lose a life when a truck hits you, you hit a land mine, you fall off a cliff,or you get hit by a falling rock.You score points (i.e. rewards) by eating seeds along the road, eating steel shot, and destroying the coyote.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=412", + "reward_description": "### Rewards\n\nScore points are your only reward. You get score points each time you:\n\n| actions | points |\n|-------------------------------------------------------|--------|\n| eat a pile of birdseed | 100 |\n| eat steel shot | 100 |\n| get the coyote hit by a mine (cannonball, rock, etc.) | 200 |\n| get the coyote hit by a truck | 1000 |\n\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=412).\n" + }, + "robotank": { + "env_description": "You control your Robot Tanks to destroy enemies and avoid enemy fire.Game ends when all of your Robot Tanks are destroyed or all 12 enemy squadrons are destroyed.The game begins with one active Robot Tank and three reserves.Your Robot Tank may get lost when it is hit by enemy rocket fire - your video scrambles with static interference when this happens - or just become damaged - sensors report the damage by flashing on your control panel (look at V/C/R/T squares).You earn one bonus Robot Tank for every enemy squadron destroyed. The maximum number of bonus Robot Tanks allowed at any one time is 12.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=416", + "reward_description": "### Rewards\n\nThe number of enemies destroyed is the only reward.\n\nA small tank appears at the top of your screen for each enemy\n you destroy. A square with the number 12 appears each time a squadron of twelve enemies are\n destroyed.\n\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=416).\n" + }, + "seaquest": { + "env_description": "You control a sub able to move in all directions and fire torpedoes.The goal is to retrieve as many divers as you can, while dodging and blasting enemy subs and killer sharks; points will be awarded accordingly.The game begins with one sub and three waiting on the horizon. Each time you increase your score by 10,000 points, an extra sub will be delivered to yourbase. You can only have six reserve subs on the screen at one time.Your sub will explode if it collides with anything except your own divers.The sub has a limited amount of oxygen that decreases at a constant rate during the game. When the oxygen tank is almost empty, you need to surface and if you don't do it in time, your sub will blow up and you'll lose one diver. Each time you're forced to surface, with less than six divers, you lose one diver as well.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=424", + "reward_description": "### Rewards\n\nScore points are your only reward.\n\nBlasting enemy sub and killer shark is worth\n20 points. Every time you surface with six divers, the value of enemy subs\nand killer sharks increases by 10, up to a maximum of 90 points each.\n\nRescued divers start at 50 points each. Then, their point value increases by 50, every\ntime you surface, up to a maximum of 1000 points each.\n\nYou'll be further rewarded with bonus points for all the oxygen you have remaining the\nmoment you surface. The more oxygen you have left, the more bonus points\nyou're given.\n\nFor a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=424).\n" + }, + "skiing": { + "env_description": "You control a skier who can move sideways.The goal is to run through all gates (between the poles) in the fastest time.You are penalized five seconds for each gate you miss.If you hit a gate or a tree, your skier will jump back up and keep going.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=434", + "reward_description": "### Rewards\n\nSeconds are your only rewards - negative rewards and penalties (e.g. missing a gate) are assigned as additional seconds.\n\nFor a more detailed documentation, see [the AtariAge page [SLALOM RACING section]](https://atariage.com/manual_html_page.php?SoftwareLabelID=434).\n" + }, + "solaris": { + "env_description": "You control a spaceship. Blast enemies before they can blast you. You can warp to different sectors. You have to defend Federation planets, and destroy Zylon forces. Keep track of your fuel, if you run out you lose a life. Warp to a Federation planet to refuel. The game ends if all your ships are destroyed or if you reach the Solaris planet.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=450", + "reward_description": "### Rewards\n\nYou gain points for destroying enemies, rescuing cadets, making it through a corridor, destroying enemy planets etc. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=450).\n" + }, + "space_invaders": { + "env_description": "Your objective is to destroy the space invaders by shooting your laser cannon at them before they reach the Earth. The game ends when all your lives are lost after taking enemy fire, or when they reach the earth.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=460", + "reward_description": "### Rewards\n\nYou gain points for destroying space invaders. The invaders in the back rows are worth more points. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=460).\n" + }, + "star_gunner": { + "env_description": "Stop the alien invasion by shooting down alien saucers and creatures while avoiding bombs.", + "atariage_url": "http://www.atarimania.com/game-atari-2600-vcs-stargunner_16921.html", + "reward_description": "### Rewards\n\nYou score points for destroying enemies. You get bonus points for clearing a wave and a level. For a more detailed documentation, see [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-stargunner_16921.html).\n" + }, + "tennis": { + "env_description": "You control the orange player playing against a computer-controlled blue player. The game follows the rules of tennis.The first player to win at least 6 games with a margin of at least two games wins the match. If the score is tied at 6-6, the first player to go 2 games up wins the match.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=555", + "reward_description": "### Rewards\n\nThe scoring is as per the sport of tennis, played till one set. For a more detailed documentation, see [the AtariAge page](https://atariage.com/manual_html_page.php?SoftwareLabelID=555).\n" + }, + "time_pilot": { + "env_description": "You control an aircraft. Use it to destroy your enemies. As you progress in the game, you encounter enemies with technology that is increasingly from the future.", + "atariage_url": "http://www.atarimania.com/game-atari-2600-vcs-time-pilot_8038.html", + "reward_description": "### Rewards\n\nYou score points for destroying enemies, gaining more points for difficult enemies. For a more detailed documentation, see [the Atari Mania page](http://www.atarimania.com/game-atari-2600-vcs-time-pilot_8038.html).\n" + }, + "tutankham": { + "env_description": "Your goal is to rack up points by finding treasures in the mazes of the tomb while eliminating its guardians.", + "atariage_url": "https://atariage.com/manual_thumbs.php?SoftwareLabelID=572", + "reward_description": "" + }, + "up_n_down": { + "env_description": "Your goal is to steer your baja bugger to collect prizes and eliminate opponents.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=574", + "reward_description": "" + }, + "venture": { + "env_description": "Your goal is to capture the treasure in every chamber of the dungeon while eliminating the monsters.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=576", + "reward_description": "" + }, + "video_pinball": { + "env_description": "Your goal is to keep the ball in play as long as possible and to score as many points as possible.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=588", + "reward_description": "" + }, + "wizard_of_wor": { + "env_description": "Your goal is to beat the Wizard using your laser and radar scanner.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=598", + "reward_description": "" + }, + "yars_revenge": { + "env_description": "The objective is to break a path through the shield and destroy the Qotile with a blast from the Zorlon Cannon.", + "atariage_url": "https://atariage.com/manual_page.php?SoftwareLabelID=603¤tPage=1&maxPages=12", + "reward_description": "" + }, + "zaxxon": { + "env_description": "Your goal is to stop the evil robot Zaxxon and its armies from enslaving the galaxy by piloting your fighter and shooting enemies.", + "atariage_url": "https://atariage.com/manual_html_page.php?SoftwareLabelID=606", + "reward_description": "" + } +} diff --git a/docs/scripts/gen_atari_table.py b/docs/scripts/gen_atari_table.py index 20b4e2822..2bc18e2bc 100644 --- a/docs/scripts/gen_atari_table.py +++ b/docs/scripts/gen_atari_table.py @@ -1,109 +1,226 @@ +import itertools +import json + import tabulate +from ale_py.roms import utils as rom_utils +from shimmy.utils.envs_configs import LEGACY_ATARI_GAMES as ALL_ATARI_GAMES from tqdm import tqdm import gymnasium -def shortened_repr(lst): - assert all(isinstance(item, int) for item in lst) - assert len(set(lst)) == len(lst) - lst = sorted(lst) +# while part of the ale-py list of roms this is not part of the roms downloaded by AutoROM +exclude_envs = ["TicTacToe3D", "Videochess", "Videocube"] - if lst[-1] - lst[0] == len(lst) - 1 and len(lst) > 3: - return f"`[{lst[0]}, ..., {lst[-1]}]`" - elif len(lst) > 3 and lst[-2] - lst[0] == len(lst) - 2: - return f"`[{lst[0]}, ..., {lst[-2]}, {lst[-1]}]`" - return f"`{str(lst)}`" +# # Generate the list of all atari games on atari.md +for rom_id in sorted(ALL_ATARI_GAMES): + if rom_utils.rom_id_to_name(rom_id) not in exclude_envs: + print(f"atari/{rom_id}") -def to_gymnasium_spelling(game): - parts = game.split("_") - return "".join([part.capitalize() for part in parts]) +def generate_value_ranges(values): + for a, b in itertools.groupby(enumerate(values), lambda pair: pair[1] - pair[0]): + b = list(b) + yield b[0][1], b[-1][1] -atari_envs = [ - "adventure", - "air_raid", - "alien", - "amidar", - "assault", - "asterix", - "asteroids", - "atlantis", - "bank_heist", - "battle_zone", - "beam_rider", - "berzerk", - "bowling", - "boxing", - "breakout", - "carnival", - "centipede", - "chopper_command", - "crazy_climber", - "defender", - "demon_attack", - "double_dunk", - "elevator_action", - "enduro", - "fishing_derby", - "freeway", - "frostbite", - "gopher", - "gravitar", - "hero", - "ice_hockey", - "jamesbond", - "journey_escape", - "kangaroo", - "krull", - "kung_fu_master", - "montezuma_revenge", - "ms_pacman", - "name_this_game", - "phoenix", - "pitfall", - "pong", - "pooyan", - "private_eye", - "qbert", - "riverraid", - "road_runner", - "robotank", - "seaquest", - "skiing", - "solaris", - "space_invaders", - "star_gunner", - "tennis", - "time_pilot", - "tutankham", - "up_n_down", - "venture", - "video_pinball", - "wizard_of_wor", - "yars_revenge", - "zaxxon", +def shortened_repr(values): + output = [] + for low, high in generate_value_ranges(values): + if high - low < 5: + output.append(", ".join(map(str, range(low, high + 1)))) + else: + output.append(f"{low}, ..., {high}") + return "[" + ", ".join(output) + "]" + + +# # Test examples +# print(shortened_repr([0])) +# print(shortened_repr([1, 2, 3])) +# print(shortened_repr([0, 1, 2, 3])) +# print(shortened_repr([0, 4, 8, 12, 16, 20, 24, 28])) +# print(shortened_repr(list(range(32)) + [128])) + + +# # Generate difficult levels table on atari.md +headers = [ + "Environment", + "Possible Modes", + "Default Mode", + "Possible Difficulties", + "Default Difficulty", ] - - -header = ["Environment", "Valid Modes", "Valid Difficulties", "Default Mode"] rows = [] -for game in tqdm(atari_envs): - env = gymnasium.make(f"ALE/{to_gymnasium_spelling(game)}-v5") - valid_modes = env.unwrapped.ale.getAvailableModes() - valid_difficulties = env.unwrapped.ale.getAvailableDifficulties() - difficulty = env.unwrapped.ale.cloneState().getDifficulty() - assert difficulty == 0, difficulty +for rom_id in tqdm(ALL_ATARI_GAMES): + env_name = rom_utils.rom_id_to_name(rom_id) + if env_name in exclude_envs: + continue + + env = gymnasium.make(f"ALE/{env_name}-v5") + + available_difficulties = env.ale.getAvailableDifficulties() + default_difficulty = env.ale.cloneState().getDifficulty() + available_modes = env.ale.getAvailableModes() + default_mode = env.ale.cloneState().getCurrentMode() + rows.append( [ - to_gymnasium_spelling(game), - shortened_repr(valid_modes), - shortened_repr(valid_difficulties), - f"`{env.unwrapped.ale.cloneState().getCurrentMode()}`", + env_name, + shortened_repr(available_modes), + default_mode, + shortened_repr(available_difficulties), + default_difficulty, ] ) + env.close() +print(tabulate.tabulate(rows, headers=headers, tablefmt="github")) -print(tabulate.tabulate(rows, headers=header, tablefmt="github")) +# Generate each pages results +with open("atari-docs.json") as file: + atari_data = json.load(file) + +for rom_id in tqdm(ALL_ATARI_GAMES): + env_name = rom_utils.rom_id_to_name(rom_id) + if env_name in exclude_envs: + continue + + env = gymnasium.make(f"ALE/{env_name}-v5") + env_data = atari_data[rom_id] + + env_description = env_data["env_description"] + if env_data["atariage_url"]: + env_url = f""" +For a more detailed documentation, see [the AtariAge page]({env_data['atariage_url']}) +""" + else: + env_url = "" + reward_description = env_data["reward_description"] + + default_action_table = tabulate.tabulate( + [ + [f"`{num}`", f"`{meaning}`"] + for num, meaning in enumerate(env.get_action_meanings()) + ], + headers=["Value", "Meaning"], + tablefmt="github", + ) + if env.action_space.n == 18: + action_description = f"""{env_name} has the action space `{env.action_space}` with the table below lists the meaning of each action's meanings. +As {env_name} uses the full set of actions then specifying `full_action_space=True` will not modify the action space of the environment if passed to `gymnasium.make`.""" + else: + action_description = f"""{env_name} has the action space `{env.action_space}` with the table below lists the meaning of each action's meanings. +As {env_name} uses a reduced set of actions for `v0`, `v4` and `v5` versions of the environment. +To enable all 18 possible actions that can be performed on an Atari 2600, specify `full_action_space=True` during +initialization or by passing `full_action_space=True` to `gymnasium.make`.""" + + # Environment variants + env_specs = sorted( + [ + env_spec + for env_spec in gymnasium.registry.values() + if env_name in env_spec.name + ], + key=lambda env_spec: f"{env_spec.version}{env_spec.name}", + ) + + env_variant_headers = [ + "Env-id", + "obs_type=", + "frameskip=", + "repeat_action_probability=", + ] + env_variant_rows = [ + [ + env_spec.id, + f'`"{env_spec.kwargs["obs_type"]}"`', + f'`{env_spec.kwargs["frameskip"]}`', + f'`{env_spec.kwargs["repeat_action_probability"]}`', + ] + for env_spec in env_specs + ] + env_variant_table = tabulate.tabulate( + env_variant_rows, headers=env_variant_headers, tablefmt="github" + ) + + # difficult and mode description + + difficulty_mode_header = [ + "Available Modes", + "Default Mode", + "Available Difficulties", + "Default Difficulty", + ] + difficulty_mode_row = [ + [ + f"`{shortened_repr(env.ale.getAvailableModes())}`", + f"`{env.ale.cloneState().getCurrentMode()}`", + f"`{shortened_repr(env.ale.getAvailableDifficulties())}`", + f"`{env.ale.cloneState().getDifficulty()}`", + ] + ] + difficulty_mode_table = tabulate.tabulate( + difficulty_mode_row, headers=difficulty_mode_header, tablefmt="github" + ) + + env.close() + + TEMPLATE = f"""--- +title: {env_name} +--- + +# {env_name} + +```{{figure}} ../../_static/videos/atari/{rom_id}.gif +:width: 120px +:name: {env_name} +``` + +This environment is part of the Atari environments. Please read that page first for general information. + +## Description + +{env_description} +{env_url} +## Actions + +{action_description} + +{default_action_table} + +## Observations + +Atari environment have two possible observation types, the observation space is listed below. +See variants section for the type of observation used by each environment id. + +- `obs_type="rgb" -> observation_space=Box(0, 255, (210, 160, 3), np.uint8)` +- `obs_type="ram" -> observation_space=Box(0, 255, (128,), np.uint8)` + +Additionally, `obs_type="grayscale"` cause the environment return a grayscale version of the rgb array for observations with the observation space being `Box(0, 255, (210, 160), np.uint8)` +{reward_description} +## Variants + +{env_name} has the following variants of the environment id which have the following differences in observation, +the number of frame-skips and the repeat action probability. + +{env_variant_table} + +## Difficulty and modes + +It is possible to specify various flavors of the environment via the keyword arguments `difficulty` and `mode`. +A flavor is a combination of a game mode and a difficulty setting. The table below lists the possible difficulty and mode values +along with the default values. + +{difficulty_mode_table} + +## Version History + +A thorough discussion of the intricate differences between the versions and configurations can be found in the general article on Atari environments. + +* v5: Stickiness was added back and stochastic frameskipping was removed. The environments are now in the "ALE" namespace. +* v4: Stickiness of actions was removed +* v0: Initial versions release +""" + with open(f"../environments/atari/{rom_id}.md", "w") as file: + file.write(TEMPLATE)