diff --git a/.buildinfo b/.buildinfo index c40858d5c..2f9352bde 100644 --- a/.buildinfo +++ b/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 309340cd494e6110e8779bfdf20952bc +config: c29ef785859083c891d7d61febe82c92 tags: d77d1c0d9ca2f4c8421862c7c5a0d620 diff --git a/404.html b/404.html index afa3c4407..0b7a7bc42 100644 --- a/404.html +++ b/404.html @@ -15,7 +15,11 @@ 404 - Page Not Found - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -465,11 +477,11 @@ const menuBtn = document.querySelector(".farama-header-menu__btn"); const menuContainer = document.querySelector(".farama-header-menu-container"); if (document.querySelector(".farama-header-menu").classList.contains("active")) { - menuBtn.ariaExpanded = "false"; - menuContainer.ariaHidden = "true"; + menuBtn.setAttribute("aria-expanded", "false"); + menuContainer.setAttribute("aria-hidden", "true"); } else { - menuBtn.ariaExpanded = "true"; - menuContainer.ariaHidden = "false"; + menuBtn.setAttribute("aria-expanded", "true"); + menuContainer.setAttribute("aria-hidden", "false"); } document.querySelector(".farama-header-menu").classList.toggle("active"); } @@ -639,7 +651,7 @@ githubUser: 'Farama-Foundation', githubRepo: 'Gymnasium', }; - fetch('/_static/versioning/versioning_menu.html').then(response => { + fetch('/main/_static/versioning/versioning_menu.html').then(response => { if (response.status === 200) { response.text().then(text => { const container = document.createElement("div"); diff --git a/README/index.html b/README/index.html index 0560feb15..6d2e32338 100644 --- a/README/index.html +++ b/README/index.html @@ -15,7 +15,11 @@ Gymnasium-docs - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -545,11 +557,11 @@ const menuBtn = document.querySelector(".farama-header-menu__btn"); const menuContainer = document.querySelector(".farama-header-menu-container"); if (document.querySelector(".farama-header-menu").classList.contains("active")) { - menuBtn.ariaExpanded = "false"; - menuContainer.ariaHidden = "true"; + menuBtn.setAttribute("aria-expanded", "false"); + menuContainer.setAttribute("aria-hidden", "true"); } else { - menuBtn.ariaExpanded = "true"; - menuContainer.ariaHidden = "false"; + menuBtn.setAttribute("aria-expanded", "true"); + menuContainer.setAttribute("aria-hidden", "false"); } document.querySelector(".farama-header-menu").classList.toggle("active"); } @@ -719,7 +731,7 @@ githubUser: 'Farama-Foundation', githubRepo: 'Gymnasium', }; - fetch('/_static/versioning/versioning_menu.html').then(response => { + fetch('/main/_static/versioning/versioning_menu.html').then(response => { if (response.status === 200) { response.text().then(text => { const container = document.createElement("div"); diff --git a/_downloads/3831a62128c6d96d80d039f936893259/environment_creation.ipynb b/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb similarity index 92% rename from _downloads/3831a62128c6d96d80d039f936893259/environment_creation.ipynb rename to _downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb index a9bc7416c..62d5c387d 100644 --- a/_downloads/3831a62128c6d96d80d039f936893259/environment_creation.ipynb +++ b/_downloads/0f28446f9f426c9833f40d61857a6f21/environment_creation.ipynb @@ -15,7 +15,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "\n# Make your own custom environment\n\nThis documentation overviews creating new environments and relevant\nuseful wrappers, utilities and tests included in Gymnasium designed for\nthe creation of new environments. You can clone gym-examples to play\nwith the code that is presented here. We recommend that you use a virtual environment:\n\n.. code:: console\n\n git clone https://github.com/Farama-Foundation/gym-examples\n cd gym-examples\n python -m venv .env\n source .env/bin/activate\n pip install -e .\n\n## Subclassing gymnasium.Env\n\nBefore learning how to create your own environment you should check out\n[the documentation of Gymnasium\u2019s API](/api/core)_.\n\nWe will be concerned with a subset of gym-examples that looks like this:\n\n.. code:: sh\n\n gym-examples/\n README.md\n setup.py\n gym_examples/\n __init__.py\n envs/\n __init__.py\n grid_world.py\n wrappers/\n __init__.py\n relative_position.py\n reacher_weighted_reward.py\n discrete_action.py\n clip_reward.py\n\nTo illustrate the process of subclassing ``gymnasium.Env``, we will\nimplement a very simplistic game, called ``GridWorldEnv``. We will write\nthe code for our custom environment in\n``gym-examples/gym_examples/envs/grid_world.py``. The environment\nconsists of a 2-dimensional square grid of fixed size (specified via the\n``size`` parameter during construction). The agent can move vertically\nor horizontally between grid cells in each timestep. The goal of the\nagent is to navigate to a target on the grid that has been placed\nrandomly at the beginning of the episode.\n\n- Observations provide the location of the target and agent.\n- There are 4 actions in our environment, corresponding to the\n movements \u201cright\u201d, \u201cup\u201d, \u201cleft\u201d, and \u201cdown\u201d.\n- A done signal is issued as soon as the agent has navigated to the\n grid cell where the target is located.\n- Rewards are binary and sparse, meaning that the immediate reward is\n always zero, unless the agent has reached the target, then it is 1.\n\nAn episode in this environment (with ``size=5``) might look like this:\n\nwhere the blue dot is the agent and the red square represents the\ntarget.\n\nLet us look at the source code of ``GridWorldEnv`` piece by piece:\n" + "\n# Make your own custom environment\n\nThis documentation overviews creating new environments and relevant\nuseful wrappers, utilities and tests included in Gymnasium designed for\nthe creation of new environments. You can clone gym-examples to play\nwith the code that is presented here. We recommend that you use a virtual environment:\n\n.. code:: console\n\n git clone https://github.com/Farama-Foundation/gym-examples\n cd gym-examples\n python -m venv .env\n source .env/bin/activate\n pip install -e .\n\n## Subclassing gymnasium.Env\n\nBefore learning how to create your own environment you should check out\n[the documentation of Gymnasium\u2019s API](/api/env)_.\n\nWe will be concerned with a subset of gym-examples that looks like this:\n\n.. code:: sh\n\n gym-examples/\n README.md\n setup.py\n gym_examples/\n __init__.py\n envs/\n __init__.py\n grid_world.py\n wrappers/\n __init__.py\n relative_position.py\n reacher_weighted_reward.py\n discrete_action.py\n clip_reward.py\n\nTo illustrate the process of subclassing ``gymnasium.Env``, we will\nimplement a very simplistic game, called ``GridWorldEnv``. We will write\nthe code for our custom environment in\n``gym-examples/gym_examples/envs/grid_world.py``. The environment\nconsists of a 2-dimensional square grid of fixed size (specified via the\n``size`` parameter during construction). The agent can move vertically\nor horizontally between grid cells in each timestep. The goal of the\nagent is to navigate to a target on the grid that has been placed\nrandomly at the beginning of the episode.\n\n- Observations provide the location of the target and agent.\n- There are 4 actions in our environment, corresponding to the\n movements \u201cright\u201d, \u201cup\u201d, \u201cleft\u201d, and \u201cdown\u201d.\n- A done signal is issued as soon as the agent has navigated to the\n grid cell where the target is located.\n- Rewards are binary and sparse, meaning that the immediate reward is\n always zero, unless the agent has reached the target, then it is 1.\n\nAn episode in this environment (with ``size=5``) might look like this:\n\nwhere the blue dot is the agent and the red square represents the\ntarget.\n\nLet us look at the source code of ``GridWorldEnv`` piece by piece:\n" ] }, { @@ -196,7 +196,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.15" + "version": "3.9.16" } }, "nbformat": 4, diff --git a/_downloads/315c4c52fb68082a731b192d944e2ede/tutorials_python.zip b/_downloads/315c4c52fb68082a731b192d944e2ede/tutorials_python.zip new file mode 100644 index 000000000..d859f3bcc Binary files /dev/null and b/_downloads/315c4c52fb68082a731b192d944e2ede/tutorials_python.zip differ diff --git a/_downloads/bee952c8ccdb09b22233577d70a62253/handling_time_limits.ipynb b/_downloads/46c1a6ff1aae3b48b3379fddfeca0650/handling_time_limits.ipynb similarity index 99% rename from _downloads/bee952c8ccdb09b22233577d70a62253/handling_time_limits.ipynb rename to _downloads/46c1a6ff1aae3b48b3379fddfeca0650/handling_time_limits.ipynb index 450213fca..4e6342272 100644 --- a/_downloads/bee952c8ccdb09b22233577d70a62253/handling_time_limits.ipynb +++ b/_downloads/46c1a6ff1aae3b48b3379fddfeca0650/handling_time_limits.ipynb @@ -49,7 +49,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.15" + "version": "3.9.16" } }, "nbformat": 4, diff --git a/_downloads/50e7c09c20b787d0a5bd70c4aeb0a515/vector_envs_tutorial.ipynb b/_downloads/50e7c09c20b787d0a5bd70c4aeb0a515/vector_envs_tutorial.ipynb new file mode 100644 index 000000000..498543f5e --- /dev/null +++ b/_downloads/50e7c09c20b787d0a5bd70c4aeb0a515/vector_envs_tutorial.ipynb @@ -0,0 +1,325 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n# Training A2C with Vector Envs and Domain Randomization\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Introduction\n\nIn this tutorial, you'll learn how to use vectorized environments to train an Advantage Actor-Critic agent.\nWe are going to use A2C, which is the synchronous version of the A3C algorithm [1].\n\nVectorized environments [3] can help to achieve quicker and more robust training by allowing multiple instances\nof the same environment to run in parallel (on multiple CPUs). This can significantly reduce the variance and thus speeds up the training.\n\nWe will implement an Advantage Actor-Critic from scratch to look at how you can feed batched states into your networks to get a vector of actions\n(one action per environment) and calculate the losses for actor and critic on minibatches of transitions.\nEach minibatch contains the transitions of one sampling phase: `n_steps_per_update` steps are executed in `n_envs` environments in parallel\n(multiply the two to get the number of transitions in a minibatch). After each sampling phase, the losses are calculated and one gradient step is executed.\nTo calculate the advantages, we are going to use the Generalized Advantage Estimation (GAE) method [2], which balances the tradeoff\nbetween variance and bias of the advantage estimates.\n\nThe A2C agent class is initialized with the number of features of the input state, the number of actions the agent can take,\nthe learning rates and the number of environments that run in parallel to collect experiences. The actor and critic networks are defined\nand their respective optimizers are initialized. The forward pass of the networks takes in a batched vector of states and returns a tensor of state values\nand a tensor of action logits. The select_action method returns a tuple of the chosen actions, the log-probs of those actions, and the state values for each action.\nIn addition, it also returns the entropy of the policy distribution, which is subtracted from the loss later (with a weighting factor `ent_coef`) to encourage exploration.\n\nThe get_losses function calculates the losses for the actor and critic networks (using GAE), which are then updated using the update_parameters function.\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "------------------------------\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# Author: Till Zemann\n# License: MIT License\n\nfrom __future__ import annotations\n\nimport os\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torch import optim\nfrom tqdm import tqdm\n\nimport gymnasium as gym" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Advantage Actor-Critic (A2C)\n\nThe Actor-Critic combines elements of value-based and policy-based methods. In A2C, the agent has two separate neural networks:\na critic network that estimates the state-value function, and an actor network that outputs logits for a categorical probability distribution over all actions.\nThe critic network is trained to minimize the mean squared error between the predicted state values and the actual returns received by the agent\n(this is equivalent to minimizing the squared advantages, because the advantage of an action is as the difference between the return and the state-value: A(s,a) = Q(s,a) - V(s).\nThe actor network is trained to maximize the expected return by selecting actions that have high expected values according to the critic network.\n\nThe focus of this tutorial will not be on the details of A2C itself. Instead, the tutorial will focus on how to use vectorized environments\nand domain randomization to accelerate the training process for A2C (and other reinforcement learning algorithms).\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "------------------------------\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "class A2C(nn.Module):\n \"\"\"\n (Synchronous) Advantage Actor-Critic agent class\n\n Args:\n n_features: The number of features of the input state.\n n_actions: The number of actions the agent can take.\n device: The device to run the computations on (running on a GPU might be quicker for larger Neural Nets,\n for this code CPU is totally fine).\n critic_lr: The learning rate for the critic network (should usually be larger than the actor_lr).\n actor_lr: The learning rate for the actor network.\n n_envs: The number of environments that run in parallel (on multiple CPUs) to collect experiences.\n \"\"\"\n\n def __init__(\n self,\n n_features: int,\n n_actions: int,\n device: torch.device,\n critic_lr: float,\n actor_lr: float,\n n_envs: int,\n ) -> None:\n \"\"\"Initializes the actor and critic networks and their respective optimizers.\"\"\"\n super().__init__()\n self.device = device\n self.n_envs = n_envs\n\n critic_layers = [\n nn.Linear(n_features, 32),\n nn.ReLU(),\n nn.Linear(32, 32),\n nn.ReLU(),\n nn.Linear(32, 1), # estimate V(s)\n ]\n\n actor_layers = [\n nn.Linear(n_features, 32),\n nn.ReLU(),\n nn.Linear(32, 32),\n nn.ReLU(),\n nn.Linear(\n 32, n_actions\n ), # estimate action logits (will be fed into a softmax later)\n ]\n\n # define actor and critic networks\n self.critic = nn.Sequential(*critic_layers).to(self.device)\n self.actor = nn.Sequential(*actor_layers).to(self.device)\n\n # define optimizers for actor and critic\n self.critic_optim = optim.RMSprop(self.critic.parameters(), lr=critic_lr)\n self.actor_optim = optim.RMSprop(self.actor.parameters(), lr=actor_lr)\n\n def forward(self, x: np.ndarray) -> tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\n Forward pass of the networks.\n\n Args:\n x: A batched vector of states.\n\n Returns:\n state_values: A tensor with the state values, with shape [n_envs,].\n action_logits_vec: A tensor with the action logits, with shape [n_envs, n_actions].\n \"\"\"\n x = torch.Tensor(x).to(self.device)\n state_values = self.critic(x) # shape: [n_envs,]\n action_logits_vec = self.actor(x) # shape: [n_envs, n_actions]\n return (state_values, action_logits_vec)\n\n def select_action(\n self, x: np.ndarray\n ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:\n \"\"\"\n Returns a tuple of the chosen actions and the log-probs of those actions.\n\n Args:\n x: A batched vector of states.\n\n Returns:\n actions: A tensor with the actions, with shape [n_steps_per_update, n_envs].\n action_log_probs: A tensor with the log-probs of the actions, with shape [n_steps_per_update, n_envs].\n state_values: A tensor with the state values, with shape [n_steps_per_update, n_envs].\n \"\"\"\n state_values, action_logits = self.forward(x)\n action_pd = torch.distributions.Categorical(\n logits=action_logits\n ) # implicitly uses softmax\n actions = action_pd.sample()\n action_log_probs = action_pd.log_prob(actions)\n entropy = action_pd.entropy()\n return (actions, action_log_probs, state_values, entropy)\n\n def get_losses(\n self,\n rewards: torch.Tensor,\n action_log_probs: torch.Tensor,\n value_preds: torch.Tensor,\n entropy: torch.Tensor,\n masks: torch.Tensor,\n gamma: float,\n lam: float,\n ent_coef: float,\n device: torch.device,\n ) -> tuple[torch.Tensor, torch.Tensor]:\n \"\"\"\n Computes the loss of a minibatch (transitions collected in one sampling phase) for actor and critic\n using Generalized Advantage Estimation (GAE) to compute the advantages (https://arxiv.org/abs/1506.02438).\n\n Args:\n rewards: A tensor with the rewards for each time step in the episode, with shape [n_steps_per_update, n_envs].\n action_log_probs: A tensor with the log-probs of the actions taken at each time step in the episode, with shape [n_steps_per_update, n_envs].\n value_preds: A tensor with the state value predictions for each time step in the episode, with shape [n_steps_per_update, n_envs].\n masks: A tensor with the masks for each time step in the episode, with shape [n_steps_per_update, n_envs].\n gamma: The discount factor.\n lam: The GAE hyperparameter. (lam=1 corresponds to Monte-Carlo sampling with high variance and no bias,\n and lam=0 corresponds to normal TD-Learning that has a low variance but is biased\n because the estimates are generated by a Neural Net).\n device: The device to run the computations on (e.g. CPU or GPU).\n\n Returns:\n critic_loss: The critic loss for the minibatch.\n actor_loss: The actor loss for the minibatch.\n \"\"\"\n T = len(rewards)\n advantages = torch.zeros(T, self.n_envs, device=device)\n\n # compute the advantages using GAE\n gae = 0.0\n for t in reversed(range(T - 1)):\n td_error = (\n rewards[t] + gamma * masks[t] * value_preds[t + 1] - value_preds[t]\n )\n gae = td_error + gamma * lam * masks[t] * gae\n advantages[t] = gae\n\n # calculate the loss of the minibatch for actor and critic\n critic_loss = advantages.pow(2).mean()\n\n # give a bonus for higher entropy to encourage exploration\n actor_loss = (\n -(advantages.detach() * action_log_probs).mean() - ent_coef * entropy.mean()\n )\n return (critic_loss, actor_loss)\n\n def update_parameters(\n self, critic_loss: torch.Tensor, actor_loss: torch.Tensor\n ) -> None:\n \"\"\"\n Updates the parameters of the actor and critic networks.\n\n Args:\n critic_loss: The critic loss.\n actor_loss: The actor loss.\n \"\"\"\n self.critic_optim.zero_grad()\n critic_loss.backward()\n self.critic_optim.step()\n\n self.actor_optim.zero_grad()\n actor_loss.backward()\n self.actor_optim.step()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using Vectorized Environments\n\nWhen you calculate the losses for the two Neural Networks over only one epoch, it might have a high variance. With vectorized environments,\nwe can play with `n_envs` in parallel and thus get up to a linear speedup (meaning that in theory, we collect samples `n_envs` times quicker)\nthat we can use to calculate the loss for the current policy and critic network. When we are using more samples to calculate the loss,\nit will have a lower variance and theirfore leads to quicker learning.\n\nA2C is a synchronous method, meaning that the parameter updates to Networks take place deterministically (after each sampling phase),\nbut we can still make use of asynchronous vector envs to spawn multiple processes for parallel environment execution.\n\nThe simplest way to create vector environments is by calling `gym.vector.make`, which creates multiple instances of the same environment:\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "envs = gym.vector.make(\"LunarLander-v2\", num_envs=3, max_episode_steps=600)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Domain Randomization\n\nIf we want to randomize the environment for training to get more robust agents (that can deal with different parameterizations of an environment\nand theirfore might have a higher degree of generalization), we can set the desired parameters manually or use a pseudo-random number generator to generate them.\n\nManually setting up 3 parallel 'LunarLander-v2' envs with different parameters:\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "envs = gym.vector.AsyncVectorEnv(\n [\n lambda: gym.make(\n \"LunarLander-v2\",\n gravity=-10.0,\n enable_wind=True,\n wind_power=15.0,\n turbulence_power=1.5,\n max_episode_steps=600,\n ),\n lambda: gym.make(\n \"LunarLander-v2\",\n gravity=-9.8,\n enable_wind=True,\n wind_power=10.0,\n turbulence_power=1.3,\n max_episode_steps=600,\n ),\n lambda: gym.make(\n \"LunarLander-v2\", gravity=-7.0, enable_wind=False, max_episode_steps=600\n ),\n ]\n)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "------------------------------\n\nRandomly generating the parameters for 3 parallel 'LunarLander-v2' envs, using `np.clip` to stay in the recommended parameter space:\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "envs = gym.vector.AsyncVectorEnv(\n [\n lambda: gym.make(\n \"LunarLander-v2\",\n gravity=np.clip(\n np.random.normal(loc=-10.0, scale=1.0), a_min=-11.99, a_max=-0.01\n ),\n enable_wind=np.random.choice([True, False]),\n wind_power=np.clip(\n np.random.normal(loc=15.0, scale=1.0), a_min=0.01, a_max=19.99\n ),\n turbulence_power=np.clip(\n np.random.normal(loc=1.5, scale=0.5), a_min=0.01, a_max=1.99\n ),\n max_episode_steps=600,\n )\n for i in range(3)\n ]\n)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "------------------------------\n\nHere we are using normal distributions with the standard parameterization of the environment as the mean and an arbitrary standard deviation (scale).\nDepending on the problem, you can experiment with higher variance and use different distributions as well.\n\nIf you are training on the same `n_envs` environments for the entire training time, and `n_envs` is a relatively low number\n(in proportion to how complex the environment is), you might still get some overfitting to the specific parameterizations that you picked.\nTo mitigate this, you can either pick a high number of randomly parameterized environments or remake your environments every couple of sampling phases\nto generate a new set of pseudo-random parameters.\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# environment hyperparams\nn_envs = 10\nn_updates = 1000\nn_steps_per_update = 128\nrandomize_domain = False\n\n# agent hyperparams\ngamma = 0.999\nlam = 0.95 # hyperparameter for GAE\nent_coef = 0.01 # coefficient for the entropy bonus (to encourage exploration)\nactor_lr = 0.001\ncritic_lr = 0.005\n\n# Note: the actor has a slower learning rate so that the value targets become\n# more stationary and are theirfore easier to estimate for the critic\n\n# environment setup\nif randomize_domain:\n envs = gym.vector.AsyncVectorEnv(\n [\n lambda: gym.make(\n \"LunarLander-v2\",\n gravity=np.clip(\n np.random.normal(loc=-10.0, scale=1.0), a_min=-11.99, a_max=-0.01\n ),\n enable_wind=np.random.choice([True, False]),\n wind_power=np.clip(\n np.random.normal(loc=15.0, scale=1.0), a_min=0.01, a_max=19.99\n ),\n turbulence_power=np.clip(\n np.random.normal(loc=1.5, scale=0.5), a_min=0.01, a_max=1.99\n ),\n max_episode_steps=600,\n )\n for i in range(n_envs)\n ]\n )\n\nelse:\n envs = gym.vector.make(\"LunarLander-v2\", num_envs=n_envs, max_episode_steps=600)\n\n\nobs_shape = envs.single_observation_space.shape[0]\naction_shape = envs.single_action_space.n\n\n# set the device\nuse_cuda = False\nif use_cuda:\n device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nelse:\n device = torch.device(\"cpu\")\n\n# init the agent\nagent = A2C(obs_shape, action_shape, device, critic_lr, actor_lr, n_envs)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Training the A2C Agent\n\nFor our training loop, we are using the `RecordEpisodeStatistics` wrapper to record the episode lengths and returns and we are also saving\nthe losses and entropies to plot them after the agent finished training.\n\nYou may notice that the don't reset the vectorized envs at the start of each episode like we would usually do.\nThis is because each environment resets automatically once the episode finishes (each environment takes a different number of timesteps to finish\nan episode because of the random seeds). As a result, we are also not collecting data in `episodes`, but rather just play a certain number of steps\n(`n_steps_per_update`) in each environment (as an example, this could mean that we play 20 timesteps to finish an episode and then\nuse the rest of the timesteps to begin a new one).\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# create a wrapper environment to save episode returns and episode lengths\nenvs_wrapper = gym.wrappers.RecordEpisodeStatistics(envs, deque_size=n_envs * n_updates)\n\ncritic_losses = []\nactor_losses = []\nentropies = []\n\n# use tqdm to get a progress bar for training\nfor sample_phase in tqdm(range(n_updates)):\n\n # we don't have to reset the envs, they just continue playing\n # until the episode is over and then reset automatically\n\n # reset lists that collect experiences of an episode (sample phase)\n ep_value_preds = torch.zeros(n_steps_per_update, n_envs, device=device)\n ep_rewards = torch.zeros(n_steps_per_update, n_envs, device=device)\n ep_action_log_probs = torch.zeros(n_steps_per_update, n_envs, device=device)\n masks = torch.zeros(n_steps_per_update, n_envs, device=device)\n\n # at the start of training reset all envs to get an initial state\n if sample_phase == 0:\n states, info = envs_wrapper.reset(seed=42)\n\n # play n steps in our parallel environments to collect data\n for step in range(n_steps_per_update):\n\n # select an action A_{t} using S_{t} as input for the agent\n actions, action_log_probs, state_value_preds, entropy = agent.select_action(\n states\n )\n\n # perform the action A_{t} in the environment to get S_{t+1} and R_{t+1}\n states, rewards, terminated, truncated, infos = envs_wrapper.step(\n actions.numpy()\n )\n\n ep_value_preds[step] = torch.squeeze(state_value_preds)\n ep_rewards[step] = torch.tensor(rewards, device=device)\n ep_action_log_probs[step] = action_log_probs\n\n # add a mask (for the return calculation later);\n # for each env the mask is 1 if the episode is ongoing and 0 if it is terminated (not by truncation!)\n masks[step] = torch.tensor([not term for term in terminated])\n\n # calculate the losses for actor and critic\n critic_loss, actor_loss = agent.get_losses(\n ep_rewards,\n ep_action_log_probs,\n ep_value_preds,\n entropy,\n masks,\n gamma,\n lam,\n ent_coef,\n device,\n )\n\n # update the actor and critic networks\n agent.update_parameters(critic_loss, actor_loss)\n\n # log the losses and entropy\n critic_losses.append(critic_loss.detach().cpu().numpy())\n actor_losses.append(actor_loss.detach().cpu().numpy())\n entropies.append(entropy.detach().mean().cpu().numpy())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Plotting\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "\"\"\" plot the results \"\"\"\n\n# %matplotlib inline\n\nrolling_length = 20\nfig, axs = plt.subplots(nrows=2, ncols=2, figsize=(12, 5))\nfig.suptitle(\n f\"Training plots for {agent.__class__.__name__} in the LunarLander-v2 environment \\n \\\n (n_envs={n_envs}, n_steps_per_update={n_steps_per_update}, randomize_domain={randomize_domain})\"\n)\n\n# episode return\naxs[0][0].set_title(\"Episode Returns\")\nepisode_returns_moving_average = (\n np.convolve(\n np.array(envs_wrapper.return_queue).flatten(),\n np.ones(rolling_length),\n mode=\"valid\",\n )\n / rolling_length\n)\naxs[0][0].plot(\n np.arange(len(episode_returns_moving_average)) / n_envs,\n episode_returns_moving_average,\n)\naxs[0][0].set_xlabel(\"Number of episodes\")\n\n# entropy\naxs[1][0].set_title(\"Entropy\")\nentropy_moving_average = (\n np.convolve(np.array(entropies), np.ones(rolling_length), mode=\"valid\")\n / rolling_length\n)\naxs[1][0].plot(entropy_moving_average)\naxs[1][0].set_xlabel(\"Number of updates\")\n\n\n# critic loss\naxs[0][1].set_title(\"Critic Loss\")\ncritic_losses_moving_average = (\n np.convolve(\n np.array(critic_losses).flatten(), np.ones(rolling_length), mode=\"valid\"\n )\n / rolling_length\n)\naxs[0][1].plot(critic_losses_moving_average)\naxs[0][1].set_xlabel(\"Number of updates\")\n\n\n# actor loss\naxs[1][1].set_title(\"Actor Loss\")\nactor_losses_moving_average = (\n np.convolve(np.array(actor_losses).flatten(), np.ones(rolling_length), mode=\"valid\")\n / rolling_length\n)\naxs[1][1].plot(actor_losses_moving_average)\naxs[1][1].set_xlabel(\"Number of updates\")\n\nplt.tight_layout()\nplt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"training_plots\"\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Performance Analysis of Synchronous and Asynchronous Vectorized Environments\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "------------------------------\n\nAsynchronous environments can lead to quicker training times and a higher speedup\nfor data collection compared to synchronous environments. This is because asynchronous environments\nallow multiple agents to interact with their environments in parallel,\nwhile synchronous environments run multiple environments serially.\nThis results in better efficiency and faster training times for asynchronous environments.\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"performance_plots\"\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "------------------------------\n\nAccording to the Karp-Flatt metric (a metric used in parallel computing to estimate the limit for the\nspeedup when scaling up the number of parallel processes, here the number of environments),\nthe estimated max. speedup for asynchronous environments is 57, while the estimated maximum speedup\nfor synchronous environments is 21. This suggests that asynchronous environments have significantly\nfaster training times compared to synchronous environments (see graphs).\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"karp_flatt_metric\"\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "------------------------------\n\nHowever, it is important to note that increasing the number of parallel vector environments\ncan lead to slower training times after a certain number of environments (see plot below, where the\nagent was trained until the mean training returns were above -120). The slower training times might occur\nbecause the gradients of the environments are good enough after a relatively low number of environments\n(especially if the environment is not very complex). In this case, increasing the number of environments\ndoes not increase the learning speed, and actually increases the runtime, possibly due to the additional time\nneeded to calculate the gradients. For LunarLander-v2, the best performing configuration used a AsyncVectorEnv\nwith 10 parallel environments, but environments with a higher complexity may require more\nparallel environments to achieve optimal performance.\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"runtime_until_threshold_plot\"\n\n\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Saving/ Loading Weights\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "save_weights = False\nload_weights = False\n\nactor_weights_path = \"weights/actor_weights.h5\"\ncritic_weights_path = \"weights/critic_weights.h5\"\n\nif not os.path.exists(\"weights\"):\n os.mkdir(\"weights\")\n\n\"\"\" save network weights \"\"\"\nif save_weights:\n torch.save(agent.actor.state_dict(), actor_weights_path)\n torch.save(agent.critic.state_dict(), critic_weights_path)\n\n\n\"\"\" load network weights \"\"\"\nif load_weights:\n agent = A2C(obs_shape, action_shape, device, critic_lr, actor_lr)\n\n agent.actor.load_state_dict(torch.load(actor_weights_path))\n agent.critic.load_state_dict(torch.load(critic_weights_path))\n agent.actor.eval()\n agent.critic.eval()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Showcase the Agent\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "\"\"\" play a couple of showcase episodes \"\"\"\n\nn_showcase_episodes = 3\n\nfor episode in range(n_showcase_episodes):\n print(f\"starting episode {episode}...\")\n\n # create a new sample environment to get new random parameters\n if randomize_domain:\n env = gym.make(\n \"LunarLander-v2\",\n render_mode=\"human\",\n gravity=np.clip(\n np.random.normal(loc=-10.0, scale=2.0), a_min=-11.99, a_max=-0.01\n ),\n enable_wind=np.random.choice([True, False]),\n wind_power=np.clip(\n np.random.normal(loc=15.0, scale=2.0), a_min=0.01, a_max=19.99\n ),\n turbulence_power=np.clip(\n np.random.normal(loc=1.5, scale=1.0), a_min=0.01, a_max=1.99\n ),\n max_episode_steps=500,\n )\n else:\n env = gym.make(\"LunarLander-v2\", render_mode=\"human\", max_episode_steps=500)\n\n # get an initial state\n state, info = env.reset()\n\n # play one episode\n done = False\n while not done:\n\n # select an action A_{t} using S_{t} as input for the agent\n with torch.no_grad():\n action, _, _, _ = agent.select_action(state[None, :])\n\n # perform the action A_{t} in the environment to get S_{t+1} and R_{t+1}\n state, reward, terminated, truncated, info = env.step(action.item())\n\n # update if the environment is done\n done = terminated or truncated\n\nenv.close()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Try playing the environment yourself\n\n\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# from gymnasium.utils.play import play\n#\n# play(gym.make('LunarLander-v2', render_mode='rgb_array'),\n# keys_to_action={'w': 2, 'a': 1, 'd': 3}, noop=0)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## References\n\n[1] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. P. Lillicrap, T. Harley, D. Silver, K. Kavukcuoglu. \"Asynchronous Methods for Deep Reinforcement Learning\" ICML (2016).\n\n[2] J. Schulman, P. Moritz, S. Levine, M. Jordan and P. Abbeel. \"High-dimensional continuous control using generalized advantage estimation.\" ICLR (2016).\n\n[3] Gymnasium Documentation: Vector environments. (URL: https://gymnasium.farama.org/api/vector/)\n\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.16" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file diff --git a/_downloads/01a413564c9bff768b24ed43b946607d/environment_creation.py b/_downloads/56585a5841cc0f2c5a3dea777f5b14f0/environment_creation.py similarity index 99% rename from _downloads/01a413564c9bff768b24ed43b946607d/environment_creation.py rename to _downloads/56585a5841cc0f2c5a3dea777f5b14f0/environment_creation.py index 9515eeacf..19e65145a 100644 --- a/_downloads/01a413564c9bff768b24ed43b946607d/environment_creation.py +++ b/_downloads/56585a5841cc0f2c5a3dea777f5b14f0/environment_creation.py @@ -20,7 +20,7 @@ Subclassing gymnasium.Env ------------------------- Before learning how to create your own environment you should check out -`the documentation of Gymnasium’s API `__. +`the documentation of Gymnasium’s API `__. We will be concerned with a subset of gym-examples that looks like this: diff --git a/_downloads/a5659940aa3f8f568547d47752a43172/tutorials_jupyter.zip b/_downloads/a5659940aa3f8f568547d47752a43172/tutorials_jupyter.zip new file mode 100644 index 000000000..6ad4d1d60 Binary files /dev/null and b/_downloads/a5659940aa3f8f568547d47752a43172/tutorials_jupyter.zip differ diff --git a/_downloads/cbfb989de54c286b47aa60ed45417281/implementing_custom_wrappers.ipynb b/_downloads/b4e38e84e3b5c293cbde554ccbba7111/implementing_custom_wrappers.ipynb similarity index 99% rename from _downloads/cbfb989de54c286b47aa60ed45417281/implementing_custom_wrappers.ipynb rename to _downloads/b4e38e84e3b5c293cbde554ccbba7111/implementing_custom_wrappers.ipynb index 48047ba88..051eb14fc 100644 --- a/_downloads/cbfb989de54c286b47aa60ed45417281/implementing_custom_wrappers.ipynb +++ b/_downloads/b4e38e84e3b5c293cbde554ccbba7111/implementing_custom_wrappers.ipynb @@ -107,7 +107,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.15" + "version": "3.9.16" } }, "nbformat": 4, diff --git a/_downloads/1ddb20b882782d2f22f1dd33432af412/handling_time_limits.py b/_downloads/bcabe4d911fd12d268dd101a2f9af983/handling_time_limits.py similarity index 100% rename from _downloads/1ddb20b882782d2f22f1dd33432af412/handling_time_limits.py rename to _downloads/bcabe4d911fd12d268dd101a2f9af983/handling_time_limits.py diff --git a/_downloads/bbd827c8ce7fa89c011383532a3cf898/reinforce_invpend_gym_v26.ipynb b/_downloads/c5364859a73a7f575f592a0ad6aa1b73/reinforce_invpend_gym_v26.ipynb similarity index 99% rename from _downloads/bbd827c8ce7fa89c011383532a3cf898/reinforce_invpend_gym_v26.ipynb rename to _downloads/c5364859a73a7f575f592a0ad6aa1b73/reinforce_invpend_gym_v26.ipynb index a2a2f0b31..68abbeb3e 100644 --- a/_downloads/bbd827c8ce7fa89c011383532a3cf898/reinforce_invpend_gym_v26.ipynb +++ b/_downloads/c5364859a73a7f575f592a0ad6aa1b73/reinforce_invpend_gym_v26.ipynb @@ -125,7 +125,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.15" + "version": "3.9.16" } }, "nbformat": 4, diff --git a/_downloads/b0e7aa582a7b3c32ccdef133cbd779c5/blackjack_tutorial.ipynb b/_downloads/d1980709c80836b9d7e8f9131878afbb/blackjack_tutorial.ipynb similarity index 94% rename from _downloads/b0e7aa582a7b3c32ccdef133cbd779c5/blackjack_tutorial.ipynb rename to _downloads/d1980709c80836b9d7e8f9131878afbb/blackjack_tutorial.ipynb index f63b4f52d..bb9f7795f 100644 --- a/_downloads/b0e7aa582a7b3c32ccdef133cbd779c5/blackjack_tutorial.ipynb +++ b/_downloads/d1980709c80836b9d7e8f9131878afbb/blackjack_tutorial.ipynb @@ -169,7 +169,7 @@ }, "outputs": [], "source": [ - "rolling_length = 500\nfig, axs = plt.subplots(ncols=3, figsize=(12, 5))\naxs[0].set_title(\"Episode rewards\")\nreward_moving_average = (\n np.convolve(\n np.array(env.return_queue).flatten(), np.ones(rolling_length), mode=\"valid\"\n )\n / rolling_length\n)\naxs[0].plot(range(len(reward_moving_average)), reward_moving_average)\naxs[1].set_title(\"Episode lengths\")\nlength_moving_average = (\n np.convolve(\n np.array(env.length_queue).flatten(), np.ones(rolling_length), mode=\"same\"\n )\n / rolling_length\n)\naxs[1].plot(range(len(length_moving_average)), length_moving_average)\naxs[2].set_title(\"Training Error\")\ntraining_error_moving_average = (\n np.convolve(np.array(agent.training_error), np.ones(rolling_length), mode=\"same\")\n / rolling_length\n)\naxs[2].plot(range(len(training_error_moving_average)), training_error_moving_average)\nplt.tight_layout()\nplt.show()" + "rolling_length = 500\nfig, axs = plt.subplots(ncols=3, figsize=(12, 5))\naxs[0].set_title(\"Episode rewards\")\n# compute and assign a rolling average of the data to provide a smoother graph\nreward_moving_average = (\n np.convolve(\n np.array(env.return_queue).flatten(), np.ones(rolling_length), mode=\"valid\"\n )\n / rolling_length\n)\naxs[0].plot(range(len(reward_moving_average)), reward_moving_average)\naxs[1].set_title(\"Episode lengths\")\nlength_moving_average = (\n np.convolve(\n np.array(env.length_queue).flatten(), np.ones(rolling_length), mode=\"same\"\n )\n / rolling_length\n)\naxs[1].plot(range(len(length_moving_average)), length_moving_average)\naxs[2].set_title(\"Training Error\")\ntraining_error_moving_average = (\n np.convolve(np.array(agent.training_error), np.ones(rolling_length), mode=\"same\")\n / rolling_length\n)\naxs[2].plot(range(len(training_error_moving_average)), training_error_moving_average)\nplt.tight_layout()\nplt.show()" ] }, { @@ -264,7 +264,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.15" + "version": "3.9.16" } }, "nbformat": 4, diff --git a/_downloads/7784aa5068fc88abcafc19118c52cbb7/blackjack_tutorial.py b/_downloads/e1249c888e952c938d27855c3210a4bb/blackjack_tutorial.py similarity index 99% rename from _downloads/7784aa5068fc88abcafc19118c52cbb7/blackjack_tutorial.py rename to _downloads/e1249c888e952c938d27855c3210a4bb/blackjack_tutorial.py index e91b951b3..31495887d 100644 --- a/_downloads/7784aa5068fc88abcafc19118c52cbb7/blackjack_tutorial.py +++ b/_downloads/e1249c888e952c938d27855c3210a4bb/blackjack_tutorial.py @@ -299,6 +299,7 @@ for episode in tqdm(range(n_episodes)): rolling_length = 500 fig, axs = plt.subplots(ncols=3, figsize=(12, 5)) axs[0].set_title("Episode rewards") +# compute and assign a rolling average of the data to provide a smoother graph reward_moving_average = ( np.convolve( np.array(env.return_queue).flatten(), np.ones(rolling_length), mode="valid" diff --git a/_downloads/e688a889564af5a98daa8accfbca806e/vector_envs_tutorial.py b/_downloads/e688a889564af5a98daa8accfbca806e/vector_envs_tutorial.py new file mode 100644 index 000000000..c62512d6b --- /dev/null +++ b/_downloads/e688a889564af5a98daa8accfbca806e/vector_envs_tutorial.py @@ -0,0 +1,710 @@ +""" +Training A2C with Vector Envs and Domain Randomization +====================================================== + +""" + + +# %% +# Introduction +# ------------ +# +# In this tutorial, you'll learn how to use vectorized environments to train an Advantage Actor-Critic agent. +# We are going to use A2C, which is the synchronous version of the A3C algorithm [1]. +# +# Vectorized environments [3] can help to achieve quicker and more robust training by allowing multiple instances +# of the same environment to run in parallel (on multiple CPUs). This can significantly reduce the variance and thus speeds up the training. +# +# We will implement an Advantage Actor-Critic from scratch to look at how you can feed batched states into your networks to get a vector of actions +# (one action per environment) and calculate the losses for actor and critic on minibatches of transitions. +# Each minibatch contains the transitions of one sampling phase: `n_steps_per_update` steps are executed in `n_envs` environments in parallel +# (multiply the two to get the number of transitions in a minibatch). After each sampling phase, the losses are calculated and one gradient step is executed. +# To calculate the advantages, we are going to use the Generalized Advantage Estimation (GAE) method [2], which balances the tradeoff +# between variance and bias of the advantage estimates. +# +# The A2C agent class is initialized with the number of features of the input state, the number of actions the agent can take, +# the learning rates and the number of environments that run in parallel to collect experiences. The actor and critic networks are defined +# and their respective optimizers are initialized. The forward pass of the networks takes in a batched vector of states and returns a tensor of state values +# and a tensor of action logits. The select_action method returns a tuple of the chosen actions, the log-probs of those actions, and the state values for each action. +# In addition, it also returns the entropy of the policy distribution, which is subtracted from the loss later (with a weighting factor `ent_coef`) to encourage exploration. +# +# The get_losses function calculates the losses for the actor and critic networks (using GAE), which are then updated using the update_parameters function. +# + + +# %% +# +# ------------------------------ +# + +# Author: Till Zemann +# License: MIT License + +from __future__ import annotations + +import os + +import matplotlib.pyplot as plt +import numpy as np +import torch +import torch.nn as nn +from torch import optim +from tqdm import tqdm + +import gymnasium as gym + + +# %% +# Advantage Actor-Critic (A2C) +# ---------------------------- +# +# The Actor-Critic combines elements of value-based and policy-based methods. In A2C, the agent has two separate neural networks: +# a critic network that estimates the state-value function, and an actor network that outputs logits for a categorical probability distribution over all actions. +# The critic network is trained to minimize the mean squared error between the predicted state values and the actual returns received by the agent +# (this is equivalent to minimizing the squared advantages, because the advantage of an action is as the difference between the return and the state-value: A(s,a) = Q(s,a) - V(s). +# The actor network is trained to maximize the expected return by selecting actions that have high expected values according to the critic network. +# +# The focus of this tutorial will not be on the details of A2C itself. Instead, the tutorial will focus on how to use vectorized environments +# and domain randomization to accelerate the training process for A2C (and other reinforcement learning algorithms). +# + + +# %% +# +# ------------------------------ +# + + +class A2C(nn.Module): + """ + (Synchronous) Advantage Actor-Critic agent class + + Args: + n_features: The number of features of the input state. + n_actions: The number of actions the agent can take. + device: The device to run the computations on (running on a GPU might be quicker for larger Neural Nets, + for this code CPU is totally fine). + critic_lr: The learning rate for the critic network (should usually be larger than the actor_lr). + actor_lr: The learning rate for the actor network. + n_envs: The number of environments that run in parallel (on multiple CPUs) to collect experiences. + """ + + def __init__( + self, + n_features: int, + n_actions: int, + device: torch.device, + critic_lr: float, + actor_lr: float, + n_envs: int, + ) -> None: + """Initializes the actor and critic networks and their respective optimizers.""" + super().__init__() + self.device = device + self.n_envs = n_envs + + critic_layers = [ + nn.Linear(n_features, 32), + nn.ReLU(), + nn.Linear(32, 32), + nn.ReLU(), + nn.Linear(32, 1), # estimate V(s) + ] + + actor_layers = [ + nn.Linear(n_features, 32), + nn.ReLU(), + nn.Linear(32, 32), + nn.ReLU(), + nn.Linear( + 32, n_actions + ), # estimate action logits (will be fed into a softmax later) + ] + + # define actor and critic networks + self.critic = nn.Sequential(*critic_layers).to(self.device) + self.actor = nn.Sequential(*actor_layers).to(self.device) + + # define optimizers for actor and critic + self.critic_optim = optim.RMSprop(self.critic.parameters(), lr=critic_lr) + self.actor_optim = optim.RMSprop(self.actor.parameters(), lr=actor_lr) + + def forward(self, x: np.ndarray) -> tuple[torch.Tensor, torch.Tensor]: + """ + Forward pass of the networks. + + Args: + x: A batched vector of states. + + Returns: + state_values: A tensor with the state values, with shape [n_envs,]. + action_logits_vec: A tensor with the action logits, with shape [n_envs, n_actions]. + """ + x = torch.Tensor(x).to(self.device) + state_values = self.critic(x) # shape: [n_envs,] + action_logits_vec = self.actor(x) # shape: [n_envs, n_actions] + return (state_values, action_logits_vec) + + def select_action( + self, x: np.ndarray + ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]: + """ + Returns a tuple of the chosen actions and the log-probs of those actions. + + Args: + x: A batched vector of states. + + Returns: + actions: A tensor with the actions, with shape [n_steps_per_update, n_envs]. + action_log_probs: A tensor with the log-probs of the actions, with shape [n_steps_per_update, n_envs]. + state_values: A tensor with the state values, with shape [n_steps_per_update, n_envs]. + """ + state_values, action_logits = self.forward(x) + action_pd = torch.distributions.Categorical( + logits=action_logits + ) # implicitly uses softmax + actions = action_pd.sample() + action_log_probs = action_pd.log_prob(actions) + entropy = action_pd.entropy() + return (actions, action_log_probs, state_values, entropy) + + def get_losses( + self, + rewards: torch.Tensor, + action_log_probs: torch.Tensor, + value_preds: torch.Tensor, + entropy: torch.Tensor, + masks: torch.Tensor, + gamma: float, + lam: float, + ent_coef: float, + device: torch.device, + ) -> tuple[torch.Tensor, torch.Tensor]: + """ + Computes the loss of a minibatch (transitions collected in one sampling phase) for actor and critic + using Generalized Advantage Estimation (GAE) to compute the advantages (https://arxiv.org/abs/1506.02438). + + Args: + rewards: A tensor with the rewards for each time step in the episode, with shape [n_steps_per_update, n_envs]. + action_log_probs: A tensor with the log-probs of the actions taken at each time step in the episode, with shape [n_steps_per_update, n_envs]. + value_preds: A tensor with the state value predictions for each time step in the episode, with shape [n_steps_per_update, n_envs]. + masks: A tensor with the masks for each time step in the episode, with shape [n_steps_per_update, n_envs]. + gamma: The discount factor. + lam: The GAE hyperparameter. (lam=1 corresponds to Monte-Carlo sampling with high variance and no bias, + and lam=0 corresponds to normal TD-Learning that has a low variance but is biased + because the estimates are generated by a Neural Net). + device: The device to run the computations on (e.g. CPU or GPU). + + Returns: + critic_loss: The critic loss for the minibatch. + actor_loss: The actor loss for the minibatch. + """ + T = len(rewards) + advantages = torch.zeros(T, self.n_envs, device=device) + + # compute the advantages using GAE + gae = 0.0 + for t in reversed(range(T - 1)): + td_error = ( + rewards[t] + gamma * masks[t] * value_preds[t + 1] - value_preds[t] + ) + gae = td_error + gamma * lam * masks[t] * gae + advantages[t] = gae + + # calculate the loss of the minibatch for actor and critic + critic_loss = advantages.pow(2).mean() + + # give a bonus for higher entropy to encourage exploration + actor_loss = ( + -(advantages.detach() * action_log_probs).mean() - ent_coef * entropy.mean() + ) + return (critic_loss, actor_loss) + + def update_parameters( + self, critic_loss: torch.Tensor, actor_loss: torch.Tensor + ) -> None: + """ + Updates the parameters of the actor and critic networks. + + Args: + critic_loss: The critic loss. + actor_loss: The actor loss. + """ + self.critic_optim.zero_grad() + critic_loss.backward() + self.critic_optim.step() + + self.actor_optim.zero_grad() + actor_loss.backward() + self.actor_optim.step() + + +# %% +# Using Vectorized Environments +# ----------------------------- +# +# When you calculate the losses for the two Neural Networks over only one epoch, it might have a high variance. With vectorized environments, +# we can play with `n_envs` in parallel and thus get up to a linear speedup (meaning that in theory, we collect samples `n_envs` times quicker) +# that we can use to calculate the loss for the current policy and critic network. When we are using more samples to calculate the loss, +# it will have a lower variance and theirfore leads to quicker learning. +# +# A2C is a synchronous method, meaning that the parameter updates to Networks take place deterministically (after each sampling phase), +# but we can still make use of asynchronous vector envs to spawn multiple processes for parallel environment execution. +# +# The simplest way to create vector environments is by calling `gym.vector.make`, which creates multiple instances of the same environment: +# + +envs = gym.vector.make("LunarLander-v2", num_envs=3, max_episode_steps=600) + + +# %% +# Domain Randomization +# -------------------- +# +# If we want to randomize the environment for training to get more robust agents (that can deal with different parameterizations of an environment +# and theirfore might have a higher degree of generalization), we can set the desired parameters manually or use a pseudo-random number generator to generate them. +# +# Manually setting up 3 parallel 'LunarLander-v2' envs with different parameters: + + +envs = gym.vector.AsyncVectorEnv( + [ + lambda: gym.make( + "LunarLander-v2", + gravity=-10.0, + enable_wind=True, + wind_power=15.0, + turbulence_power=1.5, + max_episode_steps=600, + ), + lambda: gym.make( + "LunarLander-v2", + gravity=-9.8, + enable_wind=True, + wind_power=10.0, + turbulence_power=1.3, + max_episode_steps=600, + ), + lambda: gym.make( + "LunarLander-v2", gravity=-7.0, enable_wind=False, max_episode_steps=600 + ), + ] +) + + +# %% +# +# ------------------------------ +# +# Randomly generating the parameters for 3 parallel 'LunarLander-v2' envs, using `np.clip` to stay in the recommended parameter space: +# + + +envs = gym.vector.AsyncVectorEnv( + [ + lambda: gym.make( + "LunarLander-v2", + gravity=np.clip( + np.random.normal(loc=-10.0, scale=1.0), a_min=-11.99, a_max=-0.01 + ), + enable_wind=np.random.choice([True, False]), + wind_power=np.clip( + np.random.normal(loc=15.0, scale=1.0), a_min=0.01, a_max=19.99 + ), + turbulence_power=np.clip( + np.random.normal(loc=1.5, scale=0.5), a_min=0.01, a_max=1.99 + ), + max_episode_steps=600, + ) + for i in range(3) + ] +) + + +# %% +# +# ------------------------------ +# +# Here we are using normal distributions with the standard parameterization of the environment as the mean and an arbitrary standard deviation (scale). +# Depending on the problem, you can experiment with higher variance and use different distributions as well. +# +# If you are training on the same `n_envs` environments for the entire training time, and `n_envs` is a relatively low number +# (in proportion to how complex the environment is), you might still get some overfitting to the specific parameterizations that you picked. +# To mitigate this, you can either pick a high number of randomly parameterized environments or remake your environments every couple of sampling phases +# to generate a new set of pseudo-random parameters. +# + + +# %% +# Setup +# ----- +# + +# environment hyperparams +n_envs = 10 +n_updates = 1000 +n_steps_per_update = 128 +randomize_domain = False + +# agent hyperparams +gamma = 0.999 +lam = 0.95 # hyperparameter for GAE +ent_coef = 0.01 # coefficient for the entropy bonus (to encourage exploration) +actor_lr = 0.001 +critic_lr = 0.005 + +# Note: the actor has a slower learning rate so that the value targets become +# more stationary and are theirfore easier to estimate for the critic + +# environment setup +if randomize_domain: + envs = gym.vector.AsyncVectorEnv( + [ + lambda: gym.make( + "LunarLander-v2", + gravity=np.clip( + np.random.normal(loc=-10.0, scale=1.0), a_min=-11.99, a_max=-0.01 + ), + enable_wind=np.random.choice([True, False]), + wind_power=np.clip( + np.random.normal(loc=15.0, scale=1.0), a_min=0.01, a_max=19.99 + ), + turbulence_power=np.clip( + np.random.normal(loc=1.5, scale=0.5), a_min=0.01, a_max=1.99 + ), + max_episode_steps=600, + ) + for i in range(n_envs) + ] + ) + +else: + envs = gym.vector.make("LunarLander-v2", num_envs=n_envs, max_episode_steps=600) + + +obs_shape = envs.single_observation_space.shape[0] +action_shape = envs.single_action_space.n + +# set the device +use_cuda = False +if use_cuda: + device = torch.device("cuda" if torch.cuda.is_available() else "cpu") +else: + device = torch.device("cpu") + +# init the agent +agent = A2C(obs_shape, action_shape, device, critic_lr, actor_lr, n_envs) + + +# %% +# Training the A2C Agent +# ---------------------- +# +# For our training loop, we are using the `RecordEpisodeStatistics` wrapper to record the episode lengths and returns and we are also saving +# the losses and entropies to plot them after the agent finished training. +# +# You may notice that the don't reset the vectorized envs at the start of each episode like we would usually do. +# This is because each environment resets automatically once the episode finishes (each environment takes a different number of timesteps to finish +# an episode because of the random seeds). As a result, we are also not collecting data in `episodes`, but rather just play a certain number of steps +# (`n_steps_per_update`) in each environment (as an example, this could mean that we play 20 timesteps to finish an episode and then +# use the rest of the timesteps to begin a new one). +# + +# create a wrapper environment to save episode returns and episode lengths +envs_wrapper = gym.wrappers.RecordEpisodeStatistics(envs, deque_size=n_envs * n_updates) + +critic_losses = [] +actor_losses = [] +entropies = [] + +# use tqdm to get a progress bar for training +for sample_phase in tqdm(range(n_updates)): + + # we don't have to reset the envs, they just continue playing + # until the episode is over and then reset automatically + + # reset lists that collect experiences of an episode (sample phase) + ep_value_preds = torch.zeros(n_steps_per_update, n_envs, device=device) + ep_rewards = torch.zeros(n_steps_per_update, n_envs, device=device) + ep_action_log_probs = torch.zeros(n_steps_per_update, n_envs, device=device) + masks = torch.zeros(n_steps_per_update, n_envs, device=device) + + # at the start of training reset all envs to get an initial state + if sample_phase == 0: + states, info = envs_wrapper.reset(seed=42) + + # play n steps in our parallel environments to collect data + for step in range(n_steps_per_update): + + # select an action A_{t} using S_{t} as input for the agent + actions, action_log_probs, state_value_preds, entropy = agent.select_action( + states + ) + + # perform the action A_{t} in the environment to get S_{t+1} and R_{t+1} + states, rewards, terminated, truncated, infos = envs_wrapper.step( + actions.numpy() + ) + + ep_value_preds[step] = torch.squeeze(state_value_preds) + ep_rewards[step] = torch.tensor(rewards, device=device) + ep_action_log_probs[step] = action_log_probs + + # add a mask (for the return calculation later); + # for each env the mask is 1 if the episode is ongoing and 0 if it is terminated (not by truncation!) + masks[step] = torch.tensor([not term for term in terminated]) + + # calculate the losses for actor and critic + critic_loss, actor_loss = agent.get_losses( + ep_rewards, + ep_action_log_probs, + ep_value_preds, + entropy, + masks, + gamma, + lam, + ent_coef, + device, + ) + + # update the actor and critic networks + agent.update_parameters(critic_loss, actor_loss) + + # log the losses and entropy + critic_losses.append(critic_loss.detach().cpu().numpy()) + actor_losses.append(actor_loss.detach().cpu().numpy()) + entropies.append(entropy.detach().mean().cpu().numpy()) + + +# %% +# Plotting +# -------- +# + +""" plot the results """ + +# %matplotlib inline + +rolling_length = 20 +fig, axs = plt.subplots(nrows=2, ncols=2, figsize=(12, 5)) +fig.suptitle( + f"Training plots for {agent.__class__.__name__} in the LunarLander-v2 environment \n \ + (n_envs={n_envs}, n_steps_per_update={n_steps_per_update}, randomize_domain={randomize_domain})" +) + +# episode return +axs[0][0].set_title("Episode Returns") +episode_returns_moving_average = ( + np.convolve( + np.array(envs_wrapper.return_queue).flatten(), + np.ones(rolling_length), + mode="valid", + ) + / rolling_length +) +axs[0][0].plot( + np.arange(len(episode_returns_moving_average)) / n_envs, + episode_returns_moving_average, +) +axs[0][0].set_xlabel("Number of episodes") + +# entropy +axs[1][0].set_title("Entropy") +entropy_moving_average = ( + np.convolve(np.array(entropies), np.ones(rolling_length), mode="valid") + / rolling_length +) +axs[1][0].plot(entropy_moving_average) +axs[1][0].set_xlabel("Number of updates") + + +# critic loss +axs[0][1].set_title("Critic Loss") +critic_losses_moving_average = ( + np.convolve( + np.array(critic_losses).flatten(), np.ones(rolling_length), mode="valid" + ) + / rolling_length +) +axs[0][1].plot(critic_losses_moving_average) +axs[0][1].set_xlabel("Number of updates") + + +# actor loss +axs[1][1].set_title("Actor Loss") +actor_losses_moving_average = ( + np.convolve(np.array(actor_losses).flatten(), np.ones(rolling_length), mode="valid") + / rolling_length +) +axs[1][1].plot(actor_losses_moving_average) +axs[1][1].set_xlabel("Number of updates") + +plt.tight_layout() +plt.show() + +# %% +# .. image:: /_static/img/tutorials/vector_env_a2c_training_plots.png +# :alt: training_plots +# + + +# %% +# Performance Analysis of Synchronous and Asynchronous Vectorized Environments +# ---------------------------------------------------------------------------- +# + +# %% +# +# ------------------------------ +# +# Asynchronous environments can lead to quicker training times and a higher speedup +# for data collection compared to synchronous environments. This is because asynchronous environments +# allow multiple agents to interact with their environments in parallel, +# while synchronous environments run multiple environments serially. +# This results in better efficiency and faster training times for asynchronous environments. +# + +# %% +# .. image:: /_static/img/tutorials/vector_env_performance_plots.png +# :alt: performance_plots +# + +# %% +# +# ------------------------------ +# +# According to the Karp-Flatt metric (a metric used in parallel computing to estimate the limit for the +# speedup when scaling up the number of parallel processes, here the number of environments), +# the estimated max. speedup for asynchronous environments is 57, while the estimated maximum speedup +# for synchronous environments is 21. This suggests that asynchronous environments have significantly +# faster training times compared to synchronous environments (see graphs). +# + +# %% +# .. image:: /_static/img/tutorials/vector_env_karp_flatt_plot.png +# :alt: karp_flatt_metric +# + +# %% +# +# ------------------------------ +# +# However, it is important to note that increasing the number of parallel vector environments +# can lead to slower training times after a certain number of environments (see plot below, where the +# agent was trained until the mean training returns were above -120). The slower training times might occur +# because the gradients of the environments are good enough after a relatively low number of environments +# (especially if the environment is not very complex). In this case, increasing the number of environments +# does not increase the learning speed, and actually increases the runtime, possibly due to the additional time +# needed to calculate the gradients. For LunarLander-v2, the best performing configuration used a AsyncVectorEnv +# with 10 parallel environments, but environments with a higher complexity may require more +# parallel environments to achieve optimal performance. +# + +# %% +# .. image:: /_static/img/tutorials/vector_env_runtime_until_threshold.png +# :alt: runtime_until_threshold_plot +# + + +# %% +# Saving/ Loading Weights +# ----------------------- +# + +save_weights = False +load_weights = False + +actor_weights_path = "weights/actor_weights.h5" +critic_weights_path = "weights/critic_weights.h5" + +if not os.path.exists("weights"): + os.mkdir("weights") + +""" save network weights """ +if save_weights: + torch.save(agent.actor.state_dict(), actor_weights_path) + torch.save(agent.critic.state_dict(), critic_weights_path) + + +""" load network weights """ +if load_weights: + agent = A2C(obs_shape, action_shape, device, critic_lr, actor_lr) + + agent.actor.load_state_dict(torch.load(actor_weights_path)) + agent.critic.load_state_dict(torch.load(critic_weights_path)) + agent.actor.eval() + agent.critic.eval() + + +# %% +# Showcase the Agent +# ------------------ +# + +""" play a couple of showcase episodes """ + +n_showcase_episodes = 3 + +for episode in range(n_showcase_episodes): + print(f"starting episode {episode}...") + + # create a new sample environment to get new random parameters + if randomize_domain: + env = gym.make( + "LunarLander-v2", + render_mode="human", + gravity=np.clip( + np.random.normal(loc=-10.0, scale=2.0), a_min=-11.99, a_max=-0.01 + ), + enable_wind=np.random.choice([True, False]), + wind_power=np.clip( + np.random.normal(loc=15.0, scale=2.0), a_min=0.01, a_max=19.99 + ), + turbulence_power=np.clip( + np.random.normal(loc=1.5, scale=1.0), a_min=0.01, a_max=1.99 + ), + max_episode_steps=500, + ) + else: + env = gym.make("LunarLander-v2", render_mode="human", max_episode_steps=500) + + # get an initial state + state, info = env.reset() + + # play one episode + done = False + while not done: + + # select an action A_{t} using S_{t} as input for the agent + with torch.no_grad(): + action, _, _, _ = agent.select_action(state[None, :]) + + # perform the action A_{t} in the environment to get S_{t+1} and R_{t+1} + state, reward, terminated, truncated, info = env.step(action.item()) + + # update if the environment is done + done = terminated or truncated + +env.close() + + +# %% +# Try playing the environment yourself +# ------------------------------------ +# + +# from gymnasium.utils.play import play +# +# play(gym.make('LunarLander-v2', render_mode='rgb_array'), +# keys_to_action={'w': 2, 'a': 1, 'd': 3}, noop=0) + + +# %% +# References +# ---------- +# +# [1] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. P. Lillicrap, T. Harley, D. Silver, K. Kavukcuoglu. "Asynchronous Methods for Deep Reinforcement Learning" ICML (2016). +# +# [2] J. Schulman, P. Moritz, S. Levine, M. Jordan and P. Abbeel. "High-dimensional continuous control using generalized advantage estimation." ICLR (2016). +# +# [3] Gymnasium Documentation: Vector environments. (URL: https://gymnasium.farama.org/api/vector/) diff --git a/_downloads/6f2a99877a488456d33ac44780187aa4/implementing_custom_wrappers.py b/_downloads/edd29dab6b748d30e0d5574912999157/implementing_custom_wrappers.py similarity index 100% rename from _downloads/6f2a99877a488456d33ac44780187aa4/implementing_custom_wrappers.py rename to _downloads/edd29dab6b748d30e0d5574912999157/implementing_custom_wrappers.py diff --git a/_downloads/13a63d72db29f8035b56e0c8762b90e0/reinforce_invpend_gym_v26.py b/_downloads/f89a2a42559f7e216498edbfe9cb9bc8/reinforce_invpend_gym_v26.py similarity index 100% rename from _downloads/13a63d72db29f8035b56e0c8762b90e0/reinforce_invpend_gym_v26.py rename to _downloads/f89a2a42559f7e216498edbfe9cb9bc8/reinforce_invpend_gym_v26.py diff --git a/_images/sphx_glr_blackjack_tutorial_thumb.png b/_images/sphx_glr_blackjack_tutorial_thumb.png new file mode 100644 index 000000000..ab51e6970 Binary files /dev/null and b/_images/sphx_glr_blackjack_tutorial_thumb.png differ diff --git a/_images/sphx_glr_environment_creation_thumb.png b/_images/sphx_glr_environment_creation_thumb.png new file mode 100644 index 000000000..ab51e6970 Binary files /dev/null and b/_images/sphx_glr_environment_creation_thumb.png differ diff --git a/_images/sphx_glr_handling_time_limits_thumb.png b/_images/sphx_glr_handling_time_limits_thumb.png new file mode 100644 index 000000000..ab51e6970 Binary files /dev/null and b/_images/sphx_glr_handling_time_limits_thumb.png differ diff --git a/_images/sphx_glr_implementing_custom_wrappers_thumb.png b/_images/sphx_glr_implementing_custom_wrappers_thumb.png new file mode 100644 index 000000000..ab51e6970 Binary files /dev/null and b/_images/sphx_glr_implementing_custom_wrappers_thumb.png differ diff --git a/_images/sphx_glr_reinforce_invpend_gym_v26_thumb.png b/_images/sphx_glr_reinforce_invpend_gym_v26_thumb.png new file mode 100644 index 000000000..ab51e6970 Binary files /dev/null and b/_images/sphx_glr_reinforce_invpend_gym_v26_thumb.png differ diff --git a/_images/sphx_glr_vector_envs_tutorial_thumb.png b/_images/sphx_glr_vector_envs_tutorial_thumb.png new file mode 100644 index 000000000..ab51e6970 Binary files /dev/null and b/_images/sphx_glr_vector_envs_tutorial_thumb.png differ diff --git a/_images/vector_env_a2c_training_plots.png b/_images/vector_env_a2c_training_plots.png new file mode 100644 index 000000000..1e581aea9 Binary files /dev/null and b/_images/vector_env_a2c_training_plots.png differ diff --git a/_images/vector_env_karp_flatt_plot.png b/_images/vector_env_karp_flatt_plot.png new file mode 100644 index 000000000..e411a73fa Binary files /dev/null and b/_images/vector_env_karp_flatt_plot.png differ diff --git a/_images/vector_env_performance_plots.png b/_images/vector_env_performance_plots.png new file mode 100644 index 000000000..836363dfc Binary files /dev/null and b/_images/vector_env_performance_plots.png differ diff --git a/_images/vector_env_runtime_until_threshold.png b/_images/vector_env_runtime_until_threshold.png new file mode 100644 index 000000000..55c1c9628 Binary files /dev/null and b/_images/vector_env_runtime_until_threshold.png differ diff --git a/_static/binder_badge_logo.svg b/_static/binder_badge_logo.svg new file mode 100644 index 000000000..327f6b639 --- /dev/null +++ b/_static/binder_badge_logo.svg @@ -0,0 +1 @@ + launchlaunchbinderbinder \ No newline at end of file diff --git a/_static/broken_example.png b/_static/broken_example.png new file mode 100644 index 000000000..4fea24e7d Binary files /dev/null and b/_static/broken_example.png differ diff --git a/_static/favicon.png b/_static/favicon.png index d94b34384..b1ec92701 100644 Binary files a/_static/favicon.png and b/_static/favicon.png differ diff --git a/_static/img/favicon.png b/_static/img/favicon.png index d94b34384..b1ec92701 100644 Binary files a/_static/img/favicon.png and b/_static/img/favicon.png differ diff --git a/_static/img/tutorials/vector_env_a2c_training_plots.png b/_static/img/tutorials/vector_env_a2c_training_plots.png new file mode 100644 index 000000000..1e581aea9 Binary files /dev/null and b/_static/img/tutorials/vector_env_a2c_training_plots.png differ diff --git a/_static/img/tutorials/vector_env_karp_flatt_plot.png b/_static/img/tutorials/vector_env_karp_flatt_plot.png new file mode 100644 index 000000000..e411a73fa Binary files /dev/null and b/_static/img/tutorials/vector_env_karp_flatt_plot.png differ diff --git a/_static/img/tutorials/vector_env_performance_plots.png b/_static/img/tutorials/vector_env_performance_plots.png new file mode 100644 index 000000000..836363dfc Binary files /dev/null and b/_static/img/tutorials/vector_env_performance_plots.png differ diff --git a/_static/img/tutorials/vector_env_runtime_until_threshold.png b/_static/img/tutorials/vector_env_runtime_until_threshold.png new file mode 100644 index 000000000..55c1c9628 Binary files /dev/null and b/_static/img/tutorials/vector_env_runtime_until_threshold.png differ diff --git a/_static/no_image.png b/_static/no_image.png new file mode 100644 index 000000000..8c2d48d5d Binary files /dev/null and b/_static/no_image.png differ diff --git a/_static/sg_gallery-binder.css b/_static/sg_gallery-binder.css new file mode 100644 index 000000000..a33aa4204 --- /dev/null +++ b/_static/sg_gallery-binder.css @@ -0,0 +1,6 @@ +/* CSS for binder integration */ + +div.binder-badge { + margin: 1em auto; + vertical-align: middle; +} diff --git a/_static/sg_gallery-dataframe.css b/_static/sg_gallery-dataframe.css new file mode 100644 index 000000000..25be73092 --- /dev/null +++ b/_static/sg_gallery-dataframe.css @@ -0,0 +1,46 @@ +/* Pandas dataframe css */ +/* Taken from: https://github.com/spatialaudio/nbsphinx/blob/fb3ba670fc1ba5f54d4c487573dbc1b4ecf7e9ff/src/nbsphinx.py#L587-L619 */ +html[data-theme="light"] { + --sg-text-color: #000; + --sg-tr-odd-color: #f5f5f5; + --sg-tr-hover-color: rgba(66, 165, 245, 0.2); +} +html[data-theme="dark"] { + --sg-text-color: #fff; + --sg-tr-odd-color: #373737; + --sg-tr-hover-color: rgba(30, 81, 122, 0.2); +} + +table.dataframe { + border: none !important; + border-collapse: collapse; + border-spacing: 0; + border-color: transparent; + color: var(--sg-text-color); + font-size: 12px; + table-layout: fixed; +} +table.dataframe thead { + border-bottom: 1px solid var(--sg-text-color); + vertical-align: bottom; +} +table.dataframe tr, +table.dataframe th, +table.dataframe td { + text-align: right; + vertical-align: middle; + padding: 0.5em 0.5em; + line-height: normal; + white-space: normal; + max-width: none; + border: none; +} +table.dataframe th { + font-weight: bold; +} +table.dataframe tbody tr:nth-child(odd) { + background: var(--sg-tr-odd-color); +} +table.dataframe tbody tr:hover { + background: var(--sg-tr-hover-color); +} diff --git a/_static/sg_gallery-rendered-html.css b/_static/sg_gallery-rendered-html.css new file mode 100644 index 000000000..93dc2ffb0 --- /dev/null +++ b/_static/sg_gallery-rendered-html.css @@ -0,0 +1,224 @@ +/* Adapted from notebook/static/style/style.min.css */ +html[data-theme="light"] { + --sg-text-color: #000; + --sg-background-color: #ffffff; + --sg-code-background-color: #eff0f1; + --sg-tr-hover-color: rgba(66, 165, 245, 0.2); + --sg-tr-odd-color: #f5f5f5; +} +html[data-theme="dark"] { + --sg-text-color: #fff; + --sg-background-color: #121212; + --sg-code-background-color: #2f2f30; + --sg-tr-hover-color: rgba(66, 165, 245, 0.2); + --sg-tr-odd-color: #1f1f1f; +} + +.rendered_html { + color: var(--sg-text-color); + /* any extras will just be numbers: */ +} +.rendered_html em { + font-style: italic; +} +.rendered_html strong { + font-weight: bold; +} +.rendered_html u { + text-decoration: underline; +} +.rendered_html :link { + text-decoration: underline; +} +.rendered_html :visited { + text-decoration: underline; +} +.rendered_html h1 { + font-size: 185.7%; + margin: 1.08em 0 0 0; + font-weight: bold; + line-height: 1.0; +} +.rendered_html h2 { + font-size: 157.1%; + margin: 1.27em 0 0 0; + font-weight: bold; + line-height: 1.0; +} +.rendered_html h3 { + font-size: 128.6%; + margin: 1.55em 0 0 0; + font-weight: bold; + line-height: 1.0; +} +.rendered_html h4 { + font-size: 100%; + margin: 2em 0 0 0; + font-weight: bold; + line-height: 1.0; +} +.rendered_html h5 { + font-size: 100%; + margin: 2em 0 0 0; + font-weight: bold; + line-height: 1.0; + font-style: italic; +} +.rendered_html h6 { + font-size: 100%; + margin: 2em 0 0 0; + font-weight: bold; + line-height: 1.0; + font-style: italic; +} +.rendered_html h1:first-child { + margin-top: 0.538em; +} +.rendered_html h2:first-child { + margin-top: 0.636em; +} +.rendered_html h3:first-child { + margin-top: 0.777em; +} +.rendered_html h4:first-child { + margin-top: 1em; +} +.rendered_html h5:first-child { + margin-top: 1em; +} +.rendered_html h6:first-child { + margin-top: 1em; +} +.rendered_html ul:not(.list-inline), +.rendered_html ol:not(.list-inline) { + padding-left: 2em; +} +.rendered_html ul { + list-style: disc; +} +.rendered_html ul ul { + list-style: square; + margin-top: 0; +} +.rendered_html ul ul ul { + list-style: circle; +} +.rendered_html ol { + list-style: decimal; +} +.rendered_html ol ol { + list-style: upper-alpha; + margin-top: 0; +} +.rendered_html ol ol ol { + list-style: lower-alpha; +} +.rendered_html ol ol ol ol { + list-style: lower-roman; +} +.rendered_html ol ol ol ol ol { + list-style: decimal; +} +.rendered_html * + ul { + margin-top: 1em; +} +.rendered_html * + ol { + margin-top: 1em; +} +.rendered_html hr { + color: var(--sg-text-color); + background-color: var(--sg-text-color); +} +.rendered_html pre { + margin: 1em 2em; + padding: 0px; + background-color: var(--sg-background-color); +} +.rendered_html code { + background-color: var(--sg-code-background-color); +} +.rendered_html p code { + padding: 1px 5px; +} +.rendered_html pre code { + background-color: var(--sg-background-color); +} +.rendered_html pre, +.rendered_html code { + border: 0; + color: var(--sg-text-color); + font-size: 100%; +} +.rendered_html blockquote { + margin: 1em 2em; +} +.rendered_html table { + margin-left: auto; + margin-right: auto; + border: none; + border-collapse: collapse; + border-spacing: 0; + color: var(--sg-text-color); + font-size: 12px; + table-layout: fixed; +} +.rendered_html thead { + border-bottom: 1px solid var(--sg-text-color); + vertical-align: bottom; +} +.rendered_html tr, +.rendered_html th, +.rendered_html td { + text-align: right; + vertical-align: middle; + padding: 0.5em 0.5em; + line-height: normal; + white-space: normal; + max-width: none; + border: none; +} +.rendered_html th { + font-weight: bold; +} +.rendered_html tbody tr:nth-child(odd) { + background: var(--sg-tr-odd-color); +} +.rendered_html tbody tr:hover { + color: var(--sg-text-color); + background: var(--sg-tr-hover-color); +} +.rendered_html * + table { + margin-top: 1em; +} +.rendered_html p { + text-align: left; +} +.rendered_html * + p { + margin-top: 1em; +} +.rendered_html img { + display: block; + margin-left: auto; + margin-right: auto; +} +.rendered_html * + img { + margin-top: 1em; +} +.rendered_html img, +.rendered_html svg { + max-width: 100%; + height: auto; +} +.rendered_html img.unconfined, +.rendered_html svg.unconfined { + max-width: none; +} +.rendered_html .alert { + margin-bottom: initial; +} +.rendered_html * + .alert { + margin-top: 1em; +} +[dir="rtl"] .rendered_html p { + text-align: right; +} diff --git a/_static/sg_gallery.css b/_static/sg_gallery.css new file mode 100644 index 000000000..72227837d --- /dev/null +++ b/_static/sg_gallery.css @@ -0,0 +1,342 @@ +/* +Sphinx-Gallery has compatible CSS to fix default sphinx themes +Tested for Sphinx 1.3.1 for all themes: default, alabaster, sphinxdoc, +scrolls, agogo, traditional, nature, haiku, pyramid +Tested for Read the Docs theme 0.1.7 */ + +/* Define light colors */ +:root, html[data-theme="light"], body[data-theme="light"]{ + --sg-tooltip-foreground: black; + --sg-tooltip-background: rgba(250, 250, 250, 0.9); + --sg-tooltip-border: #ccc transparent; + --sg-thumb-box-shadow-color: #6c757d40; + --sg-thumb-hover-border: #0069d9; + --sg-script-out: #888; + --sg-script-pre: #fafae2; + --sg-pytb-foreground: #000; + --sg-pytb-background: #ffe4e4; + --sg-pytb-border-color: #f66; + --sg-download-a-background-color: #ffc; + --sg-download-a-background-image: linear-gradient(to bottom, #ffc, #d5d57e); + --sg-download-a-border-color: 1px solid #c2c22d; + --sg-download-a-color: #000; + --sg-download-a-hover-background-color: #d5d57e; + --sg-download-a-hover-box-shadow-1: rgba(255, 255, 255, 0.1); + --sg-download-a-hover-box-shadow-2: rgba(0, 0, 0, 0.25); +} +@media(prefers-color-scheme: light) { + :root[data-theme="auto"], html[data-theme="auto"], body[data-theme="auto"] { + --sg-tooltip-foreground: black; + --sg-tooltip-background: rgba(250, 250, 250, 0.9); + --sg-tooltip-border: #ccc transparent; + --sg-thumb-box-shadow-color: #6c757d40; + --sg-thumb-hover-border: #0069d9; + --sg-script-out: #888; + --sg-script-pre: #fafae2; + --sg-pytb-foreground: #000; + --sg-pytb-background: #ffe4e4; + --sg-pytb-border-color: #f66; + --sg-download-a-background-color: #ffc; + --sg-download-a-background-image: linear-gradient(to bottom, #ffc, #d5d57e); + --sg-download-a-border-color: 1px solid #c2c22d; + --sg-download-a-color: #000; + --sg-download-a-hover-background-color: #d5d57e; + --sg-download-a-hover-box-shadow-1: rgba(255, 255, 255, 0.1); + --sg-download-a-hover-box-shadow-2: rgba(0, 0, 0, 0.25); + } +} + +html[data-theme="dark"], body[data-theme="dark"] { + --sg-tooltip-foreground: white; + --sg-tooltip-background: rgba(10, 10, 10, 0.9); + --sg-tooltip-border: #333 transparent; + --sg-thumb-box-shadow-color: #79848d40; + --sg-thumb-hover-border: #003975; + --sg-script-out: rgb(179, 179, 179); + --sg-script-pre: #2e2e22; + --sg-pytb-foreground: #fff; + --sg-pytb-background: #1b1717; + --sg-pytb-border-color: #622; + --sg-download-a-background-color: #443; + --sg-download-a-background-image: linear-gradient(to bottom, #443, #221); + --sg-download-a-border-color: 1px solid #3a3a0d; + --sg-download-a-color: #fff; + --sg-download-a-hover-background-color: #616135; + --sg-download-a-hover-box-shadow-1: rgba(0, 0, 0, 0.1); + --sg-download-a-hover-box-shadow-2: rgba(255, 255, 255, 0.25); +} +@media(prefers-color-scheme: dark){ + html[data-theme="auto"], body[data-theme="auto"] { + --sg-tooltip-foreground: white; + --sg-tooltip-background: rgba(10, 10, 10, 0.9); + --sg-tooltip-border: #333 transparent; + --sg-thumb-box-shadow-color: #79848d40; + --sg-thumb-hover-border: #003975; + --sg-script-out: rgb(179, 179, 179); + --sg-script-pre: #2e2e22; + --sg-pytb-foreground: #fff; + --sg-pytb-background: #1b1717; + --sg-pytb-border-color: #622; + --sg-download-a-background-color: #443; + --sg-download-a-background-image: linear-gradient(to bottom, #443, #221); + --sg-download-a-border-color: 1px solid #3a3a0d; + --sg-download-a-color: #fff; + --sg-download-a-hover-background-color: #616135; + --sg-download-a-hover-box-shadow-1: rgba(0, 0, 0, 0.1); + --sg-download-a-hover-box-shadow-2: rgba(255, 255, 255, 0.25); + } +} + +.sphx-glr-thumbnails { + width: 100%; + margin: 0px 0px 20px 0px; + + /* align thumbnails on a grid */ + justify-content: space-between; + display: grid; + /* each grid column should be at least 160px (this will determine + the actual number of columns) and then take as much of the + remaining width as possible */ + grid-template-columns: repeat(auto-fill, minmax(160px, 1fr)); + gap: 15px; +} +.sphx-glr-thumbnails .toctree-wrapper { + /* hide empty toctree divs added to the DOM + by sphinx even though the toctree is hidden + (they would fill grid places with empty divs) */ + display: none; +} +.sphx-glr-thumbcontainer { + background: transparent; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + border-radius: 5px; + box-shadow: 0 0 10px var(--sg-thumb-box-shadow-color); + + /* useful to absolutely position link in div */ + position: relative; + + /* thumbnail width should include padding and borders + and take all available space */ + box-sizing: border-box; + width: 100%; + padding: 10px; + border: 1px solid transparent; + + /* align content in thumbnail */ + display: flex; + flex-direction: column; + align-items: center; + gap: 7px; +} +.sphx-glr-thumbcontainer p { + position: absolute; + top: 0; + left: 0; +} +.sphx-glr-thumbcontainer p, +.sphx-glr-thumbcontainer p a { + /* link should cover the whole thumbnail div */ + width: 100%; + height: 100%; +} +.sphx-glr-thumbcontainer p a span { + /* text within link should be masked + (we are just interested in the href) */ + display: none; +} +.sphx-glr-thumbcontainer:hover { + border: 1px solid; + border-color: var(--sg-thumb-hover-border); + cursor: pointer; +} +.sphx-glr-thumbcontainer a.internal { + bottom: 0; + display: block; + left: 0; + box-sizing: border-box; + padding: 150px 10px 0; + position: absolute; + right: 0; + top: 0; +} +/* Next one is to avoid Sphinx traditional theme to cover all the +thumbnail with its default link Background color */ +.sphx-glr-thumbcontainer a.internal:hover { + background-color: transparent; +} + +.sphx-glr-thumbcontainer p { + margin: 0 0 0.1em 0; +} +.sphx-glr-thumbcontainer .figure { + margin: 10px; + width: 160px; +} +.sphx-glr-thumbcontainer img { + display: inline; + max-height: 112px; + max-width: 160px; +} +.sphx-glr-thumbcontainer[tooltip]:hover:after { + background: var(--sg-tooltip-background); + -webkit-border-radius: 4px; + -moz-border-radius: 4px; + border-radius: 4px; + color: var(--sg-tooltip-foreground); + content: attr(tooltip); + padding: 10px; + z-index: 98; + width: 100%; + height: 100%; + position: absolute; + pointer-events: none; + top: 0; + box-sizing: border-box; + overflow: hidden; + backdrop-filter: blur(3px); +} + +.sphx-glr-script-out { + color: var(--sg-script-out); + display: flex; + gap: 0.5em; +} +.sphx-glr-script-out::before { + content: "Out:"; + /* These numbers come from the pre style in the pydata sphinx theme. This + * turns out to match perfectly on the rtd theme, but be a bit too low for + * the pydata sphinx theme. As I could not find a dimension to use that was + * scaled the same way, I just picked one option that worked pretty close for + * both. */ + line-height: 1.4; + padding-top: 10px; +} +.sphx-glr-script-out .highlight { + background-color: transparent; + /* These options make the div expand... */ + flex-grow: 1; + /* ... but also keep it from overflowing its flex container. */ + overflow: auto; +} +.sphx-glr-script-out .highlight pre { + background-color: var(--sg-script-pre); + border: 0; + max-height: 30em; + overflow: auto; + padding-left: 1ex; + /* This margin is necessary in the pydata sphinx theme because pre has a box + * shadow which would be clipped by the overflow:auto in the parent div + * above. */ + margin: 2px; + word-break: break-word; +} +.sphx-glr-script-out + p { + margin-top: 1.8em; +} +blockquote.sphx-glr-script-out { + margin-left: 0pt; +} +.sphx-glr-script-out.highlight-pytb .highlight pre { + color: var(--sg-pytb-foreground); + background-color: var(--sg-pytb-background); + border: 1px solid var(--sg-pytb-border-color); + margin-top: 10px; + padding: 7px; +} + +div.sphx-glr-footer { + text-align: center; +} + +div.sphx-glr-download { + margin: 1em auto; + vertical-align: middle; +} + +div.sphx-glr-download a { + background-color: var(--sg-download-a-background-color); + background-image: var(--sg-download-a-background-image); + border-radius: 4px; + border: 1px solid var(--sg-download-a-border-color); + color: var(--sg-download-a-color); + display: inline-block; + font-weight: bold; + padding: 1ex; + text-align: center; +} + +div.sphx-glr-download code.download { + display: inline-block; + white-space: normal; + word-break: normal; + overflow-wrap: break-word; + /* border and background are given by the enclosing 'a' */ + border: none; + background: none; +} + +div.sphx-glr-download a:hover { + box-shadow: inset 0 1px 0 var(--sg-download-a-hover-box-shadow-1), 0 1px 5px var(--sg-download-a-hover-box-shadow-2); + text-decoration: none; + background-image: none; + background-color: var(--sg-download-a-hover-background-color); +} + +.sphx-glr-example-title:target::before { + display: block; + content: ""; + margin-top: -50px; + height: 50px; + visibility: hidden; +} + +ul.sphx-glr-horizontal { + list-style: none; + padding: 0; +} +ul.sphx-glr-horizontal li { + display: inline; +} +ul.sphx-glr-horizontal img { + height: auto !important; +} + +.sphx-glr-single-img { + margin: auto; + display: block; + max-width: 100%; +} + +.sphx-glr-multi-img { + max-width: 42%; + height: auto; +} + +div.sphx-glr-animation { + margin: auto; + display: block; + max-width: 100%; +} +div.sphx-glr-animation .animation { + display: block; +} + +p.sphx-glr-signature a.reference.external { + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + border-radius: 5px; + padding: 3px; + font-size: 75%; + text-align: right; + margin-left: auto; + display: table; +} + +.sphx-glr-clear { + clear: both; +} + +a.sphx-glr-backref-instance { + text-decoration: none; +} diff --git a/_static/styles/furo-extensions.css b/_static/styles/furo-extensions.css index c17edf89e..c2f92cc68 100644 --- a/_static/styles/furo-extensions.css +++ b/_static/styles/furo-extensions.css @@ -1,2 +1,2 @@ -#furo-sidebar-ad-placement{padding:var(--sidebar-item-spacing-vertical) var(--sidebar-item-spacing-horizontal)}#furo-sidebar-ad-placement .ethical-sidebar{background:var(--color-background-secondary);border:none;box-shadow:none}#furo-sidebar-ad-placement .ethical-sidebar:hover{background:var(--color-background-hover)}#furo-sidebar-ad-placement .ethical-sidebar a{color:var(--color-foreground-primary)}#furo-sidebar-ad-placement .ethical-callout a{color:var(--color-foreground-secondary)!important}#furo-readthedocs-versions{background:transparent;display:block;position:static;width:100%}#furo-readthedocs-versions .rst-versions{background:#1a1c1e}#furo-readthedocs-versions .rst-current-version{background:var(--color-sidebar-item-background);cursor:unset}#furo-readthedocs-versions .rst-current-version:hover{background:var(--color-sidebar-item-background)}#furo-readthedocs-versions .rst-current-version .fa-book{color:var(--color-foreground-primary)}#furo-readthedocs-versions>.rst-other-versions{padding:0}#furo-readthedocs-versions>.rst-other-versions small{opacity:1}#furo-readthedocs-versions .injected .rst-versions{position:unset}#furo-readthedocs-versions:focus-within,#furo-readthedocs-versions:hover{box-shadow:0 0 0 1px var(--color-sidebar-background-border)}#furo-readthedocs-versions:focus-within .rst-current-version,#furo-readthedocs-versions:hover .rst-current-version{background:#1a1c1e;font-size:inherit;height:auto;line-height:inherit;padding:12px;text-align:right}#furo-readthedocs-versions:focus-within .rst-current-version .fa-book,#furo-readthedocs-versions:hover .rst-current-version .fa-book{color:#fff;float:left}#furo-readthedocs-versions:focus-within .fa-caret-down,#furo-readthedocs-versions:hover .fa-caret-down{display:none}#furo-readthedocs-versions:focus-within .injected,#furo-readthedocs-versions:focus-within .rst-current-version,#furo-readthedocs-versions:focus-within .rst-other-versions,#furo-readthedocs-versions:hover .injected,#furo-readthedocs-versions:hover .rst-current-version,#furo-readthedocs-versions:hover .rst-other-versions{display:block}#furo-readthedocs-versions:focus-within>.rst-current-version,#furo-readthedocs-versions:hover>.rst-current-version{display:none}.highlight:hover button.copybtn{color:var(--color-code-foreground)}.highlight button.copybtn{align-items:center;background-color:var(--color-code-background);border:none;color:var(--color-background-item);cursor:pointer;height:1.25em;opacity:1;right:.5rem;top:.625rem;transition:color .3s,opacity .3s;width:1.25em}.highlight button.copybtn:hover{background-color:var(--color-code-background);color:var(--color-brand-content)}.highlight button.copybtn:after{background-color:transparent;color:var(--color-code-foreground);display:none}.highlight button.copybtn.success{color:#22863a;transition:color 0ms}.highlight button.copybtn.success:after{display:block}.highlight button.copybtn svg{padding:0}body{--sd-color-primary:var(--color-brand-primary);--sd-color-primary-highlight:var(--color-brand-content);--sd-color-primary-text:var(--color-background-primary);--sd-color-shadow:rgba(0,0,0,.05);--sd-color-card-border:var(--color-card-border);--sd-color-card-border-hover:var(--color-brand-content);--sd-color-card-background:var(--color-card-background);--sd-color-card-text:var(--color-foreground-primary);--sd-color-card-header:var(--color-card-marginals-background);--sd-color-card-footer:var(--color-card-marginals-background);--sd-color-tabs-label-active:var(--color-brand-content);--sd-color-tabs-label-hover:var(--color-foreground-muted);--sd-color-tabs-label-inactive:var(--color-foreground-muted);--sd-color-tabs-underline-active:var(--color-brand-content);--sd-color-tabs-underline-hover:var(--color-foreground-border);--sd-color-tabs-underline-inactive:var(--color-background-border);--sd-color-tabs-overline:var(--color-background-border);--sd-color-tabs-underline:var(--color-background-border)}.sd-tab-content{box-shadow:0 -2px var(--sd-color-tabs-overline),0 1px var(--sd-color-tabs-underline)}.sd-card{box-shadow:0 .1rem .25rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)}.sd-shadow-sm{box-shadow:0 .1rem .25rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)!important}.sd-shadow-md{box-shadow:0 .3rem .75rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)!important}.sd-shadow-lg{box-shadow:0 .6rem 1.5rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)!important}.sd-card-hover:hover{transform:none}.sd-cards-carousel{gap:.25rem;padding:.25rem}body{--tabs--label-text:var(--color-foreground-muted);--tabs--label-text--hover:var(--color-foreground-muted);--tabs--label-text--active:var(--color-brand-content);--tabs--label-text--active--hover:var(--color-brand-content);--tabs--label-background:transparent;--tabs--label-background--hover:transparent;--tabs--label-background--active:transparent;--tabs--label-background--active--hover:transparent;--tabs--padding-x:0.25em;--tabs--margin-x:1em;--tabs--border:var(--color-background-border);--tabs--label-border:transparent;--tabs--label-border--hover:var(--color-foreground-muted);--tabs--label-border--active:var(--color-brand-content);--tabs--label-border--active--hover:var(--color-brand-content)}[role=main] .container{max-width:none;padding-left:0;padding-right:0}.shadow.docutils{border:none;box-shadow:0 .2rem .5rem rgba(0,0,0,.05),0 0 .0625rem rgba(0,0,0,.1)!important}.sphinx-bs .card{background-color:var(--color-background-secondary);color:var(--color-foreground)}h1{font-size:2.2rem}h2{font-size:1.7rem}h3{font-size:1.4rem}html:has(.farama-header-menu.active){visibility:hidden}.farama-hidden[aria-hidden=true]{visibility:hidden}.farama-hidden[aria-hidden=false]{visibility:visible}.cookie-alert{background-color:var(--color-background-secondary);border-top:1px solid var(--color-background-border);bottom:0;color:var(--color-foreground-primary);display:flex;left:0;min-height:70px;position:fixed;width:100%;z-index:99999}.cookie-alert__container{align-items:center;display:flex;margin:auto;max-width:calc(100% - 28px);width:700px}.cookie-alert p{flex:1}.cookie-alert button{background-color:transparent;border:none;cursor:pointer;height:26px;padding:0;width:26px}.cookie-alert button svg .cls-1{stroke:var(--color-foreground-primary)}.cookie-alert button svg{width:100%}article[role=main]:has(.farama-env-icon-container) .farama-env-icon-container{display:flex;margin-top:7px;position:absolute}article[role=main]:has(.farama-env-icon-container) .section h1:first-child,article[role=main]:has(.farama-env-icon-container) .section h2:first-child,article[role=main]:has(.farama-env-icon-container) section h1:first-child,article[role=main]:has(.farama-env-icon-container) section h2:first-child{margin-left:34px}.farama-env-icon{height:32px}.env-grid{box-sizing:border-box;display:flex;flex-wrap:wrap;justify-content:center;width:100%}.env-grid__cell{display:flex;flex-direction:column;height:180px;padding:10px;width:180px}.cell__image-container{display:flex;height:148px;justify-content:center}.cell__image-container img{max-height:100%;-o-object-fit:contain;object-fit:contain}.cell__title{align-items:flex-end;display:flex;height:32px;justify-content:center;line-height:16px;text-align:center}.more-btn{display:block;margin:12px auto;width:240px}html:has(.farama-header-menu.active){overflow:hidden}body{--farama-header-height:52px;--farama-header-logo-margin:10px}.farama-header{background-color:var(--color-background-secondary);border-bottom:1px solid var(--color-header-border);box-sizing:border-box;display:flex;height:var(--farama-header-height);padding:0 36px 0 24px;position:absolute;width:100%;z-index:95}.farama-header .farama-header__container{display:flex;margin:0 auto;max-width:1400px;width:100%}.farama-header a{color:var(--color-foreground-primary);text-decoration:none;transition:color .125s ease}.farama-header a:hover{color:var(--color-foreground-secondary)}.farama-header .farama-header__logo{margin:var(--farama-header-logo-margin);max-height:calc(var(--farama-header-height) - var(--farama-header-logo-margin))}.farama-header .farama-header__title{align-self:center;font-size:var(--font-size--normal);font-weight:400;margin:0 0 2px;padding:0 0 0 4px}.farama-header .farama-header__left,.farama-header .farama-header__left a{display:flex}.farama-header .farama-header__left--mobile{display:none}.farama-header .farama-header__left--mobile .nav-overlay-icon svg{stroke:var(--color-foreground-primary);fill:var(--color-foreground-primary);stroke-width:2px;padding:0 6px;width:20px}.farama-header .farama-header__right{align-items:center;display:flex;flex:1;justify-content:flex-end;z-index:2}.farama-header .farama-header__right .farama-header__nav{display:flex;height:100%;list-style:none}.farama-header .farama-header__right .farama-header__nav li{align-items:center;cursor:pointer;display:flex;margin-left:20px;text-decoration:none}.farama-header .farama-header__right .farama-header__nav li a{align-items:center;display:flex;height:100%}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container{align-items:center;display:flex;height:100%;position:relative}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container:hover .farama-header__dropdown-menu{display:block}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container svg{fill:var(--color-foreground-primary);width:32px}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container .farama-header__dropdown-menu{background:var(--color-background-hover);border:1px solid var(--color-background-border);display:none;position:absolute;right:0;top:var(--farama-header-height);z-index:9999}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container .farama-header__dropdown-menu ul{display:inherit;margin:0;padding:6px 14px}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container .farama-header__dropdown-menu li{margin:0;padding:6px 0}.farama-header .farama-header__right .farama-header-menu{display:flex;justify-content:center;position:relative}.farama-header .farama-header__right .farama-header-menu .farama-header-menu__btn{background:none;border:none;cursor:pointer;display:flex}.farama-header .farama-header__right .farama-header-menu .farama-header-menu__btn img{width:26px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu__btn svg{stroke:var(--color-foreground-primary);stroke-width:2px;align-self:center;width:14px}.farama-header .farama-header__right .farama-header-menu.active .farama-header-menu-container{transform:translateY(100vh)}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container{background-color:var(--color-background-secondary);border-left:1px solid var(--color-background-border);box-sizing:border-box;height:100%;overflow:auto;position:fixed;right:0;top:-100vh;transform:translateY(0);transition:transform .2s ease-in;width:100%;z-index:99}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header{align-items:center;border-bottom:1px solid var(--color-background-border);box-sizing:border-box;display:flex;margin:0 auto;max-width:1400px;padding:7px 52px;position:relative;width:100%}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header a{align-items:center;display:flex}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header a .farama-header-menu__logo{width:36px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header a span{color:var(--color-sidebar-brand-text);padding-left:8px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header .farama-header-menu-header__right{padding-right:inherit;position:absolute;right:0}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header .farama-header-menu-header__right button{background:none;border:none;cursor:pointer;display:flex}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header .farama-header-menu-header__right button svg{color:var(--color-foreground-primary);width:20px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body{box-sizing:border-box;display:flex;flex-wrap:wrap;margin:0 auto;max-width:1400px;padding:22px 52px;width:100%}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section{margin-bottom:24px;min-width:250px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu__section-title{display:block;font-size:var(--font-size--normal);font-weight:700;margin-bottom:12px;text-transform:uppercase}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list{display:inherit;list-style:none;margin:0;padding:0}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li:hover{background-color:var(--color-background-hover)}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li a{align-items:center;display:flex;padding:12px 18px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li a:hover{color:inherit}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li a img{filter:invert(100%);margin-right:10px;width:24px}.farama-sidebar__title{align-items:center;display:flex;margin-top:var(--sidebar-search-space-above);min-height:calc(52px - var(--sidebar-search-space-above));padding-right:4px}.farama-sidebar__title img{height:calc(var(--farama-header-height) - 20px);margin:var(--farama-header-logo-margin)}.farama-sidebar__title span{color:var(--color-foreground-primary)}.farama-sidebar__title span:hover{text-decoration:none}.sidebar-brand{align-items:center;flex-direction:row;padding:var(--sidebar-item-spacing-vertical)}.sidebar-brand .sidebar-logo-container{display:flex;height:auto;max-width:55px}.sidebar-brand .sidebar-brand-text{font-size:1.3rem;padding-left:11px}.mobile-header .header-center{opacity:0;transition:opacity easy-in .2s}.mobile-header.scrolled .header-center{opacity:1}.sphx-glr-script-out{color:var(--color-foreground-secondary);display:flex;gap:.5em}.sphx-glr-script-out:before{content:"Out:";line-height:1.4;padding-top:10px}.sphx-glr-script-out .highlight{overflow-x:auto}@media(max-width:950px){.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header{padding:7px 42px}.farama-header .farama-header-menu__btn-name{display:none}}@media(max-width:600px){.farama-header{padding:0 4px}.farama-header .farama-header__title{font-size:var(--font-size--small)}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header{padding:8px 12px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body{padding:18px 12px}}@media(max-width:480px){.farama-header .farama-header__title{width:110px}.farama-header .farama-header-menu__btn-name{text-align:right;width:100px}}body[data-theme=dark] .farama-black-logo-invert,body[data-theme=light] .farama-white-logo-invert{filter:invert(1)}body[data-theme=dark] img[src*="//render.githubusercontent.com/render/math"]{filter:invert(90%)} +#furo-sidebar-ad-placement{padding:var(--sidebar-item-spacing-vertical) var(--sidebar-item-spacing-horizontal)}#furo-sidebar-ad-placement .ethical-sidebar{background:var(--color-background-secondary);border:none;box-shadow:none}#furo-sidebar-ad-placement .ethical-sidebar:hover{background:var(--color-background-hover)}#furo-sidebar-ad-placement .ethical-sidebar a{color:var(--color-foreground-primary)}#furo-sidebar-ad-placement .ethical-callout a{color:var(--color-foreground-secondary)!important}#furo-readthedocs-versions{background:transparent;display:block;position:static;width:100%}#furo-readthedocs-versions .rst-versions{background:#1a1c1e}#furo-readthedocs-versions .rst-current-version{background:var(--color-sidebar-item-background);cursor:unset}#furo-readthedocs-versions .rst-current-version:hover{background:var(--color-sidebar-item-background)}#furo-readthedocs-versions .rst-current-version .fa-book{color:var(--color-foreground-primary)}#furo-readthedocs-versions>.rst-other-versions{padding:0}#furo-readthedocs-versions>.rst-other-versions small{opacity:1}#furo-readthedocs-versions .injected .rst-versions{position:unset}#furo-readthedocs-versions:focus-within,#furo-readthedocs-versions:hover{box-shadow:0 0 0 1px var(--color-sidebar-background-border)}#furo-readthedocs-versions:focus-within .rst-current-version,#furo-readthedocs-versions:hover .rst-current-version{background:#1a1c1e;font-size:inherit;height:auto;line-height:inherit;padding:12px;text-align:right}#furo-readthedocs-versions:focus-within .rst-current-version .fa-book,#furo-readthedocs-versions:hover .rst-current-version .fa-book{color:#fff;float:left}#furo-readthedocs-versions:focus-within .fa-caret-down,#furo-readthedocs-versions:hover .fa-caret-down{display:none}#furo-readthedocs-versions:focus-within .injected,#furo-readthedocs-versions:focus-within .rst-current-version,#furo-readthedocs-versions:focus-within .rst-other-versions,#furo-readthedocs-versions:hover .injected,#furo-readthedocs-versions:hover .rst-current-version,#furo-readthedocs-versions:hover .rst-other-versions{display:block}#furo-readthedocs-versions:focus-within>.rst-current-version,#furo-readthedocs-versions:hover>.rst-current-version{display:none}.highlight:hover button.copybtn{color:var(--color-code-foreground)}.highlight button.copybtn{align-items:center;background-color:var(--color-code-background);border:none;color:var(--color-background-item);cursor:pointer;height:1.25em;opacity:1;right:.5rem;top:.625rem;transition:color .3s,opacity .3s;width:1.25em}.highlight button.copybtn:hover{background-color:var(--color-code-background);color:var(--color-brand-content)}.highlight button.copybtn:after{background-color:transparent;color:var(--color-code-foreground);display:none}.highlight button.copybtn.success{color:#22863a;transition:color 0ms}.highlight button.copybtn.success:after{display:block}.highlight button.copybtn svg{padding:0}body{--sd-color-primary:var(--color-brand-primary);--sd-color-primary-highlight:var(--color-brand-content);--sd-color-primary-text:var(--color-background-primary);--sd-color-shadow:rgba(0,0,0,.05);--sd-color-card-border:var(--color-card-border);--sd-color-card-border-hover:var(--color-brand-content);--sd-color-card-background:var(--color-card-background);--sd-color-card-text:var(--color-foreground-primary);--sd-color-card-header:var(--color-card-marginals-background);--sd-color-card-footer:var(--color-card-marginals-background);--sd-color-tabs-label-active:var(--color-brand-content);--sd-color-tabs-label-hover:var(--color-foreground-muted);--sd-color-tabs-label-inactive:var(--color-foreground-muted);--sd-color-tabs-underline-active:var(--color-brand-content);--sd-color-tabs-underline-hover:var(--color-foreground-border);--sd-color-tabs-underline-inactive:var(--color-background-border);--sd-color-tabs-overline:var(--color-background-border);--sd-color-tabs-underline:var(--color-background-border)}.sd-tab-content{box-shadow:0 -2px var(--sd-color-tabs-overline),0 1px var(--sd-color-tabs-underline)}.sd-card{box-shadow:0 .1rem .25rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)}.sd-shadow-sm{box-shadow:0 .1rem .25rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)!important}.sd-shadow-md{box-shadow:0 .3rem .75rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)!important}.sd-shadow-lg{box-shadow:0 .6rem 1.5rem var(--sd-color-shadow),0 0 .0625rem rgba(0,0,0,.1)!important}.sd-card-hover:hover{transform:none}.sd-cards-carousel{gap:.25rem;padding:.25rem}body{--tabs--label-text:var(--color-foreground-muted);--tabs--label-text--hover:var(--color-foreground-muted);--tabs--label-text--active:var(--color-brand-content);--tabs--label-text--active--hover:var(--color-brand-content);--tabs--label-background:transparent;--tabs--label-background--hover:transparent;--tabs--label-background--active:transparent;--tabs--label-background--active--hover:transparent;--tabs--padding-x:0.25em;--tabs--margin-x:1em;--tabs--border:var(--color-background-border);--tabs--label-border:transparent;--tabs--label-border--hover:var(--color-foreground-muted);--tabs--label-border--active:var(--color-brand-content);--tabs--label-border--active--hover:var(--color-brand-content)}[role=main] .container{max-width:none;padding-left:0;padding-right:0}.shadow.docutils{border:none;box-shadow:0 .2rem .5rem rgba(0,0,0,.05),0 0 .0625rem rgba(0,0,0,.1)!important}.sphinx-bs .card{background-color:var(--color-background-secondary);color:var(--color-foreground)}h1{font-size:2.2rem}h2{font-size:1.7rem}h3{font-size:1.4rem}html:has(.farama-header-menu.active){visibility:hidden}.farama-hidden[aria-hidden=true]{visibility:hidden}.farama-hidden[aria-hidden=false]{visibility:visible}.cookie-alert{background-color:var(--color-background-secondary);border-top:1px solid var(--color-background-border);bottom:0;color:var(--color-foreground-primary);display:flex;left:0;min-height:70px;position:fixed;width:100%;z-index:99999}.cookie-alert__container{align-items:center;display:flex;margin:auto;max-width:calc(100% - 28px);width:700px}.cookie-alert p{flex:1}.cookie-alert button{background-color:transparent;border:none;cursor:pointer;height:26px;padding:0;width:26px}.cookie-alert button svg .cls-1{stroke:var(--color-foreground-primary)}.cookie-alert button svg{width:100%}article[role=main]:has(.farama-env-icon-container) .farama-env-icon-container{display:flex;margin-top:7px;position:absolute}article[role=main]:has(.farama-env-icon-container) .section h1:first-child,article[role=main]:has(.farama-env-icon-container) .section h2:first-child,article[role=main]:has(.farama-env-icon-container) section h1:first-child,article[role=main]:has(.farama-env-icon-container) section h2:first-child{margin-left:34px}.farama-env-icon{height:32px}.env-grid{box-sizing:border-box;display:flex;flex-wrap:wrap;justify-content:center;width:100%}.env-grid__cell{display:flex;flex-direction:column;height:180px;padding:10px;width:180px}.cell__image-container{display:flex;height:148px;justify-content:center}.cell__image-container img{max-height:100%;-o-object-fit:contain;object-fit:contain}.cell__title{align-items:flex-end;display:flex;height:32px;justify-content:center;line-height:16px;text-align:center}.more-btn{display:block;margin:12px auto;width:240px}html:has(.farama-header-menu.active){overflow:hidden}body{--farama-header-height:52px;--farama-header-logo-margin:10px}.farama-header{background-color:var(--color-background-secondary);border-bottom:1px solid var(--color-header-border);box-sizing:border-box;display:flex;height:var(--farama-header-height);padding:0 36px 0 24px;position:absolute;width:100%;z-index:95}.farama-header .farama-header__container{display:flex;justify-content:space-between;margin:0 auto;max-width:1400px;width:100%}.farama-header a{color:var(--color-foreground-primary);text-decoration:none;transition:color .125s ease}.farama-header a:hover{color:var(--color-foreground-secondary)}.farama-header .farama-header__logo{margin:var(--farama-header-logo-margin);max-height:calc(var(--farama-header-height) - var(--farama-header-logo-margin))}.farama-header .farama-header__title{align-self:center;font-size:var(--font-size--normal);font-weight:400;margin:0 0 2px;padding:0 0 0 4px}.farama-header .farama-header__left,.farama-header .farama-header__left a{display:flex}.farama-header .farama-header__left--mobile{display:none}.farama-header .farama-header__left--mobile .nav-overlay-icon svg{stroke:var(--color-foreground-primary);fill:var(--color-foreground-primary);stroke-width:2px;padding:0 6px;width:20px}.farama-header .farama-header__right{align-items:center;display:flex;z-index:2}.farama-header .farama-header__right .farama-header__nav{display:flex;height:100%;list-style:none}.farama-header .farama-header__right .farama-header__nav li{align-items:center;cursor:pointer;display:flex;margin-left:20px;text-decoration:none}.farama-header .farama-header__right .farama-header__nav li a{align-items:center;display:flex;height:100%}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container{align-items:center;display:flex;height:100%;position:relative}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container:hover .farama-header__dropdown-menu{display:block}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container svg{fill:var(--color-foreground-primary);width:32px}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container .farama-header__dropdown-menu{background:var(--color-background-hover);border:1px solid var(--color-background-border);display:none;position:absolute;right:0;top:var(--farama-header-height);z-index:9999}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container .farama-header__dropdown-menu ul{display:inherit;margin:0;padding:6px 14px}.farama-header .farama-header__right .farama-header__nav li .farama-header__dropdown-container .farama-header__dropdown-menu li{margin:0;padding:6px 0}.farama-header .farama-header__right .farama-header-menu{display:flex;justify-content:center;position:relative}.farama-header .farama-header__right .farama-header-menu .farama-header-menu__btn{background:none;border:none;cursor:pointer;display:flex}.farama-header .farama-header__right .farama-header-menu .farama-header-menu__btn img{width:26px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu__btn svg{stroke:var(--color-foreground-primary);stroke-width:2px;align-self:center;width:14px}.farama-header .farama-header__right .farama-header-menu.active .farama-header-menu-container{transform:translateY(100vh)}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container{background-color:var(--color-background-secondary);border-left:1px solid var(--color-background-border);box-sizing:border-box;height:100%;overflow:auto;position:fixed;right:0;top:-100vh;transform:translateY(0);transition:transform .2s ease-in;width:100%;z-index:99}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header{align-items:center;border-bottom:1px solid var(--color-background-border);box-sizing:border-box;display:flex;margin:0 auto;max-width:1400px;padding:7px 52px;position:relative;width:100%}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header a{align-items:center;display:flex}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header a .farama-header-menu__logo{width:36px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header a span{color:var(--color-sidebar-brand-text);padding-left:8px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header .farama-header-menu-header__right{padding-right:inherit;position:absolute;right:0}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header .farama-header-menu-header__right button{background:none;border:none;cursor:pointer;display:flex}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header .farama-header-menu-header__right button svg{color:var(--color-foreground-primary);width:20px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body{box-sizing:border-box;display:flex;flex-wrap:wrap;margin:0 auto;max-width:1400px;padding:22px 52px;width:100%}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section{margin-bottom:24px;min-width:250px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu__section-title{display:block;font-size:var(--font-size--normal);font-weight:700;margin-bottom:12px;text-transform:uppercase}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list{display:inherit;list-style:none;margin:0;padding:0}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li:hover{background-color:var(--color-background-hover)}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li a{align-items:center;display:flex;padding:12px 18px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li a:hover{color:inherit}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body .farama-header-menu__section .farama-header-menu-list li a img{margin-right:10px;width:24px}.farama-sidebar__title{align-items:center;display:flex;margin-top:var(--sidebar-search-space-above);min-height:calc(52px - var(--sidebar-search-space-above));padding-right:4px}.farama-sidebar__title img{height:calc(var(--farama-header-height) - 20px);margin:var(--farama-header-logo-margin)}.farama-sidebar__title span{color:var(--color-foreground-primary)}.farama-sidebar__title span:hover{text-decoration:none}.sidebar-brand{align-items:center;flex-direction:row;padding:var(--sidebar-item-spacing-vertical)}.sidebar-brand .sidebar-logo-container{display:flex;height:auto;max-width:55px}.sidebar-brand .sidebar-brand-text{font-size:1.3rem;padding-left:11px}.mobile-header .header-center{opacity:0;transition:opacity easy-in .2s}.mobile-header.scrolled .header-center{opacity:1}.sphx-glr-script-out{color:var(--color-foreground-secondary);display:flex;gap:.5em}.sphx-glr-script-out:before{content:"Out:";line-height:1.4;padding-top:10px}.sphx-glr-script-out .highlight{overflow-x:auto}.sphx-glr-thumbcontainer{z-index:1}div.sphx-glr-download a{background:#0f4a65;box-sizing:border-box;max-width:100%;width:340px}div.sphx-glr-download a:hover{background:#0d3a4e;box-shadow:none}body[data-theme=dark] div.sphx-glr-download a{background:#0f4a65}body[data-theme=dark] div.sphx-glr-download a:hover{background:#0d3a4e}body[data-theme=light] div.sphx-glr-download a{background:#f9d4a1}body[data-theme=light] div.sphx-glr-download a:hover{background:#d9b481}.sphx-glr-thumbcontainer img{background-color:#fff;border-radius:4px}@media(max-width:950px){.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header{padding:7px 42px}.farama-header .farama-header-menu__btn-name{display:none}}@media(max-width:600px){.farama-header{padding:0 4px}.farama-header .farama-header__title{font-size:var(--font-size--small)}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header{padding:8px 12px}.farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__body{padding:18px 12px}}@media(max-width:480px){.farama-header .farama-header__title{width:110px}.farama-header .farama-header-menu__btn-name{text-align:right;width:100px}}body[data-theme=dark] .farama-black-logo-invert,body[data-theme=light] .farama-white-logo-invert{filter:invert(1)}body[data-theme=dark] img[src*="//render.githubusercontent.com/render/math"]{filter:invert(90%)} /*# sourceMappingURL=furo-extensions.css.map*/ \ No newline at end of file diff --git a/_static/styles/furo-extensions.css.map b/_static/styles/furo-extensions.css.map index 66fd043e8..4e982ce3c 100644 --- a/_static/styles/furo-extensions.css.map +++ b/_static/styles/furo-extensions.css.map @@ -1 +1 @@ -{"version":3,"file":"styles/furo-extensions.css","mappings":"AAGA,2BACE,oFACA,4CAKE,6CAHA,YACA,eAEA,CACA,kDACE,yCAEF,8CACE,sCAEJ,8CACE,kDAEJ,2BAGE,uBACA,cAHA,gBACA,UAEA,CAGA,yCACE,mBAEF,gDAEE,gDADA,YACA,CACA,sDACE,gDACF,yDACE,sCAEJ,+CACE,UACA,qDACE,UAGF,mDACE,eAEJ,yEAEE,4DAEA,mHASE,mBAPA,kBAEA,YADA,oBAGA,aADA,gBAIA,CAEA,qIAEE,WADA,UACA,CAEJ,uGACE,aAEF,iUAGE,cAEF,mHACE,aC1EJ,gCACE,mCAEF,0BAKE,mBAUA,8CACA,YAFA,mCAKA,eAZA,cALA,UASA,YADA,YAYA,iCAdA,YAcA,CAEA,gCAEE,8CADA,gCACA,CAEF,gCAGE,6BADA,mCADA,YAEA,CAEF,kCAEE,cADA,oBACA,CACA,wCACE,cAEJ,8BACE,UC5CN,KAEE,6CAA8C,CAC9C,uDAAwD,CACxD,uDAAwD,CAGxD,iCAAsC,CAGtC,+CAAgD,CAChD,uDAAwD,CACxD,uDAAwD,CACxD,oDAAqD,CACrD,6DAA8D,CAC9D,6DAA8D,CAG9D,uDAAwD,CACxD,yDAA0D,CAC1D,4DAA6D,CAC7D,2DAA4D,CAC5D,8DAA+D,CAC/D,iEAAkE,CAClE,uDAAwD,CACxD,wDAAyD,CAG3D,gBACE,qFAGF,SACE,6EAEF,cACE,uFAEF,cACE,uFAEF,cACE,uFAGF,qBACE,eAEF,mBACE,WACA,eChDF,KACE,gDAAiD,CACjD,uDAAwD,CACxD,qDAAsD,CACtD,4DAA6D,CAC7D,oCAAqC,CACrC,2CAA4C,CAC5C,4CAA6C,CAC7C,mDAAoD,CACpD,wBAAyB,CACzB,oBAAqB,CACrB,6CAA8C,CAC9C,gCAAiC,CACjC,yDAA0D,CAC1D,uDAAwD,CACxD,8DAA+D,CCbjE,uBACE,eACA,eACA,gBAGF,iBACE,YACA,+EAGF,iBACE,mDACA,8BCfF,GACI,iBACJ,GACI,iBACJ,GACI,iBAGJ,qCACI,kBAEJ,iCACI,kBAEJ,kCACI,mBAIJ,cAKI,mDAEA,oDACA,SAFA,sCAJA,aAOA,OALA,gBAHA,eAEA,WAOA,cAEJ,yBAEI,mBADA,aAIA,YADA,4BADA,WAEA,CAEJ,gBACI,OAEJ,qBACI,6BACA,YAIA,eAFA,YACA,UAFA,UAGA,CAEJ,gCACI,uCAEJ,yBACI,WAKA,8EAEI,aACA,eAFA,iBAEA,CAEJ,0SACI,iBAER,iBACI,YAIJ,UAKI,sBAJA,aACA,eACA,uBACA,UACA,CAEJ,gBACI,aACA,sBAEA,aACA,aAFA,WAEA,CAEJ,uBACI,aACA,aACA,uBAEJ,2BACI,gBACA,yCAEJ,aAII,qBAHA,aAIA,YAHA,uBAIA,iBAHA,iBAGA,CAEJ,UAGI,cADA,iBADA,WAEA,CAIJ,qCACI,gBAEJ,KACI,2BAA4B,CAC5B,gCAAiC,CAErC,eAMI,mDADA,mDAGA,sBANA,aAEA,mCAGA,sBANA,kBAEA,WAMA,WAEA,yCAGI,aACA,cAFA,iBADA,UAGA,CAEJ,iBACI,sCACA,qBACA,4BAEA,uBACI,wCAER,oCAEI,wCADA,+EACA,CAEJ,qCAKI,kBAJA,mCACA,gBACA,eACA,iBACA,CAKA,0EACI,aAER,4CACI,aAEA,kEAEI,uCACA,qCACA,iBACA,cAJA,UAIA,CAER,qCAEI,mBADA,aAGA,OADA,yBAEA,UAEA,yDACI,aAEA,YADA,eACA,CAEA,4DAII,mBACA,eAFA,aADA,iBADA,oBAIA,CAEA,8DAGI,mBADA,aADA,WAEA,CAEJ,+FAGI,mBADA,aAEA,YAHA,iBAGA,CAGI,mIACI,cAER,mGAEI,qCADA,UACA,CAEJ,6HAKI,yCADA,gDAGA,aANA,kBAEA,QADA,gCAIA,YACA,CAEA,gIACI,gBACA,SACA,iBAEJ,gIACI,SACA,cAEpB,yDAEI,aACA,uBAFA,iBAEA,CAEA,kFAEI,gBACA,YACA,eAHA,YAGA,CAEA,sFACI,WACJ,sFAEI,uCACA,iBACA,kBAHA,UAGA,CAGR,8FACI,4BAEJ,uFAUI,mDACA,qDAHA,sBAFA,YAMA,cAXA,eAEA,QACA,WAGA,wBAEA,iCAJA,WAHA,UAUA,CAEA,mHASI,mBAFA,uDAHA,sBAIA,aAHA,cAFA,iBAGA,iBALA,kBACA,UAOA,CAEA,qHAEI,mBADA,YACA,CAEA,+IACI,WAEJ,0HACI,sCACA,iBAER,qJAGI,sBAFA,kBACA,OACA,CAEA,4JAEI,gBACA,YACA,eAHA,YAGA,CAEA,gKAEI,sCADA,UACA,CAEhB,iHAKI,sBAJA,aAMA,eADA,cAHA,iBACA,kBAFA,UAKA,CAEA,8IAEI,mBADA,eACA,CAEA,iLACI,cACA,mCACA,gBAEA,mBADA,wBACA,CAEJ,uKACI,gBAGA,gBAFA,SACA,SACA,CAII,gLACI,+CAEJ,4KAGI,mBAFA,aACA,iBACA,CAEA,kLACI,cAEJ,gLAGI,oBADA,kBADA,UAEA,CAExC,uBAEI,mBADA,aAIA,6CADA,0DADA,iBAEA,CAEA,2BACI,gDACA,wCACJ,4BACI,sCACA,kCACI,qBAEZ,eAGI,mBAFA,mBACA,4CACA,CAEA,uCACI,aAEA,YADA,cACA,CAEJ,mCACI,iBACA,kBAGJ,8BAEI,SAAQ,CADR,8BACA,CAGJ,uCACI,UAIR,qBACI,wCACA,aACA,SAEJ,4BACI,eACA,gBACA,iBAEJ,gCACI,gBAGJ,wBACI,mHACI,iBAEJ,6CACI,cAGR,wBACI,eACI,cAEA,qCACI,kCAGA,mHACI,iBACJ,iHACI,mBAGhB,wBAEQ,qCACI,YAEJ,6CACI,iBACA,aAUR,iGACI,iBAEJ,6EACI","sources":["webpack:///./src/furo/assets/styles/extensions/_readthedocs.sass","webpack:///./src/furo/assets/styles/extensions/_copybutton.sass","webpack:///./src/furo/assets/styles/extensions/_sphinx-design.sass","webpack:///./src/furo/assets/styles/extensions/_sphinx-inline-tabs.sass","webpack:///./src/furo/assets/styles/extensions/_sphinx-panels.sass","webpack:///./src/furo/assets/styles/extensions/_farama.sass"],"sourcesContent":["// This file contains the styles used for tweaking how ReadTheDoc's embedded\n// contents would show up inside the theme.\n\n#furo-sidebar-ad-placement\n padding: var(--sidebar-item-spacing-vertical) var(--sidebar-item-spacing-horizontal)\n .ethical-sidebar\n // Remove the border and box-shadow.\n border: none\n box-shadow: none\n // Manage the background colors.\n background: var(--color-background-secondary)\n &:hover\n background: var(--color-background-hover)\n // Ensure the text is legible.\n a\n color: var(--color-foreground-primary)\n\n .ethical-callout a\n color: var(--color-foreground-secondary) !important\n\n#furo-readthedocs-versions\n position: static\n width: 100%\n background: transparent\n display: block\n\n // Make the background color fit with the theme's aesthetic.\n .rst-versions\n background: rgb(26, 28, 30)\n\n .rst-current-version\n cursor: unset\n background: var(--color-sidebar-item-background)\n &:hover\n background: var(--color-sidebar-item-background)\n .fa-book\n color: var(--color-foreground-primary)\n\n > .rst-other-versions\n padding: 0\n small\n opacity: 1\n\n .injected\n .rst-versions\n position: unset\n\n &:hover,\n &:focus-within\n box-shadow: 0 0 0 1px var(--color-sidebar-background-border)\n\n .rst-current-version\n // Undo the tweaks done in RTD's CSS\n font-size: inherit\n line-height: inherit\n height: auto\n text-align: right\n padding: 12px\n\n // Match the rest of the body\n background: #1a1c1e\n\n .fa-book\n float: left\n color: white\n\n .fa-caret-down\n display: none\n\n .rst-current-version,\n .rst-other-versions,\n .injected\n display: block\n\n > .rst-current-version\n display: none\n",".highlight\n &:hover button.copybtn\n color: var(--color-code-foreground)\n\n button.copybtn\n // Make it visible\n opacity: 1\n\n // Align things correctly\n align-items: center\n\n height: 1.25em\n width: 1.25em\n\n top: 0.625rem // $code-spacing-vertical\n right: 0.5rem\n\n // Make it look better\n color: var(--color-background-item)\n background-color: var(--color-code-background)\n border: none\n\n // Change to cursor to make it obvious that you can click on it\n cursor: pointer\n\n // Transition smoothly, for aesthetics\n transition: color 300ms, opacity 300ms\n\n &:hover\n color: var(--color-brand-content)\n background-color: var(--color-code-background)\n\n &::after\n display: none\n color: var(--color-code-foreground)\n background-color: transparent\n\n &.success\n transition: color 0ms\n color: #22863a\n &::after\n display: block\n\n svg\n padding: 0\n","body\n // Colors\n --sd-color-primary: var(--color-brand-primary)\n --sd-color-primary-highlight: var(--color-brand-content)\n --sd-color-primary-text: var(--color-background-primary)\n\n // Shadows\n --sd-color-shadow: rgba(0, 0, 0, 0.05)\n\n // Cards\n --sd-color-card-border: var(--color-card-border)\n --sd-color-card-border-hover: var(--color-brand-content)\n --sd-color-card-background: var(--color-card-background)\n --sd-color-card-text: var(--color-foreground-primary)\n --sd-color-card-header: var(--color-card-marginals-background)\n --sd-color-card-footer: var(--color-card-marginals-background)\n\n // Tabs\n --sd-color-tabs-label-active: var(--color-brand-content)\n --sd-color-tabs-label-hover: var(--color-foreground-muted)\n --sd-color-tabs-label-inactive: var(--color-foreground-muted)\n --sd-color-tabs-underline-active: var(--color-brand-content)\n --sd-color-tabs-underline-hover: var(--color-foreground-border)\n --sd-color-tabs-underline-inactive: var(--color-background-border)\n --sd-color-tabs-overline: var(--color-background-border)\n --sd-color-tabs-underline: var(--color-background-border)\n\n// Tabs\n.sd-tab-content\n box-shadow: 0 -2px var(--sd-color-tabs-overline), 0 1px var(--sd-color-tabs-underline)\n\n// Shadows\n.sd-card // Have a shadow by default\n box-shadow: 0 0.1rem 0.25rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1)\n\n.sd-shadow-sm\n box-shadow: 0 0.1rem 0.25rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n.sd-shadow-md\n box-shadow: 0 0.3rem 0.75rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n.sd-shadow-lg\n box-shadow: 0 0.6rem 1.5rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n// Cards\n.sd-card-hover:hover // Don't change scale on hover\n transform: none\n\n.sd-cards-carousel // Have a bit of gap in the carousel by default\n gap: 0.25rem\n padding: 0.25rem\n","// This file contains styles to tweak sphinx-inline-tabs to work well with Furo.\n\nbody\n --tabs--label-text: var(--color-foreground-muted)\n --tabs--label-text--hover: var(--color-foreground-muted)\n --tabs--label-text--active: var(--color-brand-content)\n --tabs--label-text--active--hover: var(--color-brand-content)\n --tabs--label-background: transparent\n --tabs--label-background--hover: transparent\n --tabs--label-background--active: transparent\n --tabs--label-background--active--hover: transparent\n --tabs--padding-x: 0.25em\n --tabs--margin-x: 1em\n --tabs--border: var(--color-background-border)\n --tabs--label-border: transparent\n --tabs--label-border--hover: var(--color-foreground-muted)\n --tabs--label-border--active: var(--color-brand-content)\n --tabs--label-border--active--hover: var(--color-brand-content)\n","// This file contains styles to tweak sphinx-panels to work well with Furo.\n\n// sphinx-panels includes Bootstrap 4, which uses .container which can conflict\n// with docutils' `.. container::` directive.\n[role=\"main\"] .container\n max-width: initial\n padding-left: initial\n padding-right: initial\n\n// Make the panels look nicer!\n.shadow.docutils\n border: none\n box-shadow: 0 0.2rem 0.5rem rgba(0, 0, 0, 0.05), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n// Make panel colors respond to dark mode\n.sphinx-bs .card\n background-color: var(--color-background-secondary)\n color: var(--color-foreground)\n","// Farama Base\n\nh1\n font-size: 2.2rem\nh2\n font-size: 1.7rem\nh3\n font-size: 1.4rem\n\n// If menu is active then all elements except the menu are not visible (i.e. only element with aria-hidden=\"true\")\nhtml:has(.farama-header-menu.active)\n visibility: hidden\n\n.farama-hidden[aria-hidden=\"true\"]\n visibility: hidden\n\n.farama-hidden[aria-hidden=\"false\"]\n visibility: visible\n\n// Cookies Alert\n\n.cookie-alert\n position: fixed\n display: flex\n width: 100%\n min-height: 70px\n background-color: var(--color-background-secondary)\n color: var(--color-foreground-primary)\n border-top: 1px solid var(--color-background-border)\n bottom: 0\n left: 0\n z-index: 99999\n\n.cookie-alert__container\n display: flex\n align-items: center\n width: 700px\n max-width: calc(100% - 28px)\n margin: auto\n\n.cookie-alert p\n flex: 1\n\n.cookie-alert button\n background-color: transparent\n border: none\n width: 26px\n height: 26px\n padding: 0\n cursor: pointer\n\n.cookie-alert button svg .cls-1\n stroke: var(--color-foreground-primary)\n\n.cookie-alert button svg\n width: 100%\n\n// Env Icons\n\narticle[role=main]:has(.farama-env-icon-container)\n .farama-env-icon-container\n position: absolute\n display: flex\n margin-top: 7px\n\n .section h1:first-child, .section h2:first-child, section h1:first-child, section h2:first-child\n margin-left: 34px\n\n.farama-env-icon\n height: 32px\n\n// Envinronments grid\n\n.env-grid\n display: flex\n flex-wrap: wrap\n justify-content: center\n width: 100%\n box-sizing: border-box\n\n.env-grid__cell\n display: flex\n flex-direction: column\n width: 180px\n height: 180px\n padding: 10px\n\n.cell__image-container\n display: flex\n height: 148px\n justify-content: center\n\n.cell__image-container img\n max-height: 100%\n object-fit: contain\n\n.cell__title\n display: flex\n justify-content: center\n text-align: center\n align-items: flex-end\n height: 32px\n line-height: 16px\n\n.more-btn\n width: 240px\n margin: 12px auto\n display: block\n\n// Farama Header\n\nhtml:has(.farama-header-menu.active)\n overflow: hidden\n\nbody\n --farama-header-height: 52px\n --farama-header-logo-margin: 10px\n\n.farama-header\n position: absolute\n display: flex\n width: 100%\n height: var(--farama-header-height)\n border-bottom: 1px solid var(--color-header-border)\n background-color: var(--color-background-secondary)\n padding: 0 36px 0 24px\n box-sizing: border-box\n z-index: 95\n\n .farama-header__container\n width: 100%\n max-width: 1400px\n display: flex\n margin: 0 auto\n\n a\n color: var(--color-foreground-primary)\n text-decoration: none\n transition: color 0.125s ease\n\n &:hover\n color: var(--color-foreground-secondary)\n\n .farama-header__logo\n max-height: calc(var(--farama-header-height) - var(--farama-header-logo-margin))\n margin: var(--farama-header-logo-margin)\n\n .farama-header__title\n font-size: var(--font-size--normal)\n font-weight: normal\n margin: 0 0 2px 0\n padding: 0 0 0 4px\n align-self: center\n\n .farama-header__left\n display: flex\n\n a\n display: flex\n\n .farama-header__left--mobile\n display: none\n\n .nav-overlay-icon svg\n width: 20px\n stroke: var(--color-foreground-primary)\n fill: var(--color-foreground-primary)\n stroke-width: 2px\n padding: 0 6px\n\n .farama-header__right\n display: flex\n align-items: center\n justify-content: flex-end\n flex: 1\n z-index: 2\n\n .farama-header__nav\n display: flex\n list-style: none\n height: 100%\n\n li\n text-decoration: none\n margin-left: 20px\n display: flex\n align-items: center\n cursor: pointer\n\n a\n height: 100%\n display: flex\n align-items: center\n\n .farama-header__dropdown-container\n position: relative\n display: flex\n align-items: center\n height: 100%\n\n &:hover\n .farama-header__dropdown-menu\n display: block\n\n svg\n width: 32px\n fill: var(--color-foreground-primary)\n\n .farama-header__dropdown-menu\n position: absolute\n top: var(--farama-header-height)\n right: 0\n border: 1px solid var(--color-background-border)\n background: var(--color-background-hover)\n z-index: 9999\n display: none\n\n ul\n display: inherit\n margin: 0\n padding: 6px 14px\n\n li\n margin: 0\n padding: 6px 0\n\n .farama-header-menu\n position: relative\n display: flex\n justify-content: center\n\n .farama-header-menu__btn\n display: flex\n background: none\n border: none\n cursor: pointer\n\n img\n width: 26px\n svg\n width: 14px\n stroke: var(--color-foreground-primary)\n stroke-width: 2px\n align-self: center\n\n\n &.active .farama-header-menu-container\n transform: translateY(100vh)\n\n .farama-header-menu-container\n position: fixed\n z-index: 99\n right: 0\n top: -100vh\n width: 100%\n height: calc(100vh - calc(100vh - 100%))\n transform: translateY(0)\n box-sizing: border-box\n transition: transform 0.2s ease-in\n background-color: var(--color-background-secondary)\n border-left: 1px solid var(--color-background-border)\n overflow: auto\n\n .farama-header-menu__header\n position: relative\n width: 100%\n max-width: 1400px\n box-sizing: border-box\n margin: 0 auto\n padding: 7px 52px\n border-bottom: 1px solid var(--color-background-border)\n display: flex\n align-items: center\n\n a\n display: flex\n align-items: center\n\n .farama-header-menu__logo\n width: 36px\n\n span\n color: var(--color-sidebar-brand-text)\n padding-left: 8px\n\n .farama-header-menu-header__right\n position: absolute\n right: 0\n padding-right: inherit\n\n button\n display: flex\n background: none\n border: none\n cursor: pointer\n\n svg\n width: 20px\n color: var(--color-foreground-primary)\n\n .farama-header-menu__body\n display: flex\n width: 100%\n max-width: 1400px\n padding: 22px 52px\n box-sizing: border-box\n margin: 0 auto\n flex-wrap: wrap\n\n .farama-header-menu__section\n min-width: 250px\n margin-bottom: 24px\n\n .farama-header-menu__section-title\n display: block\n font-size: var(--font-size--normal)\n font-weight: bold\n text-transform: uppercase\n margin-bottom: 12px\n\n .farama-header-menu-list\n display: inherit\n margin: 0\n padding: 0\n list-style: none\n\n li\n\n &:hover\n background-color: var(--color-background-hover)\n\n a\n display: flex\n padding: 12px 18px\n align-items: center\n\n &:hover\n color: inherit\n\n img\n width: 24px\n margin-right: 10px\n filter: invert(100%)\n\n.farama-sidebar__title\n display: flex\n align-items: center\n padding-right: 4px\n min-height: calc(52px - var(--sidebar-search-space-above))\n margin-top: var(--sidebar-search-space-above)\n\n img\n height: calc(var(--farama-header-height) - 20px)\n margin: var(--farama-header-logo-margin)\n span\n color: var(--color-foreground-primary)\n &:hover\n text-decoration: none\n\n.sidebar-brand\n flex-direction: row\n padding: var(--sidebar-item-spacing-vertical)\n align-items: center\n\n .sidebar-logo-container\n display: flex\n max-width: 55px\n height: auto\n\n .sidebar-brand-text\n font-size: 1.3rem\n padding-left: 11px\n\n.mobile-header\n .header-center\n transition: opacity 0.2s easy-in\n opacity: 0\n\n.mobile-header.scrolled\n .header-center\n opacity: 1\n\n// Sphinx Gallery\n\n.sphx-glr-script-out\n color: var(--color-foreground-secondary)\n display: flex\n gap: 0.5em\n\n.sphx-glr-script-out::before\n content: \"Out:\"\n line-height: 1.4\n padding-top: 10px\n\n.sphx-glr-script-out .highlight\n overflow-x: auto\n\n\n@media (max-width: 950px)\n .farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header\n padding: 7px 42px\n\n .farama-header .farama-header-menu__btn-name\n display: none\n\n\n@media (max-width: 600px)\n .farama-header\n padding: 0 4px\n\n .farama-header__title\n font-size: var(--font-size--small)\n\n .farama-header__right .farama-header-menu .farama-header-menu-container\n .farama-header-menu__header\n padding: 8px 12px\n .farama-header-menu__body\n padding: 18px 12px\n\n\n@media (max-width: 480px)\n .farama-header\n .farama-header__title\n width: 110px\n\n .farama-header-menu__btn-name\n text-align: right\n width: 100px\n\n\nbody[data-theme=\"light\"]\n .farama-white-logo-invert\n filter: invert(1)\n\n// Github math render\n\nbody[data-theme=\"dark\"]\n .farama-black-logo-invert\n filter: invert(1)\n\n img[src*=\"//render.githubusercontent.com/render/math\"]\n filter: invert(90%)\n"],"names":[],"sourceRoot":""} \ No newline at end of file +{"version":3,"file":"styles/furo-extensions.css","mappings":"AAGA,2BACE,oFACA,4CAKE,6CAHA,YACA,eAEA,CACA,kDACE,yCAEF,8CACE,sCAEJ,8CACE,kDAEJ,2BAGE,uBACA,cAHA,gBACA,UAEA,CAGA,yCACE,mBAEF,gDAEE,gDADA,YACA,CACA,sDACE,gDACF,yDACE,sCAEJ,+CACE,UACA,qDACE,UAGF,mDACE,eAEJ,yEAEE,4DAEA,mHASE,mBAPA,kBAEA,YADA,oBAGA,aADA,gBAIA,CAEA,qIAEE,WADA,UACA,CAEJ,uGACE,aAEF,iUAGE,cAEF,mHACE,aC1EJ,gCACE,mCAEF,0BAKE,mBAUA,8CACA,YAFA,mCAKA,eAZA,cALA,UASA,YADA,YAYA,iCAdA,YAcA,CAEA,gCAEE,8CADA,gCACA,CAEF,gCAGE,6BADA,mCADA,YAEA,CAEF,kCAEE,cADA,oBACA,CACA,wCACE,cAEJ,8BACE,UC5CN,KAEE,6CAA8C,CAC9C,uDAAwD,CACxD,uDAAwD,CAGxD,iCAAsC,CAGtC,+CAAgD,CAChD,uDAAwD,CACxD,uDAAwD,CACxD,oDAAqD,CACrD,6DAA8D,CAC9D,6DAA8D,CAG9D,uDAAwD,CACxD,yDAA0D,CAC1D,4DAA6D,CAC7D,2DAA4D,CAC5D,8DAA+D,CAC/D,iEAAkE,CAClE,uDAAwD,CACxD,wDAAyD,CAG3D,gBACE,qFAGF,SACE,6EAEF,cACE,uFAEF,cACE,uFAEF,cACE,uFAGF,qBACE,eAEF,mBACE,WACA,eChDF,KACE,gDAAiD,CACjD,uDAAwD,CACxD,qDAAsD,CACtD,4DAA6D,CAC7D,oCAAqC,CACrC,2CAA4C,CAC5C,4CAA6C,CAC7C,mDAAoD,CACpD,wBAAyB,CACzB,oBAAqB,CACrB,6CAA8C,CAC9C,gCAAiC,CACjC,yDAA0D,CAC1D,uDAAwD,CACxD,8DAA+D,CCbjE,uBACE,eACA,eACA,gBAGF,iBACE,YACA,+EAGF,iBACE,mDACA,8BCbF,GACI,iBACJ,GACI,iBACJ,GACI,iBAGJ,qCACI,kBAEJ,iCACI,kBAEJ,kCACI,mBAIJ,cAKI,mDAEA,oDACA,SAFA,sCAJA,aAOA,OALA,gBAHA,eAEA,WAOA,cAEJ,yBAEI,mBADA,aAIA,YADA,4BADA,WAEA,CAEJ,gBACI,OAEJ,qBACI,6BACA,YAIA,eAFA,YACA,UAFA,UAGA,CAEJ,gCACI,uCAEJ,yBACI,WAKA,8EAEI,aACA,eAFA,iBAEA,CAEJ,0SACI,iBAER,iBACI,YAIJ,UAKI,sBAJA,aACA,eACA,uBACA,UACA,CAEJ,gBACI,aACA,sBAEA,aACA,aAFA,WAEA,CAEJ,uBACI,aACA,aACA,uBAEJ,2BACI,gBACA,yCAEJ,aAII,qBAHA,aAIA,YAHA,uBAIA,iBAHA,iBAGA,CAEJ,UAGI,cADA,iBADA,WAEA,CAIJ,qCACI,gBAEJ,KACI,2BAA4B,CAC5B,gCAAiC,CAErC,eAMI,mDADA,mDAGA,sBANA,aAEA,mCAGA,sBANA,kBAEA,WAMA,WAEA,yCAGI,aAEA,8BADA,cAFA,iBADA,UAIA,CAEJ,iBACI,sCACA,qBACA,4BAEA,uBACI,wCAER,oCAEI,wCADA,+EACA,CAEJ,qCAKI,kBAJA,mCACA,gBACA,eACA,iBACA,CAKA,0EACI,aAER,4CACI,aAEA,kEAEI,uCACA,qCACA,iBACA,cAJA,UAIA,CAER,qCAEI,mBADA,aAEA,UAEA,yDACI,aAEA,YADA,eACA,CAEA,4DAII,mBACA,eAFA,aADA,iBADA,oBAIA,CAEA,8DAGI,mBADA,aADA,WAEA,CAEJ,+FAGI,mBADA,aAEA,YAHA,iBAGA,CAGI,mIACI,cAER,mGAEI,qCADA,UACA,CAEJ,6HAKI,yCADA,gDAGA,aANA,kBAEA,QADA,gCAIA,YACA,CAEA,gIACI,gBACA,SACA,iBAEJ,gIACI,SACA,cAEpB,yDAEI,aACA,uBAFA,iBAEA,CAEA,kFAEI,gBACA,YACA,eAHA,YAGA,CAEA,sFACI,WACJ,sFAEI,uCACA,iBACA,kBAHA,UAGA,CAGR,8FACI,4BAEJ,uFAUI,mDACA,qDAHA,sBAFA,YAMA,cAXA,eAEA,QACA,WAGA,wBAEA,iCAJA,WAHA,UAUA,CAEA,mHASI,mBAFA,uDAHA,sBAIA,aAHA,cAFA,iBAGA,iBALA,kBACA,UAOA,CAEA,qHAEI,mBADA,YACA,CAEA,+IACI,WAEJ,0HACI,sCACA,iBAER,qJAGI,sBAFA,kBACA,OACA,CAEA,4JAEI,gBACA,YACA,eAHA,YAGA,CAEA,gKAEI,sCADA,UACA,CAEhB,iHAKI,sBAJA,aAMA,eADA,cAHA,iBACA,kBAFA,UAKA,CAEA,8IAEI,mBADA,eACA,CAEA,iLACI,cACA,mCACA,gBAEA,mBADA,wBACA,CAEJ,uKACI,gBAGA,gBAFA,SACA,SACA,CAII,gLACI,+CAEJ,4KAGI,mBAFA,aACA,iBACA,CAEA,kLACI,cAEJ,gLAEI,kBADA,UACA,CAExC,uBAEI,mBADA,aAIA,6CADA,0DADA,iBAEA,CAEA,2BACI,gDACA,wCACJ,4BACI,sCACA,kCACI,qBAEZ,eAGI,mBAFA,mBACA,4CACA,CAEA,uCACI,aAEA,YADA,cACA,CAEJ,mCACI,iBACA,kBAGJ,8BAEI,SAAQ,CADR,8BACA,CAGJ,uCACI,UAIR,qBACI,wCACA,aACA,SAEJ,4BACI,eACA,gBACA,iBAEJ,gCACI,gBAEJ,yBACI,UAEJ,wBAII,mBADA,sBADA,eADA,WAGA,CAEJ,8BACI,mBACA,gBAGA,8CACI,mBACJ,oDACI,mBAGJ,+CACI,mBACJ,qDACI,mBAER,6BACI,sBACA,kBAGJ,wBACI,mHACI,iBAEJ,6CACI,cAGR,wBACI,eACI,cAEA,qCACI,kCAGA,mHACI,iBACJ,iHACI,mBAGhB,wBAEQ,qCACI,YAEJ,6CACI,iBACA,aAUR,iGACI,iBAEJ,6EACI","sources":["webpack:///./src/furo/assets/styles/extensions/_readthedocs.sass","webpack:///./src/furo/assets/styles/extensions/_copybutton.sass","webpack:///./src/furo/assets/styles/extensions/_sphinx-design.sass","webpack:///./src/furo/assets/styles/extensions/_sphinx-inline-tabs.sass","webpack:///./src/furo/assets/styles/extensions/_sphinx-panels.sass","webpack:///./src/furo/assets/styles/extensions/_farama.sass"],"sourcesContent":["// This file contains the styles used for tweaking how ReadTheDoc's embedded\n// contents would show up inside the theme.\n\n#furo-sidebar-ad-placement\n padding: var(--sidebar-item-spacing-vertical) var(--sidebar-item-spacing-horizontal)\n .ethical-sidebar\n // Remove the border and box-shadow.\n border: none\n box-shadow: none\n // Manage the background colors.\n background: var(--color-background-secondary)\n &:hover\n background: var(--color-background-hover)\n // Ensure the text is legible.\n a\n color: var(--color-foreground-primary)\n\n .ethical-callout a\n color: var(--color-foreground-secondary) !important\n\n#furo-readthedocs-versions\n position: static\n width: 100%\n background: transparent\n display: block\n\n // Make the background color fit with the theme's aesthetic.\n .rst-versions\n background: rgb(26, 28, 30)\n\n .rst-current-version\n cursor: unset\n background: var(--color-sidebar-item-background)\n &:hover\n background: var(--color-sidebar-item-background)\n .fa-book\n color: var(--color-foreground-primary)\n\n > .rst-other-versions\n padding: 0\n small\n opacity: 1\n\n .injected\n .rst-versions\n position: unset\n\n &:hover,\n &:focus-within\n box-shadow: 0 0 0 1px var(--color-sidebar-background-border)\n\n .rst-current-version\n // Undo the tweaks done in RTD's CSS\n font-size: inherit\n line-height: inherit\n height: auto\n text-align: right\n padding: 12px\n\n // Match the rest of the body\n background: #1a1c1e\n\n .fa-book\n float: left\n color: white\n\n .fa-caret-down\n display: none\n\n .rst-current-version,\n .rst-other-versions,\n .injected\n display: block\n\n > .rst-current-version\n display: none\n",".highlight\n &:hover button.copybtn\n color: var(--color-code-foreground)\n\n button.copybtn\n // Make it visible\n opacity: 1\n\n // Align things correctly\n align-items: center\n\n height: 1.25em\n width: 1.25em\n\n top: 0.625rem // $code-spacing-vertical\n right: 0.5rem\n\n // Make it look better\n color: var(--color-background-item)\n background-color: var(--color-code-background)\n border: none\n\n // Change to cursor to make it obvious that you can click on it\n cursor: pointer\n\n // Transition smoothly, for aesthetics\n transition: color 300ms, opacity 300ms\n\n &:hover\n color: var(--color-brand-content)\n background-color: var(--color-code-background)\n\n &::after\n display: none\n color: var(--color-code-foreground)\n background-color: transparent\n\n &.success\n transition: color 0ms\n color: #22863a\n &::after\n display: block\n\n svg\n padding: 0\n","body\n // Colors\n --sd-color-primary: var(--color-brand-primary)\n --sd-color-primary-highlight: var(--color-brand-content)\n --sd-color-primary-text: var(--color-background-primary)\n\n // Shadows\n --sd-color-shadow: rgba(0, 0, 0, 0.05)\n\n // Cards\n --sd-color-card-border: var(--color-card-border)\n --sd-color-card-border-hover: var(--color-brand-content)\n --sd-color-card-background: var(--color-card-background)\n --sd-color-card-text: var(--color-foreground-primary)\n --sd-color-card-header: var(--color-card-marginals-background)\n --sd-color-card-footer: var(--color-card-marginals-background)\n\n // Tabs\n --sd-color-tabs-label-active: var(--color-brand-content)\n --sd-color-tabs-label-hover: var(--color-foreground-muted)\n --sd-color-tabs-label-inactive: var(--color-foreground-muted)\n --sd-color-tabs-underline-active: var(--color-brand-content)\n --sd-color-tabs-underline-hover: var(--color-foreground-border)\n --sd-color-tabs-underline-inactive: var(--color-background-border)\n --sd-color-tabs-overline: var(--color-background-border)\n --sd-color-tabs-underline: var(--color-background-border)\n\n// Tabs\n.sd-tab-content\n box-shadow: 0 -2px var(--sd-color-tabs-overline), 0 1px var(--sd-color-tabs-underline)\n\n// Shadows\n.sd-card // Have a shadow by default\n box-shadow: 0 0.1rem 0.25rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1)\n\n.sd-shadow-sm\n box-shadow: 0 0.1rem 0.25rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n.sd-shadow-md\n box-shadow: 0 0.3rem 0.75rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n.sd-shadow-lg\n box-shadow: 0 0.6rem 1.5rem var(--sd-color-shadow), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n// Cards\n.sd-card-hover:hover // Don't change scale on hover\n transform: none\n\n.sd-cards-carousel // Have a bit of gap in the carousel by default\n gap: 0.25rem\n padding: 0.25rem\n","// This file contains styles to tweak sphinx-inline-tabs to work well with Furo.\n\nbody\n --tabs--label-text: var(--color-foreground-muted)\n --tabs--label-text--hover: var(--color-foreground-muted)\n --tabs--label-text--active: var(--color-brand-content)\n --tabs--label-text--active--hover: var(--color-brand-content)\n --tabs--label-background: transparent\n --tabs--label-background--hover: transparent\n --tabs--label-background--active: transparent\n --tabs--label-background--active--hover: transparent\n --tabs--padding-x: 0.25em\n --tabs--margin-x: 1em\n --tabs--border: var(--color-background-border)\n --tabs--label-border: transparent\n --tabs--label-border--hover: var(--color-foreground-muted)\n --tabs--label-border--active: var(--color-brand-content)\n --tabs--label-border--active--hover: var(--color-brand-content)\n","// This file contains styles to tweak sphinx-panels to work well with Furo.\n\n// sphinx-panels includes Bootstrap 4, which uses .container which can conflict\n// with docutils' `.. container::` directive.\n[role=\"main\"] .container\n max-width: initial\n padding-left: initial\n padding-right: initial\n\n// Make the panels look nicer!\n.shadow.docutils\n border: none\n box-shadow: 0 0.2rem 0.5rem rgba(0, 0, 0, 0.05), 0 0 0.0625rem rgba(0, 0, 0, 0.1) !important\n\n// Make panel colors respond to dark mode\n.sphinx-bs .card\n background-color: var(--color-background-secondary)\n color: var(--color-foreground)\n","// Farama Base\n\n$farama-background: linear-gradient(to right top, #765e3e, #054f5b)\n\nh1\n font-size: 2.2rem\nh2\n font-size: 1.7rem\nh3\n font-size: 1.4rem\n\n// If menu is active then all elements except the menu are not visible (i.e. only element with aria-hidden=\"true\")\nhtml:has(.farama-header-menu.active)\n visibility: hidden\n\n.farama-hidden[aria-hidden=\"true\"]\n visibility: hidden\n\n.farama-hidden[aria-hidden=\"false\"]\n visibility: visible\n\n// Cookies Alert\n\n.cookie-alert\n position: fixed\n display: flex\n width: 100%\n min-height: 70px\n background-color: var(--color-background-secondary)\n color: var(--color-foreground-primary)\n border-top: 1px solid var(--color-background-border)\n bottom: 0\n left: 0\n z-index: 99999\n\n.cookie-alert__container\n display: flex\n align-items: center\n width: 700px\n max-width: calc(100% - 28px)\n margin: auto\n\n.cookie-alert p\n flex: 1\n\n.cookie-alert button\n background-color: transparent\n border: none\n width: 26px\n height: 26px\n padding: 0\n cursor: pointer\n\n.cookie-alert button svg .cls-1\n stroke: var(--color-foreground-primary)\n\n.cookie-alert button svg\n width: 100%\n\n// Env Icons\n\narticle[role=main]:has(.farama-env-icon-container)\n .farama-env-icon-container\n position: absolute\n display: flex\n margin-top: 7px\n\n .section h1:first-child, .section h2:first-child, section h1:first-child, section h2:first-child\n margin-left: 34px\n\n.farama-env-icon\n height: 32px\n\n// Envinronments grid\n\n.env-grid\n display: flex\n flex-wrap: wrap\n justify-content: center\n width: 100%\n box-sizing: border-box\n\n.env-grid__cell\n display: flex\n flex-direction: column\n width: 180px\n height: 180px\n padding: 10px\n\n.cell__image-container\n display: flex\n height: 148px\n justify-content: center\n\n.cell__image-container img\n max-height: 100%\n object-fit: contain\n\n.cell__title\n display: flex\n justify-content: center\n text-align: center\n align-items: flex-end\n height: 32px\n line-height: 16px\n\n.more-btn\n width: 240px\n margin: 12px auto\n display: block\n\n// Farama Header\n\nhtml:has(.farama-header-menu.active)\n overflow: hidden\n\nbody\n --farama-header-height: 52px\n --farama-header-logo-margin: 10px\n\n.farama-header\n position: absolute\n display: flex\n width: 100%\n height: var(--farama-header-height)\n border-bottom: 1px solid var(--color-header-border)\n background-color: var(--color-background-secondary)\n padding: 0 36px 0 24px\n box-sizing: border-box\n z-index: 95\n\n .farama-header__container\n width: 100%\n max-width: 1400px\n display: flex\n margin: 0 auto\n justify-content: space-between\n\n a\n color: var(--color-foreground-primary)\n text-decoration: none\n transition: color 0.125s ease\n\n &:hover\n color: var(--color-foreground-secondary)\n\n .farama-header__logo\n max-height: calc(var(--farama-header-height) - var(--farama-header-logo-margin))\n margin: var(--farama-header-logo-margin)\n\n .farama-header__title\n font-size: var(--font-size--normal)\n font-weight: normal\n margin: 0 0 2px 0\n padding: 0 0 0 4px\n align-self: center\n\n .farama-header__left\n display: flex\n\n a\n display: flex\n\n .farama-header__left--mobile\n display: none\n\n .nav-overlay-icon svg\n width: 20px\n stroke: var(--color-foreground-primary)\n fill: var(--color-foreground-primary)\n stroke-width: 2px\n padding: 0 6px\n\n .farama-header__right\n display: flex\n align-items: center\n z-index: 2\n\n .farama-header__nav\n display: flex\n list-style: none\n height: 100%\n\n li\n text-decoration: none\n margin-left: 20px\n display: flex\n align-items: center\n cursor: pointer\n\n a\n height: 100%\n display: flex\n align-items: center\n\n .farama-header__dropdown-container\n position: relative\n display: flex\n align-items: center\n height: 100%\n\n &:hover\n .farama-header__dropdown-menu\n display: block\n\n svg\n width: 32px\n fill: var(--color-foreground-primary)\n\n .farama-header__dropdown-menu\n position: absolute\n top: var(--farama-header-height)\n right: 0\n border: 1px solid var(--color-background-border)\n background: var(--color-background-hover)\n z-index: 9999\n display: none\n\n ul\n display: inherit\n margin: 0\n padding: 6px 14px\n\n li\n margin: 0\n padding: 6px 0\n\n .farama-header-menu\n position: relative\n display: flex\n justify-content: center\n\n .farama-header-menu__btn\n display: flex\n background: none\n border: none\n cursor: pointer\n\n img\n width: 26px\n svg\n width: 14px\n stroke: var(--color-foreground-primary)\n stroke-width: 2px\n align-self: center\n\n\n &.active .farama-header-menu-container\n transform: translateY(100vh)\n\n .farama-header-menu-container\n position: fixed\n z-index: 99\n right: 0\n top: -100vh\n width: 100%\n height: calc(100vh - calc(100vh - 100%))\n transform: translateY(0)\n box-sizing: border-box\n transition: transform 0.2s ease-in\n background-color: var(--color-background-secondary)\n border-left: 1px solid var(--color-background-border)\n overflow: auto\n\n .farama-header-menu__header\n position: relative\n width: 100%\n max-width: 1400px\n box-sizing: border-box\n margin: 0 auto\n padding: 7px 52px\n border-bottom: 1px solid var(--color-background-border)\n display: flex\n align-items: center\n\n a\n display: flex\n align-items: center\n\n .farama-header-menu__logo\n width: 36px\n\n span\n color: var(--color-sidebar-brand-text)\n padding-left: 8px\n\n .farama-header-menu-header__right\n position: absolute\n right: 0\n padding-right: inherit\n\n button\n display: flex\n background: none\n border: none\n cursor: pointer\n\n svg\n width: 20px\n color: var(--color-foreground-primary)\n\n .farama-header-menu__body\n display: flex\n width: 100%\n max-width: 1400px\n padding: 22px 52px\n box-sizing: border-box\n margin: 0 auto\n flex-wrap: wrap\n\n .farama-header-menu__section\n min-width: 250px\n margin-bottom: 24px\n\n .farama-header-menu__section-title\n display: block\n font-size: var(--font-size--normal)\n font-weight: bold\n text-transform: uppercase\n margin-bottom: 12px\n\n .farama-header-menu-list\n display: inherit\n margin: 0\n padding: 0\n list-style: none\n\n li\n\n &:hover\n background-color: var(--color-background-hover)\n\n a\n display: flex\n padding: 12px 18px\n align-items: center\n\n &:hover\n color: inherit\n\n img\n width: 24px\n margin-right: 10px\n\n.farama-sidebar__title\n display: flex\n align-items: center\n padding-right: 4px\n min-height: calc(52px - var(--sidebar-search-space-above))\n margin-top: var(--sidebar-search-space-above)\n\n img\n height: calc(var(--farama-header-height) - 20px)\n margin: var(--farama-header-logo-margin)\n span\n color: var(--color-foreground-primary)\n &:hover\n text-decoration: none\n\n.sidebar-brand\n flex-direction: row\n padding: var(--sidebar-item-spacing-vertical)\n align-items: center\n\n .sidebar-logo-container\n display: flex\n max-width: 55px\n height: auto\n\n .sidebar-brand-text\n font-size: 1.3rem\n padding-left: 11px\n\n.mobile-header\n .header-center\n transition: opacity 0.2s easy-in\n opacity: 0\n\n.mobile-header.scrolled\n .header-center\n opacity: 1\n\n// Sphinx Gallery\n\n.sphx-glr-script-out\n color: var(--color-foreground-secondary)\n display: flex\n gap: 0.5em\n\n.sphx-glr-script-out::before\n content: \"Out:\"\n line-height: 1.4\n padding-top: 10px\n\n.sphx-glr-script-out .highlight\n overflow-x: auto\n\n.sphx-glr-thumbcontainer\n z-index: 1\n\ndiv.sphx-glr-download a\n width: 340px\n max-width: 100%\n box-sizing: border-box\n background: #0f4a65\n\ndiv.sphx-glr-download a:hover\n background: #0d3a4e\n box-shadow: none\n\nbody[data-theme=\"dark\"]\n div.sphx-glr-download a\n background: #0f4a65\n div.sphx-glr-download a:hover\n background: #0d3a4e\n\nbody[data-theme=\"light\"]\n div.sphx-glr-download a\n background: #f9d4a1\n div.sphx-glr-download a:hover\n background: #d9b481\n\n.sphx-glr-thumbcontainer img\n background-color: white\n border-radius: 4px\n\n\n@media (max-width: 950px)\n .farama-header .farama-header__right .farama-header-menu .farama-header-menu-container .farama-header-menu__header\n padding: 7px 42px\n\n .farama-header .farama-header-menu__btn-name\n display: none\n\n\n@media (max-width: 600px)\n .farama-header\n padding: 0 4px\n\n .farama-header__title\n font-size: var(--font-size--small)\n\n .farama-header__right .farama-header-menu .farama-header-menu-container\n .farama-header-menu__header\n padding: 8px 12px\n .farama-header-menu__body\n padding: 18px 12px\n\n\n@media (max-width: 480px)\n .farama-header\n .farama-header__title\n width: 110px\n\n .farama-header-menu__btn-name\n text-align: right\n width: 100px\n\n\nbody[data-theme=\"light\"]\n .farama-white-logo-invert\n filter: invert(1)\n\n// Github math render\n\nbody[data-theme=\"dark\"]\n .farama-black-logo-invert\n filter: invert(1)\n\n img[src*=\"//render.githubusercontent.com/render/math\"]\n filter: invert(90%)\n"],"names":[],"sourceRoot":""} \ No newline at end of file diff --git a/_static/versioning/versioning_menu.html b/_static/versioning/versioning_menu.html index dfb99fbee..2ead34cdf 100644 --- a/_static/versioning/versioning_menu.html +++ b/_static/versioning/versioning_menu.html @@ -113,6 +113,15 @@ menuElem.classList.remove("active"); }); + const sortVersions = (a, b) => { + if (a.replace("v", "") > b.replace("v", "")) + return -1 + else if (a.replace("v", "") < b.replace("v", "")) + return 1 + else + return 0 + } + if (githubUser !== null || repo !== null) { const basePath = ""; const mainBranchName = "main" @@ -125,7 +134,7 @@ } response.json().then(json => { const versions = json.tree.filter(v => versionRegex.test(v.path)).map(v => v.path); - versions.reverse(); + versions.sort(sortVersions); // verify if exists a directory with main version if (json.tree.filter(v => v.path == mainBranchName).length > 0) { versions.unshift(mainBranchName); diff --git a/api/env/index.html b/api/env/index.html index 1894b0b65..2207ade4d 100644 --- a/api/env/index.html +++ b/api/env/index.html @@ -15,7 +15,11 @@ Env - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -766,11 +778,11 @@ For more information see the environment creation tutorial.

const menuBtn = document.querySelector(".farama-header-menu__btn"); const menuContainer = document.querySelector(".farama-header-menu-container"); if (document.querySelector(".farama-header-menu").classList.contains("active")) { - menuBtn.ariaExpanded = "false"; - menuContainer.ariaHidden = "true"; + menuBtn.setAttribute("aria-expanded", "false"); + menuContainer.setAttribute("aria-hidden", "true"); } else { - menuBtn.ariaExpanded = "true"; - menuContainer.ariaHidden = "false"; + menuBtn.setAttribute("aria-expanded", "true"); + menuContainer.setAttribute("aria-hidden", "false"); } document.querySelector(".farama-header-menu").classList.toggle("active"); } @@ -941,7 +953,7 @@ For more information see the environment creation tutorial.

githubUser: 'Farama-Foundation', githubRepo: 'Gymnasium', }; - fetch('/_static/versioning/versioning_menu.html').then(response => { + fetch('/main/_static/versioning/versioning_menu.html').then(response => { if (response.status === 200) { response.text().then(text => { const container = document.createElement("div"); diff --git a/api/experimental/functional/index.html b/api/experimental/functional/index.html index af33a1dea..83ea16aa1 100644 --- a/api/experimental/functional/index.html +++ b/api/experimental/functional/index.html @@ -15,7 +15,11 @@ Functional Environment - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -607,11 +619,11 @@ we intend to flesh it out and officially expose it to end users.

const menuBtn = document.querySelector(".farama-header-menu__btn"); const menuContainer = document.querySelector(".farama-header-menu-container"); if (document.querySelector(".farama-header-menu").classList.contains("active")) { - menuBtn.ariaExpanded = "false"; - menuContainer.ariaHidden = "true"; + menuBtn.setAttribute("aria-expanded", "false"); + menuContainer.setAttribute("aria-hidden", "true"); } else { - menuBtn.ariaExpanded = "true"; - menuContainer.ariaHidden = "false"; + menuBtn.setAttribute("aria-expanded", "true"); + menuContainer.setAttribute("aria-hidden", "false"); } document.querySelector(".farama-header-menu").classList.toggle("active"); } @@ -781,7 +793,7 @@ we intend to flesh it out and officially expose it to end users.

githubUser: 'Farama-Foundation', githubRepo: 'Gymnasium', }; - fetch('/_static/versioning/versioning_menu.html').then(response => { + fetch('/main/_static/versioning/versioning_menu.html').then(response => { if (response.status === 200) { response.text().then(text => { const container = document.createElement("div"); diff --git a/api/experimental/index.html b/api/experimental/index.html index 02c473351..df8f444a6 100644 --- a/api/experimental/index.html +++ b/api/experimental/index.html @@ -15,7 +15,11 @@ Experimental - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -701,11 +713,11 @@ const menuBtn = document.querySelector(".farama-header-menu__btn"); const menuContainer = document.querySelector(".farama-header-menu-container"); if (document.querySelector(".farama-header-menu").classList.contains("active")) { - menuBtn.ariaExpanded = "false"; - menuContainer.ariaHidden = "true"; + menuBtn.setAttribute("aria-expanded", "false"); + menuContainer.setAttribute("aria-hidden", "true"); } else { - menuBtn.ariaExpanded = "true"; - menuContainer.ariaHidden = "false"; + menuBtn.setAttribute("aria-expanded", "true"); + menuContainer.setAttribute("aria-hidden", "false"); } document.querySelector(".farama-header-menu").classList.toggle("active"); } @@ -875,7 +887,7 @@ githubUser: 'Farama-Foundation', githubRepo: 'Gymnasium', }; - fetch('/_static/versioning/versioning_menu.html').then(response => { + fetch('/main/_static/versioning/versioning_menu.html').then(response => { if (response.status === 200) { response.text().then(text => { const container = document.createElement("div"); diff --git a/api/experimental/vector/index.html b/api/experimental/vector/index.html index 93b9575ae..79747cc6d 100644 --- a/api/experimental/vector/index.html +++ b/api/experimental/vector/index.html @@ -15,7 +15,11 @@ Vectorizing Environment - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -518,11 +530,11 @@ const menuBtn = document.querySelector(".farama-header-menu__btn"); const menuContainer = document.querySelector(".farama-header-menu-container"); if (document.querySelector(".farama-header-menu").classList.contains("active")) { - menuBtn.ariaExpanded = "false"; - menuContainer.ariaHidden = "true"; + menuBtn.setAttribute("aria-expanded", "false"); + menuContainer.setAttribute("aria-hidden", "true"); } else { - menuBtn.ariaExpanded = "true"; - menuContainer.ariaHidden = "false"; + menuBtn.setAttribute("aria-expanded", "true"); + menuContainer.setAttribute("aria-hidden", "false"); } document.querySelector(".farama-header-menu").classList.toggle("active"); } @@ -692,7 +704,7 @@ githubUser: 'Farama-Foundation', githubRepo: 'Gymnasium', }; - fetch('/_static/versioning/versioning_menu.html').then(response => { + fetch('/main/_static/versioning/versioning_menu.html').then(response => { if (response.status === 200) { response.text().then(text => { const container = document.createElement("div"); diff --git a/api/experimental/vector_wrappers/index.html b/api/experimental/vector_wrappers/index.html index c26b3bcb4..5630bdf5b 100644 --- a/api/experimental/vector_wrappers/index.html +++ b/api/experimental/vector_wrappers/index.html @@ -15,7 +15,11 @@ Vector Environment Wrappers - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -518,11 +530,11 @@ const menuBtn = document.querySelector(".farama-header-menu__btn"); const menuContainer = document.querySelector(".farama-header-menu-container"); if (document.querySelector(".farama-header-menu").classList.contains("active")) { - menuBtn.ariaExpanded = "false"; - menuContainer.ariaHidden = "true"; + menuBtn.setAttribute("aria-expanded", "false"); + menuContainer.setAttribute("aria-hidden", "true"); } else { - menuBtn.ariaExpanded = "true"; - menuContainer.ariaHidden = "false"; + menuBtn.setAttribute("aria-expanded", "true"); + menuContainer.setAttribute("aria-hidden", "false"); } document.querySelector(".farama-header-menu").classList.toggle("active"); } @@ -692,7 +704,7 @@ githubUser: 'Farama-Foundation', githubRepo: 'Gymnasium', }; - fetch('/_static/versioning/versioning_menu.html').then(response => { + fetch('/main/_static/versioning/versioning_menu.html').then(response => { if (response.status === 200) { response.text().then(text => { const container = document.createElement("div"); diff --git a/api/experimental/wrappers/index.html b/api/experimental/wrappers/index.html index eb952f70f..bde1d288d 100644 --- a/api/experimental/wrappers/index.html +++ b/api/experimental/wrappers/index.html @@ -15,7 +15,11 @@ Wrappers - Gymnasium Documentation - + + + + + @@ -363,15 +367,23 @@

Tutorials

Development

@@ -430,11 +442,13 @@ If the observations from func are outside the bounds of the env spaces, provide a observation_space.

Example

>>> import gymnasium as gym
+>>> from gymnasium.experimental.wrappers import LambdaObservationV0
 >>> import numpy as np
->>> env = gym.make('CartPole-v1')
->>> env = LambdaObservationV0(env, lambda obs: obs + 0.1 * np.random.random(obs.shape))
->>> env.reset()
-array([-0.08319338,  0.04635121, -0.07394746,  0.20877492])
+>>> np.random.seed(0)
+>>> env = gym.make("CartPole-v1")
+>>> env = LambdaObservationV0(env, lambda obs: obs + 0.1 * np.random.random(obs.shape), env.observation_space)
+>>> env.reset(seed=42)  
+(array([ 0.06199517,  0.0511615 , -0.04432538,  0.02694618]), {})

Constructor for the lambda observation wrapper.

@@ -455,17 +469,18 @@ If the observations from Example

>>> import gymnasium as gym
->>> env = gym.wrappers.TransformObservation(
-...     gym.make('CartPole-v1'), lambda obs: {'obs': obs, 'time': 0}
-... )
+>>> from gymnasium.wrappers import TransformObservation
+>>> from gymnasium.experimental.wrappers import FilterObservationV0
+>>> env = gym.make("CartPole-v1")
+>>> env = gym.wrappers.TransformObservation(env, lambda obs: {'obs': obs, 'time': 0})
 >>> env.observation_space = gym.spaces.Dict(obs=env.observation_space, time=gym.spaces.Discrete(1))
->>> env.reset()
-{'obs': array([-0.00067088, -0.01860439,  0.04772898, -0.01911527], dtype=float32), 'time': 0}
+>>> env.reset(seed=42)
+({'obs': array([ 0.0273956 , -0.00611216,  0.03585979,  0.0197368 ], dtype=float32), 'time': 0}, {})
 >>> env = FilterObservationV0(env, filter_keys=['time'])
->>> env.reset()
-{'obs': array([ 0.04560107,  0.04466959, -0.0328232 , -0.02367178], dtype=float32)}
+>>> env.reset(seed=42)
+({'time': 0}, {})
 >>> env.step(0)
-({'obs': array([ 0.04649447, -0.14996664, -0.03329664,  0.25847703], dtype=float32)}, 1.0, False, {})
+({'time': 0}, 1.0, False, False, {})

Constructor for an environment with a dictionary observation space where all filter_keys are in the observation space keys.

@@ -477,13 +492,14 @@ If the observations from Example

>>> import gymnasium as gym
->>> env = gym.make('CarRacing-v1')
+>>> from gymnasium.experimental.wrappers import FlattenObservationV0
+>>> env = gym.make("CarRacing-v2")
 >>> env.observation_space.shape
 (96, 96, 3)
 >>> env = FlattenObservationV0(env)
 >>> env.observation_space.shape
 (27648,)
->>> obs, info = env.reset()
+>>> obs, _ = env.reset()
 >>> obs.shape
 (27648,)
@@ -498,7 +514,8 @@ If the observations from keep_dim will keep the channel dimension

Example

>>> import gymnasium as gym
->>> env = gym.make("CarRacing-v1")
+>>> from gymnasium.experimental.wrappers import GrayscaleObservationV0
+>>> env = gym.make("CarRacing-v2")
 >>> env.observation_space.shape
 (96, 96, 3)
 >>> grayscale_env = GrayscaleObservationV0(env)
@@ -518,6 +535,7 @@ If the observations from Example

 
>>> import gymnasium as gym
+>>> from gymnasium.experimental.wrappers import ResizeObservationV0
 >>> env = gym.make("CarRacing-v2")
 >>> env.observation_space.shape
 (96, 96, 3)
@@ -535,7 +553,8 @@ If the observations from Example

 
>>> import gymnasium as gym
->>> env = gym.make("CarRacing-v1")
+>>> from gymnasium.experimental.wrappers import ReshapeObservationV0
+>>> env = gym.make("CarRacing-v2")
 >>> env.observation_space.shape
 (96, 96, 3)
 >>> reshape_env = ReshapeObservationV0(env, (24, 4, 96, 1, 3))
@@ -552,11 +571,13 @@ If the observations from Example

 
>>> import gymnasium as gym
+>>> from gymnasium.experimental.wrappers import RescaleObservationV0
 >>> env = gym.make("Pendulum-v1")
 >>> env.observation_space
 Box([-1. -1. -8.], [1. 1. 8.], (3,), float32)
 >>> env = RescaleObservationV0(env, np.array([-2, -1, -10]), np.array([1, 0, 1]))
-Box([-2. -1. -10.], [1. 0. 1.], (3,), float32)
+>>> env.observation_space
+Box([ -2.  -1. -10.], [1. 0. 1.], (3,), float32)
 

 

 
Constructor that requires the env observation spaces to be a Box.

@@ -628,25 +649,27 @@ as the final element in the tuple.

 
Example

 
>>> import gymnasium as gym
 >>> from gymnasium.experimental.wrappers import TimeAwareObservationV0
->>> env = gym.make('CartPole-v1')
+>>> env = gym.make("CartPole-v1")
 >>> env = TimeAwareObservationV0(env)
 >>> env.observation_space
-Dict(obs: Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), time: Box(0.0, 500, (1,), float32))
->>> _ = env.reset()
+Dict('obs': Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38], (4,), float32), 'time': Box(0.0, 1.0, (1,), float32))
+>>> _ = env.reset(seed=42)
+>>> _ = env.action_space.seed(42)
 >>> env.step(env.action_space.sample())[0]
-OrderedDict([('obs',
-...       array([ 0.02866629,  0.2310988 , -0.02614601, -0.2600732 ], dtype=float32)),
-...      ('time', array([0.002]))])
+{'obs': array([ 0.02727336, -0.20172954,  0.03625453,  0.32351476], dtype=float32), 'time': 0.002}
 

 

 

-
Flatten observation space example:
>>> env = gym.make('CartPole-v1')
+
Flatten observation space example:
>>> env = gym.make("CartPole-v1")
 >>> env = TimeAwareObservationV0(env, flatten=True)
 >>> env.observation_space
-Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38  0.0000000e+00], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38 500], (5,), float32)
->>> _ = env.reset()
+Box([-4.8000002e+00 -3.4028235e+38 -4.1887903e-01 -3.4028235e+38
+  0.0000000e+00], [4.8000002e+00 3.4028235e+38 4.1887903e-01 3.4028235e+38 1.0000000e+00], (5,), float32)
+>>> _ = env.reset(seed=42)
+>>> _ = env.action_space.seed(42)
 >>> env.step(env.action_space.sample())[0]
-array([-0.01232257,  0.19335455, -0.02244143, -0.32388705,  0.002 ], dtype=float32)
+array([ 0.02727336, -0.20172954,  0.03625453,  0.32351476,  0.002     ],
+      dtype=float32)
 

 

 

@@ -682,11 +705,12 @@ I.e. the observation returned by Example

 
>>> import gymnasium as gym
->>> env = gym.make('CarRacing-v1')
->>> env = FrameStack(env, 4)
+>>> from gymnasium.experimental.wrappers import FrameStackObservationV0
+>>> env = gym.make("CarRacing-v2")
+>>> env = FrameStackObservationV0(env, 4)
 >>> env.observation_space
-Box(4, 96, 96, 3)
->>> obs = env.reset()
+Box(0, 255, (4, 96, 96, 3), uint8)
+>>> obs, _ = env.reset()
 >>> obs.shape
 (4, 96, 96, 3)
 

@@ -792,13 +816,15 @@ optimization benefits of FrameStack Wrapper.

 
Clip the continuous action within the valid Box observation space bound.

 
Example

 
>>> import gymnasium as gym
+>>> from gymnasium.experimental.wrappers import ClipActionV0
 >>> import numpy as np
->>> env = gym.make('BipedalWalker-v3', disable_env_checker=True)
+>>> env = gym.make("Hopper-v4", disable_env_checker=True)
 >>> env = ClipActionV0(env)
 >>> env.action_space
-Box(-1.0, 1.0, (4,), float32)
->>> env.step(np.array([5.0, 2.0, -10.0, 0.0]))
-# Executes the action np.array([1.0, 1.0, -1.0, 0]) in the base environment
+Box(-inf, inf, (3,), float32)
+>>> _ = env.reset(seed=42)
+>>> _ = env.step(np.array([5.0, -2.0, 0.0]))
+... # Executes the action np.array([1.0, -1.0, 0]) in the base environment
 

 

 
A wrapper for clipping continuous actions within the valid bound.

@@ -817,13 +843,14 @@ optimization benefits of FrameStack Wrapper.

 or max_action are numpy arrays, the shape must match the shape of the environment’s action space.

 
Example

 
>>> import gymnasium as gym
+>>> from gymnasium.experimental.wrappers import RescaleActionV0
 >>> import numpy as np
->>> env = gym.make('BipedalWalker-v3', disable_env_checker=True)
+>>> env = gym.make("Hopper-v4", disable_env_checker=True)
 >>> _ = env.reset(seed=42)
->>> obs, _, _, _, _ = env.step(np.array([1,1,1,1]))
+>>> obs, _, _, _, _ = env.step(np.array([1,1,1]))
 >>> _ = env.reset(seed=42)
 >>> min_action = -0.5
->>> max_action = np.array([0.0, 0.5, 1.0, 0.75])
+>>> max_action = np.array([0.0, 0.5, 0.75])
 >>> wrapped_env = RescaleActionV0(env, min_action=min_action, max_action=max_action)
 >>> wrapped_env_obs, _, _, _, _ = wrapped_env.step(max_action)
 >>> np.alltrue(obs == wrapped_env_obs)
@@ -897,7 +924,7 @@ in Section 5.2 on page 12.

 >>> from gymnasium.experimental.wrappers import ClipRewardV0
 >>> env = gym.make("CartPole-v1")
 >>> env = ClipRewardV0(env, 0, 0.5)
->>> env.reset()
+>>> _ = env.reset()
 >>> _, rew, _, _, _ = env.step(1)
 >>> rew
 0.5
@@ -922,6 +949,9 @@ in Section 5.2 on page 12.

 class gymnasium.experimental.wrappers.NormalizeRewardV0(env: Env, gamma: float = 0.99, epsilon: float = 1e-8)#
 
This wrapper will normalize immediate rewards s.t. their exponential moving average has a fixed variance.

 
The exponential moving average will have variance \((1 - \gamma)^2\).

+
The property _update_running_mean allows to freeze/continue the running mean calculation of the reward
+statistics. If True (default), the RunningMeanStd will get updated every time self.normalize() is called.
+If False, the calculated statistics are used but not updated anymore; this may be used during evaluation.

 

 
Note

 
The scaling depends on past trajectories and rewards will not be scaled correctly if the wrapper was newly
@@ -966,16 +996,17 @@ instantiated or the policy was changed recently.

 class gymnasium.experimental.wrappers.OrderEnforcingV0(env: Env, disable_render_order_enforcing: bool = False)#
 
A wrapper that will produce an error if step() is called before an initial reset().

 
Example

-
>>> from gymnasium.envs.classic_control import CartPoleEnv
->>> env = CartPoleEnv()
+
>>> import gymnasium as gym
+>>> from gymnasium.experimental.wrappers import OrderEnforcingV0
+>>> env = gym.make("CartPole-v1", render_mode="human")
 >>> env = OrderEnforcingV0(env)
->>> env.step(0)
-ResetNeeded: Cannot call env.step() before calling env.reset()
+>>> env.step(0) 
+gymnasium.error.ResetNeeded: Cannot call env.step() before calling env.reset()
+>>> env.render() 
+gymnasium.error.ResetNeeded('Cannot call `env.render()` before calling `env.reset()`, if this is a intended action, set `disable_render_order_enforcing=True` on the OrderEnforcer wrapper.')
+>>> _ = env.reset()
 >>> env.render()
-ResetNeeded: Cannot call env.render() before calling env.reset()
->>> env.reset()
->>> env.render()
->>> env.step(0)
+>>> _ = env.step(0)
 

 

 
A wrapper that will produce an error if step() is called before an initial reset().

@@ -999,7 +1030,6 @@ using the key episo
 the episode statistics.

 
After the completion of an episode, info will look like this:

 
>>> info = {
-...     ...
 ...     "episode": {
 ...         "r": "<cumulative reward>",
 ...         "l": "<episode length>",
@@ -1010,7 +1040,10 @@ the episode statistics.

 

 
For a vectorized environments the output will be in the form of:

 
>>> infos = {
-...     ...
+...     "final_observation": "<array of length num-envs>",
+...     "_final_observation": "<boolean array of length num-envs>",
+...     "final_info": "<array of length num-envs>",
+...     "_final_info": "<boolean array of length num-envs>",
 ...     "episode": {
 ...         "r": "<array of cumulative reward>",
 ...         "l": "<array of episode length>",
@@ -1047,12 +1080,32 @@ the episode statistics.

 
Rendering Wrappers#

 

 

-class gymnasium.experimental.wrappers.RecordVideoV0(env: Env[ObsType, ActType])#

-
Record a video of an environment.

-
Wraps an environment to allow a modular transformation of the step() and reset() methods.

+class gymnasium.experimental.wrappers.RecordVideoV0(env: Env, video_folder: str, episode_trigger: Optional[Callable[[int], bool]] = None, step_trigger: Optional[Callable[[int], bool]] = None, video_length: int = 0, name_prefix: str = 'rl-video', disable_logger: bool = False)#
+
This wrapper records videos of rollouts.

+
Usually, you only want to record episodes intermittently, say every hundredth episode.
+To do this, you can specify episode_trigger or step_trigger.
+They should be functions returning a boolean that indicates whether a recording should be started at the
+current episode or step, respectively.
+If neither episode_trigger nor step_trigger is passed, a default episode_trigger will be employed,
+i.e. capped_cubic_video_schedule. This function starts a video at every episode that is a power of 3 until 1000 and
+then every 1000 episodes.
+By default, the recording will be stopped once reset is called. However, you can also create recordings of fixed
+length (possibly spanning several episodes) by passing a strictly positive value for video_length.
+This wrapper uses the value fps from metadata as the number of frames per second;
+if fps is not defined in metadata, the default value 30 is used.

+
Wrapper records videos of rollouts.

 

 
Parameters:

-
env – The environment to wrap

+
    
+
  • env – The environment that will be wrapped
    • 
+
  • video_folder (str) – The folder where the recordings will be stored
    • 
+
  • episode_trigger – Function that accepts an integer and returns True iff a recording should be started at this episode
    • 
+
  • step_trigger – Function that accepts an integer and returns True iff a recording should be started at this step
    • 
+
  • video_length (int) – The length of recorded episodes. If 0, entire episodes are recorded.
+Otherwise, snippets of the specified length are captured
    • 
+
  • name_prefix (str) – Will be prepended to the filename of the recordings
    • 
+
  • disable_logger (bool) – Whether to disable moviepy logger or not
    • 
+

 

 

 

@@ -1067,26 +1120,28 @@ If you want to use this wrapper with your environments, remember to specify 
 
The render_mode of the wrapped environment must be either 'rgb_array' or 'rgb_array_list'.

 
Example

-
>>> env = gym.make("LunarLander-v2", render_mode="rgb_array")
+
>>> import gymnasium as gym
+>>> from gymnasium.experimental.wrappers import HumanRenderingV0
+>>> env = gym.make("LunarLander-v2", render_mode="rgb_array")
 >>> wrapped = HumanRenderingV0(env)
->>> wrapped.reset()     # This will start rendering to the screen
+>>> obs, _ = wrapped.reset()     # This will start rendering to the screen
 

 

 
The wrapper can also be applied directly when the environment is instantiated, simply by passing
 render_mode="human" to make. The wrapper will only be applied if the environment does not
 implement human-rendering natively (i.e. render_mode does not contain "human").

 
Example

-
>>> env = gym.make("NoNativeRendering-v2", render_mode="human")      # NoNativeRendering-v0 doesn't implement human-rendering natively
->>> env.reset()     # This will start rendering to the screen
+
>>> env = gym.make("CartPoleJax-v1", render_mode="human")      # CartPoleJax-v1 doesn't implement human-rendering natively
+>>> obs, _ = env.reset()     # This will start rendering to the screen
 

 

 

 
Warning: If the base environment uses render_mode="rgb_array_list", its (i.e. the base environment’s) render method
will always return an empty list:

 
>>> env = gym.make("LunarLander-v2", render_mode="rgb_array_list")
 >>> wrapped = HumanRenderingV0(env)
->>> wrapped.reset()
->>> env.render()
-[]          # env.render() will always return an empty list!
+>>> obs, _ = wrapped.reset()
+>>> env.render() # env.render() will always return an empty list!
+[]
 

 

 

@@ -1302,11 +1357,11 @@ The reason for this is jax does not support non-array values, therefore numpy  {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/registry/index.html b/api/registry/index.html
index c62da8626..f069d6683 100644
--- a/api/registry/index.html
+++ b/api/registry/index.html
@@ -15,7 +15,11 @@
         Registry - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -641,11 +653,11 @@ instead of printing it to console.

         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -815,7 +827,7 @@ instead of printing it to console.

         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/spaces/composite/index.html b/api/spaces/composite/index.html
index 6bb66213c..b38c45ce6 100644
--- a/api/spaces/composite/index.html
+++ b/api/spaces/composite/index.html
@@ -15,7 +15,11 @@
         Composite Spaces - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -429,14 +441,14 @@
 
Elements of this space are (ordered) dictionaries of elements from the constituent spaces.

 
Example usage:

 
>>> from gymnasium.spaces import Dict, Discrete
->>> observation_space = Dict({"position": Discrete(2), "velocity": Discrete(3)})
+>>> observation_space = Dict({"position": Discrete(2), "velocity": Discrete(3)}, seed=42)
 >>> observation_space.sample()
-OrderedDict([('position', 1), ('velocity', 2)])
+OrderedDict([('position', 0), ('velocity', 2)])
 

 

 
Example usage [nested]:

 
>>> from gymnasium.spaces import Box, Dict, Discrete, MultiBinary, MultiDiscrete
->>> Dict(
+>>> Dict(  
 ...     {
 ...         "ext_controller": MultiDiscrete([5, 2, 2]),
 ...         "inner_state": Dict(
@@ -466,9 +478,9 @@ keyword arguments (where you will need to avoid the keys 
>>> from gymnasium.spaces import Box, Discrete
 >>> Dict({"position": Box(-1, 1, shape=(2,)), "color": Discrete(3)})
-Dict(color:Discrete(3), position:Box(-1.0, 1.0, (2,), float32))
+Dict('color': Discrete(3), 'position': Box(-1.0, 1.0, (2,), float32))
 >>> Dict(position=Box(-1, 1, shape=(2,)), color=Discrete(3))
-Dict(color:Discrete(3), position:Box(-1.0, 1.0, (2,), float32))
+Dict('position': Box(-1.0, 1.0, (2,), float32), 'color': Discrete(3))
 

 

 

@@ -524,9 +536,9 @@ keyword arguments (where you will need to avoid the keys 
>>> from gymnasium.spaces import Box, Discrete
->>> observation_space = Tuple((Discrete(2), Box(-1, 1, shape=(2,))))
+>>> observation_space = Tuple((Discrete(2), Box(-1, 1, shape=(2,))), seed=42)
 >>> observation_space.sample()
-(0, array([0.03633198, 0.42370757], dtype=float32))
+(0, array([-0.3991573 ,  0.21649833], dtype=float32))
 

 

 
Constructor of Tuple space.

@@ -585,11 +597,11 @@ expects the same number of masks as spaces

 to some space that is specified during initialization and the integer \(n\) is not fixed

 

 
Example::
>>> from gymnasium.spaces import Box
->>> space = Sequence(Box(0, 1))
->>> space.sample()
-(array([0.0259352], dtype=float32),)
->>> space.sample()
-(array([0.80977976], dtype=float32), array([0.80066574], dtype=float32), array([0.77165383], dtype=float32))
+>>> space = Sequence(Box(0, 1), seed=42)
+>>> space.sample()   
+(array([0.6369617], dtype=float32),)
+>>> space.sample()   
+(array([0.01652764], dtype=float32), array([0.8132702], dtype=float32),)
 

 

 

@@ -799,11 +811,11 @@ If no num_edges is provided then the edge_mask is mult
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -974,7 +986,7 @@ If no num_edges is provided then the edge_mask is mult
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/spaces/fundamental/index.html b/api/spaces/fundamental/index.html
index 2838f95d6..02ac86a79 100644
--- a/api/spaces/fundamental/index.html
+++ b/api/spaces/fundamental/index.html
@@ -15,7 +15,11 @@
         Fundamental Spaces - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -433,13 +445,13 @@ Each interval has the form of one of 
 
    
 
  • Identical bound for each dimension:
    
 
    >>> Box(low=-1.0, high=2.0, shape=(3, 4), dtype=np.float32)
-Box(3, 4)
+Box(-1.0, 2.0, (3, 4), float32)
 
    
 
    
 
    • 
 
  • Independent bound for each dimension:
    
 
    >>> Box(low=np.array([-1.0, -2.0]), high=np.array([2.0, 4.0]), dtype=np.float32)
-Box(2,)
+Box([-1. -2.], [2. 4.], (2,), float32)
 
    
 
    
 
    • 
@@ -522,7 +534,9 @@ that is chosen according to the form of the interval:
    
 
    This class represents a finite subset of integers, more specifically a set of the form \(\{ a, a+1, \dots, a+n-1 \}\).
    
 
    Example:
    
 
    >>> Discrete(2)            # {0, 1}
+Discrete(2)
 >>> Discrete(3, start=-1)  # {-1, 0, 1}
+Discrete(3, start=-1)
 
    
 
    
 
    Constructor of Discrete space.
    
@@ -570,14 +584,14 @@ If there are no possible actions (i.e. np.all(mask == 0)) then 
    An n-shape binary space.
    
 
    Elements of this space are binary arrays of a shape that is fixed during construction.
    
 
    Example Usage:
    
-
    >>> observation_space = MultiBinary(5)
+
    >>> observation_space = MultiBinary(5, seed=42)
 >>> observation_space.sample()
-    array([0, 1, 0, 1, 0], dtype=int8)
->>> observation_space = MultiBinary([3, 2])
+array([1, 0, 1, 0, 1], dtype=int8)
+>>> observation_space = MultiBinary([3, 2], seed=42)
 >>> observation_space.sample()
-    array([[0, 0],
-        [0, 1],
-        [1, 1]], dtype=int8)
+array([[1, 0],
+       [1, 0],
+       [1, 1]], dtype=int8)
 
    
 
    
 
    Constructor of MultiBinary space.
    
@@ -637,7 +651,7 @@ The expected mask shape is the space shape and mask dtype is np.int8Although this feature is rarely used, MultiDiscrete spaces may also have several axes
 if nvec has several axes:
    
 
    Example:
    
-
    >> d = MultiDiscrete(np.array([[1, 2], [3, 4]]))
+
    >> d = MultiDiscrete(np.array([[1, 2], [3, 4]]), seed=42)
 >> d.sample()
 array([[0, 0],
        [2, 3]])
@@ -689,11 +703,13 @@ Only mask values == 1 are possible to sample unless all mask values for an actio
 
    
 
    Example::
    >>> # {"", "B5", "hello", ...}
 >>> Text(5)
+Text(1, 5, characters=0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz)
 >>> # {"0", "42", "0123456789", ...}
 >>> import string
 >>> Text(min_length = 1,
 ...      max_length = 10,
 ...      charset = string.digits)
+Text(1, 10, characters=0123456789)
 
    
 
    
 
    
@@ -855,11 +871,11 @@ If the charlist mask is all zero then an empty string is returned no matter the
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1030,7 +1046,7 @@ If the charlist mask is all zero then an empty string is returned no matter the
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/spaces/index.html b/api/spaces/index.html
index c98b21b9a..9429f62be 100644
--- a/api/spaces/index.html
+++ b/api/spaces/index.html
@@ -15,7 +15,11 @@
         Spaces - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 

 
Tutorials

 
 
Development

 
 
@@ -683,11 +695,11 @@ Therefore, Gymnasium provides a number of additional functions used when using a
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -857,7 +869,7 @@ Therefore, Gymnasium provides a number of additional functions used when using a
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/spaces/utils/index.html b/api/spaces/utils/index.html
index 77d7235ad..75a627850 100644
--- a/api/spaces/utils/index.html
+++ b/api/spaces/utils/index.html
@@ -15,7 +15,11 @@
         Spaces Utils - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -461,9 +473,9 @@ errors or non-uniform sampling.

 
Example::
>>> from gymnasium.spaces import Box
 >>> box = Box(0.0, 1.0, shape=(3, 4, 5))
 >>> box
-Box(3, 4, 5)
+Box(0.0, 1.0, (3, 4, 5), float32)
 >>> flatten_space(box)
-Box(60,)
+Box(0.0, 1.0, (60,), float32)
 >>> flatten(box, box.sample()) in flatten_space(box)
 True
 

@@ -472,7 +484,7 @@ errors or non-uniform sampling.

 
Example that flattens a discrete space::
>>> from gymnasium.spaces import Discrete
 >>> discrete = Discrete(5)
 >>> flatten_space(discrete)
-Box(5,)
+Box(0, 1, (5,), int64)
 >>> flatten(box, box.sample()) in flatten_space(box)
 True
 

@@ -481,7 +493,7 @@ errors or non-uniform sampling.

 
Example that recursively flattens a dict::
>>> from gymnasium.spaces import Dict, Discrete, Box
 >>> space = Dict({"position": Discrete(2), "velocity": Box(0, 1, shape=(2, 2))})
 >>> flatten_space(space)
-Box(6,)
+Box(0.0, 1.0, (6,), float64)
 >>> flatten(space, space.sample()) in flatten_space(space)
 True
 

@@ -758,11 +770,11 @@ that the spacespace {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/spaces/vector_utils/index.html b/api/spaces/vector_utils/index.html
index 5e5d0e04b..23dd4aa4b 100644
--- a/api/spaces/vector_utils/index.html
+++ b/api/spaces/vector_utils/index.html
@@ -15,7 +15,11 @@
         Spaces Vector Utils - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -440,12 +452,13 @@
 
Create a (batched) space, containing multiple copies of a single space.

 
Example:

 
>>> from gymnasium.spaces import Box, Dict
+>>> import numpy as np
 >>> space = Dict({
 ...     'position': Box(low=0, high=1, shape=(3,), dtype=np.float32),
 ...     'velocity': Box(low=0, high=1, shape=(2,), dtype=np.float32)
 ... })
 >>> batch_space(space, n=5)
-Dict(position:Box(5, 3), velocity:Box(5, 2))
+Dict('position': Box(0.0, 1.0, (5, 3), float32), 'velocity': Box(0.0, 1.0, (5, 2), float32))
 

 

 

@@ -484,12 +497,13 @@
 
Concatenate multiple samples from space into a single object.

 
Example:

 
>>> from gymnasium.spaces import Box
->>> space = Box(low=0, high=1, shape=(3,), dtype=np.float32)
+>>> import numpy as np
+>>> space = Box(low=0, high=1, shape=(3,), seed=42, dtype=np.float32)
 >>> out = np.zeros((2, 3), dtype=np.float32)
 >>> items = [space.sample() for _ in range(2)]
 >>> concatenate(space, items, out)
-array([[0.6348213 , 0.28607962, 0.60760117],
-       [0.87383074, 0.192658  , 0.2148103 ]], dtype=float32)
+array([[0.77395606, 0.43887845, 0.85859793],
+       [0.697368  , 0.09417735, 0.97562236]], dtype=float32)
 

 

 

@@ -529,18 +543,17 @@
 
Iterate over the elements of a (batched) space.

 
Example:

 
>>> from gymnasium.spaces import Box, Dict
+>>> import numpy as np
 >>> space = Dict({
-... 'position': Box(low=0, high=1, shape=(2, 3), dtype=np.float32),
-... 'velocity': Box(low=0, high=1, shape=(2, 2), dtype=np.float32)})
+... 'position': Box(low=0, high=1, shape=(2, 3), seed=42, dtype=np.float32),
+... 'velocity': Box(low=0, high=1, shape=(2, 2), seed=42, dtype=np.float32)})
 >>> items = space.sample()
 >>> it = iterate(space, items)
 >>> next(it)
-{'position': array([-0.99644893, -0.08304597, -0.7238421 ], dtype=float32),
-'velocity': array([0.35848552, 0.1533453 ], dtype=float32)}
->>> next(it)
-{'position': array([-0.67958736, -0.49076623,  0.38661423], dtype=float32),
-'velocity': array([0.7975036 , 0.93317133], dtype=float32)}
+OrderedDict([('position', array([0.77395606, 0.43887845, 0.85859793], dtype=float32)), ('velocity', array([0.77395606, 0.43887845], dtype=float32))])
 >>> next(it)
+OrderedDict([('position', array([0.697368  , 0.09417735, 0.97562236], dtype=float32)), ('velocity', array([0.85859793, 0.697368  ], dtype=float32))])
+>>> next(it) 
 StopIteration
 

 

@@ -564,7 +577,7 @@
 
Shared Memory Utils#

 

 

-gymnasium.vector.utils.create_empty_array(space: Space, n: int = 1, fn: callable = np.zeros) → Union[tuple, dict, ndarray]#

+gymnasium.vector.utils.create_empty_array(space: Space, n: int = 1, fn: Callable[[...], ndarray] = np.zeros) → Union[tuple, dict, ndarray]#
 

 gymnasium.vector.utils.create_empty_array(space: ~gymnasium.spaces.multi_binary.MultiBinary, n=1, fn=<built-in function zeros>)

 

@@ -582,14 +595,14 @@
 
Create an empty (possibly nested) numpy array.

 
Example:

 
>>> from gymnasium.spaces import Box, Dict
+>>> import numpy as np
 >>> space = Dict({
 ... 'position': Box(low=0, high=1, shape=(3,), dtype=np.float32),
 ... 'velocity': Box(low=0, high=1, shape=(2,), dtype=np.float32)})
 >>> create_empty_array(space, n=2, fn=np.zeros)
 OrderedDict([('position', array([[0., 0., 0.],
-                                 [0., 0., 0.]], dtype=float32)),
-             ('velocity', array([[0., 0.],
-                                 [0., 0.]], dtype=float32))])
+       [0., 0., 0.]], dtype=float32)), ('velocity', array([[0., 0.],
+       [0., 0.]], dtype=float32))])
 

 

 

@@ -613,17 +626,17 @@
 

 gymnasium.vector.utils.create_shared_memory(space: Space, n: int = 1, ctx=mp) → Union[dict, tuple, Array]#

 

-gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.multi_binary.MultiBinary, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/multiprocessing/__init__.py'>)

+gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.multi_binary.MultiBinary, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/multiprocessing/__init__.py'>)
 

-gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.multi_discrete.MultiDiscrete, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/multiprocessing/__init__.py'>)

+gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.multi_discrete.MultiDiscrete, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/multiprocessing/__init__.py'>)
 

-gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.discrete.Discrete, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/multiprocessing/__init__.py'>)

+gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.discrete.Discrete, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/multiprocessing/__init__.py'>)
 

-gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.box.Box, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/multiprocessing/__init__.py'>)

+gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.box.Box, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/multiprocessing/__init__.py'>)
 

-gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.tuple.Tuple, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/multiprocessing/__init__.py'>)

+gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.tuple.Tuple, n: int = 1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/multiprocessing/__init__.py'>)
 

-gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.dict.Dict, n=1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/multiprocessing/__init__.py'>)

+gymnasium.vector.utils.create_shared_memory(space: ~gymnasium.spaces.dict.Dict, n=1, ctx=<module 'multiprocessing' from '/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/multiprocessing/__init__.py'>)
 
Create a shared memory object, to be shared across processes.

 
This eventually contains the observations from the vectorized environment.

 

@@ -804,11 +817,11 @@ This object is created with create_shared_memory.

         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -978,7 +991,7 @@ This object is created with create_shared_memory.

         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/utils/index.html b/api/utils/index.html
index 39f43d109..c9124c4ab 100644
--- a/api/utils/index.html
+++ b/api/utils/index.html
@@ -15,7 +15,11 @@
         Utils - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -429,7 +441,7 @@
 
Example:

 
>>> import gymnasium as gym
 >>> from gymnasium.utils.play import play
->>> play(gym.make("CarRacing-v1", render_mode="rgb_array"), keys_to_action={
+>>> play(gym.make("CarRacing-v2", render_mode="rgb_array"), keys_to_action={  
 ...                                                "w": np.array([0, 0.7, 0]),
 ...                                                "a": np.array([-1, 0, 0]),
 ...                                                "s": np.array([0, 0, 1]),
@@ -448,10 +460,11 @@ unplayable.

 gym.utils.play.PlayPlot. Here’s a sample code for plotting the reward
 for last 150 steps.

 
>>> import gymnasium as gym
+>>> from gymnasium.utils.play import PlayPlot, play
 >>> def callback(obs_t, obs_tp1, action, rew, terminated, truncated, info):
 ...        return [rew,]
->>> plotter = PlayPlot(callback, 150, ["reward"])
->>> play(gym.make("CartPole-v1"), callback=plotter.callback)
+>>> plotter = PlayPlot(callback, 150, ["reward"])             
+>>> play(gym.make("CartPole-v1"), callback=plotter.callback)  
 

 

 

@@ -476,7 +489,7 @@ points of the keys, as a tuple of characters, or as a string where each characte
 one key.
 For example if pressing ‘w’ and space at the same time is supposed
 to trigger action number 2 then key_to_action dict could look like this:

-
>>> {
+
>>> key_to_action = {
 ...    # ...
 ...    (ord('w'), ord(' ')): 2
 ...    # ...
@@ -484,7 +497,7 @@ to trigger action number 2 then 

 

 
or like this:

-
>>> {
+
>>> key_to_action = {
 ...    # ...
 ...    ("w", " "): 2
 ...    # ...
@@ -492,7 +505,7 @@ to trigger action number 2 then 

 

 
or like this:

-
>>> {
+
>>> key_to_action = {
 ...    # ...
 ...    "w ": 2
 ...    # ...
@@ -510,7 +523,7 @@ to trigger action number 2 then 
 

-class gymnasium.utils.play.PlayPlot(callback: callable, horizon_timesteps: int, plot_names: List[str])#

+class gymnasium.utils.play.PlayPlot(callback: Callable, horizon_timesteps: int, plot_names: List[str])#
 
Provides a callback to create live plots of arbitrary metrics when using play().

 

 
This class is instantiated with a function that accepts information about a single environment transition:
    
@@ -533,9 +546,9 @@ For instance, the function may look like this:
    
 
    PlayPlot provides the method callback() which will pass its arguments along to that function
 and uses the returned values to update live plots of the metrics.
    
 
    Typically, this callback() will be used in conjunction with play() to see how the metrics evolve as you play:
    
-
    >>> plotter = PlayPlot(compute_metrics, horizon_timesteps=200,
+
    >>> plotter = PlayPlot(compute_metrics, horizon_timesteps=200,                               
 ...                    plot_names=["Immediate Rew.", "Cumulative Rew.", "Action Magnitude"])
->>> play(your_env, callback=plotter.callback)
+>>> play(your_env, callback=plotter.callback)                                                
 
    
 
    
 
    Constructor of PlayPlot.
    
@@ -632,10 +645,10 @@ You need to specify either fps or duration.
    
 
    >>> import gymnasium as gym
 >>> from gymnasium.utils.save_video import save_video
 >>> env = gym.make("FrozenLake-v1", render_mode="rgb_array_list")
->>> env.reset()
+>>> _ = env.reset()
 >>> step_starting_index = 0
 >>> episode_index = 0
->>> for step_index in range(199):
+>>> for step_index in range(199): 
 ...    action = env.action_space.sample()
 ...    _, _, terminated, truncated, _ = env.step(action)
 ...
@@ -698,13 +711,16 @@ Terminated Truncated (new) step API refers to step() method returning (observati
 

 

 
>>> import gymnasium as gym
->>> env = gym.make("OldEnv")
->>> obs, rew, done, info = step_api_compatibility(env.step(action), output_truncation_bool=False)
->>> obs, rew, terminated, truncated, info = step_api_compatibility(env.step(action), output_truncation_bool=True)
+>>> env = gym.make("CartPole-v0")
+>>> _ = env.reset()
+>>> obs, rewards, done, info = step_api_compatibility(env.step(0), output_truncation_bool=False)
+>>> obs, rewards, terminated, truncated, info = step_api_compatibility(env.step(0), output_truncation_bool=True)
 

 

-
>>> vec_env = gym.vector.make("OldEnv")
->>> observations, rewards, dones, infos = step_api_compatibility(vec_env.step(action), is_vector_env=True)
+
>>> vec_env = gym.vector.make("CartPole-v0")
+>>> _ = vec_env.reset()
+>>> obs, rewards, dones, infos = step_api_compatibility(vec_env.step([0]), is_vector_env=True, output_truncation_bool=False)
+>>> obs, rewards, terminated, truncated, info = step_api_compatibility(vec_env.step([0]), is_vector_env=True, output_truncation_bool=True)
 

 

 

@@ -889,11 +905,11 @@ for more information about the API.

         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1064,7 +1080,7 @@ for more information about the API.

         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/vector/index.html b/api/vector/index.html
index 758106bb8..33f7a3390 100644
--- a/api/vector/index.html
+++ b/api/vector/index.html
@@ -15,7 +15,11 @@
         Vector - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -491,10 +503,10 @@ In other words, a vector of multiple different environments is not supported.
An example:

 
>>> import gymnasium as gym
 >>> envs = gym.vector.make("CartPole-v1", num_envs=3)
->>> envs.reset()
-(array([[-0.02240574, -0.03439831, -0.03904812,  0.02810693],
-       [ 0.01586068,  0.01929009,  0.02394426,  0.04016077],
-       [-0.01314174,  0.03893502, -0.02400815,  0.0038326 ]],
+>>> envs.reset(seed=42)
+(array([[ 0.0273956 , -0.00611216,  0.03585979,  0.0197368 ],
+       [ 0.01522993, -0.04562247, -0.04799704,  0.03392126],
+       [-0.03774345, -0.02418869, -0.00942293,  0.0469184 ]],
       dtype=float32), {})
 

 

@@ -520,15 +532,14 @@ info as “final_observation” and “final_info”.<
 

 
An example:

 
>>> envs = gym.vector.make("CartPole-v1", num_envs=3)
->>> envs.reset()
+>>> _ = envs.reset(seed=42)
 >>> actions = np.array([1, 0, 1])
 >>> observations, rewards, termination, truncation, infos = envs.step(actions)
-
 >>> observations
-array([[ 0.00122802,  0.16228443,  0.02521779, -0.23700266],
-        [ 0.00788269, -0.17490888,  0.03393489,  0.31735462],
-        [ 0.04918966,  0.19421194,  0.02938497, -0.29495203]],
-        dtype=float32)
+array([[ 0.02727336,  0.18847767,  0.03625453, -0.26141977],
+       [ 0.01431748, -0.24002443, -0.04731862,  0.3110827 ],
+       [-0.03822722,  0.1710671 , -0.00848456, -0.2487226 ]],
+      dtype=float32)
 >>> rewards
 array([1., 1., 1.])
 >>> termination
@@ -617,16 +628,16 @@ vectorized environments.

 
Making Vector Environments#

 

 

-gymnasium.vector.make(id: str, num_envs: int = 1, asynchronous: bool = True, wrappers: Optional[Union[callable, List[callable]]] = None, disable_env_checker: Optional[bool] = None, **kwargs) → VectorEnv#

+gymnasium.vector.make(id: str, num_envs: int = 1, asynchronous: bool = True, wrappers: Optional[Union[Callable[[Env], Env], List[Callable[[Env], Env]]]] = None, disable_env_checker: Optional[bool] = None, **kwargs) → VectorEnv#
 
Create a vectorized environment from multiple copies of an environment, from its id.

 
Example:

 
>>> import gymnasium as gym
 >>> env = gym.vector.make('CartPole-v1', num_envs=3)
->>> env.reset()
-array([[-0.04456399,  0.04653909,  0.01326909, -0.02099827],
-       [ 0.03073904,  0.00145001, -0.03088818, -0.03131252],
-       [ 0.03468829,  0.01500225,  0.01230312,  0.01825218]],
-      dtype=float32)
+>>> env.reset(seed=42)
+(array([[ 0.0273956 , -0.00611216,  0.03585979,  0.0197368 ],
+       [ 0.01522993, -0.04562247, -0.04799704,  0.03392126],
+       [-0.03774345, -0.02418869, -0.00942293,  0.0469184 ]],
+      dtype=float32), {})
 

 

 

@@ -652,18 +663,18 @@ vectorized environments.

 
Async Vector Env#

 

 

-class gymnasium.vector.AsyncVectorEnv(env_fns: Sequence[callable], observation_space: Optional[Space] = None, action_space: Optional[Space] = None, shared_memory: bool = True, copy: bool = True, context: Optional[str] = None, daemon: bool = True, worker: Optional[callable] = None)#

+class gymnasium.vector.AsyncVectorEnv(env_fns: Sequence[Callable[[], Env]], observation_space: Optional[Space] = None, action_space: Optional[Space] = None, shared_memory: bool = True, copy: bool = True, context: Optional[str] = None, daemon: bool = True, worker: Optional[Callable] = None)#
 
Vectorized environment that runs multiple environments in parallel.

 
It uses multiprocessing processes, and pipes for communication.

 
Example:

 
>>> import gymnasium as gym
 >>> env = gym.vector.AsyncVectorEnv([
-...     lambda: gym.make("Pendulum-v0", g=9.81),
-...     lambda: gym.make("Pendulum-v0", g=1.62)
+...     lambda: gym.make("Pendulum-v1", g=9.81),
+...     lambda: gym.make("Pendulum-v1", g=1.62)
 ... ])
->>> env.reset()
-array([[-0.8286432 ,  0.5597771 ,  0.90249056],
-       [-0.85009176,  0.5266346 ,  0.60007906]], dtype=float32)
+>>> env.reset(seed=42)
+(array([[-0.14995256,  0.9886932 , -0.12224312],
+       [ 0.5760367 ,  0.8174238 , -0.91244936]], dtype=float32), {})
 

 

 
Vectorized environment that runs multiple environments in parallel.

@@ -715,12 +726,12 @@ from the code for _
 
Example:

 
>>> import gymnasium as gym
 >>> env = gym.vector.SyncVectorEnv([
-...     lambda: gym.make("Pendulum-v0", g=9.81),
-...     lambda: gym.make("Pendulum-v0", g=1.62)
+...     lambda: gym.make("Pendulum-v1", g=9.81),
+...     lambda: gym.make("Pendulum-v1", g=1.62)
 ... ])
->>> env.reset()
-array([[-0.8286432 ,  0.5597771 ,  0.90249056],
-       [-0.85009176,  0.5266346 ,  0.60007906]], dtype=float32)
+>>> env.reset(seed=42)
+(array([[-0.14995256,  0.9886932 , -0.12224312],
+       [ 0.5760367 ,  0.8174238 , -0.91244936]], dtype=float32), {})
 

 

 
Vectorized environment that serially runs multiple environments.

@@ -850,11 +861,11 @@ then the action space of the first environment is taken.

         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1024,7 +1035,7 @@ then the action space of the first environment is taken.

         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/wrappers/action_wrappers/index.html b/api/wrappers/action_wrappers/index.html
index 7f02640a8..06809924f 100644
--- a/api/wrappers/action_wrappers/index.html
+++ b/api/wrappers/action_wrappers/index.html
@@ -15,7 +15,11 @@
         Action Wrappers - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -459,12 +471,14 @@ the 
Clip the continuous action within the valid Box observation space bound.

 
Example

 
>>> import gymnasium as gym
->>> env = gym.make('Bipedal-Walker-v3')
+>>> from gymnasium.wrappers import ClipAction
+>>> env = gym.make("Hopper-v4")
 >>> env = ClipAction(env)
 >>> env.action_space
-Box(-1.0, 1.0, (4,), float32)
->>> env.step(np.array([5.0, 2.0, -10.0, 0.0]))
-# Executes the action np.array([1.0, 1.0, -1.0, 0]) in the base environment
+Box(-1.0, 1.0, (3,), float32)
+>>> _ = env.reset(seed=42)
+>>> _ = env.step(np.array([5.0, -2.0, 0.0]))
+... # Executes the action np.array([1.0, -1.0, 0]) in the base environment
 

 

 
A wrapper for clipping continuous actions within the valid bound.

@@ -483,15 +497,17 @@ the max_action are numpy arrays, the shape must match the shape of the environment’s action space.

 
Example

 
>>> import gymnasium as gym
->>> env = gym.make('BipedalWalker-v3')
->>> env.action_space
-Box(-1.0, 1.0, (4,), float32)
+>>> from gymnasium.wrappers import RescaleAction
+>>> import numpy as np
+>>> env = gym.make("Hopper-v4")
+>>> _ = env.reset(seed=42)
+>>> obs, _, _, _, _ = env.step(np.array([1,1,1]))
+>>> _ = env.reset(seed=42)
 >>> min_action = -0.5
->>> max_action = np.array([0.0, 0.5, 1.0, 0.75])
->>> env = RescaleAction(env, min_action=min_action, max_action=max_action)
->>> env.action_space
-Box(-0.5, [0.   0.5  1.   0.75], (4,), float32)
->>> RescaleAction(env, min_action, max_action).action_space == gym.spaces.Box(min_action, max_action)
+>>> max_action = np.array([0.0, 0.5, 0.75])
+>>> wrapped_env = RescaleAction(env, min_action=min_action, max_action=max_action)
+>>> wrapped_env_obs, _, _, _, _ = wrapped_env.step(max_action)
+>>> np.alltrue(obs == wrapped_env_obs)
 True
 

 

@@ -598,11 +614,11 @@ or 
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -772,7 +788,7 @@ or 
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/wrappers/index.html b/api/wrappers/index.html
index b78498b79..391ff9e56 100644
--- a/api/wrappers/index.html
+++ b/api/wrappers/index.html
@@ -15,7 +15,11 @@
         Wrappers - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -431,29 +443,29 @@ Most environments that are generated via 
>>> import gymnasium as gym
 >>> from gymnasium.wrappers import RescaleAction
->>> base_env = gym.make("BipedalWalker-v3")
+>>> base_env = gym.make("Hopper-v4")
 >>> base_env.action_space
-Box([-1. -1. -1. -1.], [1. 1. 1. 1.], (4,), float32)
+Box(-1.0, 1.0, (3,), float32)
 >>> wrapped_env = RescaleAction(base_env, min_action=0, max_action=1)
 >>> wrapped_env.action_space
-Box([0. 0. 0. 0.], [1. 1. 1. 1.], (4,), float32)
+Box(-1.0, 1.0, (3,), float32)
 

 

 
You can access the environment underneath the first wrapper by using the gymnasium.Wrapper.env attribute.
 As the gymnasium.Wrapper class inherits from gymnasium.Env then gymnasium.Wrapper.env can be another wrapper.

 
>>> wrapped_env
-<RescaleAction<TimeLimit<OrderEnforcing<BipedalWalker<BipedalWalker-v3>>>>>
+<RescaleAction<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>
 >>> wrapped_env.env
-<TimeLimit<OrderEnforcing<BipedalWalker<BipedalWalker-v3>>>>
+<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>
 

 

 
If you want to get to the environment underneath all of the layers of wrappers, you can use the
 gymnasium.Wrapper.unwrapped attribute.
 If the environment is already a bare environment, the gymnasium.Wrapper.unwrapped attribute will just return itself.

 
>>> wrapped_env
-<RescaleAction<TimeLimit<OrderEnforcing<BipedalWalker<BipedalWalker-v3>>>>>
->>> wrapped_env.unwrapped
-<gymnasium.envs.box2d.bipedal_walker.BipedalWalker object at 0x7f87d70712d0>
+<RescaleAction<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>
+>>> wrapped_env.unwrapped 
+<gymnasium.envs.mujoco.hopper_v4.HopperEnv object at 0x7fbb5efd0490>
 

 

 
There are three common things you might want a wrapper to do:

@@ -779,11 +791,11 @@ wrapper in the page on the wrapper type

         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -953,7 +965,7 @@ wrapper in the page on the wrapper type

         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/wrappers/misc_wrappers/index.html b/api/wrappers/misc_wrappers/index.html
index 4dfbe3e12..c71c59960 100644
--- a/api/wrappers/misc_wrappers/index.html
+++ b/api/wrappers/misc_wrappers/index.html
@@ -15,7 +15,11 @@
         Misc Wrappers - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -538,12 +550,14 @@ New step API refers to step() method returning (observation, reward, terminated,
 

 

 
Examples

-
>>> env = gym.make("CartPole-v1")
+
>>> import gymnasium as gym
+>>> from gymnasium.wrappers import StepAPICompatibility
+>>> env = gym.make("CartPole-v1")
 >>> env # wrapper not applied by default, set to new API
 <TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v1>>>>>
->>> env = gym.make("CartPole-v1", apply_api_compatibility=True) # set to old API
+>>> env = StepAPICompatibility(gym.make("CartPole-v1"))
+>>> env
 <StepAPICompatibility<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v1>>>>>>
->>> env = StepAPICompatibility(CustomEnv(), output_truncation_bool=False) # manually using wrapper on unregistered envs
 

 

 
A wrapper which can transform an environment from new step API to old and vice-versa.

@@ -574,26 +588,28 @@ If you want to use this wrapper with your environments, remember to specify 
 
The render_mode of the wrapped environment must be either 'rgb_array' or 'rgb_array_list'.

 
Example

-
>>> env = gym.make("LunarLander-v2", render_mode="rgb_array")
+
>>> import gymnasium as gym
+>>> from gymnasium.wrappers import HumanRendering
+>>> env = gym.make("LunarLander-v2", render_mode="rgb_array")
 >>> wrapped = HumanRendering(env)
->>> wrapped.reset()     # This will start rendering to the screen
+>>> obs, _ = wrapped.reset()     # This will start rendering to the screen
 

 

 
The wrapper can also be applied directly when the environment is instantiated, simply by passing
 render_mode="human" to make. The wrapper will only be applied if the environment does not
 implement human-rendering natively (i.e. render_mode does not contain "human").

 
Example

-
>>> env = gym.make("NoNativeRendering-v2", render_mode="human")      # NoNativeRendering-v0 doesn't implement human-rendering natively
->>> env.reset()     # This will start rendering to the screen
+
>>> env = gym.make("CartPoleJax-v1", render_mode="human")      # CartPoleJax-v1 doesn't implement human-rendering natively
+>>> obs, _ = env.reset()     # This will start rendering to the screen
 

 

 

 
Warning: If the base environment uses render_mode="rgb_array_list", its (i.e. the base environment’s) render method
will always return an empty list:

 
>>> env = gym.make("LunarLander-v2", render_mode="rgb_array_list")
 >>> wrapped = HumanRendering(env)
->>> wrapped.reset()
->>> env.render()
-[]          # env.render() will always return an empty list!
+>>> obs, _ = wrapped.reset()
+>>> env.render()     # env.render() will always return an empty list!
+[]
 

 

 

@@ -611,16 +627,17 @@ implement human-rendering natively (i.e. class gymnasium.wrappers.OrderEnforcing(env: Env, disable_render_order_enforcing: bool = False)#
 
A wrapper that will produce an error if step() is called before an initial reset().

 
Example

-
>>> from gymnasium.envs.classic_control import CartPoleEnv
->>> env = CartPoleEnv()
+
>>> import gymnasium as gym
+>>> from gymnasium.wrappers import OrderEnforcing
+>>> env = gym.make("CartPole-v1", render_mode="human")
 >>> env = OrderEnforcing(env)
->>> env.step(0)
-ResetNeeded: Cannot call env.step() before calling env.reset()
+>>> env.step(0) 
+gymnasium.error.ResetNeeded: Cannot call env.step() before calling env.reset()
+>>> env.render() 
+gymnasium.error.ResetNeeded('Cannot call `env.render()` before calling `env.reset()`, if this is a intended action, set `disable_render_order_enforcing=True` on the OrderEnforcer wrapper.')
+>>> _ = env.reset()
 >>> env.render()
-ResetNeeded: Cannot call env.render() before calling env.reset()
->>> env.reset()
->>> env.render()
->>> env.step(0)
+>>> _ = env.step(0)
 

 

 
A wrapper that will produce an error if step() is called before an initial reset().

@@ -644,7 +661,6 @@ using the key episo
 the episode statistics.

 
After the completion of an episode, info will look like this:

 
>>> info = {
-...     ...
 ...     "episode": {
 ...         "r": "<cumulative reward>",
 ...         "l": "<episode length>",
@@ -655,7 +671,10 @@ the episode statistics.

 

 
For a vectorized environments the output will be in the form of:

 
>>> infos = {
-...     ...
+...     "final_observation": "<array of length num-envs>",
+...     "_final_observation": "<boolean array of length num-envs>",
+...     "final_info": "<array of length num-envs>",
+...     "_final_info": "<boolean array of length num-envs>",
 ...     "episode": {
 ...         "r": "<array of cumulative reward>",
 ...         "l": "<array of episode length>",
@@ -740,9 +759,9 @@ Otherwise, snippets of the specified length are captured

 
If a truncation is not defined inside the environment itself, this is the only place that the truncation signal is issued.
 Critically, this is different from the terminated signal that originates from the underlying environment as part of the MDP.

 
Example

-
>>> from gymnasium.envs.classic_control import CartPoleEnv
+
>>> import gymnasium as gym
 >>> from gymnasium.wrappers import TimeLimit
->>> env = CartPoleEnv()
+>>> env = gym.make("CartPole-v1")
 >>> env = TimeLimit(env, max_episode_steps=1000)
 

 

@@ -768,16 +787,33 @@ environments. If using other wrappers that perform
 operation on info like RecordEpisodeStatistics this
 need to be the outermost wrapper.

 
i.e. VectorListInfo(RecordEpisodeStatistics(envs))

-
Example:

-
>>> # actual
->>> {
-...      "k": np.array[0., 0., 0.5, 0.3],
-...      "_k": np.array[False, False, True, True]
-...  }
->>> # classic
->>> [{}, {}, {k: 0.5}, {k: 0.3}]
+

+
Example::
>>> # As dict:
+>>> infos = {
+...     "final_observation": "<array of length num-envs>",
+...     "_final_observation": "<boolean array of length num-envs>",
+...     "final_info": "<array of length num-envs>",
+...     "_final_info": "<boolean array of length num-envs>",
+...     "episode": {
+...         "r": "<array of cumulative reward>",
+...         "l": "<array of episode length>",
+...         "t": "<array of elapsed time since beginning of episode>"
+...     },
+...     "_episode": "<boolean array of length num-envs>"
+... }
+>>> # As list:
+>>> infos = [
+...     {
+...         "episode": {"r": "<cumulative reward>", "l": "<episode length>", "t": "<elapsed time since beginning of episode>"},
+...         "final_observation": "<observation>",
+...         "final_info": {},
+...     },
+...     ...,
+... ]
 

 

+

+

 
This wrapper will convert the info into the list format.

 

 
Parameters:

@@ -876,11 +912,11 @@ need to be the outermost wrapper.

         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1050,7 +1086,7 @@ need to be the outermost wrapper.

         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/wrappers/observation_wrappers/index.html b/api/wrappers/observation_wrappers/index.html
index e0ae23d96..b8074a71d 100644
--- a/api/wrappers/observation_wrappers/index.html
+++ b/api/wrappers/observation_wrappers/index.html
@@ -15,7 +15,11 @@
         Observation Wrappers - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -461,11 +473,13 @@ index of the timestep to the observation.

 
If the transformation you wish to apply to observations returns values in a different space, you should subclass ObservationWrapper, implement the transformation, and set the new observation space accordingly. If you were to use this wrapper instead, the observation space would be set incorrectly.

 
Example

 
>>> import gymnasium as gym
+>>> from gymnasium.wrappers import TransformObservation
 >>> import numpy as np
->>> env = gym.make('CartPole-v1')
->>> env = TransformObservation(env, lambda obs: obs + 0.1*np.random.randn(*obs.shape))
->>> env.reset()
-array([-0.08319338,  0.04635121, -0.07394746,  0.20877492])
+>>> np.random.seed(0)
+>>> env = gym.make("CartPole-v1")
+>>> env = TransformObservation(env, lambda obs: obs + 0.1 * np.random.randn(*obs.shape))
+>>> env.reset(seed=42)
+(array([0.20380084, 0.03390356, 0.13373359, 0.24382612]), {})
 

 

 
Initialize the TransformObservation wrapper with an environment and a transform function f.

@@ -485,17 +499,17 @@ index of the timestep to the observation.

 
Filter Dict observation space by the keys.

 
Example

 
>>> import gymnasium as gym
->>> env = gym.wrappers.TransformObservation(
-...     gym.make('CartPole-v1'), lambda obs: {'obs': obs, 'time': 0}
-... )
+>>> from gymnasium.wrappers import TransformObservation
+>>> env = gym.make("CartPole-v1")
+>>> env = TransformObservation(env, lambda obs: {'obs': obs, 'time': 0})
 >>> env.observation_space = gym.spaces.Dict(obs=env.observation_space, time=gym.spaces.Discrete(1))
->>> env.reset()
-{'obs': array([-0.00067088, -0.01860439,  0.04772898, -0.01911527], dtype=float32), 'time': 0}
+>>> env.reset(seed=42)
+({'obs': array([ 0.0273956 , -0.00611216,  0.03585979,  0.0197368 ], dtype=float32), 'time': 0}, {})
 >>> env = FilterObservation(env, filter_keys=['obs'])
->>> env.reset()
-{'obs': array([ 0.04560107,  0.04466959, -0.0328232 , -0.02367178], dtype=float32)}
+>>> env.reset(seed=42)
+({'obs': array([ 0.0273956 , -0.00611216,  0.03585979,  0.0197368 ], dtype=float32)}, {})
 >>> env.step(0)
-({'obs': array([ 0.04649447, -0.14996664, -0.03329664,  0.25847703], dtype=float32)}, 1.0, False, {})
+({'obs': array([ 0.02727336, -0.20172954,  0.03625453,  0.32351476], dtype=float32)}, 1.0, False, False, {})
 

 

 
A wrapper that filters dictionary observations by their keys.

@@ -521,13 +535,14 @@ index of the timestep to the observation.

 
Observation wrapper that flattens the observation.

 
Example

 
>>> import gymnasium as gym
->>> env = gym.make('CarRacing-v1')
+>>> from gymnasium.wrappers import FlattenObservation
+>>> env = gym.make("CarRacing-v2")
 >>> env.observation_space.shape
 (96, 96, 3)
 >>> env = FlattenObservation(env)
 >>> env.observation_space.shape
 (27648,)
->>> obs, info = env.reset()
+>>> obs, _ = env.reset()
 >>> obs.shape
 (27648,)
 

@@ -560,11 +575,12 @@ I.e. the observation returned by Example

 
>>> import gymnasium as gym
->>> env = gym.make('CarRacing-v1')
+>>> from gymnasium.wrappers import FrameStack
+>>> env = gym.make("CarRacing-v2")
 >>> env = FrameStack(env, 4)
 >>> env.observation_space
-Box(4, 96, 96, 3)
->>> obs = env.reset()
+Box(0, 255, (4, 96, 96, 3), uint8)
+>>> obs, _ = env.reset()
 >>> obs.shape
 (4, 96, 96, 3)
 

@@ -586,13 +602,15 @@ I.e. the observation returned by class gymnasium.wrappers.GrayScaleObservation(env: Env, keep_dim: bool = False)#
 
Convert the image observation from RGB to gray scale.

 
Example

-
>>> env = gym.make('CarRacing-v1')
+
>>> import gymnasium as gym
+>>> from gymnasium.wrappers import GrayScaleObservation
+>>> env = gym.make("CarRacing-v2")
 >>> env.observation_space
 Box(0, 255, (96, 96, 3), uint8)
->>> env = GrayScaleObservation(gym.make('CarRacing-v1'))
+>>> env = GrayScaleObservation(gym.make("CarRacing-v2"))
 >>> env.observation_space
 Box(0, 255, (96, 96), uint8)
->>> env = GrayScaleObservation(gym.make('CarRacing-v1'), keep_dim=True)
+>>> env = GrayScaleObservation(gym.make("CarRacing-v2"), keep_dim=True)
 >>> env.observation_space
 Box(0, 255, (96, 96, 1), uint8)
 

@@ -641,22 +659,23 @@ space is of type 
 
Example

 
>>> import gymnasium as gym
->>> env = PixelObservationWrapper(gym.make('CarRacing-v1', render_mode="rgb_array"))
->>> obs = env.reset()
+>>> from gymnasium.wrappers import PixelObservationWrapper
+>>> env = PixelObservationWrapper(gym.make("CarRacing-v2", render_mode="rgb_array"))
+>>> obs, _ = env.reset()
 >>> obs.keys()
 odict_keys(['pixels'])
 >>> obs['pixels'].shape
 (400, 600, 3)
->>> env = PixelObservationWrapper(gym.make('CarRacing-v1', render_mode="rgb_array"), pixels_only=False)
->>> obs = env.reset()
+>>> env = PixelObservationWrapper(gym.make("CarRacing-v2", render_mode="rgb_array"), pixels_only=False)
+>>> obs, _ = env.reset()
 >>> obs.keys()
 odict_keys(['state', 'pixels'])
 >>> obs['state'].shape
 (96, 96, 3)
 >>> obs['pixels'].shape
 (400, 600, 3)
->>> env = PixelObservationWrapper(gym.make('CarRacing-v1', render_mode="rgb_array"), pixel_keys=('obs',))
->>> obs = env.reset()
+>>> env = PixelObservationWrapper(gym.make("CarRacing-v2", render_mode="rgb_array"), pixel_keys=('obs',))
+>>> obs, _ = env.reset()
 >>> obs.keys()
 odict_keys(['obs'])
 >>> obs['obs'].shape
@@ -696,14 +715,18 @@ Defaults to (pixels,).

 
 

 

-class gymnasium.wrappers.ResizeObservation(env: Env, shape: Union[tuple, int])#

+class gymnasium.wrappers.ResizeObservation(env: gym.Env, shape: tuple[int, int] | int)#
 
Resize the image observation.

-
This wrapper works on environments with image observations (or more generally observations of shape AxBxC) and resizes
-the observation to the shape given by the 2-tuple shape. The argument shape may also be an integer.
-In that case, the observation is scaled to a square of side-length shape.

+
This wrapper works on environments with image observations. More generally,
+the input can either be two-dimensional (AxB, e.g. grayscale images) or
+three-dimensional (AxBxC, e.g. color images). This resizes the observation
+to the shape given by the 2-tuple shape.
+The argument shape may also be an integer, in which case, the
+observation is scaled to a square of side-length shape.

 
Example

 
>>> import gymnasium as gym
->>> env = gym.make('CarRacing-v1')
+>>> from gymnasium.wrappers import ResizeObservation
+>>> env = gym.make("CarRacing-v2")
 >>> env.observation_space.shape
 (96, 96, 3)
 >>> env = ResizeObservation(env, 64)
@@ -730,12 +753,14 @@ In that case, the observation is scaled to a square of side-length Example

 
>>> import gymnasium as gym
->>> env = gym.make('CartPole-v1')
+>>> from gymnasium.wrappers import TimeAwareObservation
+>>> env = gym.make("CartPole-v1")
 >>> env = TimeAwareObservation(env)
->>> env.reset()
-array([ 0.03810719,  0.03522411,  0.02231044, -0.01088205,  0.        ])
+>>> env.reset(seed=42)
+(array([ 0.0273956 , -0.00611216,  0.03585979,  0.0197368 ,  0.        ]), {})
+>>> _ = env.action_space.seed(42)
 >>> env.step(env.action_space.sample())[0]
-array([ 0.03881167, -0.16021058,  0.0220928 ,  0.28875574,  1.        ])
+array([ 0.02727336, -0.20172954,  0.03625453,  0.32351476,  1.        ])
 

 

 
Initialize TimeAwareObservation that requires an environment with a flat Box observation space.

@@ -844,11 +869,11 @@ In particular, pixel observations are not supported. This wrapper will append th
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1018,7 +1043,7 @@ In particular, pixel observations are not supported. This wrapper will append th
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/api/wrappers/reward_wrappers/index.html b/api/wrappers/reward_wrappers/index.html
index 3bf5cde0c..35dced2df 100644
--- a/api/wrappers/reward_wrappers/index.html
+++ b/api/wrappers/reward_wrappers/index.html
@@ -15,7 +15,11 @@
         Reward Wrappers - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -461,9 +473,10 @@ you can simply define 
 
Example

 
>>> import gymnasium as gym
->>> env = gym.make('CartPole-v1')
+>>> from gymnasium.wrappers import TransformReward
+>>> env = gym.make("CartPole-v1")
 >>> env = TransformReward(env, lambda r: 0.01*r)
->>> env.reset()
+>>> _ = env.reset()
 >>> observation, reward, terminated, truncated, info = env.step(env.action_space.sample())
 >>> reward
 0.01
@@ -593,11 +606,11 @@ instantiated or the policy was changed recently.

         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -768,7 +781,7 @@ instantiated or the policy was changed recently.

         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/content/basic_usage/index.html b/content/basic_usage/index.html
index 214d6ff1f..1a05e0f40 100644
--- a/content/basic_usage/index.html
+++ b/content/basic_usage/index.html
@@ -15,7 +15,11 @@
         Basic Usage - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -513,8 +525,8 @@ Values can be shifted to {a, a+1, …, a+n-1} using an optional argument.

 
More information#

 
 
@@ -594,11 +606,11 @@ Values can be shifted to {a, a+1, …, a+n-1} using an optional argument.
 {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/content/gym_compatibility/index.html b/content/gym_compatibility/index.html
index 77d9a4c27..49dbcd349 100644
--- a/content/gym_compatibility/index.html
+++ b/content/gym_compatibility/index.html
@@ -15,7 +15,11 @@
         Compatibility with Gym - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -527,11 +539,11 @@ To allow backward compatibility, Gym and Gymnasium v0.26+ include an  {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/content/migration-guide/index.html b/content/migration-guide/index.html
index 418fe5455..4f777549b 100644
--- a/content/migration-guide/index.html
+++ b/content/migration-guide/index.html
@@ -15,7 +15,11 @@
         v21 to v26 Migration Guide - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
Tutorials

 
 
Development

 
 
@@ -492,6 +504,11 @@ However, environments that have reasons for episode truncation rather than termi
 For users looping through an environment, they should modify done = terminated or truncated as is show in the example code.
 For training libraries, the primary difference is to change done to terminated, indicating whether bootstrapping should or shouldn’t happen.

 
+

+
TimeLimit Wrapper#

+
In v21, the TimeLimit wrapper added an extra key in the info dictionary TimeLimit.truncated whenever the agent reached the time limit without reaching a terminal state.

+
In v26, this information is instead communicated through the truncated return value described in the previous section, which is True if the agent reaches the time limit, whether or not it reaches a terminal state. The old dictionary entry is equivalent to truncated and not terminated

+

 

 
Environment Render#

 
In v26, a new render API was introduced such that the render mode is fixed at initialisation as some environments don’t allow on-the-fly render mode changes. Therefore, users should now specify the render_mode within gym.make as shown in the v26 example code above.

@@ -572,6 +589,7 @@ For training libraries, the primary difference is to change Seed and random number generator
 
  • Environment Reset
    • 
 
  • Environment Step
    • 
+
  • TimeLimit Wrapper
    • 
 
  • Environment Render
    • 
 
  • Removed code
    • 
 
@@ -591,11 +609,11 @@ For training libraries, the primary difference is to change Adventure - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -592,11 +604,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -766,7 +778,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/air_raid/index.html b/environments/atari/air_raid/index.html
index 2e69ab58f..7ed43cf97 100644
--- a/environments/atari/air_raid/index.html
+++ b/environments/atari/air_raid/index.html
@@ -15,7 +15,11 @@
         Air Raid - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -632,11 +644,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -806,7 +818,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/alien/index.html b/environments/atari/alien/index.html
index d78476ee4..3f322dd53 100644
--- a/environments/atari/alien/index.html
+++ b/environments/atari/alien/index.html
@@ -15,7 +15,11 @@
         Alien - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -612,11 +624,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -786,7 +798,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/amidar/index.html b/environments/atari/amidar/index.html
index 7a0f8fac4..64bdc4587 100644
--- a/environments/atari/amidar/index.html
+++ b/environments/atari/amidar/index.html
@@ -15,7 +15,11 @@
         Amidar - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -653,11 +665,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -827,7 +839,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/assault/index.html b/environments/atari/assault/index.html
index 728728068..69aaa87d2 100644
--- a/environments/atari/assault/index.html
+++ b/environments/atari/assault/index.html
@@ -15,7 +15,11 @@
         Assault - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -633,11 +645,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -807,7 +819,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/asterix/index.html b/environments/atari/asterix/index.html
index 452819929..f33432a1f 100644
--- a/environments/atari/asterix/index.html
+++ b/environments/atari/asterix/index.html
@@ -15,7 +15,11 @@
         Asterix - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -650,11 +662,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -824,7 +836,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/asteroids/index.html b/environments/atari/asteroids/index.html
index 75d3c4210..783f96e1e 100644
--- a/environments/atari/asteroids/index.html
+++ b/environments/atari/asteroids/index.html
@@ -15,7 +15,11 @@
         Asteroids - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -667,11 +679,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -841,7 +853,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/atlantis/index.html b/environments/atari/atlantis/index.html
index 0e3df5932..09fb2e0fe 100644
--- a/environments/atari/atlantis/index.html
+++ b/environments/atari/atlantis/index.html
@@ -15,7 +15,11 @@
         Atlantis - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -638,11 +650,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -812,7 +824,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/bank_heist/index.html b/environments/atari/bank_heist/index.html
index 3963bf6ff..1022e140a 100644
--- a/environments/atari/bank_heist/index.html
+++ b/environments/atari/bank_heist/index.html
@@ -15,7 +15,11 @@
         Bank Heist - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -614,11 +626,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -788,7 +800,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/battle_zone/index.html b/environments/atari/battle_zone/index.html
index c3ac87f52..544d07874 100644
--- a/environments/atari/battle_zone/index.html
+++ b/environments/atari/battle_zone/index.html
@@ -15,7 +15,11 @@
         Battle Zone - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -611,11 +623,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -785,7 +797,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/beam_rider/index.html b/environments/atari/beam_rider/index.html
index b6e8e6bfa..529308404 100644
--- a/environments/atari/beam_rider/index.html
+++ b/environments/atari/beam_rider/index.html
@@ -15,7 +15,11 @@
         Beam Rider - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -650,11 +662,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -824,7 +836,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/berzerk/index.html b/environments/atari/berzerk/index.html
index 18948c2b3..74ea02234 100644
--- a/environments/atari/berzerk/index.html
+++ b/environments/atari/berzerk/index.html
@@ -15,7 +15,11 @@
         Berzerk - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -610,11 +622,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -784,7 +796,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/bowling/index.html b/environments/atari/bowling/index.html
index 78bce969b..f78a6dafe 100644
--- a/environments/atari/bowling/index.html
+++ b/environments/atari/bowling/index.html
@@ -15,7 +15,11 @@
         Bowling - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -644,11 +656,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -818,7 +830,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/boxing/index.html b/environments/atari/boxing/index.html
index b0d369342..113eb0812 100644
--- a/environments/atari/boxing/index.html
+++ b/environments/atari/boxing/index.html
@@ -15,7 +15,11 @@
         Boxing - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -610,11 +622,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -784,7 +796,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/breakout/index.html b/environments/atari/breakout/index.html
index df92537dd..3a09d244d 100644
--- a/environments/atari/breakout/index.html
+++ b/environments/atari/breakout/index.html
@@ -15,7 +15,11 @@
         Breakout - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -636,11 +648,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -810,7 +822,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/carnival/index.html b/environments/atari/carnival/index.html
index 5b8e2d486..76b4295ab 100644
--- a/environments/atari/carnival/index.html
+++ b/environments/atari/carnival/index.html
@@ -15,7 +15,11 @@
         Carnival - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -643,11 +655,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -817,7 +829,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/centipede/index.html b/environments/atari/centipede/index.html
index f37818f7c..691356ab9 100644
--- a/environments/atari/centipede/index.html
+++ b/environments/atari/centipede/index.html
@@ -15,7 +15,11 @@
         Centipede - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -613,11 +625,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -787,7 +799,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/chopper_command/index.html b/environments/atari/chopper_command/index.html
index d9d8cc4a1..0cc5b9e97 100644
--- a/environments/atari/chopper_command/index.html
+++ b/environments/atari/chopper_command/index.html
@@ -15,7 +15,11 @@
         Chopper Command - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -611,11 +623,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -785,7 +797,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/complete_list/index.html b/environments/atari/complete_list/index.html
index 2153d8083..202ca5acb 100644
--- a/environments/atari/complete_list/index.html
+++ b/environments/atari/complete_list/index.html
@@ -15,7 +15,11 @@
         Complete List - Atari - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -1211,11 +1223,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1385,7 +1397,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/crazy_climber/index.html b/environments/atari/crazy_climber/index.html
index 03363217f..59de612f4 100644
--- a/environments/atari/crazy_climber/index.html
+++ b/environments/atari/crazy_climber/index.html
@@ -15,7 +15,11 @@
         Crazy Climber - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -652,11 +664,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -826,7 +838,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/defender/index.html b/environments/atari/defender/index.html
index 8f767c00e..fb92f9f4f 100644
--- a/environments/atari/defender/index.html
+++ b/environments/atari/defender/index.html
@@ -15,7 +15,11 @@
         Defender - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -612,11 +624,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -786,7 +798,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/demon_attack/index.html b/environments/atari/demon_attack/index.html
index 1a044cd98..8e67d7730 100644
--- a/environments/atari/demon_attack/index.html
+++ b/environments/atari/demon_attack/index.html
@@ -15,7 +15,11 @@
         Demon Attack - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -654,11 +666,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -828,7 +840,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/double_dunk/index.html b/environments/atari/double_dunk/index.html
index 28dd04c41..47364f0fe 100644
--- a/environments/atari/double_dunk/index.html
+++ b/environments/atari/double_dunk/index.html
@@ -15,7 +15,11 @@
         Double Dunk - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -652,11 +664,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -826,7 +838,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/elevator_action/index.html b/environments/atari/elevator_action/index.html
index 741857bb9..142d17a31 100644
--- a/environments/atari/elevator_action/index.html
+++ b/environments/atari/elevator_action/index.html
@@ -15,7 +15,11 @@
         Elevator Action - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -655,11 +667,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -829,7 +841,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/enduro/index.html b/environments/atari/enduro/index.html
index e39bec3b9..f4c426750 100644
--- a/environments/atari/enduro/index.html
+++ b/environments/atari/enduro/index.html
@@ -15,7 +15,11 @@
         Enduro - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -650,11 +662,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -824,7 +836,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/fishing_derby/index.html b/environments/atari/fishing_derby/index.html
index 607b98562..5431907c3 100644
--- a/environments/atari/fishing_derby/index.html
+++ b/environments/atari/fishing_derby/index.html
@@ -15,7 +15,11 @@
         FishingDerby - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -703,11 +715,11 @@ customize the environment using the arguments above, if necessary.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -877,7 +889,7 @@ customize the environment using the arguments above, if necessary.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/freeway/index.html b/environments/atari/freeway/index.html
index 38edd9273..6c203fa8a 100644
--- a/environments/atari/freeway/index.html
+++ b/environments/atari/freeway/index.html
@@ -15,7 +15,11 @@
         Freeway - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -703,11 +715,11 @@ customize the environment using the arguments above, if necessary.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -877,7 +889,7 @@ customize the environment using the arguments above, if necessary.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/frostbite/index.html b/environments/atari/frostbite/index.html
index aa211f3d3..d15a72f91 100644
--- a/environments/atari/frostbite/index.html
+++ b/environments/atari/frostbite/index.html
@@ -15,7 +15,11 @@
         Frostbite - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -703,11 +715,11 @@ customize the environment using the arguments above, if necessary.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -877,7 +889,7 @@ customize the environment using the arguments above, if necessary.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/gopher/index.html b/environments/atari/gopher/index.html
index 9a7ae6a06..348ab2db7 100644
--- a/environments/atari/gopher/index.html
+++ b/environments/atari/gopher/index.html
@@ -15,7 +15,11 @@
         Gopher - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -703,11 +715,11 @@ customize the environment using the arguments above, if necessary.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -877,7 +889,7 @@ customize the environment using the arguments above, if necessary.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/gravitar/index.html b/environments/atari/gravitar/index.html
index 56f592233..ea9aced03 100644
--- a/environments/atari/gravitar/index.html
+++ b/environments/atari/gravitar/index.html
@@ -15,7 +15,11 @@
         Gravitar - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -703,11 +715,11 @@ customize the environment using the arguments above, if necessary.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -877,7 +889,7 @@ customize the environment using the arguments above, if necessary.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/hero/index.html b/environments/atari/hero/index.html
index 8a2fee7e2..bfb4f8a0a 100644
--- a/environments/atari/hero/index.html
+++ b/environments/atari/hero/index.html
@@ -15,7 +15,11 @@
         Hero - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -613,11 +625,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -787,7 +799,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/ice_hockey/index.html b/environments/atari/ice_hockey/index.html
index 5a72980aa..fdc825587 100644
--- a/environments/atari/ice_hockey/index.html
+++ b/environments/atari/ice_hockey/index.html
@@ -15,7 +15,11 @@
         IceHockey - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -612,11 +624,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -786,7 +798,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/index.html b/environments/atari/index.html
index 6ce752915..3f600c6e1 100644
--- a/environments/atari/index.html
+++ b/environments/atari/index.html
@@ -15,7 +15,11 @@
         Atari - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -1140,11 +1152,11 @@ URL: https://jair.org/index.php/jair/article/view/11182
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1314,7 +1326,7 @@ URL: https://jair.org/index.php/jair/article/view/11182
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/jamesbond/index.html b/environments/atari/jamesbond/index.html
index 582fe5f2d..83470fe38 100644
--- a/environments/atari/jamesbond/index.html
+++ b/environments/atari/jamesbond/index.html
@@ -15,7 +15,11 @@
         Jamesbond - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -613,11 +625,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -787,7 +799,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/journey_escape/index.html b/environments/atari/journey_escape/index.html
index 18ec5e3b2..db2d97cc7 100644
--- a/environments/atari/journey_escape/index.html
+++ b/environments/atari/journey_escape/index.html
@@ -15,7 +15,11 @@
         JourneyEscape - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -672,11 +684,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -846,7 +858,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/kangaroo/index.html b/environments/atari/kangaroo/index.html
index 716ccad0a..b8f169294 100644
--- a/environments/atari/kangaroo/index.html
+++ b/environments/atari/kangaroo/index.html
@@ -15,7 +15,11 @@
         Kangaroo - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -611,11 +623,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -785,7 +797,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/krull/index.html b/environments/atari/krull/index.html
index a92274062..8a7403981 100644
--- a/environments/atari/krull/index.html
+++ b/environments/atari/krull/index.html
@@ -15,7 +15,11 @@
         Krull - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -611,11 +623,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -785,7 +797,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/kung_fu_master/index.html b/environments/atari/kung_fu_master/index.html
index 69cddc078..415d45ce0 100644
--- a/environments/atari/kung_fu_master/index.html
+++ b/environments/atari/kung_fu_master/index.html
@@ -15,7 +15,11 @@
         Kung Fu Master - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -649,11 +661,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -823,7 +835,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/montezuma_revenge/index.html b/environments/atari/montezuma_revenge/index.html
index c7a7be906..2fd6867d7 100644
--- a/environments/atari/montezuma_revenge/index.html
+++ b/environments/atari/montezuma_revenge/index.html
@@ -15,7 +15,11 @@
         Montezuma Revenge - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -598,11 +610,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -772,7 +784,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/ms_pacman/index.html b/environments/atari/ms_pacman/index.html
index 37ce8c6ef..6c4e2f9fe 100644
--- a/environments/atari/ms_pacman/index.html
+++ b/environments/atari/ms_pacman/index.html
@@ -15,7 +15,11 @@
         Ms Pacman - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -634,11 +646,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -808,7 +820,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/name_this_game/index.html b/environments/atari/name_this_game/index.html
index 16d739fd9..dc1b5c1fe 100644
--- a/environments/atari/name_this_game/index.html
+++ b/environments/atari/name_this_game/index.html
@@ -15,7 +15,11 @@
         Name This Game - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -625,11 +637,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -799,7 +811,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/phoenix/index.html b/environments/atari/phoenix/index.html
index ebe7a6264..496ce4154 100644
--- a/environments/atari/phoenix/index.html
+++ b/environments/atari/phoenix/index.html
@@ -15,7 +15,11 @@
         Phoenix - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -631,11 +643,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -805,7 +817,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/pitfall/index.html b/environments/atari/pitfall/index.html
index 957d47749..234169051 100644
--- a/environments/atari/pitfall/index.html
+++ b/environments/atari/pitfall/index.html
@@ -15,7 +15,11 @@
         Pitfall - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -607,11 +619,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -781,7 +793,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/pong/index.html b/environments/atari/pong/index.html
index ad01b873c..a97423476 100644
--- a/environments/atari/pong/index.html
+++ b/environments/atari/pong/index.html
@@ -15,7 +15,11 @@
         Pong - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -638,11 +650,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -812,7 +824,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/pooyan/index.html b/environments/atari/pooyan/index.html
index b9ed9518c..e24a0f773 100644
--- a/environments/atari/pooyan/index.html
+++ b/environments/atari/pooyan/index.html
@@ -15,7 +15,11 @@
         Pooyan - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -638,11 +650,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -812,7 +824,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/private_eye/index.html b/environments/atari/private_eye/index.html
index 411a7d747..24b05ff74 100644
--- a/environments/atari/private_eye/index.html
+++ b/environments/atari/private_eye/index.html
@@ -15,7 +15,11 @@
         PrivateEye - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -607,11 +619,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -781,7 +793,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/qbert/index.html b/environments/atari/qbert/index.html
index 390d95bc2..c890be281 100644
--- a/environments/atari/qbert/index.html
+++ b/environments/atari/qbert/index.html
@@ -15,7 +15,11 @@
         Qbert - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -638,11 +650,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -812,7 +824,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/riverraid/index.html b/environments/atari/riverraid/index.html
index b8bdb4687..9bda17a95 100644
--- a/environments/atari/riverraid/index.html
+++ b/environments/atari/riverraid/index.html
@@ -15,7 +15,11 @@
         Riverraid - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -636,11 +648,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -810,7 +822,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/road_runner/index.html b/environments/atari/road_runner/index.html
index 7a07c1512..e3d85bee1 100644
--- a/environments/atari/road_runner/index.html
+++ b/environments/atari/road_runner/index.html
@@ -15,7 +15,11 @@
         Road Runner - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -637,11 +649,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -811,7 +823,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/robotank/index.html b/environments/atari/robotank/index.html
index 303eaaa5a..aaa76ebca 100644
--- a/environments/atari/robotank/index.html
+++ b/environments/atari/robotank/index.html
@@ -15,7 +15,11 @@
         Robot Tank - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -618,11 +630,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -792,7 +804,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/seaquest/index.html b/environments/atari/seaquest/index.html
index ec987d36e..8f4181e06 100644
--- a/environments/atari/seaquest/index.html
+++ b/environments/atari/seaquest/index.html
@@ -15,7 +15,11 @@
         Seaquest - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -626,11 +638,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -800,7 +812,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/skiing/index.html b/environments/atari/skiing/index.html
index 6b062de8b..2e73c116a 100644
--- a/environments/atari/skiing/index.html
+++ b/environments/atari/skiing/index.html
@@ -15,7 +15,11 @@
         Skiings - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -635,11 +647,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -809,7 +821,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/solaris/index.html b/environments/atari/solaris/index.html
index 5475dde51..8b946bc20 100644
--- a/environments/atari/solaris/index.html
+++ b/environments/atari/solaris/index.html
@@ -15,7 +15,11 @@
         Solaris - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -605,11 +617,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -779,7 +791,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/space_invaders/index.html b/environments/atari/space_invaders/index.html
index d3fc178f1..d45e24812 100644
--- a/environments/atari/space_invaders/index.html
+++ b/environments/atari/space_invaders/index.html
@@ -15,7 +15,11 @@
         SpaceInvaders - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -636,11 +648,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -810,7 +822,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/star_gunner/index.html b/environments/atari/star_gunner/index.html
index ee7de4192..1c8103cc4 100644
--- a/environments/atari/star_gunner/index.html
+++ b/environments/atari/star_gunner/index.html
@@ -15,7 +15,11 @@
         StarGunner - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -636,11 +648,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -810,7 +822,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/tennis/index.html b/environments/atari/tennis/index.html
index ad124c3d7..16666fc3c 100644
--- a/environments/atari/tennis/index.html
+++ b/environments/atari/tennis/index.html
@@ -15,7 +15,11 @@
         Tennis - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -606,11 +618,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -780,7 +792,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/time_pilot/index.html b/environments/atari/time_pilot/index.html
index 881b80c3e..479e39226 100644
--- a/environments/atari/time_pilot/index.html
+++ b/environments/atari/time_pilot/index.html
@@ -15,7 +15,11 @@
         TimePilot - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -664,11 +676,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -838,7 +850,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/tutankham/index.html b/environments/atari/tutankham/index.html
index 64bac0ece..08237dbe7 100644
--- a/environments/atari/tutankham/index.html
+++ b/environments/atari/tutankham/index.html
@@ -15,7 +15,11 @@
         Tutankham - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -631,11 +643,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -805,7 +817,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/up_n_down/index.html b/environments/atari/up_n_down/index.html
index 8d8a08e66..144635b69 100644
--- a/environments/atari/up_n_down/index.html
+++ b/environments/atari/up_n_down/index.html
@@ -15,7 +15,11 @@
         Up n’ Down - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -627,11 +639,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -801,7 +813,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/venture/index.html b/environments/atari/venture/index.html
index 88ea12f5d..160703c73 100644
--- a/environments/atari/venture/index.html
+++ b/environments/atari/venture/index.html
@@ -15,7 +15,11 @@
         Venture - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -598,11 +610,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -772,7 +784,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/video_pinball/index.html b/environments/atari/video_pinball/index.html
index f64113274..652a8a97c 100644
--- a/environments/atari/video_pinball/index.html
+++ b/environments/atari/video_pinball/index.html
@@ -15,7 +15,11 @@
         Video Pinball - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -636,11 +648,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -810,7 +822,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/wizard_of_wor/index.html b/environments/atari/wizard_of_wor/index.html
index 0997ed5a9..05a4f863a 100644
--- a/environments/atari/wizard_of_wor/index.html
+++ b/environments/atari/wizard_of_wor/index.html
@@ -15,7 +15,11 @@
         Wizard of Wor - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -639,11 +651,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -813,7 +825,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/atari/zaxxon/index.html b/environments/atari/zaxxon/index.html
index 97e425e37..dc29bd786 100644
--- a/environments/atari/zaxxon/index.html
+++ b/environments/atari/zaxxon/index.html
@@ -15,7 +15,11 @@
         Zaxxon - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -588,11 +600,11 @@ general article on Atari environments.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -762,7 +774,7 @@ general article on Atari environments.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/box2d/bipedal_walker/index.html b/environments/box2d/bipedal_walker/index.html
index a20bfb7c3..216966090 100644
--- a/environments/box2d/bipedal_walker/index.html
+++ b/environments/box2d/bipedal_walker/index.html
@@ -15,7 +15,11 @@
         Bipedal Walker - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -585,11 +597,11 @@ speed; ground has higher friction; lidar rendered less nervously.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -759,7 +771,7 @@ speed; ground has higher friction; lidar rendered less nervously.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/box2d/car_racing/index.html b/environments/box2d/car_racing/index.html
index 3ed20a7f2..a0f2eed02 100644
--- a/environments/box2d/car_racing/index.html
+++ b/environments/box2d/car_racing/index.html
@@ -15,7 +15,11 @@
         Car Racing - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -626,11 +638,11 @@ Example usage:
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -800,7 +812,7 @@ Example usage:
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/box2d/index.html b/environments/box2d/index.html
index 217bd8476..eeb9422d1 100644
--- a/environments/box2d/index.html
+++ b/environments/box2d/index.html
@@ -15,7 +15,11 @@
         Box2D - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -465,7 +477,7 @@
     
 
    These environments all involve toy games based around physics control, using box2d based physics and PyGame-based rendering. These environments were contributed back in the early days of OpenAI Gym by Oleg Klimov, and have become popular toy benchmarks ever since. All environments are highly configurable via arguments specified in each environment’s documentation.
    
 
    The unique dependencies for this set of environments can be installed via:
    
-
    pip install gymnasium[box2d]
+
    pip install gymnasium[box2d]
 
    
 
    
 
    
@@ -510,11 +522,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -684,7 +696,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/box2d/lunar_lander/index.html b/environments/box2d/lunar_lander/index.html
index 2c39272e5..9bbaf5901 100644
--- a/environments/box2d/lunar_lander/index.html
+++ b/environments/box2d/lunar_lander/index.html
@@ -15,7 +15,11 @@
         Lunar Lander - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -639,11 +651,11 @@ renormalized to 200; harder initial random push.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -813,7 +825,7 @@ renormalized to 200; harder initial random push.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/classic_control/acrobot/index.html b/environments/classic_control/acrobot/index.html
index be3a7c67c..2b8e9f242 100644
--- a/environments/classic_control/acrobot/index.html
+++ b/environments/classic_control/acrobot/index.html
@@ -15,7 +15,11 @@
         Acrobot - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -681,11 +693,11 @@ MIT Press. https://proceedings.neurips.cc/paper/1995/file/8f1d43620bc6bb580df6e8
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -855,7 +867,7 @@ MIT Press. https://proceedings.neurips.cc/paper/1995/file/8f1d43620bc6bb580df6e8
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/classic_control/cart_pole/index.html b/environments/classic_control/cart_pole/index.html
index f5527e3f8..10da70411 100644
--- a/environments/classic_control/cart_pole/index.html
+++ b/environments/classic_control/cart_pole/index.html
@@ -15,7 +15,11 @@
         Cart Pole - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -615,11 +627,11 @@ the new random state.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -789,7 +801,7 @@ the new random state.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/classic_control/index.html b/environments/classic_control/index.html
index 494317935..0b75880da 100644
--- a/environments/classic_control/index.html
+++ b/environments/classic_control/index.html
@@ -15,7 +15,11 @@
         Classic Control - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -488,7 +500,7 @@
 
     
 
    The unique dependencies for this set of environments can be installed via:
    
-
    pip install gymnasium[classic_control]
+
    pip install gymnasium[classic-control]
 
    
 
    
 
    There are five classic control environments: Acrobot, CartPole, Mountain Car, Continuous Mountain Car, and Pendulum. All of these environments are stochastic in terms of their initial state, within a given range. In addition, Acrobot has noise applied to the taken action. Also, regarding both mountain car environments, the cars are underpowered to climb the mountain, so it takes some effort to reach the top.
    
@@ -536,11 +548,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -710,7 +722,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/classic_control/mountain_car/index.html b/environments/classic_control/mountain_car/index.html
index 693fe20db..ba7325768 100644
--- a/environments/classic_control/mountain_car/index.html
+++ b/environments/classic_control/mountain_car/index.html
@@ -15,7 +15,11 @@
         Mountain Car - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -625,11 +637,11 @@ the new random state.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -799,7 +811,7 @@ the new random state.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/classic_control/mountain_car_continuous/index.html b/environments/classic_control/mountain_car_continuous/index.html
index 59e12135b..41d2becc4 100644
--- a/environments/classic_control/mountain_car_continuous/index.html
+++ b/environments/classic_control/mountain_car_continuous/index.html
@@ -15,7 +15,11 @@
         Mountain Car Continuous - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -622,11 +634,11 @@ the new random state.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -796,7 +808,7 @@ the new random state.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/classic_control/pendulum/index.html b/environments/classic_control/pendulum/index.html
index 14e08ed7e..fb7ffdcb0 100644
--- a/environments/classic_control/pendulum/index.html
+++ b/environments/classic_control/pendulum/index.html
@@ -15,7 +15,11 @@
         Pendulum - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -624,11 +636,11 @@ the new random state.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -798,7 +810,7 @@ the new random state.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/ant/index.html b/environments/mujoco/ant/index.html
index e9bb8e7ef..aad2c16e1 100644
--- a/environments/mujoco/ant/index.html
+++ b/environments/mujoco/ant/index.html
@@ -15,7 +15,11 @@
         Ant - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -541,11 +553,11 @@ followed by the velocities of those individual parts (their derivatives) with al
 the positions ordered before all the velocities.
    
 
    By default, observations do not include the x- and y-coordinates of the ant’s torso. These may
 be included by passing exclude_current_positions_from_observation=False during construction.
-In that case, the observation space will have 113 dimensions where the first two dimensions
+In that case, the observation space will have 29 dimensions where the first two dimensions
 represent the x- and y- coordinates of the ant’s torso.
 Regardless of whether exclude_current_positions_from_observation was set to true or false, the x- and y-coordinates
 of the torso will be returned in info with keys "x_position" and "y_position", respectively.
    
-
    However, by default, an observation is a ndarray with shape (111,)
+
    However, by default, an observation is a ndarray with shape (27,)
 where the elements correspond to the following:
    
 
    
 
@@ -808,7 +820,10 @@ where ctr_cost_
 force is too large. It is calculated contact_cost_weight * sum(clip(external contact
 force to contact_force_range)2).
    
 
-
    The total reward returned is reward = healthy_reward + forward_reward - ctrl_cost - contact_cost and info will also contain the individual reward terms.
    
+
    The total reward returned is reward = healthy_reward + forward_reward - ctrl_cost.
    
+
    But if use_contact_forces=True
+The total reward returned is reward = healthy_reward + forward_reward - ctrl_cost - contact_cost.
    
+
    In either case info will also contain the individual reward terms.
    
 
 
    
 
    Starting State#
    
@@ -870,7 +885,7 @@ the episode ends when any of the following happens:
    
 
    
-
+
@@ -997,11 +1012,11 @@ the episode ends when any of the following happens:
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1171,7 +1186,7 @@ the episode ends when any of the following happens:
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/half_cheetah/index.html b/environments/mujoco/half_cheetah/index.html
index a03339946..961c34e47 100644
--- a/environments/mujoco/half_cheetah/index.html
+++ b/environments/mujoco/half_cheetah/index.html
@@ -15,7 +15,11 @@
         Half Cheetah - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -862,11 +874,11 @@ initial velocity values of all zeros.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1036,7 +1048,7 @@ initial velocity values of all zeros.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/hopper/index.html b/environments/mujoco/hopper/index.html
index e9a3f7eae..b705efb0d 100644
--- a/environments/mujoco/hopper/index.html
+++ b/environments/mujoco/hopper/index.html
@@ -15,7 +15,11 @@
         Hopper - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -483,7 +495,7 @@ connecting the four body parts.
    
     
-
+
@@ -825,11 +837,11 @@ the episode ends when any of the following happens:
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -999,7 +1011,7 @@ the episode ends when any of the following happens:
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/humanoid/index.html b/environments/mujoco/humanoid/index.html
index 3dcf7b736..0c25aff26 100644
--- a/environments/mujoco/humanoid/index.html
+++ b/environments/mujoco/humanoid/index.html
@@ -15,7 +15,11 @@
         Humanoid - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -467,7 +479,7 @@ elbows respectively). The goal of the environment is to walk forward as fast as
     
-
+
@@ -475,7 +487,7 @@ elbows respectively). The goal of the environment is to walk forward as fast as
 
-
+
@@ -483,7 +495,7 @@ elbows respectively). The goal of the environment is to walk forward as fast as
 
-
+
@@ -1224,11 +1236,11 @@ the episode ends when any of the following happens:
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1398,7 +1410,7 @@ the episode ends when any of the following happens:
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/humanoid_standup/index.html b/environments/mujoco/humanoid_standup/index.html
index 15b1e56b5..05b30c54c 100644
--- a/environments/mujoco/humanoid_standup/index.html
+++ b/environments/mujoco/humanoid_standup/index.html
@@ -15,7 +15,11 @@
         Humanoid Standup - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -471,7 +483,7 @@ represents the numerical torques applied at the hinge joints.
    
     
-
+
@@ -479,7 +491,7 @@ represents the numerical torques applied at the hinge joints.
    
     
-
+
@@ -487,7 +499,7 @@ represents the numerical torques applied at the hinge joints.
    
     
-
+
@@ -1160,11 +1172,11 @@ beyond take gymnasium.make kwargs such as xml_file, ctrl_cost_weight, reset_nois
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1334,7 +1346,7 @@ beyond take gymnasium.make kwargs such as xml_file, ctrl_cost_weight, reset_nois
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/index.html b/environments/mujoco/index.html
index 332439db3..fdd5329a4 100644
--- a/environments/mujoco/index.html
+++ b/environments/mujoco/index.html
@@ -15,7 +15,11 @@
         MuJoCo - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -561,7 +573,7 @@
     
 
    MuJoCo stands for Multi-Joint dynamics with Contact. It is a physics engine for facilitating research and development in robotics, biomechanics, graphics and animation, and other areas where fast and accurate simulation is needed.
    
 
    The unique dependencies for this set of environments can be installed via:
    
-
    pip install gymnasium[mujoco]
+
    pip install gymnasium[mujoco]
 
    
 
    
 
    These environments also require that the MuJoCo engine be installed. As of October 2021 DeepMind has acquired MuJoCo and is open-sourcing it in 2022, making it free for everyone. Instructions on installing the MuJoCo engine can be found on their website and GitHub repository. Using MuJoCo with Gymnasium also requires that the framework mujoco be installed (this dependency is installed with the above command).
    
@@ -611,11 +623,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -785,7 +797,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/inverted_double_pendulum/index.html b/environments/mujoco/inverted_double_pendulum/index.html
index 14983fb5b..109dd3524 100644
--- a/environments/mujoco/inverted_double_pendulum/index.html
+++ b/environments/mujoco/inverted_double_pendulum/index.html
@@ -15,7 +15,11 @@
         Inverted Double Pendulum - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -742,11 +754,11 @@ beyond take gymnasi
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -916,7 +928,7 @@ beyond take gymnasi
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/inverted_pendulum/index.html b/environments/mujoco/inverted_pendulum/index.html
index 60d8e9eef..cf9f505f2 100644
--- a/environments/mujoco/inverted_pendulum/index.html
+++ b/environments/mujoco/inverted_pendulum/index.html
@@ -15,7 +15,11 @@
         Inverted Pendulum - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -667,11 +679,11 @@ v3 and beyond take 
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -841,7 +853,7 @@ v3 and beyond take 
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/pusher/index.html b/environments/mujoco/pusher/index.html
index eb4bc3f85..f7d9bd779 100644
--- a/environments/mujoco/pusher/index.html
+++ b/environments/mujoco/pusher/index.html
@@ -15,7 +15,11 @@
         Pusher - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -887,11 +899,11 @@ beyond take gymnasm
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1061,7 +1073,7 @@ beyond take gymnasm
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/reacher/index.html b/environments/mujoco/reacher/index.html
index 96824900e..653f0163e 100644
--- a/environments/mujoco/reacher/index.html
+++ b/environments/mujoco/reacher/index.html
@@ -15,7 +15,11 @@
         Reacher - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -524,7 +536,7 @@ target that is spawned at a random position.
    
 
    
-
+
@@ -532,7 +544,7 @@ target that is spawned at a random position.
    
     
-
+
@@ -585,7 +597,7 @@ target that is spawned at a random position.
    
     
-
+
@@ -794,11 +806,11 @@ beyond take gymnasi
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -968,7 +980,7 @@ beyond take gymnasi
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/swimmer/index.html b/environments/mujoco/swimmer/index.html
index 9049aedc8..0a42af0c9 100644
--- a/environments/mujoco/swimmer/index.html
+++ b/environments/mujoco/swimmer/index.html
@@ -15,7 +15,11 @@
         Swimmer - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -770,11 +782,11 @@ control and has a default value of 1e-4
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -944,7 +956,7 @@ control and has a default value of 1e-4
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/mujoco/walker2d/index.html b/environments/mujoco/walker2d/index.html
index c69a5f6be..0358e843a 100644
--- a/environments/mujoco/walker2d/index.html
+++ b/environments/mujoco/walker2d/index.html
@@ -15,7 +15,11 @@
         Walker2D - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -882,11 +894,11 @@ the episode ends when any of the following happens:
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1056,7 +1068,7 @@ the episode ends when any of the following happens:
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/third_party_environments/index.html b/environments/third_party_environments/index.html
index 73ce0f6a7..5d69c7563 100644
--- a/environments/third_party_environments/index.html
+++ b/environments/third_party_environments/index.html
@@ -8,14 +8,18 @@
     
     
     
-
+
         
 
     
         Third-party Environments - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -608,12 +620,12 @@
       
    
         
         
    
-          
+          
               
    
                 
    
                   Next
                 
    
-                
    Solving Blackjack with Q-Learning
    
+                
    Gymnasium Basics
    
               
    
               
                 
@@ -683,11 +695,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -857,7 +869,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/toy_text/blackjack/index.html b/environments/toy_text/blackjack/index.html
index 7a29487d8..4141039c5 100644
--- a/environments/toy_text/blackjack/index.html
+++ b/environments/toy_text/blackjack/index.html
@@ -15,7 +15,11 @@
         Blackjack - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -632,11 +644,11 @@ An Introduction” 2020. [Online]. Available:  {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/toy_text/cliff_walking/index.html b/environments/toy_text/cliff_walking/index.html
index 809a4d393..c767fe832 100644
--- a/environments/toy_text/cliff_walking/index.html
+++ b/environments/toy_text/cliff_walking/index.html
@@ -15,7 +15,11 @@
         Cliff Walking - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
     
 
    Development
    
 
 
@@ -594,11 +606,11 @@ An Introduction” 2020. [Online]. Available:  {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/toy_text/frozen_lake/index.html b/environments/toy_text/frozen_lake/index.html
index bcb76f36e..b2c150d3e 100644
--- a/environments/toy_text/frozen_lake/index.html
+++ b/environments/toy_text/frozen_lake/index.html
@@ -15,7 +15,11 @@
         Frozen Lake - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
     
 
    Development
    
 
 
@@ -649,11 +661,11 @@ equal probability of 1/3 in both directions.
    
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -823,7 +835,7 @@ equal probability of 1/3 in both directions.
    
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/toy_text/index.html b/environments/toy_text/index.html
index 9e0970567..227c65c5d 100644
--- a/environments/toy_text/index.html
+++ b/environments/toy_text/index.html
@@ -15,7 +15,11 @@
         Toy Text - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -519,11 +531,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -693,7 +705,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/environments/toy_text/taxi/index.html b/environments/toy_text/taxi/index.html
index 9d386e631..fce2061fa 100644
--- a/environments/toy_text/taxi/index.html
+++ b/environments/toy_text/taxi/index.html
@@ -15,7 +15,11 @@
         Taxi - Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -643,11 +655,11 @@ Journal of Artificial Intelligence Research, vol. 13, pp. 227–303, Nov. 2000,
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -817,7 +829,7 @@ Journal of Artificial Intelligence Research, vol. 13, pp. 227–303, Nov. 2000,
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/genindex/index.html b/genindex/index.html
index 69748877a..e73f31048 100644
--- a/genindex/index.html
+++ b/genindex/index.html
@@ -13,7 +13,11 @@
     Index - Gymnasium Documentation
 
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -361,15 +365,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -1062,11 +1074,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -1236,7 +1248,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/index.html b/index.html
index 0893d77c1..747fcaaad 100644
--- a/index.html
+++ b/index.html
@@ -15,7 +15,11 @@
         Gymnasium Documentation
       
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -363,15 +367,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -488,11 +500,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -662,7 +674,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/objects.inv b/objects.inv
index 29ee2efa5..d7e7d1e3d 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/py-modindex/index.html b/py-modindex/index.html
index 5276f493f..c3207f3ea 100644
--- a/py-modindex/index.html
+++ b/py-modindex/index.html
@@ -13,7 +13,11 @@
     Python Module Index - Gymnasium Documentation
 
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -361,15 +365,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -478,11 +490,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -652,7 +664,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/release_notes/index.html b/release_notes/index.html
new file mode 100644
index 000000000..4f68f44bb
--- /dev/null
+++ b/release_notes/index.html
@@ -0,0 +1,889 @@
+
+
+  
+    
+    
+    
+    
+    
+    
+    
+
+        
+
+    
+        Release Notes - Gymnasium Documentation
+      
+    
+    
+    
+    
+    
+    
+    
+    
+
+
+
+  
+    
    
+      
    
+        
    
+          
+        
    
+        
    
+          
+              
+              
+            Gymnasium Documentation
+          
+        
    
+        
    
+          
    
+            
+            
+          
    
+        
    
+      
    
+    
    
+
+    
+    
+    
+
+
+  
+    Contents
+    
+      
+    
+  
+  
+    Menu
+    
+      
+      
+      
+    
+  
+  
+    Expand
+    
+      
+    
+  
+  
+    Light mode
+    
+      
+      
+      
+      
+      
+      
+      
+      
+      
+    
+  
+  
+    Dark mode
+    
+      
+      
+    
+  
+  
+    Auto light/dark mode
+    
+      
+      
+      
+      
+      
+      
+      
+    
+  
+
+
+
+
+
+
+
+
+
+
    
+   
+  
+  
    
+    
    
+      
    
+        
+          
+            
+          
+          Back to top
+        
+        
    
+          
+    
+    
    
+  
+    
+    
+  
+
    
+            
+          
    
+          
+        
    
+        
    
+          
+          
    
+
    Release Notes#
    
+
    
+
    v0.27.1#
    
+
    Released on 2023-01-20 - GitHub - PyPI
    
+
    Release Notes
    
+
    Bugs fixed
    
+
+
    New features/improvements
    
+
      
+
    • Improve LunarLander-v2 step performance by >1.5x by @PaulMest in #235
      • 
+
    • Added vector env support to StepAPICompatibility wrapper by @nidhishs in #238
      • 
+
    • Allow sequence to accept stacked np arrays if the feature space is Box by @jjshoots in #241
      • 
+
    • Improve the warning when an error is raised from a plugin by @pseudo-rnd-thoughts in #225
      • 
+
    • Add changelog (release notes) to the website by @mgoulao in #257
      • 
+
    • Implement RecordVideoV0 by @younik in #246
      • 
+
    • Add explicit error messages when unflatten discrete and multidiscrete fail by @PierreMardon in #267
      • 
+
    
+
    Documentation updates
    
+
+
    Thanks to the new contributors to Gymnasium, if you want to get involved, join our discord server. Linked in the readme.
    
+
+
    Full Changelog: v0.27.0...v0.27.1
    
+
    
+
    v0.27.0#
    
+
    Released on 2022-12-12 - GitHub - PyPI
    
+
    Release Notes
    
+
    Gymnasium 0.27.0 is our first major release of Gymnasium. It has several significant new features, and numerous small bug fixes and code quality improvements as we work through our backlog. There should be no breaking changes beyond dropping Python 3.6 support and remove the mujoco Viewer class in favor of a MujocoRendering class. You should be able to upgrade your code that's using Gymnasium 0.26.x to 0.27.0 with little-to-no-effort.
    
+
    Like always, our development roadmap is publicly available here so you can follow our future plans. The only large breaking changes that are still planned are switching selected environments to use hardware accelerated physics engines and our long standing plans for overhauling the vector API and built-in wrappers.
    
+
    This release notably includes an entirely new part of the library: gymnasium.experimental. We are adding new features, wrappers and functional environment API discussed below for users to test and try out to find bugs and provide feedback.
    
+
    New Wrappers
    
+
    These new wrappers, accessible in gymnasium.experimental.wrappers, see the full list in https://gymnasium.farama.org/main/api/experimental/ are aimed to replace the wrappers in gymnasium v0.30.0 and contain several improvements
    
+
      
+
    • (Work in progress) Support arbitrarily complex observation / action spaces. As RL has advanced, action and observation spaces are becoming more complex and the current wrappers were not implemented with this mind.
      • 
+
    • Support for Jax-based environments. With hardware accelerated environments, i.e. Brax, written in Jax and similar PyTorch based programs, NumPy is not the only game in town anymore for writing environments. Therefore, these upgrades will use Jumpy, a project developed by Farama Foundation to provide automatic compatibility for NumPy, Jax and in the future PyTorch data for a large subset of the NumPy functions.
      • 
+
    • More wrappers. Projects like Supersuit aimed to bring more wrappers for RL, however, many users were not aware of the wrappers, so we plan to move the wrappers into Gymnasium. If we are missing common wrappers from the list provided above, please create an issue and we would be interested in adding it.
      • 
+
    • Versioning. Like environments, the implementation details of wrappers can cause changes in agent performance. Therefore, we propose adding version numbers to all wrappers, i.e., LambaActionV0. We don't expect these version numbers to change regularly and will act similarly to environment version numbers. This should ensure that all users know when significant changes could affect your agent's performance for environments and wrappers. Additionally, we hope that this will improve reproducibility of RL in the future, which is critical for academia.
      • 
+
    • In v28, we aim to rewrite the VectorEnv to not inherit from Env, as a result new vectorized versions of the wrappers will be provided.
      • 
+
    
+
    Core developers: @gianlucadecola, @RedTachyon, @pseudo-rnd-thoughts
    
+
    Functional API
    
+
    The Env class provides a very generic structure for environments to be written in allowing high flexibility in the program structure. However, this limits the ability to efficiently vectorize environments, compartmentalize the environment code, etc. Therefore, the gymnasium.experimental.FuncEnv provides a much more strict structure for environment implementation with stateless functions, for every stage of the environment implementation. This class does not inherit from Env and requires a translation / compatibility class for doing this. We already provide a FuncJaxEnv for converting jax-based FuncEnv to Env. We hope this will help improve the readability of environment implementations along with potential speed-ups for users that vectorize their code.
    
+
    This API is very experimental so open to changes in the future. We are interested in feedback from users who try to use the API which we believe will be in particular interest to users exploring RL planning, model-based RL and modifying environment functions like the rewards.
    
+
    Core developers: @RedTachyon, @pseudo-rnd-thoughts, @balisujohn
    
+
    Other Major changes
    
+
      
+
    • Refactor Mujoco Rendering mechanisms to use a separate thread for OpenGL. Remove Viewer in favor of MujocoRenderer which offscreen, human and other render mode can use by @rodrigodelazcano in #112
      • 
+
    • Add deprecation warning to gym.make(..., apply_env_compatibility=True) in favour of gym.make("GymV22Environment", env_id="...") by @pseudo-rnd-thoughts in #125
      • 
+
    • Add gymnasium.pprint_registry() for pretty printing the gymnasium registry by @kad99kev in #124
      • 
+
    • Changes discrete dtype to np.int64 such that samples are np.int64 not python ints. by @pseudo-rnd-thoughts in #141
      • 
+
    • Add migration guide for OpenAI Gym v21 to v26 by @pseudo-rnd-thoughts in #72
      • 
+
    • Add complete type hinting of core.py for Env, Wrapper and more by @pseudo-rnd-thoughts in #39
      • 
+
    • Add complete type hinting for all spaces in gymnasium.spaces by @pseudo-rnd-thoughts in #37
      • 
+
    • Make window in play() resizable by @Markus28 in #190
      • 
+
    • Add REINFORCE implementation tutorial by @siddarth-c in #155
      • 
+
    
+
    Bug fixes and documentation changes
    
+
+
    Behind-the-scenes changes
    
+
    
+
    
+
    v0.26.3#
    
+
    Released on 2022-10-24 - GitHub - PyPI
    
+
    Release Notes
    
+
    Note: ale-py (atari) has not updated to Gymnasium yet. Therefore pip install gymnasium[atari] will fail, this will be fixed in v0.27. In the meantime, use pip install shimmy[atari] for the fix.
    
+
    Bug Fixes
    
+
      
+
    • Added Gym-Gymnasium compatibility converter to allow users to use Gym environments in Gymnasium by @RedTachyon in #61
      • 
+
    • Modify metadata in the HumanRendering and RenderCollection wrappers to have the correct metadata by @RedTachyon in #35
      • 
+
    • Simplified EpisodeStatisticsRecorder wrapper  by @DavidSlayback in #31
      • 
+
    • Fix integer overflow in MultiDiscrete.flatten() by @olipinski in #55
      • 
+
    • Re-add the ability to specify the XML file for Mujoco environments by @Kallinteris-Andreas in #70
      • 
+
    
+
    Documentation change
    
+
+
    Full Changelog: v0.26.2...v0.26.3
    
+
    Thank you for the new contributors
    
+
    
+
    
+
    v0.26.2: 0.26.2 #
    
+
    Released on 2022-10-05 - GitHub - PyPI
    
+
    This Release is an upstreamed version of Gym v26.2
    
+
    Bugs Fixes
    
+
      
+
    • As reset now returns (obs, info) then in the vector environments, this caused the final step's info to be overwritten. Now, the final observation and info are contained within the info as "final_observation" and "final_info" @pseudo-rnd-thoughts
      • 
+
    • Adds warnings when trying to render without specifying the render_mode @younik
      • 
+
    • Updates Atari Preprocessing such that the wrapper can be pickled @vermouth1992
      • 
+
    • Github CI was hardened to such that the CI just has read permissions @sashashura
      • 
+
    • Clarify and fix typo in GraphInstance @ekalosak
      • 
+
    
+
    
+
    v0.26.1: 0.26.1#
    
+
    Released on 2022-09-16 - GitHub - PyPI
    
+
    This Release is an upstreamed version of Gym v26.1
    
+
    In addition, the gym docs repo has been merged in with the new website https://gymnasium.farama.org/
    
+
    
+
    v0.26.0: Initial Release#
    
+
    Released on 2022-09-13 - GitHub - PyPI
    
+
    This is the first release of Gymnasium, a maintained fork of OpenAI Gym
    
+
    This release is identical to the Gym v0.26.0 except for the project name (Gymnasium) and Code of Conduct
    
+
    Read #12 for the roadmap of changes
    
+
    
+
    Gym Release Notes#
    
+
+
    
+
    
+
+        
    
+      
    
+      
    
+        
+        
    
+          
+            
+        
    
+        
    
+          
    
+            
    
+                Copyright © 2022 Farama Foundation
+            
    
+            
+          
    
+          
    
+            
    
+              
+            
    
+          
    
+        
    
+        
+      
    
+    
    
+    
+  
    
+
    
+    
+
+    
+    
+
+      
+      
+
+    
+    
+    
+    
+    
+    
+    
+    
+    
+
+    
+    
+
\ No newline at end of file
diff --git a/search/index.html b/search/index.html
index ecedcbdec..17687bb86 100644
--- a/search/index.html
+++ b/search/index.html
@@ -12,7 +12,11 @@
 
     Search - Gymnasium Documentation
     
-    
+    
+    
+    
+    
+    
     
     
 
@@ -360,15 +364,23 @@
 
 
    Tutorials
    
 
 
    Development
    
 
 
@@ -456,11 +468,11 @@
         const menuBtn = document.querySelector(".farama-header-menu__btn");
         const menuContainer = document.querySelector(".farama-header-menu-container");
         if (document.querySelector(".farama-header-menu").classList.contains("active")) {
-          menuBtn.ariaExpanded = "false";
-          menuContainer.ariaHidden = "true";
+          menuBtn.setAttribute("aria-expanded", "false");
+          menuContainer.setAttribute("aria-hidden", "true");
         } else {
-          menuBtn.ariaExpanded = "true";
-          menuContainer.ariaHidden = "false";
+          menuBtn.setAttribute("aria-expanded", "true");
+          menuContainer.setAttribute("aria-hidden", "false");
         }
         document.querySelector(".farama-header-menu").classList.toggle("active");
       }
@@ -632,7 +644,7 @@
         githubUser: 'Farama-Foundation',
         githubRepo: 'Gymnasium',
       };
-      fetch('/_static/versioning/versioning_menu.html').then(response => {
+      fetch('/main/_static/versioning/versioning_menu.html').then(response => {
         if (response.status === 200) {
             response.text().then(text => {
                 const container = document.createElement("div");
diff --git a/searchindex.js b/searchindex.js
index fec8b458c..e9fd376fe 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["404", "README", "api/env", "api/experimental", "api/experimental/functional", "api/experimental/vector", "api/experimental/vector_wrappers", "api/experimental/wrappers", "api/registry", "api/spaces", "api/spaces/composite", "api/spaces/fundamental", "api/spaces/utils", "api/spaces/vector_utils", "api/utils", "api/vector", "api/wrappers", "api/wrappers/action_wrappers", "api/wrappers/misc_wrappers", "api/wrappers/observation_wrappers", "api/wrappers/reward_wrappers", "content/basic_usage", "content/gym_compatibility", "content/migration-guide", "environments/atari", "environments/atari/adventure", "environments/atari/air_raid", "environments/atari/alien", "environments/atari/amidar", "environments/atari/assault", "environments/atari/asterix", "environments/atari/asteroids", "environments/atari/atlantis", "environments/atari/bank_heist", "environments/atari/battle_zone", "environments/atari/beam_rider", "environments/atari/berzerk", "environments/atari/bowling", "environments/atari/boxing", "environments/atari/breakout", "environments/atari/carnival", "environments/atari/centipede", "environments/atari/chopper_command", "environments/atari/complete_list", "environments/atari/crazy_climber", "environments/atari/defender", "environments/atari/demon_attack", "environments/atari/double_dunk", "environments/atari/elevator_action", "environments/atari/enduro", "environments/atari/fishing_derby", "environments/atari/freeway", "environments/atari/frostbite", "environments/atari/gopher", "environments/atari/gravitar", "environments/atari/hero", "environments/atari/ice_hockey", "environments/atari/jamesbond", "environments/atari/journey_escape", "environments/atari/kangaroo", "environments/atari/krull", "environments/atari/kung_fu_master", "environments/atari/montezuma_revenge", "environments/atari/ms_pacman", "environments/atari/name_this_game", "environments/atari/phoenix", "environments/atari/pitfall", "environments/atari/pong", "environments/atari/pooyan", "environments/atari/private_eye", "environments/atari/qbert", "environments/atari/riverraid", "environments/atari/road_runner", "environments/atari/robotank", "environments/atari/seaquest", "environments/atari/skiing", "environments/atari/solaris", "environments/atari/space_invaders", "environments/atari/star_gunner", "environments/atari/tennis", "environments/atari/time_pilot", "environments/atari/tutankham", "environments/atari/up_n_down", "environments/atari/venture", "environments/atari/video_pinball", "environments/atari/wizard_of_wor", "environments/atari/zaxxon", "environments/box2d", "environments/box2d/bipedal_walker", "environments/box2d/car_racing", "environments/box2d/lunar_lander", "environments/classic_control", "environments/classic_control/acrobot", "environments/classic_control/cart_pole", "environments/classic_control/mountain_car", "environments/classic_control/mountain_car_continuous", "environments/classic_control/pendulum", "environments/mujoco", "environments/mujoco/ant", "environments/mujoco/half_cheetah", "environments/mujoco/hopper", "environments/mujoco/humanoid", "environments/mujoco/humanoid_standup", "environments/mujoco/inverted_double_pendulum", "environments/mujoco/inverted_pendulum", "environments/mujoco/pusher", "environments/mujoco/reacher", "environments/mujoco/swimmer", "environments/mujoco/walker2d", "environments/third_party_environments", "environments/toy_text", "environments/toy_text/blackjack", "environments/toy_text/cliff_walking", "environments/toy_text/frozen_lake", "environments/toy_text/taxi", "index", "tutorials/blackjack_tutorial", "tutorials/environment_creation", "tutorials/handling_time_limits", "tutorials/implementing_custom_wrappers", "tutorials/reinforce_invpend_gym_v26"], "filenames": ["404.md", "README.md", "api/env.md", "api/experimental.md", "api/experimental/functional.md", "api/experimental/vector.md", "api/experimental/vector_wrappers.md", "api/experimental/wrappers.md", "api/registry.md", "api/spaces.md", "api/spaces/composite.md", "api/spaces/fundamental.md", "api/spaces/utils.md", "api/spaces/vector_utils.md", "api/utils.md", "api/vector.md", "api/wrappers.md", "api/wrappers/action_wrappers.md", "api/wrappers/misc_wrappers.md", "api/wrappers/observation_wrappers.md", "api/wrappers/reward_wrappers.md", "content/basic_usage.md", "content/gym_compatibility.md", "content/migration-guide.md", "environments/atari.md", "environments/atari/adventure.md", "environments/atari/air_raid.md", "environments/atari/alien.md", "environments/atari/amidar.md", "environments/atari/assault.md", "environments/atari/asterix.md", "environments/atari/asteroids.md", "environments/atari/atlantis.md", "environments/atari/bank_heist.md", "environments/atari/battle_zone.md", "environments/atari/beam_rider.md", "environments/atari/berzerk.md", "environments/atari/bowling.md", "environments/atari/boxing.md", "environments/atari/breakout.md", "environments/atari/carnival.md", "environments/atari/centipede.md", "environments/atari/chopper_command.md", "environments/atari/complete_list.md", "environments/atari/crazy_climber.md", "environments/atari/defender.md", "environments/atari/demon_attack.md", "environments/atari/double_dunk.md", "environments/atari/elevator_action.md", "environments/atari/enduro.md", "environments/atari/fishing_derby.md", "environments/atari/freeway.md", "environments/atari/frostbite.md", "environments/atari/gopher.md", "environments/atari/gravitar.md", "environments/atari/hero.md", "environments/atari/ice_hockey.md", "environments/atari/jamesbond.md", "environments/atari/journey_escape.md", "environments/atari/kangaroo.md", "environments/atari/krull.md", "environments/atari/kung_fu_master.md", "environments/atari/montezuma_revenge.md", "environments/atari/ms_pacman.md", "environments/atari/name_this_game.md", "environments/atari/phoenix.md", "environments/atari/pitfall.md", "environments/atari/pong.md", "environments/atari/pooyan.md", "environments/atari/private_eye.md", "environments/atari/qbert.md", "environments/atari/riverraid.md", "environments/atari/road_runner.md", "environments/atari/robotank.md", "environments/atari/seaquest.md", "environments/atari/skiing.md", "environments/atari/solaris.md", "environments/atari/space_invaders.md", "environments/atari/star_gunner.md", "environments/atari/tennis.md", "environments/atari/time_pilot.md", "environments/atari/tutankham.md", "environments/atari/up_n_down.md", "environments/atari/venture.md", "environments/atari/video_pinball.md", "environments/atari/wizard_of_wor.md", "environments/atari/zaxxon.md", "environments/box2d.md", "environments/box2d/bipedal_walker.md", "environments/box2d/car_racing.md", "environments/box2d/lunar_lander.md", "environments/classic_control.md", "environments/classic_control/acrobot.md", "environments/classic_control/cart_pole.md", "environments/classic_control/mountain_car.md", "environments/classic_control/mountain_car_continuous.md", "environments/classic_control/pendulum.md", "environments/mujoco.md", "environments/mujoco/ant.md", "environments/mujoco/half_cheetah.md", "environments/mujoco/hopper.md", "environments/mujoco/humanoid.md", "environments/mujoco/humanoid_standup.md", "environments/mujoco/inverted_double_pendulum.md", "environments/mujoco/inverted_pendulum.md", "environments/mujoco/pusher.md", "environments/mujoco/reacher.md", "environments/mujoco/swimmer.md", "environments/mujoco/walker2d.md", "environments/third_party_environments.md", "environments/toy_text.md", "environments/toy_text/blackjack.md", "environments/toy_text/cliff_walking.md", "environments/toy_text/frozen_lake.md", "environments/toy_text/taxi.md", "index.md", "tutorials/blackjack_tutorial.rst", "tutorials/environment_creation.rst", "tutorials/handling_time_limits.rst", "tutorials/implementing_custom_wrappers.rst", "tutorials/reinforce_invpend_gym_v26.rst"], "titles": ["404 - Page Not Found", "Gymnasium-docs", "Env", "Experimental", "Functional Environment", "Vectorizing Environment", "Vector Environment Wrappers", "Wrappers", "Registry", "Spaces", "Composite Spaces", "Fundamental Spaces", "Spaces Utils", "Spaces Vector Utils", "Utils", "Vector", "Wrappers", "Action Wrappers", "Misc Wrappers", "Observation Wrappers", "Reward Wrappers", "Basic Usage", "Compatibility with Gym", "v21 to v26 Migration Guide", "Atari", "Adventure", "Air Raid", "Alien", "Amidar", "Assault", "Asterix", "Asteroids", "Atlantis", "Bank Heist", "Battle Zone", "Beam Rider", "Berzerk", "Bowling", "Boxing", "Breakout", "Carnival", "Centipede", "Chopper Command", "Complete List - Atari", "Crazy Climber", "Defender", "Demon Attack", "Double Dunk", "Elevator Action", "Enduro", "FishingDerby", "Freeway", "Frostbite", "Gopher", "Gravitar", "Hero", "IceHockey", "Jamesbond", "JourneyEscape", "Kangaroo", "Krull", "Kung Fu Master", "Montezuma Revenge", "Ms Pacman", "Name This Game", "Phoenix", "Pitfall", "Pong", "Pooyan", "PrivateEye", "Qbert", "Riverraid", "Road Runner", "Robot Tank", "Seaquest", "Skiings", "Solaris", "SpaceInvaders", "StarGunner", "Tennis", "TimePilot", "Tutankham", "Up n\u2019 Down", "Venture", "Video Pinball", "Wizard of Wor", "Zaxxon", "Box2D", "Bipedal Walker", "Car Racing", "Lunar Lander", "Classic Control", "Acrobot", "Cart Pole", "Mountain Car", "Mountain Car Continuous", "Pendulum", "MuJoCo", "Ant", "Half Cheetah", "Hopper", "Humanoid", "Humanoid Standup", "Inverted Double Pendulum", "Inverted Pendulum", "Pusher", "Reacher", "Swimmer", "Walker2D", "Third-party Environments", "Toy Text", "Blackjack", "Cliff Walking", "Frozen Lake", "Taxi", "Gymnasium is a standard API for reinforcement learning, and a diverse collection of reference environments", "Solving Blackjack with Q-Learning", "Make your own custom environment", "Handling Time Limits", "Implementing Custom Wrappers", "Training using REINFORCE for Mujoco"], "terms": {"thi": [1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120], "folder": [1, 14, 18, 105, 106], "contain": [1, 2, 3, 7, 9, 12, 13, 15, 16, 18, 19, 21, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 105, 106, 107, 108, 109, 111, 114, 116, 117, 120], "If": [1, 2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 30, 33, 38, 41, 50, 51, 52, 53, 54, 68, 75, 79, 88, 89, 90, 95, 98, 100, 101, 102, 106, 108, 111, 112, 113, 116, 117, 119], "you": [1, 2, 7, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 98, 101, 102, 105, 106, 109, 115, 117, 118, 119], "ar": [1, 2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 116, 117, 118, 119, 120], "pleas": [1, 2, 3, 14, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "pr": [1, 23, 109], "repo": [1, 24], "otherwis": [1, 2, 7, 8, 10, 11, 14, 15, 18, 19, 24, 37, 109, 116], "follow": [1, 4, 7, 8, 9, 14, 18, 21, 24, 37, 47, 49, 50, 51, 52, 53, 54, 79, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 117, 118, 119, 120], "below": [1, 11, 16, 21, 50, 88, 90, 96, 105, 108, 116, 118], "directli": [1, 7, 10, 16, 18, 21, 92, 109, 116, 117, 120], "markdown": 1, "file": [1, 92, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 117], "repositori": [1, 97], "fork": [1, 23, 109, 115], "docstr": 1, "": [1, 2, 3, 4, 7, 10, 11, 14, 15, 16, 17, 18, 19, 20, 21, 24, 44, 50, 51, 52, 53, 54, 56, 57, 60, 61, 62, 67, 68, 70, 87, 88, 89, 90, 91, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120], "python": [1, 7, 8, 13, 18, 21, 88, 89, 90, 115, 116, 117, 118, 119, 120], "Then": [1, 16, 21, 74, 107, 119], "pip": [1, 24, 87, 91, 97, 117], "instal": [1, 7, 14, 18, 32, 87, 91, 97, 117], "your": [1, 2, 3, 7, 9, 15, 16, 17, 18, 19, 20, 21, 27, 28, 30, 32, 33, 35, 37, 38, 39, 41, 47, 49, 50, 51, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 69, 70, 71, 72, 73, 74, 75, 76, 77, 80, 81, 82, 83, 84, 85, 86, 89, 98, 101, 102, 115, 116, 119], "run": [1, 2, 7, 8, 15, 18, 33, 55, 58, 66, 71, 72, 75, 76, 88, 90, 92, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 112, 117], "script": [1, 116], "gen_md": 1, "py": [1, 13, 24, 88, 89, 90, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 116, 117, 118, 119, 120], "automat": [1, 2, 3, 7, 8, 15, 16, 18, 22, 23, 117], "gener": [1, 2, 7, 9, 10, 11, 14, 15, 16, 18, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 117], "For": [1, 2, 4, 7, 11, 12, 14, 18, 19, 21, 22, 23, 24, 25, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 45, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 89, 90, 97, 112, 113, 114, 116, 117, 119], "add": [1, 7, 8, 14, 15, 19, 21, 89, 97, 101, 102, 114, 116, 117, 119], "complet": [1, 7, 18, 21, 23, 24, 44, 57, 69, 70, 89, 90, 116, 118, 119], "ensur": [1, 2, 3, 8, 10, 12, 14, 21, 116, 117], "i": [1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 117, 118, 119, 120], "ha": [1, 2, 3, 7, 9, 11, 12, 14, 16, 18, 19, 20, 21, 23, 24, 54, 74, 88, 89, 90, 91, 97, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 111, 115, 116, 117, 118, 120], "properli": [1, 9], "format": [1, 9, 14, 16, 18, 19, 21], "md": [1, 117], "correspond": [1, 2, 8, 16, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 114, 117, 120], "gif": [1, 92], "_static": 1, "video": [1, 7, 16, 18, 43, 73, 88], "env_typ": 1, "where": [1, 3, 7, 8, 10, 11, 12, 14, 16, 18, 21, 23, 24, 47, 89, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120], "categori": 1, "e": [1, 2, 3, 7, 9, 11, 12, 13, 15, 16, 18, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 105, 106, 109, 111, 113, 116, 117, 119, 120], "g": [1, 2, 9, 11, 12, 13, 15, 75, 92, 96, 109, 113, 114, 116, 117, 119, 120], "mujoco": [1, 21, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 119], "snake_cas": 1, "name": [1, 3, 4, 8, 16, 43, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 117], "convent": [1, 2, 24], "altern": [1, 24], "gen_gif": 1, "index": [1, 7, 13, 14, 18, 19, 21, 24, 93, 102], "toctre": 1, "requir": [1, 2, 7, 9, 19, 97, 116, 120], "packag": [1, 7, 18, 109], "cd": [1, 117], "r": [1, 7, 11, 18, 20, 73, 92, 96, 109, 111, 112, 114, 119, 120], "txt": 1, "To": [1, 7, 8, 13, 15, 16, 18, 19, 21, 22, 23, 24, 32, 42, 70, 88, 89, 90, 92, 114, 116, 117, 119, 120], "builder": 1, "onc": [1, 18, 31, 41, 48, 55, 103, 106, 114, 116, 117], "make": [1, 2, 7, 10, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 116, 119, 120], "dirhtml": 1, "_build": 1, "rebuild": [1, 32], "everi": [1, 9, 14, 18, 21, 41, 42, 46, 51, 73, 74, 83, 89, 90, 93, 98, 100, 101, 108, 109, 117, 120], "time": [1, 2, 7, 8, 10, 14, 15, 16, 18, 19, 24, 40, 43, 46, 47, 48, 52, 56, 58, 59, 66, 70, 71, 72, 73, 74, 75, 88, 89, 96, 98, 99, 100, 101, 102, 107, 108, 109, 112, 114, 116, 117, 120], "chang": [1, 2, 3, 4, 7, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 52, 60, 70, 89, 92, 93, 94, 95, 96, 97, 105, 106, 109, 114, 116, 117, 119, 120], "made": [1, 3, 105, 106], "sphinx": 1, "autobuild": 1, "b": [1, 11, 105, 106, 114], "we": [1, 2, 3, 4, 7, 11, 18, 19, 21, 23, 24, 98, 101, 102, 116, 117, 118, 119, 120], "us": [1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 113, 114, 116, 118, 119], "galleri": 1, "insid": [1, 10, 11, 18, 117], "directori": 1, "check": [1, 2, 7, 9, 11, 16, 18, 114, 116, 117, 119], "demo": 1, "see": [1, 2, 9, 14, 15, 21, 23, 24, 25, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 45, 55, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 90, 92, 98, 99, 100, 101, 107, 108, 109, 113, 117], "exampl": [1, 2, 7, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 89, 92, 111, 112, 113, 116, 117, 118, 119], "more": [1, 2, 3, 9, 10, 11, 14, 15, 16, 19, 23, 24, 25, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 45, 50, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 88, 90, 97, 98, 101, 102, 103, 104, 105, 106, 107, 109, 116, 117, 118, 119, 120], "inform": [1, 2, 7, 9, 10, 11, 14, 15, 16, 18, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 116, 117, 118, 120], "convert": [1, 7, 8, 9, 10, 16, 18, 19, 22, 23, 89, 116], "jupyt": [1, 116, 117, 118, 119, 120], "notebook": [1, 116, 117, 118, 119, 120], "can": [1, 2, 3, 7, 8, 9, 10, 11, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 117, 119, 120], "want": [1, 2, 7, 10, 15, 16, 18, 21, 33, 55, 116, 117, 119], "execut": [1, 7, 14, 17, 24, 47, 114, 117, 120], "which": [1, 2, 7, 8, 9, 10, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 44, 46, 48, 62, 89, 90, 92, 93, 97, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 109, 112, 113, 114, 117, 118, 119, 120], "output": [1, 7, 13, 14, 18, 21, 116, 120], "plot": [1, 14, 116], "should": [1, 2, 3, 7, 9, 11, 14, 15, 18, 19, 21, 23, 105, 106, 116, 117, 119], "start": [1, 2, 7, 9, 11, 14, 15, 18, 34, 47, 48, 54, 58, 59, 74, 116, 120], "run_": 1, "note": [1, 7, 9, 13, 18, 92, 93, 98, 101, 102, 114, 116, 118, 120], "so": [1, 3, 7, 14, 15, 18, 19, 21, 24, 52, 75, 90, 91, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 113, 116, 117, 118, 119], "sure": [1, 2, 18, 117, 119], "doesn": [1, 2, 7, 8, 18, 24, 90, 118], "t": [1, 2, 3, 7, 8, 10, 12, 14, 16, 18, 19, 20, 23, 24, 40, 50, 51, 52, 53, 54, 73, 74, 89, 90, 94, 95, 114, 116, 117, 118], "take": [1, 2, 4, 7, 8, 11, 12, 14, 15, 17, 18, 21, 23, 24, 77, 91, 93, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 114, 116, 119, 120], "than": [1, 23, 50, 56, 74, 90, 92, 93, 94, 95, 99, 103, 104, 105, 107, 109, 111, 116], "few": [1, 92, 116], "second": [1, 10, 14, 15, 37, 68, 75, 90, 99, 103, 106, 107], "class": [2, 4, 7, 10, 11, 14, 15, 16, 18, 21, 97, 116, 117, 119, 120], "The": [2, 3, 4, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 116, 117, 118, 120], "main": [2, 90, 100, 108], "reinforc": [2, 9, 21, 92, 107, 110, 111, 112, 114, 116, 118], "learn": [2, 7, 9, 10, 18, 19, 20, 21, 24, 50, 51, 52, 53, 54, 88, 89, 90, 92, 93, 94, 95, 99, 102, 103, 104, 107, 110, 111, 112, 114, 117], "agent": [2, 3, 7, 9, 18, 21, 23, 24, 48, 57, 88, 89, 90, 94, 98, 99, 100, 101, 102, 103, 104, 107, 108, 117, 118, 119], "encapsul": 2, "an": [2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 111, 112, 113, 114, 117, 118, 119], "arbitrari": [2, 8, 11, 14, 19, 20, 109], "behind": [2, 68], "scene": [2, 68, 109], "dynam": [2, 24, 39, 50, 51, 52, 53, 54, 92, 96, 97, 103], "through": [2, 7, 8, 9, 15, 22, 23, 24, 39, 55, 58, 61, 62, 66, 75, 76, 101, 102, 118], "step": [2, 7, 8, 15, 16, 17, 18, 19, 20, 21, 24, 88, 90, 92, 93, 96, 112, 113, 114, 115, 116, 118, 119, 120], "reset": [2, 7, 8, 14, 15, 16, 18, 19, 20, 21, 92, 93, 94, 95, 96, 109, 112, 113, 114, 115, 116, 120], "function": [2, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 90, 96, 106, 109, 113, 114, 116, 118, 119, 120], "partial": 2, "fulli": 2, "observ": [2, 4, 9, 10, 13, 14, 15, 16, 17, 18, 20, 23, 109, 115, 118, 119, 120], "singl": [2, 4, 7, 9, 10, 11, 12, 13, 14, 15, 19, 21, 100, 101, 102, 108, 117, 120], "multi": [2, 11, 97, 101, 102, 105], "pettingzoo": 2, "api": [2, 4, 7, 16, 18, 21, 23, 117, 118], "user": [2, 3, 4, 8, 15, 21, 23, 24, 92, 93, 94, 95, 96, 109, 117, 120], "need": [2, 8, 9, 10, 14, 16, 17, 18, 19, 21, 23, 42, 44, 49, 55, 59, 69, 74, 88, 90, 93, 97, 116, 117, 118, 119], "know": [2, 3, 18, 21, 117], "updat": [2, 7, 14, 18, 19, 21, 22, 23, 114, 116, 117, 118, 119, 120], "action": [2, 4, 8, 9, 10, 11, 14, 15, 16, 18, 23, 43, 91, 109, 110, 115, 117, 119, 120], "return": [2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 112, 113, 114, 116, 117, 118, 119, 120], "next": [2, 4, 13, 21, 23, 44, 46, 48, 120], "reward": [2, 4, 8, 14, 15, 16, 18, 21, 23, 97, 109, 115, 116, 117, 118, 119, 120], "termin": [2, 4, 7, 8, 14, 15, 16, 18, 20, 21, 22, 23, 92, 93, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 111, 112, 113, 114, 115, 116, 117, 119, 120], "truncat": [2, 7, 8, 14, 15, 16, 18, 20, 21, 22, 23, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 113, 114, 115, 116, 117, 119, 120], "due": [2, 23, 113, 118], "latest": [2, 120], "from": [2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 26, 40, 47, 52, 53, 56, 58, 67, 68, 80, 86, 88, 89, 90, 92, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 118, 120], "about": [2, 4, 14, 19, 21, 69, 92, 98, 101, 102, 116, 117, 118, 120], "metric": [2, 14, 23], "debug": [2, 9, 14, 110], "info": [2, 4, 7, 14, 15, 16, 18, 19, 20, 21, 23, 98, 99, 100, 101, 105, 106, 107, 108, 114, 115, 116, 117, 118, 119, 120], "initi": [2, 4, 7, 8, 10, 11, 15, 16, 17, 18, 19, 20, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 90, 91, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 119, 120], "state": [2, 4, 7, 16, 18, 19, 21, 23, 24, 91, 97, 109, 110, 116, 118, 120], "befor": [2, 7, 8, 14, 15, 16, 17, 18, 19, 20, 32, 44, 58, 69, 76, 77, 89, 98, 99, 100, 101, 102, 103, 104, 106, 107, 108, 117, 119, 120], "call": [2, 7, 8, 12, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 37, 56, 105, 106, 111, 113, 116, 117], "first": [2, 14, 15, 16, 18, 19, 21, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 117, 120], "episod": [2, 7, 8, 14, 16, 18, 19, 23, 116, 117, 118, 120], "render": [2, 4, 15, 16, 18, 19, 21, 24, 87, 88, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 116, 119], "help": [2, 9, 21, 27, 59, 62, 105, 116], "visualis": [2, 21], "what": [2, 9, 18, 24, 112], "mode": [2, 15, 16, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 116, 117], "human": [2, 7, 10, 16, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 101, 102, 105, 115, 117], "rgb_arrai": [2, 7, 14, 16, 18, 19, 24, 117], "ansi": [2, 117], "text": [2, 9, 10, 12, 21, 111, 112, 113, 114], "close": [2, 4, 11, 14, 15, 16, 21, 23, 44, 69, 98, 100, 101, 106, 108, 115, 116, 119], "import": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 117, 119, 120], "when": [2, 3, 7, 9, 10, 12, 13, 14, 15, 16, 18, 19, 22, 24, 27, 40, 44, 45, 46, 48, 50, 51, 52, 53, 54, 56, 57, 71, 72, 73, 74, 77, 89, 90, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 113, 114, 117, 118, 119, 120], "extern": [2, 23, 98, 101, 102, 118], "softwar": [2, 109], "pygam": [2, 14, 23, 87, 117], "databas": 2, "have": [2, 7, 8, 9, 11, 12, 15, 16, 17, 18, 20, 21, 22, 23, 24, 25, 27, 30, 32, 33, 37, 39, 41, 42, 44, 45, 50, 51, 52, 53, 54, 55, 58, 59, 60, 64, 66, 68, 74, 76, 87, 88, 89, 92, 98, 99, 100, 101, 102, 104, 105, 107, 108, 109, 111, 113, 114, 116, 117, 118, 119, 120], "understand": [2, 12, 15, 105, 109, 120], "action_spac": [2, 7, 9, 14, 15, 16, 17, 19, 20, 21, 23, 114, 115, 116, 117, 119, 120], "space": [2, 3, 7, 14, 15, 16, 17, 18, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 97, 109, 110, 117, 119, 120], "object": [2, 9, 13, 16, 21, 25, 30, 44, 50, 51, 59, 71, 77, 105, 116, 120], "valid": [2, 7, 8, 9, 11, 13, 15, 16, 17, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 116], "all": [2, 3, 7, 9, 10, 11, 15, 16, 18, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 51, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89, 90, 91, 92, 93, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 116, 119, 120], "within": [2, 7, 9, 17, 19, 21, 23, 62, 66, 90, 91, 103, 104], "observation_spac": [2, 7, 9, 10, 11, 15, 16, 19, 21, 117, 119, 120], "reward_rang": [2, 16, 20, 119], "A": [2, 4, 7, 9, 10, 11, 12, 14, 15, 17, 18, 19, 20, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 92, 93, 95, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 112, 113, 116, 117, 118, 119, 120], "tupl": [2, 4, 7, 9, 11, 12, 13, 14, 16, 19, 21, 23, 24, 98, 100, 101, 108, 111, 116, 117, 120], "minimum": [2, 7, 11, 96], "maximum": [2, 7, 8, 11, 14, 18, 21, 73, 74, 90, 92, 96, 103, 117], "possibl": [2, 9, 10, 11, 12, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 94, 95, 99, 101, 103, 104, 107, 108, 109, 112, 113, 114, 116, 117], "over": [2, 7, 13, 18, 27, 33, 56, 66, 71, 99, 101, 111, 113, 116, 119], "default": [2, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 92, 96, 98, 99, 100, 101, 105, 106, 107, 108, 117, 118], "rang": [2, 7, 13, 14, 16, 17, 18, 20, 21, 22, 23, 24, 56, 88, 91, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 116, 117, 119, 120], "set": [2, 7, 9, 10, 11, 15, 17, 18, 19, 23, 24, 25, 26, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 82, 83, 84, 85, 87, 90, 91, 92, 94, 95, 97, 98, 99, 100, 101, 105, 106, 107, 108, 109, 113, 116, 117, 118, 119, 120], "infti": [2, 11], "spec": [2, 15, 16, 18], "metadata": [2, 7, 14, 16, 18, 24, 117, 119], "fp": [2, 14, 24, 117], "np_random": [2, 14, 16, 18, 23, 117], "random": [2, 7, 9, 10, 11, 14, 18, 19, 21, 24, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 106, 107, 108, 109, 113, 114, 116, 117, 120], "number": [2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 24, 26, 27, 28, 29, 30, 31, 32, 35, 36, 37, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 73, 75, 77, 78, 80, 81, 82, 84, 85, 89, 90, 92, 100, 101, 102, 107, 108, 109, 111, 113, 117, 120], "assign": [2, 75, 93, 94, 95, 105, 106], "dure": [2, 7, 10, 11, 15, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 107, 108, 114, 117, 119], "super": [2, 16, 23, 117, 119, 120], "seed": [2, 7, 8, 9, 10, 11, 15, 16, 21, 72, 115, 117, 120], "assess": 2, "self": [2, 4, 7, 9, 10, 16, 17, 18, 19, 20, 95, 109, 116, 117, 119, 120], "modifi": [2, 7, 9, 15, 16, 17, 19, 20, 23, 92, 105, 106, 114, 117, 119], "extend": [2, 98], "wrapper": [2, 8, 10, 11, 14, 15, 21, 22, 23, 105, 106, 109, 113, 114, 115, 116, 120], "acttyp": [2, 4, 7, 14, 16, 17, 19, 20], "obstyp": [2, 4, 7, 14, 16, 17, 19, 20, 23, 120], "supportsfloat": [2, 7, 8, 11, 16, 20, 23, 119, 120], "bool": [2, 7, 8, 9, 11, 14, 15, 16, 18, 19, 23, 24, 90, 98, 99, 100, 101, 107, 108, 116, 117, 120], "dict": [2, 4, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 19, 21, 23, 112, 113, 114, 117, 120], "str": [2, 4, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 19, 23, 24, 98, 99, 100, 101, 107, 108, 116, 120], "ani": [2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 23, 24, 46, 50, 51, 52, 53, 54, 55, 73, 90, 93, 98, 100, 101, 102, 103, 104, 105, 106, 107, 108, 113, 116, 117, 118, 120], "one": [2, 10, 11, 12, 14, 15, 16, 18, 19, 21, 22, 23, 26, 27, 28, 32, 37, 44, 47, 48, 60, 69, 70, 71, 73, 74, 79, 92, 93, 94, 95, 96, 98, 100, 102, 103, 104, 106, 107, 111, 114, 116, 118, 119, 120], "timestep": [2, 7, 18, 19, 21, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 117, 120], "end": [2, 4, 7, 16, 18, 21, 32, 41, 42, 44, 46, 48, 49, 57, 58, 59, 69, 73, 76, 77, 88, 96, 116, 118, 120], "reach": [2, 7, 16, 32, 34, 44, 47, 48, 65, 68, 76, 77, 91, 92, 93, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 112, 113, 114, 116, 117, 118, 120], "necessari": [2, 9, 24, 50, 51, 52, 53, 54], "version": [2, 3, 8, 9, 16, 93, 116, 117, 118], "0": [2, 3, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 117, 118, 119, 120], "26": [2, 22, 98, 101, 102, 117, 118, 120], "wa": [2, 7, 14, 15, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 107, 108, 118, 119, 120], "remov": [2, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 114], "done": [2, 8, 14, 16, 18, 21, 22, 23, 98, 100, 101, 102, 103, 104, 108, 116, 117, 118, 120], "favor": 2, "clearer": 2, "had": 2, "critic": [2, 3, 18], "bootstrap": [2, 23, 118], "algorithm": [2, 9, 97, 110, 114, 116, 118, 120], "paramet": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 107, 108, 117, 119, 120], "provid": [2, 3, 7, 9, 10, 11, 14, 15, 16, 17, 18, 19, 21, 22, 24, 30, 44, 46, 50, 51, 52, 53, 54, 88, 92, 109, 117], "element": [2, 7, 9, 10, 11, 13, 15, 19, 21, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 117], "numpi": [2, 3, 4, 7, 9, 11, 13, 15, 16, 17, 19, 23, 109, 116, 117, 119, 120], "arrai": [2, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 90, 109, 116, 117, 119, 120], "posit": [2, 10, 11, 12, 13, 14, 18, 21, 24, 35, 56, 88, 89, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 113, 114, 116, 117, 119], "veloc": [2, 10, 12, 13, 88, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "pole": [2, 75, 91, 103, 104, 109, 120], "cartpol": [2, 7, 8, 14, 15, 18, 19, 20, 21, 91, 93, 103, 104, 109, 120], "result": [2, 3, 12, 14, 15, 21, 92, 98, 101, 102, 110, 112, 114, 116], "whether": [2, 4, 7, 8, 9, 11, 14, 18, 23, 37, 90, 98, 99, 100, 101, 107, 108, 111, 116, 117, 118, 120], "defin": [2, 4, 9, 12, 17, 18, 19, 20, 21, 96, 101, 117, 118, 119], "under": [2, 7, 19, 20, 116, 120], "mdp": [2, 18, 21, 94, 95], "task": [2, 10, 16, 21, 60, 66, 69, 89, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114, 117, 118, 119, 120], "neg": [2, 11, 14, 21, 75, 95, 98, 99, 101, 102, 103, 105, 106, 114, 120], "goal": [2, 27, 30, 35, 37, 39, 41, 44, 56, 61, 62, 63, 64, 65, 67, 70, 72, 74, 75, 81, 82, 83, 84, 85, 86, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 117, 120], "move": [2, 3, 7, 16, 20, 21, 23, 26, 29, 30, 39, 40, 54, 57, 68, 71, 74, 75, 88, 90, 93, 98, 99, 100, 102, 103, 104, 105, 106, 107, 108, 109, 112, 113, 114, 117, 118, 120], "lava": [2, 21, 55], "sutton": [2, 92, 93, 103, 104, 111, 112, 116], "barton": 2, "gridworld": [2, 109, 112, 117], "true": [2, 7, 8, 12, 14, 15, 17, 18, 19, 21, 22, 24, 50, 51, 52, 53, 54, 88, 89, 90, 98, 99, 100, 101, 107, 108, 111, 113, 116, 117, 120], "condit": [2, 116, 118, 120], "outsid": [2, 7, 89, 90, 118], "scope": [2, 118], "satisfi": 2, "typic": [2, 14, 114], "timelimit": [2, 8, 16, 18, 21, 117, 118], "could": [2, 3, 14, 21, 24, 116, 117, 118, 119, 120], "also": [2, 7, 8, 11, 14, 15, 16, 18, 19, 21, 24, 36, 57, 58, 69, 70, 72, 88, 89, 90, 91, 97, 98, 99, 100, 101, 107, 108, 109, 116, 117, 118, 119, 120], "indic": [2, 7, 14, 18, 23, 89, 92, 93, 98, 101, 102, 111, 112, 113, 114, 116, 118, 120], "physic": [2, 87, 97, 103, 104, 120], "go": [2, 48, 75, 79, 89, 111, 116, 118, 120], "out": [2, 4, 13, 16, 24, 33, 38, 55, 58, 66, 71, 76, 117, 118, 119, 120], "bound": [2, 7, 11, 16, 17, 21, 23, 90, 92, 93, 94, 95, 96, 118], "prematur": 2, "auxiliari": [2, 117], "diagnost": 2, "log": [2, 120], "might": [2, 9, 11, 12, 16, 17, 18, 20, 23, 116, 117, 119, 120], "instanc": [2, 4, 7, 8, 9, 10, 12, 13, 14, 15, 18, 21, 109, 119], "describ": [2, 9, 10, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 93, 107, 111, 119], "perform": [2, 3, 7, 9, 14, 18, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 106, 109, 120], "variabl": [2, 7, 9, 11, 15, 18, 98, 100, 107, 108, 109, 116, 117, 118, 120], "hidden": [2, 97, 120], "individu": [2, 3, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 117, 119, 120], "term": [2, 91, 92, 97, 98, 99, 100, 101, 105, 106, 107, 108, 117, 119], "combin": [2, 9, 14, 16, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 106], "produc": [2, 7, 16, 18, 102, 117], "total": [2, 44, 88, 89, 90, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 114, 120], "In": [2, 3, 7, 11, 14, 15, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 91, 92, 98, 99, 100, 101, 107, 108, 114, 116, 117, 118, 119, 120], "openai": [2, 8, 15, 16, 21, 23, 87, 115, 116, 118, 120], "gym": [2, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 87, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120], "v26": [2, 120], "distinguish": [2, 117, 118], "howev": [2, 3, 7, 8, 9, 10, 11, 12, 15, 17, 18, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 105, 106, 116, 117, 118, 119], "deprec": 2, "favour": [2, 22, 23], "boolean": [2, 7, 9, 14, 18, 23, 90, 116, 120], "valu": [2, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 21, 23, 24, 74, 88, 90, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 118, 120], "case": [2, 4, 7, 9, 10, 11, 17, 19, 21, 23, 57, 69, 89, 98, 99, 100, 101, 107, 108, 114, 117, 118, 119, 120], "further": [2, 23, 74, 90, 105, 106, 120], "undefin": 2, "signal": [2, 7, 16, 18, 21, 98, 100, 101, 108, 114, 117, 118], "mai": [2, 7, 9, 11, 12, 14, 15, 16, 17, 18, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 107, 108, 113, 117], "emit": [2, 16, 18], "differ": [2, 9, 10, 14, 15, 17, 18, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 96, 98, 99, 100, 101, 102, 103, 104, 108, 109, 117, 118, 119], "reason": [2, 7, 23, 90], "mayb": 2, "underli": [2, 16, 18, 21, 92], "solv": [2, 88, 91, 93, 97, 103, 104, 117, 120], "successfulli": [2, 114], "certain": [2, 47, 49, 103, 104, 106, 116], "exceed": [2, 16, 18, 21, 98, 100, 101, 108, 116], "simul": [2, 24, 50, 51, 52, 53, 54, 97, 101, 102, 103, 104, 116, 120], "enter": [2, 14, 31, 33, 60, 90, 112], "invalid": [2, 11, 116], "int": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 24, 111, 112, 113, 114, 116, 117, 120], "none": [2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 23, 113, 116, 117], "option": [2, 4, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 23, 89, 92, 93, 94, 95, 96, 117], "intern": [2, 4, 15, 19], "new": [2, 3, 7, 9, 16, 17, 18, 19, 21, 22, 23, 28, 31, 33, 46, 92, 93, 94, 95, 96, 98, 106, 109, 114, 116, 117, 118, 119], "often": [2, 9, 97, 117], "some": [2, 9, 10, 11, 15, 16, 18, 21, 23, 24, 40, 48, 66, 89, 91, 92, 97, 106, 107, 114, 116, 117, 119], "explor": [2, 9, 54, 109, 116], "generalis": 2, "polici": [2, 7, 19, 20, 21, 23, 91, 97, 98, 99, 100, 101, 102, 107, 108, 109, 115, 118], "control": [2, 9, 11, 21, 23, 24, 26, 29, 31, 32, 33, 34, 35, 40, 42, 45, 52, 53, 54, 57, 59, 66, 67, 69, 71, 72, 73, 74, 75, 76, 79, 80, 87, 89, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 119], "alreadi": [2, 3, 16, 19, 21], "rng": [2, 4, 9, 10, 11, 117], "therefor": [2, 3, 7, 9, 15, 23], "right": [2, 11, 24, 26, 28, 29, 30, 31, 35, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 57, 58, 59, 61, 63, 64, 65, 67, 70, 75, 77, 78, 80, 81, 84, 85, 88, 89, 90, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 107, 108, 112, 113, 114, 116, 117, 120], "after": [2, 4, 7, 8, 14, 15, 18, 19, 21, 32, 36, 41, 47, 51, 55, 77, 90, 98, 99, 100, 101, 102, 107, 108, 111, 114, 116, 117, 118, 120], "never": 2, "again": [2, 90, 117], "custom": [2, 7, 9, 14, 15, 16, 19, 21, 23, 24, 50, 51, 52, 53, 54, 107, 109, 113], "line": [2, 9, 23, 47, 92, 114, 117], "correctli": [2, 7, 19, 20, 117], "v0": [2, 3, 7, 8, 15, 16, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 117, 118, 120], "25": [2, 12, 24, 50, 51, 52, 53, 54, 98, 100, 101, 102, 108, 114], "return_info": 2, "now": [2, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 116, 117, 118, 120], "expect": [2, 3, 9, 10, 11, 15, 18, 21, 120], "prng": [2, 9, 10, 11], "doe": [2, 7, 8, 10, 12, 14, 15, 18, 22, 36, 52, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 117, 118, 119], "pass": [2, 4, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 90, 98, 99, 100, 101, 107, 108, 117, 119], "chosen": [2, 11, 24], "sourc": [2, 97, 109, 116, 117, 118, 119, 120], "entropi": 2, "timestamp": 2, "dev": [2, 120], "urandom": 2, "integ": [2, 7, 10, 11, 12, 14, 18, 19, 23, 24, 117], "even": [2, 8, 25, 27, 33, 34, 36, 38, 41, 42, 45, 55, 56, 57, 59, 60, 62, 66, 69, 71, 72, 73, 74, 76, 79, 83, 86, 102, 116, 117], "exist": [2, 8, 9, 15, 16, 21], "usual": [2, 10, 11, 18, 24, 50, 51, 52, 53, 54, 56, 100, 116, 117, 119], "been": [2, 14, 18, 21, 23, 98, 101, 102, 109, 117, 118], "refer": [2, 14, 18, 21, 23, 117, 118], "minim": [2, 117, 118, 120], "abov": [2, 3, 10, 11, 14, 22, 23, 24, 50, 51, 52, 53, 54, 92, 93, 96, 97, 120], "paradigm": 2, "specifi": [2, 7, 9, 10, 11, 14, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 90, 91, 96, 100, 101, 105, 106, 108, 110, 113, 114, 117, 119], "how": [2, 9, 14, 15, 21, 56, 58, 59, 102, 103, 105, 106, 116, 117, 118, 119, 120], "depend": [2, 7, 10, 14, 19, 20, 24, 26, 27, 28, 29, 30, 31, 32, 35, 36, 37, 39, 40, 42, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 59, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 87, 91, 93, 97, 98, 99, 100, 101, 107, 108, 109, 113], "specif": [2, 7, 8, 11, 18, 118, 120], "analog": [2, 24], "dictionari": [2, 7, 9, 10, 14, 15, 16, 18, 19, 21, 116, 117, 119, 120], "complement": 2, "It": [2, 7, 8, 10, 11, 14, 15, 18, 21, 24, 25, 26, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 82, 83, 84, 85, 88, 97, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 116, 117, 120], "renderfram": [2, 14], "list": [2, 3, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 24, 116], "comput": [2, 14, 67, 79, 98, 103, 105, 106, 109, 116, 117], "frame": [2, 7, 14, 18, 19, 24, 37, 89, 90, 102, 105], "render_mod": [2, 7, 14, 15, 16, 18, 19, 21, 23, 24, 115, 117], "wai": [2, 10, 16, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 116, 118, 119], "most": [2, 7, 9, 16, 18, 19, 21, 23, 24, 50, 51, 52, 53, 54, 106, 116, 117, 119], "achiev": [2, 24, 27, 92, 111], "appli": [2, 7, 8, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 88, 90, 91, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 116, 117, 119, 120], "collect": [2, 3, 7, 15, 18, 27, 30, 48, 62, 63, 66, 82, 109, 116], "As": [2, 3, 15, 16, 21, 80, 92, 97, 98, 99, 101, 102, 110, 112, 114, 119, 120], "known": [2, 18, 31], "__init__": [2, 10, 13, 16, 17, 19, 20, 116, 117, 119, 120], "initialis": [2, 7, 18, 23, 111], "By": [2, 7, 8, 18, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 107, 108], "continu": [2, 7, 9, 16, 17, 21, 29, 30, 89, 90, 91, 94, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 116, 118, 120], "current": [2, 3, 11, 14, 16, 18, 19, 24, 89, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 116, 117, 120], "displai": [2, 8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 93, 117], "consumpt": 2, "occur": [2, 7, 18, 92, 93, 117], "repres": [2, 7, 10, 11, 14, 21, 23, 90, 92, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 112, 113, 114, 115, 117, 120], "np": [2, 7, 8, 9, 10, 11, 13, 14, 15, 17, 18, 19, 90, 114, 116, 117, 119, 120], "ndarrai": [2, 4, 7, 8, 10, 11, 12, 13, 14, 17, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114, 120], "shape": [2, 7, 9, 10, 11, 12, 13, 16, 17, 18, 19, 21, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 117, 119, 120], "x": [2, 9, 12, 90, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 109, 112, 117, 120], "y": [2, 24, 90, 96, 98, 99, 101, 102, 103, 105, 106, 107, 114, 120], "3": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 117, 120], "rgb": [2, 7, 16, 18, 19, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 98, 99, 100, 101, 102, 103, 104, 107, 108], "pixel": [2, 7, 16, 19, 89, 109, 117], "imag": [2, 7, 9, 15, 16, 18, 19, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89], "string": [2, 7, 8, 9, 11, 14, 19], "stringio": 2, "style": [2, 15, 18, 109, 120], "represent": [2, 21, 118], "each": [2, 3, 7, 8, 9, 10, 11, 14, 15, 16, 17, 19, 21, 24, 44, 46, 47, 48, 49, 52, 60, 67, 69, 70, 71, 72, 73, 74, 75, 87, 88, 90, 91, 92, 93, 94, 95, 98, 101, 102, 103, 104, 105, 109, 110, 112, 114, 116, 117], "includ": [2, 7, 8, 19, 21, 22, 23, 24, 93, 98, 99, 100, 101, 103, 104, 105, 106, 107, 108, 109, 114, 117, 118], "newlin": 2, "escap": [2, 43, 58, 62], "sequenc": [2, 7, 8, 9, 11, 12, 15, 19, 21, 24, 109], "color": [2, 10, 28, 39, 52, 70, 114], "rgb_array_list": [2, 7, 14, 16, 18], "ansi_list": 2, "base": [2, 3, 4, 7, 10, 15, 16, 18, 21, 48, 57, 74, 87, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 114, 116, 117, 119, 120], "except": [2, 71, 74, 117], "rendercollect": [2, 3, 7, 16, 18], "pop": [2, 106], "kei": [2, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 24, 25, 56, 98, 99, 100, 101, 105, 106, 107, 108, 112, 113, 114, 116, 117, 118, 119], "support": [2, 3, 7, 9, 10, 12, 14, 15, 16, 18, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 117], "longer": [2, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 100, 101, 102, 103, 104, 105, 106, 108, 109, 120], "accept": [2, 14, 18, 24, 117], "rather": [2, 23, 89, 109, 116], "v1": [2, 7, 8, 14, 15, 18, 19, 20, 21, 88, 89, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 113, 114, 116], "type": [2, 7, 9, 10, 11, 15, 16, 17, 19, 21, 23, 46, 89, 98, 99, 100, 101, 107, 108, 109, 117, 119, 120], "discret": [2, 7, 9, 10, 12, 13, 15, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 90, 92, 93, 94, 95, 110, 111, 112, 113, 114, 116, 117, 119], "give": [2, 27, 28, 46, 47, 90, 105, 111, 114, 116, 120], "2": [2, 7, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 37, 39, 40, 42, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 79, 80, 81, 82, 84, 85, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 117, 119, 120], "mean": [2, 21, 23, 92, 98, 99, 100, 105, 117, 120], "two": [2, 7, 9, 10, 11, 14, 15, 18, 23, 24, 26, 37, 79, 88, 90, 92, 94, 95, 97, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 116, 117], "1": [2, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 117, 118, 119, 120], "box": [2, 7, 9, 10, 12, 13, 15, 16, 17, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 117, 119], "4028234663852886e": 2, "38": [2, 7, 101, 102], "4": [2, 7, 11, 12, 14, 15, 16, 17, 18, 19, 21, 24, 26, 28, 29, 30, 31, 33, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 58, 61, 63, 64, 65, 67, 68, 69, 70, 77, 78, 80, 81, 82, 84, 85, 88, 89, 90, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 117, 119, 120], "float32": [2, 7, 10, 11, 12, 13, 15, 16, 17, 19, 21, 88, 89, 90, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "denot": [2, 93], "well": [2, 9, 31, 74, 92, 98, 101, 102, 109, 117], "high": [2, 3, 10, 11, 12, 13, 15, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 119], "8000002e": [2, 7], "00": [2, 7], "4028235e": [2, 7], "1887903e": [2, 7], "01": [2, 7, 20, 90, 98, 99, 102, 103, 104, 105, 106, 107, 116], "dtype": [2, 7, 9, 10, 11, 12, 13, 15, 19, 90, 117], "low": [2, 10, 11, 12, 13, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 119], "etc": [2, 9, 69, 72, 76, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 117, 118, 119], "determin": [2, 11, 24, 90, 92, 93, 94, 95, 96, 109, 113, 117, 118], "inf": [2, 93, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 119], "envspec": [2, 8, 16], "normal": [2, 7, 11, 16, 18, 19, 20, 88, 89, 96, 98, 99, 103, 120], "finish": [2, 89, 90], "code": [2, 4, 7, 9, 10, 14, 15, 16, 18, 19, 20, 92, 109, 116, 117, 119, 120], "clean": 2, "up": [2, 10, 11, 15, 19, 24, 25, 27, 28, 29, 30, 31, 33, 34, 35, 37, 40, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 57, 58, 61, 63, 68, 70, 71, 72, 74, 75, 78, 79, 80, 81, 84, 85, 88, 90, 97, 98, 101, 102, 106, 111, 112, 113, 114, 116, 117], "window": [2, 14, 44, 89, 117], "http": [2, 14, 24, 89, 92, 111, 112, 113, 116, 117, 120], "connect": [2, 10, 92, 98, 99, 100, 106, 107, 108], "properti": [2, 9, 16, 118], "unwrap": [2, 16, 21, 92], "non": [2, 7, 9, 12, 14, 113, 117, 118, 120], "wrap": [2, 7, 8, 14, 15, 16, 18, 19, 20, 21, 117, 119, 120], "_np_random": 2, "meth": [2, 7], "much": [2, 24, 50, 51, 52, 53, 54, 58, 59, 102, 117, 118], "creat": [2, 3, 8, 11, 13, 14, 15, 18, 21, 34, 88, 89, 90, 105, 106, 116, 119, 120], "creation": [2, 15, 117], "tutori": [2, 16, 21, 23, 116, 119, 120], "gymnasium": [3, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 118, 120], "env": [3, 4, 7, 8, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 113, 114, 115, 116, 118, 119, 120], "flexibl": [3, 15, 109], "implement": [3, 7, 9, 10, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 96, 103, 109, 110, 114, 117, 120], "complic": [3, 11, 16, 119], "parallel": [3, 15], "propos": [3, 7, 109], "funcenv": 3, "part": [3, 18, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 118], "its": [3, 7, 14, 15, 17, 18, 39, 46, 69, 81, 86, 90, 96, 97, 107, 109, 120], "own": [3, 15, 16, 23, 39, 74, 119], "relat": [3, 120], "larg": [3, 15, 23, 95, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 119], "believ": 3, "improv": [3, 15], "work": [3, 9, 14, 16, 18, 19, 24, 89, 90, 92, 99, 100, 103, 104, 108], "progress": [3, 10, 80], "arbitrarili": 3, "complex": [3, 9, 10, 101, 102, 103, 104, 120], "rl": [3, 14, 18, 115, 116, 118, 120], "advanc": [3, 15, 92], "becom": [3, 73, 87, 118], "were": [3, 19, 23, 41, 87, 92, 117], "mind": 3, "jax": [3, 7, 109], "With": [3, 112, 116], "hardwar": 3, "acceler": [3, 89, 94, 95, 96], "brax": [3, 109], "written": [3, 14, 16, 23], "similar": [3, 10, 23, 28, 39, 105, 117], "pytorch": [3, 7, 120], "program": [3, 15], "onli": [3, 7, 8, 9, 10, 11, 12, 15, 16, 18, 19, 23, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 61, 63, 64, 65, 67, 68, 70, 71, 72, 73, 74, 75, 77, 78, 80, 81, 82, 84, 85, 94, 95, 98, 99, 100, 101, 103, 105, 106, 108, 109, 116, 117, 119], "game": [3, 7, 11, 14, 18, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 87, 111, 112, 113, 116, 117], "town": 3, "anymor": 3, "upgrad": 3, "jumpi": 3, "project": [3, 21, 23, 109, 116], "develop": [3, 97, 120], "farama": [3, 14, 109, 116, 117, 120], "foundat": [3, 109, 117], "compat": [3, 16, 19, 21, 109, 115], "futur": [3, 4, 80, 109], "subset": [3, 11, 24, 50, 51, 52, 53, 54, 117, 119], "like": [3, 4, 7, 9, 11, 14, 16, 17, 18, 19, 20, 21, 26, 27, 28, 29, 30, 31, 32, 33, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 66, 67, 68, 69, 70, 75, 77, 78, 80, 81, 82, 84, 85, 88, 90, 98, 99, 101, 102, 104, 105, 106, 108, 114, 117, 119], "supersuit": 3, "aim": [3, 56, 100, 103, 107, 108, 120], "bring": 3, "mani": [3, 7, 11, 18, 19, 21, 24, 30, 33, 37, 56, 59, 74, 84, 116, 117], "awar": [3, 23], "plan": [3, 109], "miss": [3, 21, 75, 92, 114], "issu": [3, 18, 21, 23, 24, 98, 100, 101, 102, 108, 117], "detail": [3, 9, 14, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 109, 117], "caus": [3, 18, 21, 57, 114], "ad": [3, 7, 16, 18, 19, 21, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 52, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 114, 117], "lambaactionv0": 3, "don": [3, 10, 16, 23, 24, 40, 50, 51, 52, 53, 54, 74, 89, 94, 117, 118], "regularli": 3, "signific": [3, 22], "affect": 3, "addition": [3, 101, 102], "hope": 3, "reproduc": [3, 10, 23], "academia": 3, "v28": 3, "rewrit": 3, "vectorenv": [3, 9], "inherit": [3, 9, 16, 17, 19, 20, 21, 117], "replac": [3, 16, 23, 111, 116], "30": [3, 7, 14, 18, 24, 68, 71, 101, 102], "old": [3, 7, 15, 16, 18, 22, 115], "transformobserv": [3, 7, 16, 19], "lambdaobservationv0": [3, 7], "filterobserv": [3, 16, 19], "filterobservationv0": [3, 7], "flattenobserv": [3, 10, 16, 19, 117], "flattenobservationv0": [3, 7], "grayscaleobserv": [3, 16, 19], "grayscaleobservationv0": [3, 7], "resizeobserv": [3, 16, 19], "resizeobservationv0": [3, 7], "reshape_v0": 3, "reshapeobservationv0": [3, 7], "Not": [3, 7, 18], "rescaleobservationv0": [3, 7], "dtype_v0": 3, "dtypeobservationv0": [3, 7], "pixelobservationwrapp": [3, 16, 19], "pixelobservationv0": [3, 7], "normalizeobserv": [3, 16, 19], "normalizeobservationv0": [3, 7], "timeawareobserv": [3, 16, 19, 21], "timeawareobservationv0": [3, 7], "framestack": [3, 7, 16, 18, 19], "framestackobservationv0": [3, 7], "delay_observations_v0": 3, "delayobservationv0": [3, 7], "action_lambda_v1": 3, "lambdaactionv0": [3, 7], "clipact": [3, 16, 17, 21], "clipactionv0": [3, 7], "rescaleact": [3, 7, 16, 17, 21], "rescaleactionv0": [3, 7], "sticky_actions_v0": 3, "stickyactionv0": [3, 7], "transformreward": [3, 16, 20], "lambdarewardv0": [3, 7], "clip_reward_v0": 3, "cliprewardv0": [3, 7], "normalizereward": [3, 16, 20], "normalizerewardv0": [3, 7], "autoresetwrapp": [3, 8, 16, 18, 117], "autoresetv0": [3, 7], "passiveenvcheck": [3, 18, 21], "passiveenvcheckerv0": [3, 7], "orderenforc": [3, 16, 18, 21, 117], "orderenforcingv0": [3, 7], "envcompat": [3, 16, 18, 22, 23], "shimmi": 3, "recordepisodestatist": [3, 16, 18, 23, 116, 120], "recordepisodestatisticsv0": [3, 7], "ataripreprocess": [3, 16, 18], "ataripreprocessingv0": [3, 7], "recordvideo": [3, 16, 18, 23], "recordvideov0": [3, 7], "humanrend": [3, 7, 16, 18], "humanrenderingv0": [3, 7], "rendercollectionv0": [3, 7], "jaxtonumpyv0": [3, 7], "jaxtotorchv0": [3, 7], "numpytotorchv0": [3, 7], "These": [3, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 97, 98, 101, 107, 116, 120], "28": [3, 24, 33, 39, 92, 101, 102], "templat": 4, "meant": 4, "stateless": 4, "manner": [4, 7, 11, 16, 19, 56], "being": [4, 7, 8, 12, 18, 45, 94, 95, 96, 98, 101, 102, 106, 116], "around": [4, 18, 34, 87, 106, 109], "explicitli": [4, 117, 118], "That": [4, 93], "said": [4, 98, 100, 101, 108], "noth": [4, 89, 90, 120], "here": [4, 14, 92, 117], "prevent": [4, 11, 15, 47, 109], "statefulli": 4, "just": [4, 16, 21, 50, 56, 73, 104, 106], "recommend": [4, 7, 8, 15, 18, 23, 24, 50, 51, 52, 53, 54, 90, 98, 101, 102, 116, 117], "consist": [4, 7, 11, 19, 21, 24, 37, 88, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 116, 117, 119, 120], "method": [4, 7, 10, 14, 17, 18, 19, 20, 22, 117, 119, 120], "pomdp": 4, "given": [4, 7, 8, 11, 16, 19, 71, 74, 88, 90, 91, 92, 94, 95, 98, 100, 101, 108, 116, 118], "transit": [4, 14, 21, 24, 50, 51, 52, 53, 54, 101, 102, 112, 113, 114, 118], "next_stat": [4, 116], "state_info": 4, "step_info": 4, "structur": [4, 9, 10, 12, 117], "serv": [4, 9, 98, 99, 100, 101, 107, 108, 120], "purpos": [4, 9, 120], "allow": [4, 7, 8, 9, 10, 14, 16, 21, 22, 23, 24, 55, 73, 92, 93, 94, 95, 96, 103, 104, 105, 106, 109, 117, 119, 120], "constant": [4, 35, 74, 90, 95, 98, 100, 101, 108], "them": [4, 7, 9, 16, 18, 22, 23, 24, 26, 27, 28, 31, 32, 36, 40, 47, 68, 77, 90, 98, 99, 106, 114, 119], "itself": [4, 12, 15, 16, 18, 21, 22], "moment": [4, 74], "predominantli": 4, "intend": [4, 18, 113], "flesh": 4, "offici": 4, "expos": 4, "statetyp": 4, "rewardtyp": 4, "terminaltyp": 4, "full": [4, 21, 24, 50, 51, 52, 53, 54, 89, 90, 116], "transform": [4, 7, 9, 12, 14, 16, 17, 18, 19, 20, 106, 119], "func": [4, 7], "callabl": [4, 7, 8, 13, 14, 15, 18, 19, 20], "render_imag": 4, "render_st": 4, "renderstatetyp": 4, "show": [4, 19, 23, 24, 34, 40, 111, 116, 119, 120], "render_init": 4, "kwarg": [4, 8, 14, 15, 22, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 117], "render_clos": 4, "experiment": [7, 92], "via": [7, 9, 10, 16, 18, 19, 20, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 91, 97, 109, 110, 117, 120], "lambda": [7, 15, 19, 20, 116], "ob": [7, 14, 19, 114, 116, 119, 120], "08319338": [7, 19], "04635121": [7, 19], "07394746": [7, 19], "20877492": [7, 19], "constructor": [7, 8, 9, 10, 11, 14, 16, 17, 19, 20, 21, 117], "assum": [7, 11, 19, 117], "same": [7, 10, 12, 14, 15, 19, 56, 88, 89, 92, 105, 106, 107, 114, 116, 118], "filter_kei": [7, 19], "filter": [7, 16, 19], "00067088": [7, 19], "01860439": [7, 19], "04772898": [7, 19], "01911527": [7, 19], "04560107": [7, 19], "04466959": [7, 19], "0328232": [7, 19], "02367178": [7, 19], "04649447": [7, 19], "14996664": [7, 19], "03329664": [7, 19], "25847703": [7, 19], "fals": [7, 8, 14, 15, 18, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 90, 98, 99, 100, 101, 107, 108, 111, 116, 117, 120], "flatten": [7, 9, 12, 16, 19, 97, 116, 117], "carrac": [7, 8, 14, 19, 89], "96": [7, 19, 24, 89], "27648": [7, 19], "util": [7, 21, 22, 117], "flatten_spac": [7, 9, 12], "keep_dim": [7, 19], "grayscal": [7, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "keep": [7, 14, 16, 18, 32, 45, 64, 67, 75, 76, 84, 93, 102, 117], "channel": [7, 18], "dimens": [7, 9, 11, 12, 19, 98, 99, 100, 101, 107, 108, 109, 120], "grayscale_env": 7, "resiz": [7, 16, 18, 19], "opencv": [7, 18], "v2": [7, 8, 18, 21, 23, 88, 89, 90, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114, 115, 119], "resized_env": 7, "32": [7, 24, 39, 56, 101, 102, 111, 120], "reshap": 7, "reshape_env": 7, "24": [7, 24, 33, 39, 64, 88, 89, 90, 93, 98, 101, 102, 116], "product": [7, 10, 11], "equal": [7, 94, 95, 103, 111, 113], "min_ob": 7, "float": [7, 8, 11, 12, 14, 16, 17, 19, 20, 24, 55, 68, 90, 98, 99, 100, 101, 107, 108, 116, 117, 120], "max_ob": 7, "linearli": [7, 92, 103, 104, 120], "rescal": [7, 16, 17, 21], "between": [7, 10, 11, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 116, 117, 118, 120], "pendulum": [7, 8, 15, 19, 21, 91, 92, 93, 97, 120], "8": [7, 14, 15, 19, 20, 22, 24, 26, 28, 30, 31, 33, 35, 39, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 80, 84, 85, 89, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 113, 120], "10": [7, 10, 11, 17, 24, 31, 37, 50, 51, 52, 53, 54, 61, 68, 71, 74, 88, 90, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 112, 114, 116, 117, 120], "multidiscret": [7, 9, 10, 12, 13, 15, 21, 117], "multibinari": [7, 9, 10, 12, 13, 21], "pixels_onli": [7, 19], "pixels_kei": 7, "obs_kei": 7, "augment": [7, 16, 19], "choos": [7, 19, 117], "origin": [7, 12, 15, 17, 18, 19, 23, 89, 92, 103, 105, 106], "discard": [7, 19], "both": [7, 11, 15, 18, 19, 21, 22, 88, 91, 92, 106, 108, 109, 111, 112, 113, 114, 117, 118, 120], "epsilon": [7, 19, 20, 116], "1e": [7, 19, 20, 101, 107, 108, 120], "coordin": [7, 11, 16, 19, 88, 90, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "center": [7, 16, 19, 89, 90, 93, 96, 98, 99, 101, 102], "unit": [7, 9, 16, 19, 58, 92, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "varianc": [7, 16, 19, 20], "past": [7, 19, 20], "trajectori": [7, 16, 19, 20, 90, 101, 102], "newli": [7, 19, 20], "instanti": [7, 10, 14, 18, 19, 20, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 117], "recent": [7, 18, 19, 20, 22], "stabil": [7, 19, 20, 101, 102, 119, 120], "scale": [7, 16, 18, 19, 20, 24, 90, 98, 99, 100, 101, 107, 108, 109], "normalize_tim": 7, "dict_time_kei": 7, "remain": [7, 55, 58, 59, 74, 112, 117, 118], "final": [7, 14, 15, 18, 57, 116, 117, 118], "500": [7, 48, 71, 92, 93, 114, 116], "_": [7, 13, 14, 15, 21, 115, 119], "sampl": [7, 9, 10, 11, 12, 13, 14, 15, 19, 20, 21, 23, 90, 114, 115, 116, 117, 120], "ordereddict": [7, 10, 13, 19], "02866629": 7, "2310988": 7, "02614601": 7, "2600732": 7, "002": [7, 100, 108], "0000000e": 7, "5": [7, 10, 11, 12, 13, 17, 18, 24, 26, 28, 29, 30, 31, 34, 35, 37, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 77, 78, 80, 81, 82, 84, 85, 88, 89, 90, 92, 94, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 114, 116, 117, 120], "01232257": 7, "19335455": 7, "02244143": 7, "32388705": 7, "stack_siz": 7, "stack": [7, 16, 19], "roll": [7, 16, 19, 37, 105], "process": [7, 13, 14, 15, 19, 21, 92, 105, 117, 118], "buffer": [7, 18, 19, 89], "fill": [7, 19, 33, 117], "num_stack": [7, 19], "ident": [7, 11, 15, 19], "delai": [7, 117], "delayobserv": 7, "zero": [7, 11, 13, 96, 99, 109, 116, 117, 118, 120], "noop_max": [7, 18], "frame_skip": [7, 18, 98, 99, 100, 101, 107, 108], "screen_siz": [7, 18], "84": [7, 18, 98, 101, 102], "terminal_on_life_loss": [7, 18], "grayscale_ob": [7, 18], "grayscale_newaxi": [7, 18], "scale_ob": [7, 18], "atari": [7, 16, 18, 21, 22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 116], "2600": [7, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "preprocess": [7, 14, 16, 18], "guidelin": [7, 18], "machado": [7, 18, 24], "et": [7, 18, 24], "al": [7, 18, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 116], "2018": [7, 18, 24, 92], "revisit": [7, 18, 24], "arcad": [7, 18, 24, 31, 48, 50, 51, 52, 53, 54, 109], "evalu": [7, 18, 24, 116], "protocol": [7, 18, 24], "open": [7, 18, 24, 37, 97, 109, 117], "problem": [7, 18, 24, 90, 93, 96, 103, 104, 107, 109, 111, 114, 115, 118], "stage": [7, 18], "noop": [7, 11, 14, 18, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 114], "obtain": [7, 16, 18, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 96, 111, 118, 120], "op": [7, 18], "max": [7, 11, 17, 18, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 113, 116, 120], "skip": [7, 14, 18, 24, 117], "pool": [7, 18, 107], "life": [7, 18, 30, 44, 48, 72, 76, 120], "lost": [7, 18, 33, 41, 60, 73, 77], "loss": [7, 18, 111, 120], "turn": [7, 15, 18, 23, 27, 28, 89, 90, 103, 109], "off": [7, 18, 23, 41, 56, 64, 72, 89, 90, 112, 114], "squar": [7, 18, 19, 73, 105, 106, 114, 117], "210x180": [7, 18], "84x84": [7, 18], "colour": [7, 18, 89], "greyscal": [7, 18], "255": [7, 18, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 117], "No": [7, 18, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 120], "taken": [7, 15, 18, 48, 54, 91, 93, 117, 120], "effect": [7, 12, 13, 16, 18, 19, 21, 24, 50, 51, 52, 53, 54, 90, 103, 104, 114, 116, 120], "frequenc": [7, 18], "experi": [7, 18, 98, 101, 102, 103, 104, 116, 120], "whenev": [7, 18, 30, 33, 117], "grai": [7, 16, 18, 19], "axi": [7, 18, 94, 95, 99, 101, 102, 107, 116], "dimension": [7, 9, 18, 21, 28, 90, 98, 99, 100, 106, 107, 108, 117], "limit": [7, 8, 18, 21, 40, 48, 55, 56, 69, 74, 103, 104, 116, 120], "memori": [7, 18, 19, 94, 95], "optim": [7, 18, 24, 88, 90, 101, 102, 109, 116, 120], "benefit": [7, 18], "rais": [7, 8, 11, 12, 13, 14, 15, 18, 19, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "dependencynotinstal": [7, 14, 18], "valueerror": [7, 11, 12, 13, 15, 18, 19], "disabl": [7, 8, 18, 113], "wrapperacttyp": [7, 16, 17], "lambdaact": 7, "clip": [7, 16, 17, 21, 94, 95, 98, 101, 117, 119], "bipedalwalk": [7, 8, 16, 17, 21, 88], "v3": [7, 8, 16, 17, 21, 88, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114], "disable_env_check": [7, 8, 15], "min_act": [7, 16, 17, 21], "max_act": [7, 16, 17, 21], "affin": [7, 17, 90], "must": [7, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 21, 23, 24, 25, 26, 29, 31, 32, 33, 34, 36, 40, 41, 42, 45, 48, 49, 58, 62, 64, 65, 70, 89, 100, 108, 117, 118], "match": [7, 12, 15, 17, 79], "42": [7, 10, 11, 93, 101, 102, 115], "75": [7, 17, 98], "wrapped_env": [7, 16, 18, 21, 117, 119, 120], "wrapped_env_ob": 7, "alltru": 7, "min": [7, 11, 17, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111], "scalar": [7, 11, 17], "repeat_action_prob": [7, 24, 50, 51, 52, 53, 54], "probabl": [7, 9, 24, 112, 113, 114, 116, 118, 120], "repeat": [7, 24, 105], "previou": [7, 18, 98, 101, 102, 118, 119], "section": [7, 24, 75, 98, 99, 100, 101, 107, 108, 116, 117], "page": [7, 16, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "12": [7, 12, 24, 31, 33, 37, 39, 50, 51, 52, 53, 54, 58, 61, 73, 90, 92, 93, 98, 99, 100, 101, 102, 105, 108, 111, 112, 116], "stickyact": 7, "rew": [7, 14, 118], "min_reward": [7, 119], "max_reward": [7, 119], "upper": [7, 11, 21, 101, 102], "lower": [7, 11, 21, 98, 99, 100, 101, 102, 105, 106, 107, 108, 116], "cliprewardsv0": 7, "union": [7, 10, 11, 12, 13, 14, 15, 17, 19], "higher": [7, 88], "gamma": [7, 20, 118, 120], "99": [7, 20, 120], "immedi": [7, 14, 16, 20, 111, 117], "exponenti": [7, 11, 16, 20], "averag": [7, 16, 20, 101, 118, 120], "fix": [7, 9, 10, 11, 16, 18, 20, 21, 23, 57, 92, 93, 96, 97, 98, 99, 100, 101, 103, 104, 108, 111, 113, 117, 120], "discount": [7, 20, 116, 118, 120], "factor": [7, 20, 116, 120], "passiv": [7, 18], "checker": [7, 8, 15, 18], "surround": [7, 18], "thei": [7, 9, 15, 18, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 106, 110, 111, 112, 113, 117], "test": [7, 18, 88, 109, 117, 120], "disable_render_order_enforc": [7, 18], "error": [7, 8, 11, 12, 14, 16, 18, 116, 118], "classic_control": [7, 18, 23, 91], "cartpoleenv": [7, 18], "resetneed": [7, 18], "cannot": [7, 12, 13, 18, 21, 23, 112, 117, 118], "order": [7, 8, 9, 10, 16, 18, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 97, 98, 99, 100, 101, 102, 103, 104, 105, 108, 109, 117], "enforc": [7, 8, 18], "buffer_length": 7, "100": [7, 10, 12, 18, 38, 48, 71, 72, 88, 89, 90, 92, 95, 100, 105, 112, 113], "stats_kei": 7, "track": [7, 14, 16, 18, 76, 89, 93, 98, 99, 100, 101, 102, 103, 104, 107, 108], "cumul": [7, 14, 16, 18], "length": [7, 8, 10, 11, 14, 16, 18, 19, 88, 92, 93, 94, 95, 99, 107, 109, 113, 114, 116], "At": [7, 18, 21, 33, 44, 47, 58], "statist": [7, 14, 18, 120], "vector": [7, 11, 14, 16, 18, 88, 90, 102, 103, 104, 105, 106, 120], "_episod": [7, 18], "respect": [7, 11, 15, 16, 18, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 98, 101, 102, 107, 119], "look": [7, 9, 14, 16, 18, 21, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 73, 75, 77, 78, 80, 81, 82, 84, 85, 117, 118, 119], "l": [7, 18, 107], "elaps": [7, 18], "sinc": [7, 18, 22, 24, 50, 51, 52, 53, 54, 87, 93, 97, 106, 109, 114, 117, 118], "begin": [7, 18, 23, 30, 33, 46, 71, 72, 73, 74, 116, 117], "form": [7, 9, 10, 11, 18, 92, 107, 117], "num": [7, 18, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "moreov": [7, 9, 16, 18, 24, 27, 37, 117, 119], "store": [7, 14, 15, 18, 120], "access": [7, 16, 18, 23, 55, 119], "return_queu": [7, 18, 116, 120], "length_queu": [7, 18, 116], "episode_reward_buff": 7, "last": [7, 14, 18, 46, 47, 57, 99, 102, 105, 106, 117, 118], "deque_s": [7, 18, 116], "episode_length_buff": 7, "size": [7, 9, 18, 113, 117], "record": [7, 14, 16, 18, 88, 120], "modular": [7, 16, 21, 109, 117, 119], "particularli": [7, 14, 18, 93], "haven": [7, 18], "screen": [7, 18, 24, 26, 30, 34, 39, 40, 42, 51, 57, 63, 73, 74, 98, 99, 100, 101, 102, 103, 104, 107, 108], "rememb": [7, 15, 18, 89, 117, 119], "render_fp": [7, 14, 18, 117], "either": [7, 10, 11, 12, 14, 18, 21, 22, 24, 47, 57, 94, 95, 111, 113, 120], "lunarland": [7, 8, 18, 21, 23, 90, 115], "simpli": [7, 17, 18, 19, 20, 24, 117], "nativ": [7, 18], "nonativerend": [7, 18], "warn": [7, 10, 14, 15, 18], "alwai": [7, 11, 12, 18, 24, 90, 94, 95, 98, 101, 102, 105, 107, 111, 112, 113, 114, 117], "empti": [7, 11, 13, 18, 74, 116, 120], "pop_fram": [7, 18], "reset_clean": [7, 18], "renderedfram": 7, "clear": [7, 18, 78], "interact": [7, 9, 14, 24, 109, 116, 117], "guarante": [7, 10, 24], "roundtrip": 7, "vice": [7, 13, 18], "versa": [7, 13, 18], "int_32": 7, "devicearrai": 7, "jnp": 7, "int23": 7, "input": [7, 14, 15, 21, 116], "devic": 7, "tensor": [7, 120], "torch": [7, 120], "load": 8, "pre": [8, 113], "sever": [8, 11, 18, 21, 23, 24, 54, 109], "id": [8, 15, 113, 117], "liter": 8, "mountaincar": [8, 94], "mountaincarcontinu": [8, 95], "acrobot": [8, 91], "lunarlandercontinu": [8, 119], "bipedalwalkerhardcor": 8, "blackjack": [8, 21, 110], "frozenlak": [8, 14, 113], "frozenlake8x8": 8, "cliffwalk": [8, 112], "taxi": [8, 110], "reacher": [8, 97, 105, 119], "v4": [8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 120], "pusher": [8, 97], "invertedpendulum": [8, 97, 103, 104, 120], "inverteddoublependulum": [8, 103], "halfcheetah": [8, 97, 99], "hopper": [8, 97, 108], "swimmer": [8, 97], "walker2d": [8, 97], "ant": [8, 97, 100, 108], "humanoidstandup": [8, 97, 102], "humanoid": [8, 45, 97, 98], "accord": [8, 10, 11, 15, 90], "find": [8, 24, 25, 50, 51, 52, 53, 54, 60, 69, 81, 103, 106, 117], "avail": [8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 109, 111, 112, 117], "modul": [8, 9, 13, 16, 117, 119, 120], "eg": [8, 14, 117], "max_episode_step": [8, 18, 117], "autoreset": [8, 15, 117], "apply_api_compat": [8, 18, 22], "stepapicompat": [8, 16, 18, 22], "argument": [8, 9, 10, 11, 12, 14, 15, 19, 21, 22, 87, 91, 110, 116, 117], "addit": [8, 9, 15, 18, 21, 23, 24, 30, 41, 71, 75, 90, 91, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 114, 116, 117, 120], "entry_point": [8, 117], "reward_threshold": [8, 98, 99, 100, 101, 102, 105, 106, 107, 108, 117], "nondeterminist": [8, 117], "order_enforc": [8, 117], "syntax": 8, "namespac": [8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 117], "env_nam": [8, 22], "v": [8, 73, 94, 103], "keyword": [8, 10, 15, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 111, 116, 117], "entri": 8, "point": [8, 12, 14, 16, 24, 27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 45, 46, 47, 48, 49, 51, 55, 56, 59, 66, 67, 68, 69, 70, 71, 72, 74, 76, 77, 78, 80, 81, 84, 88, 89, 90, 92, 93, 96, 106, 111, 118], "threshold": [8, 92, 93, 114, 117], "consid": [8, 89, 90, 91, 97, 98, 100, 101, 108, 117, 119], "learnt": 8, "knowledg": 8, "enabl": [8, 16, 24, 89, 109], "correct": [8, 114, 117, 118], "pprint_registri": 8, "env_id": [8, 22], "retriev": [8, 18, 45, 48, 74], "global": 8, "_registri": 8, "num_col": 8, "exclude_namespac": 8, "disable_print": 8, "column": [8, 120], "arrang": 8, "exclud": [8, 98, 99, 100, 101, 107, 108], "instead": [8, 15, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 92, 108, 117], "consol": [8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "variou": [9, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 102, 109, 120], "mathemat": [9, 120], "environ": [9, 11, 13, 16, 17, 18, 19, 20, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 110, 111, 112, 113, 114, 118, 119, 120], "three": [9, 16, 24, 27, 30, 32, 45, 53, 59, 66, 71, 72, 73, 74, 98, 100, 101, 102, 107, 108, 117], "cube": [9, 70], "superclass": [9, 17, 19, 20], "npt": [9, 10, 11], "dtypelik": 9, "crucial": 9, "clearli": 9, "u": [9, 24, 116, 117, 119, 120], "highli": [9, 23, 24, 87, 91, 109, 117], "data": [9, 12, 14, 16, 23, 116, 117, 119, 120], "painlessli": 9, "flat": [9, 10, 11, 12, 19], "especi": [9, 117, 119], "hierarch": [9, 114], "build": [9, 26, 44, 48, 103, 108, 119], "express": [9, 14, 98, 101, 102], "cover": [9, 116], "parametr": [9, 120], "distribut": [9, 10, 11, 107, 109, 113, 120], "batch": [9, 13, 15], "handl": [9, 15, 103, 111, 116], "care": [9, 75], "immut": 9, "is_np_flatten": 9, "mask": [9, 10, 11, 23, 114], "t_cov": 9, "randomli": [9, 10, 90, 113, 117], "uniform": [9, 11, 12, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "bounded": 9, "int8": [9, 11], "member": [9, 58], "possibli": [9, 10, 11, 13, 16, 18, 21, 109], "subspac": [9, 10, 11, 12], "to_json": 9, "sample_n": 9, "jsonabl": 9, "from_json": 9, "matric": [9, 109], "binari": [9, 11, 21, 117], "hold": [9, 111, 116], "down": [9, 11, 24, 28, 30, 31, 37, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 57, 58, 61, 63, 65, 66, 68, 70, 78, 80, 81, 84, 85, 89, 102, 111, 112, 113, 114, 116, 117, 118], "button": [9, 11, 14, 21], "multipl": [9, 10, 11, 13, 15], "ax": [9, 11, 116, 117], "messag": 9, "mission": [9, 57, 59, 60], "join": 9, "togeth": [9, 24, 97], "vectoris": [9, 109], "separ": [9, 10, 23, 117], "readabl": [9, 10], "unord": 9, "entiti": 9, "select": [9, 11, 47, 98, 101, 102, 105, 106], "graph": [9, 12, 21], "node": [9, 10, 12], "edg": [9, 10, 12, 93], "unflatten": [9, 12], "neural": [9, 12, 92, 107, 109, 120], "network": [9, 12, 107], "flatdim": [9, 12], "revers": [9, 12, 111, 120], "batch_spac": [9, 13], "concaten": [9, 12, 13, 97], "iter": [9, 10, 13, 15], "create_empty_arrai": [9, 13], "create_shared_memori": [9, 13], "read_from_shared_memori": [9, 13], "write_to_shared_memori": [9, 13], "spaces_kwarg": 10, "constitu": 10, "usag": [10, 11, 12, 89, 119], "nest": [10, 13, 109], "ext_control": 10, "inner_st": 10, "charg": 10, "system_check": 10, "job_statu": 10, "conveni": [10, 16, 21, 117], "easili": [10, 16], "deal": 10, "avoid": [10, 13, 16, 21, 24, 27, 28, 30, 35, 36, 44, 46, 62, 63, 65, 70, 72, 73, 78, 112, 114], "independ": [10, 11, 15, 100, 106, 107, 108], "attr": 10, "uniqu": [10, 87, 91, 97, 103], "sub": [10, 15, 74], "precis": 10, "cartesian": [10, 11, 96], "03633198": 10, "42370757": 10, "involv": [10, 87, 103, 104, 109, 112, 113, 114, 117, 120], "draw": [10, 111, 116, 117], "54": 10, "finit": [10, 11, 98, 102, 103, 104, 105, 106, 108, 118, 119, 120], "a_0": 10, "dot": [10, 11, 117], "a_n": 10, "a_i": 10, "belong": [10, 13], "n": [10, 11, 12, 13, 21, 24, 43, 89, 92, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 116, 120], "0259352": 10, "80977976": 10, "80066574": 10, "77165383": 10, "length_mask": 10, "sample_mask": 10, "drawn": [10, 11, 105, 107, 111], "geometr": 10, "featur": [10, 11, 109], "feature_spac": 10, "node_spac": [10, 12], "edge_spac": [10, 12], "seri": [10, 21], "adjac": 10, "matrix": 10, "edge_link": [10, 12], "num_nod": 10, "num_edg": 10, "graphinst": [10, 12], "edge_mask": 10, "multipli": [10, 95], "attribut": [10, 21, 23, 105, 106, 117, 119], "unbound": 11, "mathbb": 11, "interv": [11, 21, 98, 100, 101, 108], "There": [11, 16, 21, 56, 57, 69, 88, 90, 91, 94, 95, 97, 102, 103, 104, 105, 106, 109, 112, 114, 117], "common": [11, 16, 21, 118, 119, 120], "construct": [11, 24, 50, 51, 52, 53, 54, 92, 98, 99, 100, 101, 107, 108], "across": [11, 13, 30, 40, 51, 52, 55], "infer": [11, 97], "essenti": 11, "shift": [11, 21], "unsupport": 11, "is_bound": 11, "sens": [11, 109], "One": [11, 21, 98, 101, 102], "neither": [11, 18, 111], "nor": [11, 18, 111, 112], "smallest": 11, "maskndarrai": 11, "uniformli": [11, 24, 92, 93, 105, 106, 117], "infeas": 11, "sort": 11, "fair": 11, "coin": 11, "toss": 11, "per": [11, 15, 37, 56, 79, 114, 118], "nvec": 11, "int64": [11, 12], "keyboard": [11, 14], "nintendo": 11, "conceptu": 11, "arrow": [11, 68], "left": [11, 24, 26, 28, 29, 30, 31, 35, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 57, 58, 61, 63, 64, 65, 67, 70, 74, 75, 77, 78, 80, 81, 84, 85, 88, 89, 90, 93, 94, 98, 101, 102, 103, 104, 108, 112, 113, 114, 116, 117, 120], "param": 11, "press": [11, 14, 21, 89, 92], "although": 11, "rare": 11, "d": [11, 14, 16, 92, 105, 106], "categor": 11, "count": [11, 88, 90, 111, 116], "kind": 11, "unless": [11, 16, 111, 112, 113, 114, 117], "max_length": 11, "min_length": 11, "charset": 11, "alphanumer": 11, "compris": 11, "charact": [11, 14, 58, 69], "b5": 11, "hello": 11, "0123456789": 11, "digit": [11, 109], "inclus": [11, 24], "english": 11, "alphabet": 11, "plu": [11, 40, 112], "latin": 11, "charlist": 11, "matter": 11, "attempt": [12, 90, 102, 103, 114], "compound": 12, "equival": [12, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "oper": [12, 18], "boundari": 12, "while": [12, 23, 24, 27, 28, 44, 59, 62, 63, 64, 65, 68, 70, 72, 74, 78, 81, 83, 90, 92, 93, 96, 98, 99, 101, 102, 105, 106, 107, 111, 112, 116, 117, 118, 120], "exactli": [12, 107], "necessarili": 12, "hot": 12, "encod": [12, 114, 117], "60": [12, 58, 71, 109], "recurs": 12, "6": [12, 24, 28, 29, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 65, 79, 80, 81, 84, 85, 92, 94, 95, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 112, 114, 120], "notimplementederror": 12, "int32": 12, "15": [12, 13, 24, 47, 50, 51, 52, 53, 54, 58, 77, 90, 98, 99, 101, 102, 105, 108, 109, 113], "datapoint": 12, "k": [12, 14, 18, 90, 107], "m": [12, 24, 43, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 117], "would": [12, 17, 19, 20, 98, 99, 100, 101, 102, 107, 108, 115, 117, 119], "copi": [13, 15, 117], "item": [13, 14, 109, 116], "6348213": 13, "28607962": 13, "60760117": 13, "87383074": 13, "192658": 13, "2148103": 13, "99644893": 13, "08304597": 13, "7238421": 13, "35848552": 13, "1533453": 13, "67958736": 13, "49076623": 13, "38661423": 13, "7975036": 13, "93317133": 13, "stopiter": 13, "fn": 13, "multi_binari": 13, "built": [13, 23, 109], "multi_discret": 13, "ctx": 13, "mp": 13, "multiprocess": [13, 15], "opt": 13, "hostedtoolcach": 13, "9": [13, 15, 24, 28, 31, 36, 45, 50, 51, 52, 53, 54, 58, 61, 71, 80, 85, 92, 96, 98, 99, 100, 101, 102, 103, 105, 106, 108, 111, 116], "x64": 13, "lib": [13, 112], "python3": 13, "eventu": 13, "shared_memori": [13, 15], "customspaceerror": 13, "read": [13, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "forward": [13, 35, 57, 88, 98, 99, 100, 101, 102, 107, 108, 120], "side": [13, 19, 26, 39, 54, 56, 90], "write": [13, 15, 117], "num_env": [13, 15], "plai": [14, 21, 34, 47, 60, 79, 84, 89, 109, 116, 117], "transpos": [14, 117], "zoom": 14, "callback": 14, "keys_to_act": 14, "w": [14, 98, 101, 102], "7": [14, 24, 28, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 65, 80, 81, 84, 85, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "dw": 14, "verifi": 14, "level": [14, 16, 21, 24, 31, 48, 70, 78, 109], "unplay": 14, "wish": [14, 19, 23], "real": [14, 56, 99], "playplot": 14, "150": 14, "def": [14, 116, 117, 119, 120], "obs_t": 14, "obs_tp1": 14, "plotter": 14, "amount": [14, 44, 46, 47, 49, 55, 74, 88, 93, 103, 104, 120], "receiv": [14, 15, 21, 34, 37, 41, 44, 45, 51, 57, 60, 89, 90, 95, 108, 114, 116, 118, 120], "map": [14, 19, 21, 42, 113, 114, 116, 117, 120], "unicod": 14, "suppos": 14, "trigger": [14, 114], "key_to_act": 14, "ord": [14, 117], "unknown": [14, 109], "horizon_timestep": 14, "plot_nam": 14, "live": [14, 27, 30, 33, 34, 36, 39, 44, 45, 48, 55, 59, 60, 66, 72, 77, 116], "compute_metr": 14, "obs_tp": 14, "cumulative_reward": 14, "linalg": [14, 117], "norm": [14, 105, 106, 117], "along": [14, 16, 21, 22, 58, 59, 61, 72, 89, 93, 94, 95, 99, 103, 104, 107, 112], "conjunct": 14, "evolv": 14, "200": [14, 49, 72, 90, 92, 93, 94, 96, 113, 114], "magnitud": [14, 90, 95, 103, 104, 120], "your_env": 14, "len": [14, 116, 119], "horizon": [14, 74, 100, 108, 118], "titl": [14, 94, 95, 116, 120], "matplotlib": [14, 116, 120], "playablegam": 14, "process_ev": 14, "event": [14, 117], "particular": [14, 16, 19, 21, 22, 90, 118], "exit": [14, 15], "save_video": 14, "video_fold": [14, 18], "episode_trigg": [14, 18], "step_trigg": [14, 18], "video_length": [14, 18], "name_prefix": [14, 18], "episode_index": 14, "step_starting_index": 14, "extract": 14, "compos": 14, "iff": [14, 18, 117], "isn": 14, "entir": [14, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 100, 108, 109], "snippet": [14, 18], "captur": [14, 18, 83], "Will": [14, 18], "prepend": [14, 18], "filenam": [14, 18], "moviepi": [14, 18], "imagesequenceclip": 14, "durat": [14, 98, 100, 101, 102, 103, 104, 105, 106, 108, 120], "step_index": 14, "199": 14, "capped_cubic_video_schedul": 14, "episode_id": 14, "27": [14, 92, 98, 101, 102], "729": 14, "1000": [14, 18, 21, 72, 74, 89, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 115, 120], "2000": [14, 88, 114], "3000": 14, "schedul": [14, 113], "step_api_compat": [14, 22], "step_return": 14, "output_truncation_bool": [14, 18], "is_vector_env": 14, "doc": [14, 18, 90, 119], "interfac": [14, 115], "conflict": [14, 109], "desir": [14, 107, 108, 114], "oldenv": 14, "vec_env": 14, "convert_to_terminated_truncated_step_api": [14, 22], "irrespect": 14, "convert_to_done_step_api": 14, "omit": [14, 97, 98, 99, 100, 101, 102, 107, 108], "env_check": 14, "check_env": 14, "skip_render_check": 14, "invas": [14, 78], "org": [14, 24, 116, 120], "content": 14, "environment_cr": [14, 117], "ignor": [14, 24, 111, 116], "ci": 14, "linear": [15, 90, 92, 103, 104, 107, 120], "speed": [15, 35, 88, 89], "wait": [15, 48, 50, 74, 118], "until": [15, 16, 47, 60, 105, 111, 112, 113, 117, 120], "overwritten": [15, 16], "final_observ": [15, 18], "final_info": [15, 18], "asyncvectorenv": 15, "syncvectorenv": 15, "single_observation_spac": 15, "single_action_spac": 15, "v25": [15, 23], "vectorlistinfo": [15, 16, 18], "share": [15, 120], "other": [15, 17, 18, 19, 22, 30, 39, 42, 50, 56, 90, 96, 97, 99, 103, 105, 106, 108, 113, 114, 116, 117, 119, 120], "word": [15, 105, 120], "02240574": 15, "03439831": 15, "03904812": 15, "02810693": 15, "01586068": 15, "01929009": 15, "02394426": 15, "04016077": 15, "01314174": 15, "03893502": 15, "02400815": 15, "0038326": 15, "00122802": 15, "16228443": 15, "02521779": 15, "23700266": 15, "00788269": 15, "17490888": 15, "03393489": 15, "31735462": 15, "04918966": 15, "19421194": 15, "02938497": 15, "29495203": 15, "releas": [15, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 117], "resourc": [15, 116, 117], "viewer": 15, "close_extra": 15, "synchron": 15, "asynchron": 15, "garbag": 15, "04456399": 15, "04653909": 15, "01326909": 15, "02099827": 15, "03073904": 15, "00145001": 15, "03088818": 15, "03131252": 15, "03468829": 15, "01500225": 15, "01230312": 15, "01825218": 15, "registri": [15, 21], "env_fn": 15, "context": [15, 120], "daemon": 15, "worker": [15, 68], "pipe": 15, "commun": 15, "81": [15, 96], "62": 15, "8286432": 15, "5597771": 15, "90249056": 15, "85009176": 15, "5266346": 15, "60007906": 15, "back": [15, 16, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 52, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 98, 99], "effici": [15, 19, 94, 95, 103], "subprocess": 15, "flag": [15, 94, 109], "quit": [15, 117], "head": [15, 99], "spawn": [15, 106], "children": 15, "overrid": [15, 119], "inner": 15, "logic": [15, 89, 117], "degre": [15, 101, 102, 103, 119], "chanc": 15, "shoot": [15, 31, 40, 42, 47, 48, 55, 56, 65, 68, 72, 77, 78, 86], "yourself": [15, 48, 68, 89, 116, 117], "foot": [15, 99, 100, 108], "thu": [15, 24, 50, 51, 52, 53, 54, 105, 106, 109], "_worker": 15, "_worker_shared_memori": 15, "runtimeerror": 15, "serial": 15, "without": [16, 21, 46, 101, 111, 113, 116, 117], "alter": [16, 21, 24, 117], "lot": [16, 21, 116, 119, 120], "boilerpl": [16, 21, 117, 119], "chain": [16, 21, 92, 107], "base_env": [16, 21], "underneath": [16, 21, 93], "anoth": [16, 39, 101, 102, 104, 105, 106, 108, 114, 116, 118, 120], "get": [16, 21, 24, 44, 47, 48, 49, 56, 66, 67, 68, 69, 71, 72, 73, 78, 88, 90, 98, 100, 101, 103, 111, 116, 118], "layer": [16, 21, 120], "bare": 16, "box2d": [16, 21, 88, 89, 90], "bipedal_walk": [16, 21, 88], "0x7f87d70712d0": [16, 21], "thing": [16, 24, 119], "do": [16, 18, 19, 24, 37, 49, 70, 74, 75, 89, 90, 92, 98, 99, 100, 101, 107, 108, 117, 119, 120], "Such": [16, 119], "actionwrapp": [16, 17], "observationwrapp": [16, 19, 117], "rewardwrapp": [16, 20], "behavior": [16, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 96, 98, 99, 100, 101, 102, 107, 108, 109, 117, 119], "forget": [16, 117], "wrapperobstyp": [16, 19], "commonli": [16, 21], "found": [16, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 97, 109, 116, 120], "descript": [16, 117], "misc": 16, "tp": 16, "21": [16, 22, 98, 101, 102, 105, 111, 116], "request": [16, 111], "anobserv": 16, "rollout": [16, 18, 114], "save": [16, 18, 23, 119], "axbxc": [16, 19], "append": [16, 19, 116, 120], "th": 16, "overwrit": [17, 19, 20], "domain": [17, 89, 92, 94, 95, 114, 119], "among": [17, 19, 91, 97], "biped": [17, 87, 101, 102], "walker": [17, 87, 97, 106, 108], "new_ob": 18, "final_reward": 18, "final_termin": 18, "final_trunc": 18, "final_don": 18, "prior": 18, "alongsid": 18, "re": [18, 56, 71, 74, 117, 120], "old_env": 18, "legacyenv": 18, "retun": 18, "modern": [18, 109], "convers": [18, 22, 109], "customenv": 18, "manual": [18, 21, 23, 24, 50, 51, 52, 53, 54], "unregist": 18, "disable_logg": 18, "intermitt": 18, "sai": [18, 119], "hundredth": 18, "emploi": 18, "stop": [18, 21, 68, 78, 86, 111, 116, 118], "span": 18, "strictli": 18, "logger": 18, "place": [18, 28, 93, 94, 95, 117], "\u01f9one": 18, "outermost": 18, "actual": [18, 114, 117], "_k": 18, "classic": [18, 21, 23, 90, 92, 93, 94, 95, 96, 100, 104, 107, 108, 118], "reflect": [19, 93], "f": [19, 20, 116, 120], "ideal": 19, "subclass": 19, "accordingli": [19, 74, 117], "incorrectli": [19, 118], "randn": 19, "lz4_compress": 19, "lazyfram": 19, "lz4": 19, "compress": 19, "uint8": [19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "singleton": 19, "axbx1": 19, "axb": 19, "render_kwarg": 19, "pixel_kei": 19, "odict_kei": 19, "400": [19, 114], "600": 19, "assertionerror": 19, "typeerror": 19, "unexpect": 19, "64": [19, 24], "03810719": 19, "03522411": 19, "02231044": 19, "01088205": 19, "03881167": 19, "16021058": 19, "0220928": 19, "28875574": 19, "invari": 20, "incorrect": [20, 114, 118], "mountain": [21, 91], "car": [21, 33, 49, 87, 91, 109], "four": [21, 33, 44, 89, 90, 98, 100, 108, 114, 116], "introduc": [21, 22, 23, 98, 101, 102], "core": 21, "markov": [21, 118], "decis": [21, 23, 109, 118], "theori": [21, 96, 103], "perfect": 21, "reconstruct": 21, "compon": [21, 117], "veri": [21, 90, 105, 117], "easi": [21, 60, 110], "loop": [21, 23, 109, 116], "pictur": 21, "simplifi": [21, 96], "someth": 21, "spaceship": [21, 31, 45, 76], "land": [21, 38, 72, 90], "safe": [21, 68, 90], "document": [21, 24, 25, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 91, 98, 101, 102, 103, 110, 116, 117], "imagin": [21, 109, 119], "robot": [21, 23, 36, 86, 88, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 118], "destroi": [21, 27, 29, 31, 32, 33, 34, 35, 36, 39, 40, 41, 42, 45, 46, 54, 60, 71, 72, 73, 76, 77, 78, 80, 90], "enemi": [21, 28, 29, 32, 34, 35, 36, 42, 45, 46, 48, 61, 70, 71, 73, 74, 76, 77, 78, 80, 86], "exchang": 21, "crash": [21, 90], "succeed": 21, "similarli": [21, 90], "restart": 21, "train": [21, 23], "major": 21, "our": [21, 116, 117], "simpl": [21, 88, 110, 115, 118, 120], "coupl": 21, "nich": 21, "li": [21, 109], "lie": 21, "aspect": [21, 118], "regist": [22, 24], "sole": 22, "special": [22, 27, 57, 120], "gymv26environ": 22, "relev": [22, 117], "pong": [22, 24, 39, 43], "v5": [22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "introduct": [22, 92, 111, 112], "signatur": 22, "previous": [22, 24, 33], "backward": [22, 99, 120], "compliant": [22, 23], "oldv21env": 22, "break": [23, 31, 39, 89], "briefli": 23, "outlin": [23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 111, 116], "still": [23, 24, 116, 119], "stuck": [23, 27, 36, 55], "123": 23, "becaus": [23, 116, 117, 119], "emul": [23, 24], "randint": 23, "utilis": 23, "contrast": 23, "older": [23, 97], "definit": [23, 98, 100, 101, 118], "expand": [23, 24, 50, 51, 52, 53, 54], "blog": 23, "post": [23, 32], "soon": [23, 117], "extra": [23, 28, 33, 34, 36, 40, 42, 55, 74, 109, 116, 117], "address": 23, "associ": 23, "librari": [23, 115, 117], "primari": 23, "shouldn": [23, 117], "happen": [23, 73, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 111, 113, 114, 118], "fly": [23, 26, 31, 55, 71, 90], "shown": [23, 59, 89, 92, 118], "explan": [23, 116, 120], "summari": 23, "goalenv": 23, "reimplement": 23, "system": [23, 54, 92, 96, 103, 104, 109], "monitor": 23, "stella": 24, "adventur": [24, 43], "air": [24, 43], "raid": [24, 43], "alien": [24, 43, 45, 65, 78], "amidar": [24, 32, 43], "assault": [24, 43], "asterix": [24, 43], "asteroid": [24, 43], "atlanti": [24, 43], "licens": [24, 109, 116, 120], "download": [24, 116, 117, 118, 119, 120], "locat": [24, 60, 109, 112, 113, 114, 117], "agre": 24, "todo": [24, 114], "legal": [24, 50, 51, 52, 53, 54], "fire": [24, 26, 28, 29, 31, 32, 35, 37, 39, 40, 50, 51, 52, 53, 54, 57, 64, 65, 67, 68, 70, 71, 73, 74, 77, 78, 80, 82, 84, 85, 90], "upright": [24, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 63, 93, 96, 103, 104], "upleft": [24, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 63], "downright": [24, 30, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63], "downleft": [24, 30, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63], "upfir": [24, 28, 31, 37, 50, 51, 52, 53, 54, 68, 80, 81, 82, 84, 85], "11": [24, 31, 50, 51, 52, 53, 54, 58, 61, 98, 99, 100, 101, 102, 103, 105, 106, 108, 111, 112, 116], "rightfir": [24, 26, 28, 29, 31, 32, 35, 40, 50, 51, 52, 53, 54, 58, 61, 64, 65, 67, 77, 80, 81, 84, 85], "leftfir": [24, 26, 28, 29, 31, 32, 40, 50, 51, 52, 53, 54, 58, 61, 64, 65, 67, 77, 80, 81, 84, 85], "13": [24, 31, 50, 51, 52, 53, 54, 58, 61, 98, 99, 101, 102, 105, 108, 114], "downfir": [24, 28, 31, 37, 50, 51, 52, 53, 54, 58, 61, 65, 68, 80, 82, 85], "14": [24, 50, 51, 52, 53, 54, 58, 88, 90, 98, 99, 101, 102, 105, 108, 116], "uprightfir": [24, 31, 50, 51, 52, 53, 54, 58, 61], "upleftfir": [24, 31, 50, 51, 52, 53, 54, 58, 61], "16": [24, 33, 36, 39, 45, 50, 51, 52, 53, 54, 58, 96, 98, 99, 101, 102, 105, 108, 113, 116, 120], "downrightfir": [24, 50, 51, 52, 53, 54, 58, 61], "17": [24, 36, 50, 51, 52, 53, 54, 58, 98, 99, 101, 102, 105, 108, 111], "downleftfir": [24, 50, 51, 52, 53, 54, 58, 61], "smaller": [24, 29, 31, 50, 51, 52, 53, 54], "enumer": [24, 50, 51, 52, 53, 54], "full_action_spac": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "reduc": [24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 93, 116], "difficulti": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 109], "choic": [24, 120], "player": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 109, 111, 112, 113, 114, 116], "128": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "byte": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "ram": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "exact": [24, 37, 50, 51, 52, 53, 54, 111, 116], "atariag": [24, 25, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 79, 81, 82, 83, 84, 85, 86], "determinist": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 94, 95, 117], "art": 24, "memor": 24, "sticki": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "small": [24, 54, 73, 88, 109, 110, 120], "On": [24, 65, 92, 93, 94, 95, 96], "top": [24, 39, 44, 51, 57, 59, 73, 89, 90, 91, 94, 95, 100, 103, 104, 108, 117, 120], "frameskip": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "exclus": 24, "tabl": [24, 27, 28, 30, 44, 46, 101, 102, 105], "obs_typ": 24, "Its": 24, "stick": [24, 33, 55, 111, 116], "ll": [24, 57, 74, 116], "sound": 24, "lock": 24, "proper": 24, "audio": 24, "framer": [24, 102, 105, 106, 117], "best": [24, 50, 51, 52, 53, 54, 116], "practic": [24, 50, 51, 52, 53, 54, 109, 116, 118], "configur": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 91, 97, 110, 116, 117], "let": [24, 39, 109, 116, 117, 119, 120], "variat": 24, "amidardeterminist": 24, "amidarnoframeskip": 24, "ramdeterminist": 24, "ramnoframeskip": 24, "suffix": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "noframeskip": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "airraid": [24, 26], "31": [24, 31, 101, 102], "bankheist": [24, 33], "20": [24, 33, 39, 66, 74, 90, 98, 101, 102, 105, 114, 116], "battlezon": [24, 34], "beamrid": [24, 35], "berzerk": [24, 43], "18": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 101, 102, 105, 108], "bowl": [24, 43], "breakout": [24, 43], "36": [24, 39, 101, 102, 112], "40": [24, 39, 64, 101, 102], "44": [24, 39, 101, 102], "carniv": [24, 43], "centiped": [24, 43], "22": [24, 41, 98, 101, 102, 105, 116], "86": [24, 41], "choppercommand": [24, 42], "crazyclimb": [24, 44], "defend": [24, 32, 43, 48, 64, 76], "demonattack": [24, 46], "doubledunk": [24, 47], "elevatoract": [24, 48], "enduro": [24, 43], "fishingderbi": 24, "freewai": [24, 43], "frostbit": [24, 43], "gopher": [24, 43], "gravitar": [24, 43], "hero": [24, 43], "icehockei": 24, "jamesbond": [24, 43], "journeyescap": 24, "kangaroo": [24, 43], "krull": [24, 43], "kungfumast": [24, 61], "montezumareveng": [24, 62], "mspacman": [24, 63], "namethisgam": [24, 64], "phoenix": [24, 43], "pitfal": [24, 43, 88], "pooyan": [24, 43], "50": [24, 58, 68, 74, 90, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 120], "70": [24, 68], "privateey": 24, "qbert": [24, 43], "riverraid": [24, 43], "roadrunn": [24, 72], "robotank": [24, 43, 73], "seaquest": [24, 43], "ski": [24, 43], "solari": [24, 43], "spaceinvad": 24, "stargunn": 24, "tenni": [24, 43], "timepilot": 24, "tutankham": [24, 43], "upndown": [24, 82], "ventur": [24, 43], "videopinbal": [24, 84, 85], "wizardofwor": [24, 85], "yarsreveng": 24, "zaxxon": [24, 43], "oppos": 24, "ai": [24, 109], "mg": 24, "bellemar": 24, "naddaf": 24, "j": [24, 120], "veness": 24, "platform": 24, "journal": [24, 114], "artifici": [24, 114], "intellig": [24, 114], "research": [24, 97, 103, 114], "2012": 24, "url": 24, "jair": [24, 114], "php": 24, "articl": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "view": [24, 54], "11182": 24, "250": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "160": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "enchant": [25, 41], "chalic": 25, "golden": 25, "castl": 25, "pick": [25, 48, 59, 72, 114, 116], "sword": 25, "bridg": [25, 71, 109], "magnet": 25, "fight": [25, 38, 61, 64, 109], "outmanoeuvr": 25, "dragon": 25, "flavor": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "switch": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 109], "thorough": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "discuss": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "intric": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "stochast": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 91, 92, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 106, 107, 108, 112, 114], "ship": [26, 27, 29, 35, 45, 54, 76], "sidewai": [26, 29, 35, 71, 75], "protect": [26, 32, 41, 42, 53, 58, 68], "saucer": [26, 31, 78], "try": [26, 27, 28, 37, 39, 44, 67, 68, 116], "drop": [26, 33, 114], "bomb": [26, 45, 57, 65, 78], "those": [26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 92, 97, 98, 99, 100, 101, 102, 103, 104, 108, 119], "meaning": [26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85], "210": [27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 42, 45, 46, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "maze": [27, 28, 33, 36, 62, 81], "egg": 27, "scatter": [27, 69], "simultan": [27, 28], "kill": [27, 36, 55, 60], "flamethrow": 27, "awai": [27, 67, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 119], "tricki": 27, "situat": [27, 114], "occasion": [27, 31], "power": [27, 55, 89, 90, 95, 103, 104, 120], "pulsar": 27, "temporari": 27, "abil": 27, "score": [27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 74, 78, 79, 80, 84, 88, 90], "prize": [27, 82], "caught": [27, 33], "lose": [27, 30, 32, 41, 44, 45, 48, 55, 57, 66, 67, 69, 71, 72, 74, 75, 76, 90, 111, 116], "consult": [27, 56, 57, 58, 59, 60], "pac": 28, "man": 28, "visit": [28, 48, 89], "grid": [28, 112, 113, 114, 116, 117], "chicken": [28, 40, 51], "catch": [28, 44, 50, 72], "travers": 28, "vehicl": [29, 34, 49, 58], "big": [29, 50, 117], "mother": [29, 59, 68], "circl": [29, 117], "overhead": [29, 90], "deploi": 29, "drone": 29, "dodg": [29, 35, 74], "attack": [29, 31, 32, 35, 43, 45, 68], "horizont": [30, 40, 88, 90, 92, 117], "vertic": [30, 88, 103, 104, 117], "lyre": 30, "guid": [30, 51, 109], "suffici": [30, 32, 34, 36, 41, 116], "award": [30, 36, 41, 44, 48, 74, 103, 104], "field": 31, "appear": [31, 33, 36, 73, 94, 95], "satellit": 31, "ufo": 31, "job": [32, 117], "submerg": 32, "citi": [32, 33, 69], "slowli": 32, "descend": 32, "toward": [32, 56, 107], "strike": [32, 37], "distanc": [32, 49, 99, 117], "defens": [32, 47], "manag": [32, 37], "seven": 32, "fought": 32, "wave": [32, 42, 46, 58, 65, 78], "outer": [32, 92], "robber": 33, "natur": [33, 111, 113, 116], "rob": 33, "getawai": 33, "navig": [33, 69, 114, 117, 119], "polic": 33, "chase": [33, 36], "dynamit": [33, 55], "ga": [33, 89], "tank": [33, 34, 74], "nine": [33, 98], "leav": [33, 93, 97, 103, 117], "person": [34, 92, 109], "perspect": [34, 109], "3d": [34, 98, 101, 102, 109, 116], "illus": 34, "radar": [34, 85], "gain": [34, 70, 76, 77, 80, 119], "travel": 35, "steer": [35, 82, 89], "debri": 35, "leftifir": 35, "evil": [36, 61, 86], "touch": [36, 69, 117], "wall": [36, 39, 55, 94, 95, 114], "undefeat": 36, "otto": 36, "tri": 37, "knock": [37, 38], "pin": 37, "spar": 37, "spare": 37, "300": [37, 49, 88, 117], "oppon": [38, 50, 56, 67, 82], "ring": 38, "hit": [38, 39, 40, 41, 46, 68, 69, 72, 73, 75, 111, 116], "punch": [38, 59], "famou": 39, "paddl": [39, 67], "ball": [39, 56, 67, 84], "brick": 39, "wreak": 39, "havoc": 39, "five": [39, 69, 75, 91, 120], "214": 40, "em": 40, "target": [40, 92, 105, 106, 117, 118, 119], "gun": [40, 48], "suppli": [40, 64, 118], "ammunit": 40, "steal": [40, 50], "bullet": [40, 109], "subtract": 40, "minu": 40, "sign": [40, 103, 104, 120], "elf": [41, 113], "magic": 41, "wand": 41, "fend": 41, "spider": 41, "flea": 41, "mushroom": 41, "forest": 41, "bitten": 41, "tempor": 41, "paralyz": 41, "scorpion": 41, "round": 41, "helicopt": [42, 44, 71], "truck": [42, 72], "convoi": 42, "aircraft": [42, 80], "mini": 42, "bottom": [42, 57, 89, 94, 95, 100, 108, 112], "plane": 42, "surviv": [42, 46], "bank": [43, 71], "heist": 43, "battl": 43, "zone": 43, "beam": [43, 55], "rider": 43, "chopper": 43, "command": [43, 97], "crazi": 43, "climber": 43, "demon": 43, "doubl": [43, 97], "dunk": 43, "elev": 43, "fish": 43, "derbi": 43, "ic": [43, 46, 52, 56, 113], "hockei": [43, 56], "journei": [43, 58, 116], "kung": 43, "fu": 43, "master": [43, 112, 120], "montezuma": 43, "reveng": 43, "pacman": 43, "privat": [43, 69, 117], "ey": [43, 64, 69], "road": [43, 89], "runner": 43, "invad": [43, 77], "star": 43, "gunner": 43, "pilot": [43, 65, 86], "pinbal": 43, "wizard": [43, 61], "Of": 43, "wor": 43, "yar": 43, "builid": 44, "obstacl": [44, 58, 59], "fall": [44, 46, 66, 72, 88, 101, 112, 113, 118], "damag": [44, 73], "climb": [44, 59, 91], "fast": [44, 89, 97, 99, 101, 103, 107, 109], "least": [44, 79, 90], "row": [44, 77, 112, 113], "earth": [45, 77], "rescu": [45, 55, 59, 60, 61, 74, 76], "smart": 45, "shot": [45, 47, 48, 56, 57, 72, 109], "abduct": 45, "unlimit": 45, "laser": [45, 55, 77, 85], "missil": [45, 71], "aliv": [45, 101, 108], "face": [46, 98, 101, 102, 111, 116], "planet": [46, 54, 76], "krybor": 46, "accumul": 46, "reserv": [46, 71, 73, 74], "bunker": [46, 54], "increas": [46, 74, 90, 92, 93, 100, 107, 108, 116], "grant": [46, 90], "slai": 46, "2v2": 47, "basketbal": 47, "possess": 47, "rival": 47, "team": 47, "rule": [47, 79, 109, 111, 116], "foul": 47, "success": [47, 92, 114, 118], "secret": [48, 69], "ground": [48, 68, 88, 90, 98, 102, 109], "stair": 48, "equip": 48, "against": [48, 67, 79, 109], "floor": 48, "gather": [48, 69, 117], "apart": [48, 117], "mark": 48, "red": [48, 54, 114, 117], "door": 48, "unreleas": 48, "prototyp": 48, "racer": 49, "nation": 49, "long": [49, 84, 93, 103, 104], "endur": 49, "race": [49, 72, 75, 87], "overtak": 49, "dai": [49, 87], "stai": 49, "meet": 49, "quota": 49, "sunfish": 50, "But": [50, 75], "fisherman": 50, "black": [50, 60, 92, 116], "shark": [50, 64, 74], "lurk": [50, 70], "surfac": [50, 74, 103, 104, 117], "lane": [51, 109], "busi": 51, "rush": 51, "hour": 51, "traffic": [51, 109], "cross": [51, 112, 113], "bailei": 52, "who": [52, 53, 58, 59, 75], "hop": [52, 70, 100, 108], "forth": 52, "arctic": 52, "river": [52, 71], "block": 52, "white": 52, "blue": [52, 54, 79, 92, 114, 117], "he": [52, 54], "hi": [52, 54, 69], "igloo": 52, "shovel": 53, "wield": 53, "farmer": 53, "crop": 53, "carrot": 53, "spacecraft": 54, "fiction": 54, "solar": 54, "landscap": 54, "miner": 55, "mine": [55, 72], "shaft": 55, "tool": [55, 62], "propel": 55, "backpack": 55, "wherev": 55, "blast": [55, 74, 76], "vermin": 55, "raft": 55, "stretch": 55, "critter": 55, "standard": [56, 98, 99, 103, 109, 120], "minut": [56, 66], "period": [56, 100, 108], "puck": 56, "angl": [56, 57, 88, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "extrem": [56, 109, 110], "rink": 56, "realli": [56, 116], "mr": 57, "bond": 57, "design": [57, 92, 98, 101, 102, 109, 110, 114, 117], "multipurpos": 57, "craft": [57, 90], "varieti": [57, 109], "motion": [57, 109], "slightli": [57, 88, 98], "jump": [57, 72, 75], "dive": 57, "lob": 57, "rate": [57, 74, 116, 120], "highest": 57, "novic": 57, "006": 57, "007": 57, "lead": 58, "peski": 58, "backstag": 58, "scarab": 58, "000": [58, 71, 74], "concert": 58, "cash": 58, "grasp": 58, "groupi": 58, "photograph": 58, "promot": 58, "encount": [58, 59, 80], "her": [59, 68], "preciou": 59, "babi": 59, "ladder": [59, 88], "bonu": [59, 73, 74, 78, 108], "fruit": 59, "throw": [59, 119], "monkei": 59, "corner": 59, "beast": 60, "fortress": [60, 62], "princess": [60, 61], "lyssa": 60, "sunris": 60, "monster": [60, 83], "templ": 61, "victoria": 61, "defeat": [61, 70], "acquir": [62, 97], "treasur": [62, 64, 66, 81, 83], "chamber": [62, 83], "emperor": 62, "deadli": 62, "creatur": [62, 70, 78, 109], "valuabl": 62, "pellet": 63, "ghost": 63, "discov": 64, "octopu": 64, "oxygen": [64, 74], "elimin": [65, 81, 82, 83], "war": 65, "bird": 65, "harri": 66, "jungl": 66, "die": [66, 68, 89], "misfortun": 66, "hole": [66, 113], "compet": 67, "deflect": 67, "pig": 68, "piglet": 68, "wolv": 68, "rope": 68, "balloon": 68, "guard": 68, "eat": [68, 72], "bait": 68, "wolf": 68, "stone": 68, "rock": [68, 72], "french": 69, "pierr": 69, "street": 69, "park": 69, "passag": 69, "dead": 69, "search": 69, "ringlead": 69, "henri": 69, "le": 69, "fiend": 69, "gang": 69, "evid": 69, "stolen": 69, "good": [69, 116], "statut": 69, "expir": 69, "nab": 69, "question": [69, 109], "auto": 69, "pothol": 69, "q": [70, 114, 118], "bert": 70, "pyramid": 70, "destin": [70, 114], "nasti": 70, "jet": 71, "fli": 71, "fuel": [71, 76, 90], "depot": 71, "collid": [71, 74, 90], "squadron": [71, 73], "tanker": 71, "80": [71, 113], "tm": 72, "direct": [72, 74, 92, 93, 94, 95, 98, 100, 101, 103, 104, 108, 112, 113, 114, 117, 120], "outrun": 72, "wile": 72, "coyot": 72, "hazard": 72, "desert": 72, "rocket": [72, 73, 90], "cannon": [72, 77], "cliff": [72, 110], "steel": 72, "pile": 72, "birdse": 72, "cannonbal": 72, "activ": [73, 117], "scrambl": 73, "static": 73, "interfer": 73, "sensor": [73, 89], "report": [73, 88, 98, 101, 102], "flash": 73, "panel": 73, "c": [73, 90, 92, 120], "earn": 73, "twelv": 73, "abl": [74, 118], "torpedo": 74, "diver": 74, "killer": 74, "deliv": [74, 114], "six": [74, 108], "explod": 74, "anyth": 74, "decreas": [74, 90], "almost": 74, "yoursub": 74, "blow": 74, "forc": [74, 76, 90, 93, 94, 95, 98, 101, 102, 103, 104, 120], "less": [74, 88, 90, 103, 107], "worth": [74, 77], "90": [74, 116], "skier": 75, "gate": 75, "fastest": 75, "penal": [75, 119], "tree": 75, "slalom": 75, "penalti": [75, 114], "warp": 76, "sector": 76, "feder": 76, "zylon": 76, "refuel": 76, "cadet": 76, "corridor": 76, "mania": [78, 80], "orang": 79, "win": [79, 111, 116], "margin": 79, "ti": 79, "sport": 79, "till": [79, 116], "technologi": 80, "increasingli": 80, "difficult": [80, 93, 97, 103, 104], "rack": 81, "tomb": 81, "guardian": 81, "baja": 82, "bugger": 82, "dungeon": 83, "beat": [85, 111], "scanner": 85, "armi": 86, "enslav": 86, "galaxi": 86, "fighter": 86, "lunar": 87, "lander": 87, "toi": [87, 111, 112, 113, 114], "contribut": 87, "earli": [87, 116, 120], "oleg": [87, 88, 89, 90], "klimov": [87, 88, 89, 90], "popular": [87, 109, 116], "benchmark": 87, "ever": 87, "joint": [88, 90, 92, 93, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "uneven": 88, "terrain": [88, 109], "hardcor": 88, "stump": 88, "1600": 88, "heurist": [88, 90], "demonstr": [88, 109], "motor": [88, 107], "hip": [88, 98, 101, 102], "knee": [88, 101, 102], "hull": 88, "angular": [88, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "leg": [88, 90, 98, 100, 101, 102, 108], "contact": [88, 90, 97, 98, 101, 102, 103], "lidar": 88, "rangefind": 88, "measur": [88, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "far": [88, 89, 103, 105, 106, 113], "torqu": [88, 92, 96, 98, 99, 100, 101, 102, 105, 106, 107, 108], "cost": [88, 99, 100, 107, 108, 109], "better": 88, "stand": [88, 97, 98, 101, 102, 103, 104, 116], "slight": 88, "exce": [88, 111], "closest": 88, "trace": 88, "furthest": 88, "faster": [88, 90, 102], "energi": [88, 90, 93, 109, 119], "spent": [88, 90], "friction": [88, 107, 109], "nervous": 88, "easiest": 89, "ab": 89, "wheel": 89, "gyroscop": 89, "car_rac": 89, "rear": 89, "drive": 89, "brake": 89, "96x96": 89, "tile": 89, "732": 89, "926": 89, "rest": [89, 90, 100, 108], "playfield": 89, "lap_complete_perc": 89, "dictat": [89, 90], "percentag": [89, 116], "lap": 89, "domain_random": 89, "variant": [89, 117], "scenario": [89, 109], "background": 89, "demand": 89, "correspondingli": 89, "init": [89, 117], "scheme": [89, 109], "chri": 89, "campbel": 89, "2014": 89, "www": [89, 111, 112, 113], "iforce2d": 89, "net": [89, 111, 112, 120], "b2dtut": 89, "pontryagin": 90, "principl": 90, "engin": [90, 97], "throttl": 90, "why": 90, "pad": 90, "infinit": [90, 100, 108, 111, 116, 118], "lunar_land": 90, "orient": [90, 98, 101, 102], "sum": [90, 98, 99, 100, 101, 102, 105, 106, 107, 108, 111, 116], "closer": [90, 111], "slower": 90, "tilt": 90, "03": 90, "solut": [90, 116], "viewport": 90, "mass": [90, 98, 99, 101, 102, 107], "bodi": [90, 97, 98, 99, 100, 101, 102, 103, 104, 107, 108], "moon": 90, "greater": [90, 92, 93, 94, 95, 99, 104, 105, 107, 111, 116], "awak": 90, "group": 90, "come": [90, 98, 99, 100, 101, 102, 103, 104, 107, 108], "sleep": 90, "littl": [90, 102], "cpu": 90, "wake": 90, "attach": [90, 93, 96, 98, 108], "graviti": [90, 93, 94, 96, 103, 104, 120], "enable_wind": 90, "wind_pow": 90, "turbulence_pow": 90, "later": [90, 116, 120], "booster": 90, "gravit": 90, "wind": 90, "tanh": [90, 120], "sin": [90, 96, 103], "pi": [90, 92, 96], "9999": 90, "rotat": [90, 92, 98, 101, 102, 105, 116], "turbul": 90, "renorm": 90, "harder": 90, "push": [90, 93, 103, 104, 109, 120], "cart": [91, 103, 104, 120], "nois": [91, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "regard": 91, "underpow": 91, "effort": 91, "easier": 91, "ones": [91, 97, 116], "57": 92, "spars": [92, 117], "coars": 92, "barto": [92, 93, 103, 104, 111, 112, 116], "book": [92, 111, 112, 116], "link": [92, 98, 99, 100, 101, 102, 106, 107], "actuat": [92, 93, 101, 102], "swing": [92, 96], "free": [92, 96, 97, 98, 101, 102, 103, 104, 109, 116, 120], "height": [92, 100, 103, 108], "hang": 92, "downward": 92, "seen": [92, 99], "green": [92, 114], "cosin": [92, 103, 106], "theta1": 92, "sine": [92, 103, 106], "theta2": 92, "567": 92, "274": 92, "rel": [92, 101, 102, 117, 119], "4\u03c0": 92, "9\u03c0": 92, "rad": [92, 93, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "incur": [92, 112, 114], "co": [92, 94, 95, 96, 103, 106, 109], "book_or_nip": 92, "neurip": 92, "paper": [92, 103, 114], "nip": 92, "equat": [92, 96], "present": [92, 109, 116, 117, 118], "confirm": 92, "radian": [92, 93, 96, 104], "1996": 92, "touretzki": 92, "mozer": 92, "hasselmo": 92, "ed": 92, "vol": [92, 114], "mit": [92, 116, 120], "proceed": 92, "cc": 92, "1995": 92, "8f1d43620bc6bb580df6e80b0dc05c48": 92, "pdf": [92, 111, 112], "anderson": [93, 103, 104], "neuronlik": [93, 103, 104], "adapt": [93, 103, 104, 112], "un": 93, "frictionless": 93, "balanc": [93, 103, 104, 120], "vari": [93, 103, 104, 120], "418": 93, "untermin": 93, "2095": 93, "allot": 93, "475": 93, "05": [93, 98, 99, 105], "07": [94, 95], "sinusoid": [94, 95], "vallei": [94, 95], "strateg": [94, 95], "hill": [94, 95], "andrew": [94, 95], "moor": [94, 95], "phd": [94, 95, 107], "thesi": [94, 95, 107], "1990": [94, 95], "techreport": [94, 95], "moore90efficientmemori": [94, 95], "author": [94, 95, 109, 116, 120], "william": [94, 95, 120], "institut": [94, 95], "univers": [94, 95], "cambridg": [94, 95], "year": [94, 95], "001": [94, 96, 100, 103, 108, 116], "0025": [94, 95], "collis": [94, 95], "inelast": [94, 95], "upon": [94, 95], "quickli": [94, 116], "penalis": [94, 95, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "0015": 95, "45": [95, 102, 105], "999": 95, "invert": [96, 97, 120], "swingup": 96, "diagram": 96, "meter": 96, "theta": 96, "tau": 96, "counter": 96, "clockwis": 96, "theta_dt": 96, "heta": 96, "2736044": 96, "calcul": [96, 97, 98, 101, 102, 103, 105, 106, 112, 113, 114], "math": 96, "half": 97, "cheetah": 97, "standup": 97, "facilit": [97, 109], "biomechan": 97, "graphic": [97, 109], "anim": 97, "area": 97, "accur": 97, "octob": 97, "2021": [97, 109], "deepmind": 97, "2022": 97, "everyon": 97, "instruct": [97, 117], "websit": [97, 109], "github": [97, 112, 117], "framework": 97, "ten": [97, 111, 116], "iverteddoublependulum": 97, "gaussian": 97, "mjsim": 97, "qpo": 97, "qvel": [97, 101, 102], "indirectli": 97, "xml": [97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "tweak": [97, 107], "schulman": 98, "moritz": 98, "levin": 98, "jordan": 98, "abbeel": 98, "advantag": [98, 109, 117], "estim": [98, 118, 120], "torso": [98, 99, 100, 101, 102, 108], "eight": 98, "hing": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "rotor": [98, 99, 100, 101, 102, 107, 108], "hip_4": 98, "right_back_leg": 98, "angle_4": 98, "front": [98, 99, 107], "hip_1": [98, 101, 102], "front_left_leg": [98, 101, 102], "angle_1": [98, 101, 102], "hip_2": [98, 101, 102], "front_right_leg": [98, 101, 102], "angle_2": 98, "hip_3": 98, "back_leg": 98, "angle_3": 98, "deriv": [98, 99, 100, 101, 102, 103, 104, 107, 108, 120], "exclude_current_positions_from_observ": [98, 99, 100, 101, 107, 108], "113": 98, "regardless": [98, 99, 100, 101, 107, 108], "x_posit": [98, 99, 100, 101, 107, 108], "y_posit": [98, 101, 107], "111": 98, "z": [98, 99, 100, 101, 102, 105, 106, 108], "centr": [98, 101, 102], "ankle_1": 98, "ankle_2": 98, "ankle_3": 98, "ankle_4": 98, "19": [98, 101, 102, 105], "23": [98, 101, 102, 105, 109], "use_contact_forc": 98, "translat": [98, 101, 102, 117], "dof": [98, 101, 102], "quaternion": [98, 101, 102], "healthy_reward": [98, 100, 101, 108], "healthi": [98, 100, 101, 108], "forward_reward": [98, 99, 100, 101, 107, 108], "dt": [98, 99, 100, 101, 105, 106, 107, 108], "frametim": [98, 99, 100, 101, 107, 108], "ctrl_cost": [98, 99, 100, 101, 107, 108], "too": [98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "ctrl_cost_weight": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "ctr_cost_weight": 98, "contact_cost": [98, 101], "contact_cost_weight": [98, 101], "contact_force_rang": 98, "reset_noise_scal": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "deviat": [98, 99, 103, 107, 120], "intention": [98, 101, 102], "therebi": [98, 101, 102], "unhealthi": [98, 100, 101, 108], "healthy_z_rang": [98, 100, 101, 108], "terminate_when_unhealthi": [98, 100, 101, 108], "xml_file": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "path": [98, 99, 100, 101, 105, 106, 107, 108, 113], "model": [98, 99, 100, 101, 107, 108, 109, 116], "weight": [98, 99, 100, 101, 105, 106, 107, 108, 109, 118, 119, 120], "5e": [98, 100, 101, 108], "perturb": [98, 99, 100, 101, 107, 108], "induct": [98, 99, 100, 101, 107, 108], "bia": [98, 99, 100, 101, 107, 108], "induc": [98, 99, 100, 101, 107, 108], "agnost": [98, 99, 100, 101, 102, 107, 108], "bind": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "restor": 98, "camera": [98, 99, 100, 101, 102, 103, 104, 107, 108], "max_time_step": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "p": [99, 109, 112, 113, 114], "wawrzy\u0144ski": 99, "cat": 99, "paw": 99, "alloc": 99, "thigh": [99, 100, 101, 102, 108], "shin": [99, 101, 102], "feet": [99, 108], "bthigh": 99, "bshin": 99, "bfoot": 99, "fthigh": 99, "fshin": 99, "ffoot": 99, "tip": [99, 103, 107], "rootz": [99, 100, 108], "slide": [99, 100, 103, 104, 105, 106, 107, 108], "rooti": [99, 100, 108], "rootx": [99, 100, 108], "forward_reward_weight": [99, 100, 101, 107, 108], "half_cheetah": 99, "erez": [100, 101, 102, 108], "tassa": [100, 101, 102, 108], "todorov": [100, 101, 102, 108], "predict": [100, 108, 120], "nonlinear": [100, 108], "compar": [100, 107, 108, 116], "figur": [100, 108, 116, 120], "middl": [100, 108], "thigh_joint": [100, 108], "leg_joint": [100, 108], "foot_joint": [100, 108], "008": [100, 108], "els": [100, 108, 113, 116, 117], "healthy_state_rang": 100, "fallen": 100, "healthy_angle_rang": [100, 108], "376": [101, 102], "synthesi": [101, 102], "onlin": [101, 102, 109, 111, 112], "abdomen": [101, 102], "pair": [101, 102], "arm": [101, 102, 105, 106, 109], "elbow": [101, 102, 105], "walk": [101, 108, 110, 113, 117], "right_hip_x": [101, 102], "right_thigh": [101, 102], "right_hip_z": [101, 102], "right_hip_i": [101, 102], "right_kne": [101, 102], "left_hip_x": [101, 102], "left_thigh": [101, 102], "left_hip_z": [101, 102], "left_hip_i": [101, 102], "left_kne": [101, 102], "right_shoulder1": [101, 102], "right_shoulder2": [101, 102], "right_elbow": [101, 102], "left_shoulder1": [101, 102], "left_shoulder2": [101, 102], "left_elbow": [101, 102], "378": 101, "root": [101, 102], "lower_waist": [101, 102], "abdomen_z": [101, 102], "abdomen_i": [101, 102], "pelvi": [101, 102], "abdomen_x": [101, 102], "right_upper_arm": [101, 102], "right_lower_arm": [101, 102], "left_upper_arm": [101, 102], "left_lower_arm": [101, 102], "anglular": [101, 102, 104], "29": [101, 102], "aanglular": [101, 102], "33": [101, 102], "34": 101, "35": [101, 102], "37": [101, 102], "39": [101, 102], "41": [101, 102], "43": [101, 102], "velocitti": 101, "cinert": [101, 102], "inertia": [101, 102], "rigid": [101, 102], "intermedi": [101, 102], "nbodi": [101, 102], "henc": [101, 102, 110, 120], "140": [101, 102], "cvel": [101, 102], "qfrc_actuat": [101, 102], "constraint": [101, 102, 103, 109], "nv": [101, 102], "cfrc_ext": [101, 102], "freedom": [101, 102, 103, 119], "dim": [101, 102], "003": 101, "015": 101, "nu": [101, 102], "contact_cost_rang": 101, "lai": 102, "numer": [102, 103, 104, 111, 119, 120], "state_spac": 102, "uph_cost": 102, "upward": 102, "absolut": [102, 104, 108], "overal": 102, "atom": 102, "movement": [102, 117], "though": 102, "inflat": 102, "quad_ctrl_cost": 102, "quad_impact_cost": 102, "000001": 102, "105": 102, "unlik": [102, 103, 104, 105, 106, 120], "beyond": [102, 103, 104, 105, 106, 108], "slider": [103, 104], "unitless": [103, 106], "hinge2": 103, "realist": [103, 109], "accuraci": 103, "approach": [103, 116, 117, 120], "analyt": 103, "alive_bonu": 103, "distance_penalti": 103, "velocity_penalti": 103, "005": [103, 105, 106], "y_coordin": 103, "196": 103, "perpendicularli": 103, "cylind": 105, "effector": [105, 106], "fingertip": [105, 106], "shoulder": 105, "forearm": 105, "wrist": 105, "pan": 105, "r_shoulder_pan_joint": 105, "lift": 105, "r_shoulder_lift_joint": 105, "r_upper_arm_roll_joint": 105, "flex": 105, "r_elbow_flex_joint": 105, "r_forearm_roll_joint": 105, "r_wrist_flex_joint": 105, "r_wrist_roll_joint": 105, "analogi": 105, "tips_arm": 105, "obj_slidex": 105, "obj_slidei": 105, "goal_slidex": 105, "goal_slidei": 105, "sphere": 105, "reward_near": 105, "unattach": [105, 106], "reward_dist": [105, 106, 119], "reward_control": [105, 106], "euclidean": [105, 106], "reward_ctrl": [105, 106, 119], "perman": 105, "323": 105, "rise": 105, "modif": [105, 106, 119], "asset": [105, 106, 109, 113], "gymnasmium": 105, "fixtur": 106, "joint0": 106, "joint1": 106, "target_x": 106, "target_i": 106, "position_fingertip": 106, "position_target": 106, "na": 106, "2d": [106, 119], "disk": 106, "radiu": 106, "everyth": 106, "02": 106, "r\u00e9mi": 107, "coulom": 107, "applic": [107, 109], "segment": 107, "articul": [107, 109], "suspend": 107, "subject": 107, "fluid": 107, "viscou": 107, "coeffici": 107, "motor1_rot": 107, "motor2_rot": 107, "\u03b8": 107, "free_body_rot": 107, "slider1": 107, "slider2": 107, "04": 107, "swim": 107, "split": 108, "thigh_left_joint": 108, "leg_left_joint": 108, "foot_left_joint": 108, "ctr_cost": 108, "procedur": [109, 120], "clone": [109, 117], "huge": 109, "3v3": 109, "hundr": 109, "roughli": 109, "tweakabl": 109, "monei": 109, "slime": 109, "volleybal": 109, "motiv": 109, "premad": 109, "foss": 109, "interoper": 109, "extens": [109, 120], "light": 109, "poli": 109, "pinocchio": 109, "meshcat": 109, "web": 109, "bitcraz": 109, "crazyfli": 109, "nanoquadrotor": 109, "endeavor": 109, "gap": 109, "deep": [109, 118, 120], "methodologi": 109, "coverag": 109, "laboratori": 109, "internet": 109, "quadrotor": 109, "casadi": 109, "symbol": 109, "priori": 109, "isaac": 109, "omnivers": 109, "cours": 109, "wide": 109, "account": 109, "yield": 109, "voltag": 109, "duti": 109, "cycl": 109, "emphasi": 109, "percept": 109, "social": 109, "driver": 109, "whose": [109, 111], "uncertain": 109, "merg": 109, "intersect": 109, "roundabout": 109, "latent": 109, "transfer": 109, "miniatur": 109, "racecar": 109, "llvm": 109, "phase": 109, "gcc": 109, "cuda": 109, "p_0": 109, "morl": 109, "cell": [109, 112, 117], "warehous": 109, "storag": 109, "overfit": 109, "predefin": [109, 117], "hyperparamet": [109, 116, 120], "cheap": 109, "surrog": 109, "evolutionari": 109, "tag": 109, "answer": 109, "classif": 109, "dataset": 109, "autom": 109, "prover": 109, "vampir": 109, "theoret": 109, "workshop": 109, "minatari": 109, "bsuit": 109, "great": [109, 116, 119], "focu": 109, "simplic": 109, "comprehens": 109, "crypto": 109, "omg": 109, "toolbox": 109, "minimalist": 109, "synthes": 109, "neuro": 109, "architectur": 109, "integr": 109, "twin": 109, "concept": 109, "seamless": 109, "firmwar": 109, "neuroflight": 109, "frozen": 110, "lake": 110, "suitabl": 110, "card": [111, 116], "dealer": [111, 116], "deck": [111, 116], "jack": 111, "queen": 111, "king": 111, "ac": [111, 116], "usabl": [111, 116], "held": 111, "decid": 111, "bust": [111, 116], "reveal": 111, "facedown": 111, "goe": 111, "outcom": 111, "hand": [111, 120], "sab": [111, 116], "2020": [111, 112], "incompleteidea": [111, 112], "rlbook2020": [111, 112], "48": 112, "4x12": 112, "world": [112, 113, 114], "132": 112, "inspir": 112, "com": [112, 113, 117, 120], "dennybritz": 112, "blob": 112, "cliff_walk": 112, "latter": 112, "current_row": [112, 113], "nrow": [112, 113], "current_col": [112, 113], "col": [112, 113], "47": 112, "proabil": [112, 114], "slipperi": 113, "extent": 113, "4x4": 113, "perpendicular": 113, "sometim": [113, 117, 119], "is_slipperi": 113, "stool": 113, "franuka": 113, "itch": 113, "io": 113, "rpg": 113, "snow": 113, "tileset": 113, "mel": 113, "tilleri": 113, "cyaneu": 113, "ncol": [113, 116], "time_limit": [113, 114], "8x8": 113, "desc": 113, "map_nam": 113, "preload": 113, "sfff": 113, "fhfh": 113, "fffh": 113, "hffg": 113, "generate_random_map": 113, "toy_text": [113, 116], "frozen_lak": 113, "sfffffff": 113, "ffffffff": 113, "fffhffff": 113, "fffffhff": 113, "fhhfffhf": 113, "fhffhfhf": 113, "fffhfffg": 113, "bug": 113, "passeng": 114, "yellow": 114, "5x5": 114, "maxq": 114, "decomposit": 114, "tom": 114, "dietterich": 114, "pickup": 114, "south": 114, "north": 114, "east": 114, "west": 114, "letter": 114, "taxi_row": 114, "taxi_col": 114, "passenger_loc": 114, "404": 114, "reachabl": 114, "illeg": 114, "action_mask": 114, "fickl": 114, "Or": 114, "argmax": [114, 116], "q_valu": [114, 116], "pp": 114, "227": 114, "303": 114, "nov": 114, "doi": 114, "1613": 114, "639": 114, "cleaner": 114, "disallow": 114, "loc": 114, "passidx": 114, "maintain": 115, "capabl": 115, "insert": [115, 120], "casino": 116, "infam": 116, "beatabl": 116, "won": 116, "viabl": 116, "strategi": 116, "favorit": 116, "zemann": 116, "__future__": [116, 120], "annot": [116, 120], "defaultdict": 116, "pyplot": [116, 120], "plt": [116, 120], "seaborn": [116, 120], "sn": [116, 120], "patch": 116, "tqdm": 116, "explanatori": 116, "tell": [116, 118], "idea": [116, 118], "slow": 116, "showcas": 116, "respond": 116, "harm": 116, "greedi": 116, "blackjackag": 116, "learning_r": [116, 120], "initial_epsilon": 116, "epsilon_decai": 116, "final_epsilon": 116, "discount_factor": 116, "95": 116, "arg": [116, 120], "decai": 116, "lr": [116, 120], "training_error": 116, "get_act": 116, "act": [116, 119], "greedili": 116, "exploit": 116, "next_ob": [116, 120], "future_q_valu": 116, "temporal_differ": 116, "decay_epsilon": 116, "readi": 116, "n_episod": 116, "100_000": 116, "start_epsilon": 116, "decent": [116, 120], "converg": 116, "10x": 116, "rolling_length": 116, "fig": 116, "subplot": 116, "figsiz": [116, 120], "set_titl": 116, "reward_moving_averag": 116, "convolv": 116, "length_moving_averag": 116, "training_error_moving_averag": 116, "tight_layout": 116, "create_grid": 116, "usable_ac": 116, "state_valu": 116, "action_valu": 116, "player_count": 116, "dealer_count": 116, "meshgrid": 116, "arang": 116, "apply_along_axi": 116, "arr": 116, "dstack": 116, "value_grid": 116, "policy_grid": 116, "create_plot": 116, "figaspect": 116, "suptitl": 116, "fontsiz": 116, "ax1": 116, "add_subplot": 116, "plot_surfac": 116, "rstride": 116, "cstride": 116, "cmap": 116, "viridi": 116, "edgecolor": 116, "xtick": 116, "ytick": 116, "set_xlabel": 116, "set_ylabel": 116, "zaxi": 116, "set_rotate_label": 116, "set_zlabel": 116, "view_init": 116, "220": 116, "ax2": 116, "heatmap": 116, "linewidth": 116, "accent_r": 116, "cbar": 116, "set_xticklabel": 116, "set_yticklabel": 116, "legend": 116, "legend_el": 116, "facecolor": 116, "lightgreen": 116, "label": 116, "grei": 116, "bbox_to_anchor": 116, "fig1": 116, "fig2": 116, "hopefulli": 116, "grip": 116, "challeng": 116, "mont": [116, 120], "carlo": [116, 120], "fun": [116, 120], "blackjack_tutori": 116, "ipynb": [116, 117, 118, 119, 120], "overview": [117, 120], "virtual": 117, "git": 117, "venv": 117, "bin": 117, "concern": [117, 120], "readm": 117, "setup": 117, "gym_exampl": 117, "grid_world": 117, "relative_posit": 117, "reacher_weighted_reward": 117, "discrete_act": 117, "clip_reward": 117, "illustr": [117, 118], "simplist": 117, "gridworldenv": 117, "piec": 117, "abstract": 117, "window_s": 117, "512": 117, "_action_to_direct": 117, "assert": 117, "clock": 117, "_get_ob": 117, "mandatori": 117, "_agent_loc": 117, "_target_loc": 117, "manhattan": 117, "_get_info": 117, "oftentim": 117, "worri": 117, "coincid": 117, "earlier": 117, "array_equ": 117, "_render_fram": 117, "trivial": 117, "skeleton": 117, "set_mod": 117, "canva": 117, "pix_square_s": 117, "rect": 117, "gridlin": 117, "width": 117, "visibl": 117, "blit": 117, "get_rect": 117, "pump": 117, "stabl": 117, "tick": 117, "surfarrai": 117, "pixels3d": 117, "bother": 117, "detect": 117, "put": 117, "registr": 117, "appropri": [117, 120], "entrypoint": 117, "mere": 117, "third": 117, "parti": 117, "codebas": 117, "edit": 117, "setuptool": 117, "install_requir": 117, "local": 117, "perfectli": 117, "fine": 117, "print": [117, 119, 120], "relativeposit": [117, 119], "did": 118, "failur": 118, "notabl": 118, "inher": 118, "preserv": 118, "obviou": 118, "forev": 118, "forcibli": 118, "halt": 118, "formal": 118, "q_": 118, "o_t": 118, "a_t": 118, "r_t": 118, "max_a": 118, "o_": 118, "a_": 118, "q_target": 118, "distinct": 118, "frequent": 118, "differenti": 118, "vf_target": 118, "vf_next_stat": 118, "onward": 118, "handling_time_limit": 118, "agent_posit": 119, "target_posit": 119, "discreteact": 119, "disc_to_cont": 119, "__name__": 119, "__main__": 119, "intrins": 119, "clipreward": 119, "nevertheless": 119, "reacherrewardwrapp": 119, "reward_dist_weight": 119, "reward_ctrl_weight": 119, "implementing_custom_wrapp": 119, "scratch": 120, "earliest": 120, "gradient": 120, "burden": 120, "maxim": 120, "inverted_pendulum": 120, "1d": 120, "panda": 120, "pd": 120, "nn": 120, "rcparam": 120, "parameter": 120, "policy_network": 120, "obs_space_dim": 120, "action_space_dim": 120, "hidden_space1": 120, "feel": 120, "hidden_space2": 120, "shared_net": 120, "sequenti": 120, "policy_mean_net": 120, "std": 120, "policy_stddev_net": 120, "action_mean": 120, "action_stddev": 120, "shared_featur": 120, "exp": 120, "mention": 120, "fact": 120, "reinfroc": 120, "acronym": 120, "ward": 120, "ncrement": 120, "actor": 120, "o": 120, "ffset": 120, "einforc": 120, "haracterist": 120, "ligibl": 120, "tune": 120, "ep": 120, "prob": 120, "adamw": 120, "sample_act": 120, "distrib": 120, "log_prob": 120, "running_g": 120, "delta": 120, "zip": 120, "zero_grad": 120, "centric": 120, "reiniti": 120, "fairli": 120, "brittl": 120, "spinningup": 120, "en": 120, "html": 120, "total_num_episod": 120, "5e3": 120, "rewards_over_se": 120, "fibonacci": 120, "manual_se": 120, "reward_over_episod": 120, "avg_reward": 120, "rewards_to_plot": 120, "df1": 120, "datafram": 120, "melt": 120, "renam": 120, "inplac": 120, "darkgrid": 120, "talk": 120, "palett": 120, "rainbow": 120, "lineplot": 120, "siddarth": 120, "chandrasekar": 120, "ronald": 120, "connectionist": 120, "machin": 120, "2004": 120, "229": 120, "256": 120, "reinforce_invpend_gym_v26": 120}, "objects": {"": [[15, 0, 1, "", "action_space"], [15, 0, 1, "", "observation_space"], [15, 0, 1, "", "single_action_space"], [15, 0, 1, "", "single_observation_space"]], "gymnasium": [[17, 1, 1, "", "ActionWrapper"], [2, 1, 1, "", "Env"], [19, 1, 1, "", "ObservationWrapper"], [20, 1, 1, "", "RewardWrapper"], [16, 1, 1, "", "Wrapper"], [8, 3, 1, "", "make"], [8, 3, 1, "", "pprint_registry"], [8, 3, 1, "", "register"], [9, 5, 0, "-", "spaces"], [8, 3, 1, "", "spec"], [16, 5, 0, "-", "wrappers"]], "gymnasium.ActionWrapper": [[17, 2, 1, "", "action"]], "gymnasium.Env": [[2, 0, 1, "", "action_space"], [2, 3, 1, "", "close"], [2, 0, 1, "", "metadata"], [2, 4, 1, "", "np_random"], [2, 0, 1, "", "observation_space"], [2, 3, 1, "", "render"], [2, 0, 1, "", "render_mode"], [2, 3, 1, "", "reset"], [2, 0, 1, "", "reward_range"], [2, 0, 1, "", "spec"], [2, 3, 1, "", "step"], [2, 4, 1, "", "unwrapped"]], "gymnasium.ObservationWrapper": [[19, 2, 1, "", "observation"]], "gymnasium.RewardWrapper": [[20, 2, 1, "", "reward"]], "gymnasium.Wrapper": [[16, 4, 1, "", "action_space"], [16, 3, 1, "", "close"], [16, 4, 1, "", "metadata"], [16, 4, 1, "", "np_random"], [16, 4, 1, "", "observation_space"], [16, 3, 1, "", "reset"], [16, 4, 1, "", "reward_range"], [16, 4, 1, "", "spec"], [16, 3, 1, "", "step"], [16, 4, 1, "", "unwrapped"]], "gymnasium.experimental": [[4, 1, 1, "", "FuncEnv"]], "gymnasium.experimental.FuncEnv": [[4, 3, 1, "", "initial"], [4, 3, 1, "", "observation"], [4, 3, 1, "", "render_close"], [4, 3, 1, "", "render_image"], [4, 3, 1, "", "render_init"], [4, 3, 1, "", "reward"], [4, 3, 1, "", "state_info"], [4, 3, 1, "", "step_info"], [4, 3, 1, "", "terminal"], [4, 3, 1, "", "transform"], [4, 3, 1, "", "transition"]], "gymnasium.experimental.wrappers": [[7, 1, 1, "", "AtariPreprocessingV0"], [7, 1, 1, "", "AutoresetV0"], [7, 1, 1, "", "ClipActionV0"], [7, 1, 1, "", "ClipRewardV0"], [7, 1, 1, "", "DelayObservationV0"], [7, 1, 1, "", "DtypeObservationV0"], [7, 1, 1, "", "FilterObservationV0"], [7, 1, 1, "", "FlattenObservationV0"], [7, 1, 1, "", "FrameStackObservationV0"], [7, 1, 1, "", "GrayscaleObservationV0"], [7, 1, 1, "", "HumanRenderingV0"], [7, 1, 1, "", "JaxToNumpyV0"], [7, 1, 1, "", "JaxToTorchV0"], [7, 1, 1, "", "LambdaActionV0"], [7, 1, 1, "", "LambdaObservationV0"], [7, 1, 1, "", "LambdaRewardV0"], [7, 1, 1, "", "NormalizeObservationV0"], [7, 1, 1, "", "NormalizeRewardV0"], [7, 1, 1, "", "NumpyToTorchV0"], [7, 1, 1, "", "OrderEnforcingV0"], [7, 1, 1, "", "PassiveEnvCheckerV0"], [7, 1, 1, "", "PixelObservationV0"], [7, 1, 1, "", "RecordEpisodeStatisticsV0"], [7, 1, 1, "", "RecordVideoV0"], [7, 1, 1, "", "RenderCollectionV0"], [7, 1, 1, "", "RescaleActionV0"], [7, 1, 1, "", "RescaleObservationV0"], [7, 1, 1, "", "ReshapeObservationV0"], [7, 1, 1, "", "ResizeObservationV0"], [7, 1, 1, "", "StickyActionV0"], [7, 1, 1, "", "TimeAwareObservationV0"]], "gymnasium.spaces": [[11, 1, 1, "", "Box"], [10, 1, 1, "", "Dict"], [11, 1, 1, "", "Discrete"], [10, 1, 1, "", "Graph"], [11, 1, 1, "", "MultiBinary"], [11, 1, 1, "", "MultiDiscrete"], [10, 1, 1, "", "Sequence"], [9, 1, 1, "", "Space"], [11, 1, 1, "", "Text"], [10, 1, 1, "", "Tuple"]], "gymnasium.spaces.Box": [[11, 2, 1, "", "is_bounded"], [11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Dict": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.Discrete": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Graph": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.MultiBinary": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.MultiDiscrete": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Sequence": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.Space": [[9, 4, 1, "", "dtype"]], "gymnasium.spaces.Text": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Tuple": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.space.Space": [[9, 3, 1, "", "contains"], [9, 3, 1, "", "from_jsonable"], [9, 4, 1, "", "is_np_flattenable"], [9, 3, 1, "", "sample"], [9, 3, 1, "", "seed"], [9, 4, 1, "", "shape"], [9, 3, 1, "", "to_jsonable"]], "gymnasium.spaces.utils": [[12, 3, 1, "", "flatdim"], [12, 3, 1, "", "flatten"], [12, 3, 1, "", "flatten_space"], [12, 3, 1, "", "unflatten"]], "gymnasium.utils.env_checker": [[14, 3, 1, "", "check_env"]], "gymnasium.utils.play": [[14, 1, 1, "", "PlayPlot"], [14, 1, 1, "", "PlayableGame"], [14, 3, 1, "", "play"]], "gymnasium.utils.play.PlayPlot": [[14, 2, 1, "", "callback"]], "gymnasium.utils.play.PlayableGame": [[14, 2, 1, "", "process_event"]], "gymnasium.utils.save_video": [[14, 3, 1, "", "capped_cubic_video_schedule"], [14, 3, 1, "", "save_video"]], "gymnasium.utils.seeding": [[14, 3, 1, "", "np_random"]], "gymnasium.utils.step_api_compatibility": [[14, 3, 1, "", "convert_to_done_step_api"], [14, 3, 1, "", "convert_to_terminated_truncated_step_api"], [14, 3, 1, "", "step_api_compatibility"]], "gymnasium.vector": [[15, 1, 1, "", "AsyncVectorEnv"], [15, 1, 1, "", "SyncVectorEnv"], [15, 1, 1, "", "VectorEnv"], [15, 3, 1, "", "make"]], "gymnasium.vector.VectorEnv": [[15, 2, 1, "", "close"], [15, 2, 1, "", "reset"], [15, 2, 1, "", "step"]], "gymnasium.vector.utils": [[13, 3, 1, "", "batch_space"], [13, 3, 1, "", "concatenate"], [13, 3, 1, "", "create_empty_array"], [13, 3, 1, "", "create_shared_memory"], [13, 3, 1, "", "iterate"], [13, 3, 1, "", "read_from_shared_memory"], [13, 3, 1, "", "write_to_shared_memory"]], "gymnasium.wrappers": [[18, 1, 1, "", "AtariPreprocessing"], [18, 1, 1, "", "AutoResetWrapper"], [17, 1, 1, "", "ClipAction"], [18, 1, 1, "", "EnvCompatibility"], [19, 1, 1, "", "FilterObservation"], [19, 1, 1, "", "FlattenObservation"], [19, 1, 1, "", "FrameStack"], [19, 1, 1, "", "GrayScaleObservation"], [18, 1, 1, "", "HumanRendering"], [19, 1, 1, "", "NormalizeObservation"], [20, 1, 1, "", "NormalizeReward"], [18, 1, 1, "", "OrderEnforcing"], [18, 1, 1, "", "PassiveEnvChecker"], [19, 1, 1, "", "PixelObservationWrapper"], [18, 1, 1, "", "RecordEpisodeStatistics"], [18, 1, 1, "", "RecordVideo"], [18, 1, 1, "", "RenderCollection"], [17, 1, 1, "", "RescaleAction"], [19, 1, 1, "", "ResizeObservation"], [18, 1, 1, "", "StepAPICompatibility"], [19, 1, 1, "", "TimeAwareObservation"], [18, 1, 1, "", "TimeLimit"], [19, 1, 1, "", "TransformObservation"], [20, 1, 1, "", "TransformReward"], [18, 1, 1, "", "VectorListInfo"]], "gymnasium.wrappers.gymnasium.Wrapper": [[16, 0, 1, "", "env"]]}, "objtypes": {"0": "py:attribute", "1": "py:class", "2": "py:method", "3": "py:function", "4": "py:property", "5": "py:module"}, "objnames": {"0": ["py", "attribute", "Python attribute"], "1": ["py", "class", "Python class"], "2": ["py", "method", "Python method"], "3": ["py", "function", "Python function"], "4": ["py", "property", "Python property"], "5": ["py", "module", "Python module"]}, "titleterms": {"404": 0, "page": [0, 1], "Not": 0, "found": 0, "The": [0, 9], "request": 0, "could": 0, "gymnasium": [1, 2, 4, 5, 15, 16, 115, 117, 119], "doc": 1, "instruct": 1, "modifi": [1, 21], "environ": [1, 2, 3, 4, 5, 6, 7, 8, 14, 15, 21, 22, 23, 109, 115, 116, 117], "edit": 1, "an": [1, 116, 120], "ad": 1, "new": [1, 14], "atari": [1, 24, 43], "env": [1, 2, 15, 109, 117], "non": 1, "other": [1, 7, 109], "step": [1, 14, 22, 23, 117], "build": [1, 116, 120], "document": 1, "write": 1, "tutori": 1, "method": [2, 9, 15, 16], "attribut": [2, 9, 15, 16], "addit": 2, "implement": [2, 119], "experiment": [3, 4, 5], "function": [3, 4], "wrapper": [3, 6, 7, 16, 17, 18, 19, 20, 117, 119], "observ": [3, 6, 7, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 117], "action": [3, 6, 7, 17, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116], "reward": [3, 6, 7, 20, 24, 27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "common": [3, 6, 24], "render": [3, 7, 14, 23, 117], "data": [3, 7], "convers": [3, 7], "vector": [3, 5, 6, 9, 13, 15], "funcenv": 4, "func2env": 4, "functionaljaxcompatibilityenv": 4, "vectorenv": [5, 15], "asyncvectorenv": 5, "syncvectorenv": 5, "custom": [5, 117, 119], "envpool": 5, "lambda": 6, "onli": 6, "registri": 8, "make": [8, 15, 117], "regist": [8, 117], "all": 8, "spec": 8, "pretti": 8, "print": 8, "space": [9, 10, 11, 12, 13, 21, 24, 50, 51, 52, 53, 54, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "base": [9, 17, 19, 20], "class": [9, 17, 19, 20], "fundament": [9, 11], "composit": [9, 10], "util": [9, 12, 13, 14], "dict": 10, "tupl": 10, "sequenc": 10, "graph": 10, "box": [11, 38], "discret": [11, 109], "multibinari": 11, "multidiscret": 11, "text": [11, 110], "share": 13, "memori": 13, "visual": [14, 116], "save": 14, "video": [14, 84, 109], "old": 14, "api": [14, 22, 115], "compat": [14, 22], "seed": [14, 23], "check": 14, "async": 15, "sync": 15, "avail": [17, 19, 20], "misc": 18, "basic": 21, "usag": 21, "initi": [21, 117], "interact": 21, "explain": 21, "code": [21, 23, 118], "more": 21, "inform": [21, 111, 112, 113, 114], "gym": [22, 109], "load": 22, "openai": [22, 109], "v0": 22, "22": 22, "v21": 23, "v26": 23, "migrat": 23, "guid": 23, "exampl": 23, "random": 23, "number": 23, "gener": 23, "reset": [23, 89, 117], "remov": 23, "autorom": 24, "instal": 24, "rom": 24, "stochast": 24, "argument": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "version": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114], "histori": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "name": [24, 64], "scheme": 24, "flavor": 24, "refer": [24, 89, 92, 111, 112, 114, 115, 120], "adventur": 25, "descript": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "air": 26, "raid": 26, "alien": 27, "amidar": 28, "assault": 29, "asterix": 30, "asteroid": 31, "atlanti": 32, "bank": 33, "heist": 33, "battl": 34, "zone": 34, "beam": 35, "rider": 35, "berzerk": 36, "bowl": 37, "breakout": 39, "carniv": 40, "centiped": 41, "chopper": 42, "command": 42, "complet": 43, "list": 43, "crazi": 44, "climber": 44, "defend": 45, "demon": 46, "attack": 46, "doubl": [47, 103], "dunk": 47, "elev": 48, "enduro": 49, "fishingderbi": 50, "freewai": 51, "frostbit": 52, "gopher": 53, "gravitar": 54, "hero": 55, "icehockei": 56, "jamesbond": 57, "journeyescap": 58, "kangaroo": 59, "krull": 60, "kung": 61, "fu": 61, "master": 61, "montezuma": 62, "reveng": 62, "m": 63, "pacman": 63, "thi": 64, "game": [64, 109], "phoenix": 65, "pitfal": 66, "pong": 67, "pooyan": 68, "privateey": 69, "qbert": 70, "riverraid": 71, "road": 72, "runner": 72, "robot": [73, 109], "tank": 73, "seaquest": 74, "ski": 75, "solari": 76, "spaceinvad": 77, "stargunn": 78, "tenni": 79, "timepilot": 80, "tutankham": 81, "up": 82, "n": 82, "down": 82, "ventur": 83, "pinbal": 84, "wizard": 85, "wor": 85, "zaxxon": 86, "box2d": 87, "biped": 88, "walker": 88, "start": [88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "state": [88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 117], "episod": [88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "termin": [88, 89, 90, 118], "credit": [88, 89, 90], "car": [89, 94, 95], "race": [89, 109], "lunar": 90, "lander": 90, "classic": [91, 109], "control": [91, 109], "acrobot": 92, "end": [92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "cart": 93, "pole": 93, "mountain": [94, 95], "transit": [94, 95], "dynam": [94, 95, 109], "continu": 95, "pendulum": [96, 103, 104], "truncat": [96, 118], "mujoco": [97, 120], "ant": 98, "half": 99, "cheetah": 99, "hopper": 100, "humanoid": [101, 102], "standup": 102, "invert": [103, 104], "pusher": 105, "reacher": 106, "swimmer": 107, "note": 107, "walker2d": 108, "third": 109, "parti": 109, "stabl": 109, "retro": 109, "maintain": 109, "flappi": 109, "bird": 109, "A": 109, "derk": 109, "gpu": 109, "acceler": 109, "moba": 109, "slimevolleygym": 109, "simpl": 109, "singl": 109, "multi": 109, "agent": [109, 116, 120], "reinforc": [109, 115, 120], "learn": [109, 115, 116, 118, 120], "uniti": 109, "ml": 109, "engin": 109, "pge": 109, "parallel": 109, "jimini": 109, "train": [109, 116, 120], "pybullet": 109, "drone": 109, "quadcopt": 109, "marsexplor": 109, "mar": 109, "panda": 109, "us": [109, 117, 120], "physic": 109, "robo": 109, "real": 109, "world": 109, "simul": 109, "offworld": 109, "safe": 109, "softrobot": 109, "soft": 109, "igibson": 109, "dexteroushand": 109, "dual": 109, "dexter": 109, "hand": 109, "manipul": 109, "task": 109, "omniisaacgymenv": 109, "autonom": 109, "drive": 109, "sumo": 109, "rl": 109, "duckietown": 109, "electr": 109, "motor": 109, "highwai": 109, "commonroad": 109, "tmrl": 109, "trackmania": 109, "2020": 109, "through": 109, "racing_dream": 109, "l2r": 109, "multimod": 109, "where": 109, "how": 109, "racecar_gym": 109, "compilergym": 109, "optimis": 109, "compil": 109, "carl": 109, "context": 109, "adapt": 109, "matrix": 109, "mdp": 109, "easili": 109, "creat": [109, 117], "mo": 109, "object": 109, "cellular": 109, "automata": 109, "sokoban": 109, "2d": 109, "transport": 109, "puzzl": 109, "dacbench": 109, "benchmark": 109, "librari": 109, "algorithm": 109, "configur": 109, "nlpgym": 109, "toolkit": 109, "develop": 109, "solv": [109, 116], "nlp": 109, "satur": 109, "prove": 109, "theorem": 109, "shinrl": 109, "evalu": 109, "gymnax": 109, "hardwar": 109, "anytrad": 109, "financi": 109, "trade": 109, "forex": 109, "stock": 109, "mtsim": 109, "metatrad": 109, "5": 109, "platform": 109, "openmodelica": 109, "microgrid": 109, "power": 109, "electron": 109, "convert": 109, "mobil": 109, "coordin": 109, "wireless": 109, "network": [109, 120], "gymfc": 109, "flight": 109, "tune": 109, "framework": 109, "toi": 110, "blackjack": [111, 116], "cliff": 112, "walk": 112, "frozen": 113, "lake": 113, "taxi": 114, "i": 115, "standard": 115, "divers": 115, "collect": 115, "q": 116, "import": [116, 118], "setup": 116, "execut": 116, "visualis": 116, "polici": [116, 120], "think": 116, "you": 116, "can": 116, "do": 116, "better": 116, "your": 117, "own": 117, "subclass": 117, "declar": 117, "construct": 117, "from": [117, 119], "close": 117, "packag": 117, "instanc": 117, "handl": 118, "time": 118, "limit": 118, "solut": 118, "inherit": 119, "observationwrapp": 119, "actionwrapp": 119, "rewardwrapp": 119, "plot": 120, "curv": 120}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx": 57}, "alltitles": {"404 - Page Not Found": [[0, "page-not-found"]], "The requested page could not be found.": [[0, "the-requested-page-could-not-be-found"]], "Gymnasium-docs": [[1, "gymnasium-docs"]], "Instructions for modifying environment pages": [[1, "instructions-for-modifying-environment-pages"]], "Editing an environment page": [[1, "editing-an-environment-page"]], "Adding a new environment": [[1, "adding-a-new-environment"]], "Atari env": [[1, "atari-env"]], "Non-Atari env": [[1, "non-atari-env"]], "Other steps": [[1, "other-steps"]], "Build the Documentation": [[1, "build-the-documentation"]], "Writing Tutorials": [[1, "writing-tutorials"]], "Env": [[2, "env"]], "gymnasium.Env": [[2, "gymnasium-env"]], "Methods": [[2, "methods"], [9, "methods"], [15, "methods"], [16, "methods"]], "Attributes": [[2, "attributes"], [9, "attributes"], [15, "attributes"], [16, "attributes"]], "Additional Methods": [[2, "additional-methods"]], "Implementing environments": [[2, "implementing-environments"]], "Experimental": [[3, "experimental"]], "Functional Environments": [[3, "functional-environments"]], "Wrappers": [[3, "wrappers"], [7, "wrappers"], [16, "wrappers"]], "Observation Wrappers": [[3, "observation-wrappers"], [7, "observation-wrappers"], [19, "observation-wrappers"]], "Action Wrappers": [[3, "action-wrappers"], [7, "action-wrappers"], [17, "action-wrappers"]], "Reward Wrappers": [[3, "reward-wrappers"], [7, "reward-wrappers"], [20, "reward-wrappers"]], "Common Wrappers": [[3, "common-wrappers"]], "Rendering Wrappers": [[3, "rendering-wrappers"], [7, "rendering-wrappers"]], "Environment data conversion": [[3, "environment-data-conversion"], [7, "environment-data-conversion"]], "Vector Environment": [[3, "vector-environment"]], "Wrappers for Vector Environments": [[3, "wrappers-for-vector-environments"]], "Functional Environment": [[4, "functional-environment"]], "gymnasium.experimental.FuncEnv": [[4, "gymnasium-experimental-funcenv"]], "gymnasium.experimental.func2env.FunctionalJaxCompatibilityEnv": [[4, "gymnasium-experimental-func2env-functionaljaxcompatibilityenv"]], "Vectorizing Environment": [[5, "vectorizing-environment"]], "gymnasium.experimental.VectorEnv": [[5, "gymnasium-experimental-vectorenv"]], "gymnasium.experimental.vector.AsyncVectorEnv": [[5, "gymnasium-experimental-vector-asyncvectorenv"]], "gymnasium.experimental.vector.SyncVectorEnv": [[5, "gymnasium-experimental-vector-syncvectorenv"]], "Custom Vector environments": [[5, "custom-vector-environments"]], "EnvPool": [[5, "envpool"]], "Vector Environment Wrappers": [[6, "vector-environment-wrappers"]], "Vector Lambda Observation Wrappers": [[6, "vector-lambda-observation-wrappers"]], "Vector Lambda Action Wrappers": [[6, "vector-lambda-action-wrappers"]], "Vector Lambda Reward Wrappers": [[6, "vector-lambda-reward-wrappers"]], "Vector Common Wrappers": [[6, "vector-common-wrappers"]], "Vector Only Wrappers": [[6, "vector-only-wrappers"]], "Other Wrappers": [[7, "other-wrappers"]], "Registry": [[8, "registry"]], "Make": [[8, "make"]], "Register": [[8, "register"]], "All registered environments": [[8, "all-registered-environments"]], "Spec": [[8, "spec"]], "Pretty print registry": [[8, "pretty-print-registry"]], "Spaces": [[9, "spaces"]], "The Base Class": [[9, "the-base-class"]], "Fundamental Spaces": [[9, "fundamental-spaces"], [11, "fundamental-spaces"]], "Composite Spaces": [[9, "composite-spaces"], [10, "composite-spaces"]], "Utils": [[9, "utils"], [14, "utils"]], "Vector Utils": [[9, "vector-utils"]], "Dict": [[10, "dict"]], "Tuple": [[10, "tuple"]], "Sequence": [[10, "sequence"]], "Graph": [[10, "graph"]], "Box": [[11, "box"]], "Discrete": [[11, "discrete"]], "MultiBinary": [[11, "multibinary"]], "MultiDiscrete": [[11, "multidiscrete"]], "Text": [[11, "text"]], "Spaces Utils": [[12, "spaces-utils"]], "Spaces Vector Utils": [[13, "spaces-vector-utils"]], "Shared Memory Utils": [[13, "shared-memory-utils"]], "Visualization": [[14, "visualization"]], "Save Rendering Videos": [[14, "save-rendering-videos"]], "Old to New Step API Compatibility": [[14, "old-to-new-step-api-compatibility"]], "Seeding": [[14, "seeding"]], "Environment Checking": [[14, "environment-checking"]], "Vector": [[15, "vector"]], "Gymnasium.vector.VectorEnv": [[15, "gymnasium-vector-vectorenv"]], "Making Vector Environments": [[15, "making-vector-environments"]], "Async Vector Env": [[15, "async-vector-env"]], "Sync Vector Env": [[15, "sync-vector-env"]], "gymnasium.Wrapper": [[16, "gymnasium-wrapper"]], "Gymnasium Wrappers": [[16, "gymnasium-wrappers"]], "Base Class": [[17, "base-class"], [19, "base-class"], [20, "base-class"]], "Available Action Wrappers": [[17, "available-action-wrappers"]], "Misc Wrappers": [[18, "misc-wrappers"]], "Available Observation Wrappers": [[19, "available-observation-wrappers"]], "Available Reward Wrappers": [[20, "available-reward-wrappers"]], "Basic Usage": [[21, "basic-usage"]], "Initializing Environments": [[21, "initializing-environments"]], "Interacting with the Environment": [[21, "interacting-with-the-environment"]], "Explaining the code": [[21, "explaining-the-code"]], "Action and observation spaces": [[21, "action-and-observation-spaces"]], "Modifying the environment": [[21, "modifying-the-environment"]], "More information": [[21, "more-information"]], "Compatibility with Gym": [[22, "compatibility-with-gym"]], "Loading OpenAI Gym environments": [[22, "loading-openai-gym-environments"]], "Gym v0.22 Environment Compatibility": [[22, "gym-v0-22-environment-compatibility"]], "Step API Compatibility": [[22, "step-api-compatibility"]], "v21 to v26 Migration Guide": [[23, "v21-to-v26-migration-guide"]], "Example code for v21": [[23, "example-code-for-v21"]], "Example code for v26": [[23, "example-code-for-v26"]], "Seed and random number generator": [[23, "seed-and-random-number-generator"]], "Environment Reset": [[23, "environment-reset"]], "Environment Step": [[23, "environment-step"]], "Environment Render": [[23, "environment-render"]], "Removed code": [[23, "removed-code"]], "Atari": [[24, "atari"]], "AutoROM (installing the ROMs)": [[24, "autorom-installing-the-roms"]], "Action Space": [[24, "action-space"], [50, "action-space"], [51, "action-space"], [52, "action-space"], [53, "action-space"], [54, "action-space"], [88, "action-space"], [89, "action-space"], [90, "action-space"], [92, "action-space"], [93, "action-space"], [94, "action-space"], [95, "action-space"], [96, "action-space"], [98, "action-space"], [99, "action-space"], [100, "action-space"], [101, "action-space"], [102, "action-space"], [103, "action-space"], [104, "action-space"], [105, "action-space"], [106, "action-space"], [107, "action-space"], [108, "action-space"], [111, "action-space"], [112, "action-space"], [113, "action-space"], [114, "action-space"]], "Observation Space": [[24, "observation-space"], [88, "observation-space"], [89, "observation-space"], [90, "observation-space"], [92, "observation-space"], [93, "observation-space"], [94, "observation-space"], [95, "observation-space"], [96, "observation-space"], [98, "observation-space"], [99, "observation-space"], [100, "observation-space"], [101, "observation-space"], [102, "observation-space"], [103, "observation-space"], [104, "observation-space"], [105, "observation-space"], [106, "observation-space"], [107, "observation-space"], [108, "observation-space"], [111, "observation-space"], [112, "observation-space"], [113, "observation-space"], [114, "observation-space"]], "Rewards": [[24, "rewards"], [27, "rewards"], [28, "rewards"], [30, "rewards"], [31, "rewards"], [32, "rewards"], [33, "rewards"], [34, "rewards"], [35, "rewards"], [36, "rewards"], [37, "rewards"], [38, "rewards"], [39, "rewards"], [40, "rewards"], [41, "rewards"], [42, "rewards"], [44, "rewards"], [45, "rewards"], [46, "rewards"], [47, "rewards"], [48, "rewards"], [49, "rewards"], [50, "rewards"], [51, "rewards"], [52, "rewards"], [53, "rewards"], [54, "rewards"], [55, "rewards"], [56, "rewards"], [57, "rewards"], [58, "rewards"], [59, "rewards"], [60, "rewards"], [66, "rewards"], [67, "rewards"], [68, "rewards"], [69, "rewards"], [70, "rewards"], [71, "rewards"], [72, "rewards"], [73, "rewards"], [74, "rewards"], [75, "rewards"], [76, "rewards"], [77, "rewards"], [78, "rewards"], [79, "rewards"], [80, "rewards"], [88, "rewards"], [89, "rewards"], [90, "rewards"], [92, "rewards"], [93, "rewards"], [96, "rewards"], [98, "rewards"], [99, "rewards"], [100, "rewards"], [101, "rewards"], [102, "rewards"], [103, "rewards"], [104, "rewards"], [105, "rewards"], [106, "rewards"], [107, "rewards"], [108, "rewards"], [111, "rewards"], [113, "rewards"], [114, "rewards"]], "Stochasticity": [[24, "stochasticity"]], "Common Arguments": [[24, "common-arguments"]], "Version History and Naming Schemes": [[24, "version-history-and-naming-schemes"]], "Flavors": [[24, "flavors"]], "References": [[24, "references"], [89, "references"], [92, "references"], [111, "references"], [112, "references"], [114, "references"], [120, "references"]], "Adventure": [[25, "adventure"]], "Description": [[25, "description"], [26, "description"], [27, "description"], [28, "description"], [29, "description"], [30, "description"], [31, "description"], [32, "description"], [33, "description"], [34, "description"], [35, "description"], [36, "description"], [37, "description"], [38, "description"], [39, "description"], [40, "description"], [41, "description"], [42, "description"], [44, "description"], [45, "description"], [46, "description"], [47, "description"], [48, "description"], [49, "description"], [50, "description"], [51, "description"], [52, "description"], [53, "description"], [54, "description"], [55, "description"], [56, "description"], [57, "description"], [58, "description"], [59, "description"], [60, "description"], [61, "description"], [62, "description"], [63, "description"], [64, "description"], [65, "description"], [66, "description"], [67, "description"], [68, "description"], [69, "description"], [70, "description"], [71, "description"], [72, "description"], [73, "description"], [74, "description"], [75, "description"], [76, "description"], [77, "description"], [78, "description"], [79, "description"], [80, "description"], [81, "description"], [82, "description"], [83, "description"], [84, "description"], [85, "description"], [86, "description"], [88, "description"], [89, "description"], [90, "description"], [92, "description"], [93, "description"], [94, "description"], [95, "description"], [96, "description"], [98, "description"], [99, "description"], [100, "description"], [101, "description"], [102, "description"], [103, "description"], [104, "description"], [105, "description"], [106, "description"], [107, "description"], [108, "description"], [111, "description"], [112, "description"], [113, "description"], [114, "description"]], "Actions": [[25, "actions"], [26, "actions"], [27, "actions"], [28, "actions"], [29, "actions"], [30, "actions"], [31, "actions"], [32, "actions"], [33, "actions"], [34, "actions"], [35, "actions"], [36, "actions"], [37, "actions"], [38, "actions"], [39, "actions"], [40, "actions"], [41, "actions"], [42, "actions"], [44, "actions"], [45, "actions"], [46, "actions"], [47, "actions"], [48, "actions"], [49, "actions"], [55, "actions"], [56, "actions"], [57, "actions"], [58, "actions"], [59, "actions"], [60, "actions"], [61, "actions"], [62, "actions"], [63, "actions"], [64, "actions"], [65, "actions"], [66, "actions"], [67, "actions"], [68, "actions"], [69, "actions"], [70, "actions"], [71, "actions"], [72, "actions"], [73, "actions"], [74, "actions"], [75, "actions"], [76, "actions"], [77, "actions"], [78, "actions"], [79, "actions"], [80, "actions"], [81, "actions"], [82, "actions"], [83, "actions"], [84, "actions"], [85, "actions"], [86, "actions"]], "Observations": [[25, "observations"], [26, "observations"], [27, "observations"], [28, "observations"], [29, "observations"], [30, "observations"], [31, "observations"], [32, "observations"], [33, "observations"], [34, "observations"], [35, "observations"], [36, "observations"], [37, "observations"], [38, "observations"], [39, "observations"], [40, "observations"], [41, "observations"], [42, "observations"], [44, "observations"], [45, "observations"], [46, "observations"], [47, "observations"], [48, "observations"], [49, "observations"], [50, "observations"], [51, "observations"], [52, "observations"], [53, "observations"], [54, "observations"], [55, "observations"], [56, "observations"], [57, "observations"], [58, "observations"], [59, "observations"], [60, "observations"], [61, "observations"], [62, "observations"], [63, "observations"], [64, "observations"], [65, "observations"], [66, "observations"], [67, "observations"], [68, "observations"], [69, "observations"], [70, "observations"], [71, "observations"], [72, "observations"], [73, "observations"], [74, "observations"], [75, "observations"], [76, "observations"], [77, "observations"], [78, "observations"], [79, "observations"], [80, "observations"], [81, "observations"], [82, "observations"], [83, "observations"], [84, "observations"], [85, "observations"], [86, "observations"]], "Arguments": [[25, "arguments"], [26, "arguments"], [27, "arguments"], [28, "arguments"], [29, "arguments"], [30, "arguments"], [31, "arguments"], [32, "arguments"], [33, "arguments"], [34, "arguments"], [35, "arguments"], [36, "arguments"], [37, "arguments"], [38, "arguments"], [39, "arguments"], [40, "arguments"], [41, "arguments"], [42, "arguments"], [44, "arguments"], [45, "arguments"], [46, "arguments"], [47, "arguments"], [48, "arguments"], [49, "arguments"], [50, "arguments"], [51, "arguments"], [52, "arguments"], [53, "arguments"], [54, "arguments"], [55, "arguments"], [56, "arguments"], [57, "arguments"], [58, "arguments"], [59, "arguments"], [60, "arguments"], [61, "arguments"], [62, "arguments"], [63, "arguments"], [64, "arguments"], [65, "arguments"], [66, "arguments"], [67, "arguments"], [68, "arguments"], [69, "arguments"], [70, "arguments"], [71, "arguments"], [72, "arguments"], [73, "arguments"], [74, "arguments"], [75, "arguments"], [76, "arguments"], [77, "arguments"], [78, "arguments"], [79, "arguments"], [80, "arguments"], [81, "arguments"], [82, "arguments"], [83, "arguments"], [84, "arguments"], [85, "arguments"], [86, "arguments"], [88, "arguments"], [89, "arguments"], [90, "arguments"], [92, "arguments"], [93, "arguments"], [94, "arguments"], [95, "arguments"], [96, "arguments"], [98, "arguments"], [99, "arguments"], [100, "arguments"], [101, "arguments"], [102, "arguments"], [103, "arguments"], [104, "arguments"], [105, "arguments"], [106, "arguments"], [107, "arguments"], [108, "arguments"], [111, "arguments"], [112, "arguments"], [113, "arguments"], [114, "arguments"]], "Version History": [[25, "version-history"], [26, "version-history"], [27, "version-history"], [28, "version-history"], [29, "version-history"], [30, "version-history"], [31, "version-history"], [32, "version-history"], [33, "version-history"], [34, "version-history"], [35, "version-history"], [36, "version-history"], [37, "version-history"], [38, "version-history"], [39, "version-history"], [40, "version-history"], [41, "version-history"], [42, "version-history"], [44, "version-history"], [45, "version-history"], [46, "version-history"], [47, "version-history"], [48, "version-history"], [49, "version-history"], [50, "version-history"], [51, "version-history"], [52, "version-history"], [53, "version-history"], [54, "version-history"], [55, "version-history"], [56, "version-history"], [57, "version-history"], [58, "version-history"], [59, "version-history"], [60, "version-history"], [61, "version-history"], [62, "version-history"], [63, "version-history"], [64, "version-history"], [65, "version-history"], [66, "version-history"], [67, "version-history"], [68, "version-history"], [69, "version-history"], [70, "version-history"], [71, "version-history"], [72, "version-history"], [73, "version-history"], [74, "version-history"], [75, "version-history"], [76, "version-history"], [77, "version-history"], [78, "version-history"], [79, "version-history"], [80, "version-history"], [81, "version-history"], [82, "version-history"], [83, "version-history"], [84, "version-history"], [85, "version-history"], [86, "version-history"], [88, "version-history"], [89, "version-history"], [90, "version-history"], [92, "version-history"], [94, "version-history"], [95, "version-history"], [96, "version-history"], [98, "version-history"], [99, "version-history"], [100, "version-history"], [101, "version-history"], [102, "version-history"], [103, "version-history"], [104, "version-history"], [105, "version-history"], [106, "version-history"], [107, "version-history"], [108, "version-history"], [111, "version-history"], [112, "version-history"], [113, "version-history"], [114, "version-history"]], "Air Raid": [[26, "air-raid"]], "Alien": [[27, "alien"]], "Amidar": [[28, "amidar"]], "Assault": [[29, "assault"]], "Asterix": [[30, "asterix"]], "Asteroids": [[31, "asteroids"]], "Atlantis": [[32, "atlantis"]], "Bank Heist": [[33, "bank-heist"]], "Battle Zone": [[34, "battle-zone"]], "Beam Rider": [[35, "beam-rider"]], "Berzerk": [[36, "berzerk"]], "Bowling": [[37, "bowling"]], "Boxing": [[38, "boxing"]], "Breakout": [[39, "breakout"]], "Carnival": [[40, "carnival"]], "Centipede": [[41, "centipede"]], "Chopper Command": [[42, "chopper-command"]], "Complete List - Atari": [[43, "complete-list-atari"]], "Crazy Climber": [[44, "crazy-climber"]], "Defender": [[45, "defender"]], "Demon Attack": [[46, "demon-attack"]], "Double Dunk": [[47, "double-dunk"]], "Elevator Action": [[48, "elevator-action"]], "Enduro": [[49, "enduro"]], "FishingDerby": [[50, "fishingderby"]], "Freeway": [[51, "freeway"]], "Frostbite": [[52, "frostbite"]], "Gopher": [[53, "gopher"]], "Gravitar": [[54, "gravitar"]], "Hero": [[55, "hero"]], "IceHockey": [[56, "icehockey"]], "Jamesbond": [[57, "jamesbond"]], "JourneyEscape": [[58, "journeyescape"]], "Kangaroo": [[59, "kangaroo"]], "Krull": [[60, "krull"]], "Kung Fu Master": [[61, "kung-fu-master"]], "Montezuma Revenge": [[62, "montezuma-revenge"]], "Ms Pacman": [[63, "ms-pacman"]], "Name This Game": [[64, "name-this-game"]], "Phoenix": [[65, "phoenix"]], "Pitfall": [[66, "pitfall"]], "Pong": [[67, "pong"]], "Pooyan": [[68, "pooyan"]], "PrivateEye": [[69, "privateeye"]], "Qbert": [[70, "qbert"]], "Riverraid": [[71, "riverraid"]], "Road Runner": [[72, "road-runner"]], "Robot Tank": [[73, "robot-tank"]], "Seaquest": [[74, "seaquest"]], "Skiings": [[75, "skiings"]], "Solaris": [[76, "solaris"]], "SpaceInvaders": [[77, "spaceinvaders"]], "StarGunner": [[78, "stargunner"]], "Tennis": [[79, "tennis"]], "TimePilot": [[80, "timepilot"]], "Tutankham": [[81, "tutankham"]], "Up n\u2019 Down": [[82, "up-n-down"]], "Venture": [[83, "venture"]], "Video Pinball": [[84, "video-pinball"]], "Wizard of Wor": [[85, "wizard-of-wor"]], "Zaxxon": [[86, "zaxxon"]], "Box2D": [[87, "box2d"]], "Bipedal Walker": [[88, "bipedal-walker"]], "Starting State": [[88, "starting-state"], [89, "starting-state"], [90, "starting-state"], [92, "starting-state"], [93, "starting-state"], [94, "starting-state"], [95, "starting-state"], [96, "starting-state"], [98, "starting-state"], [99, "starting-state"], [100, "starting-state"], [101, "starting-state"], [102, "starting-state"], [103, "starting-state"], [104, "starting-state"], [105, "starting-state"], [106, "starting-state"], [107, "starting-state"], [108, "starting-state"], [111, "starting-state"], [112, "starting-state"], [113, "starting-state"], [114, "starting-state"]], "Episode Termination": [[88, "episode-termination"], [89, "episode-termination"], [90, "episode-termination"]], "Credits": [[88, "credits"], [89, "credits"], [90, "credits"]], "Car Racing": [[89, "car-racing"]], "Reset Arguments": [[89, "reset-arguments"]], "Lunar Lander": [[90, "lunar-lander"]], "Classic Control": [[91, "classic-control"]], "Acrobot": [[92, "acrobot"]], "Episode End": [[92, "episode-end"], [93, "episode-end"], [94, "episode-end"], [95, "episode-end"], [98, "episode-end"], [99, "episode-end"], [100, "episode-end"], [101, "episode-end"], [102, "episode-end"], [103, "episode-end"], [104, "episode-end"], [105, "episode-end"], [106, "episode-end"], [107, "episode-end"], [108, "episode-end"], [111, "episode-end"], [112, "episode-end"], [113, "episode-end"], [114, "episode-end"]], "Cart Pole": [[93, "cart-pole"]], "Mountain Car": [[94, "mountain-car"]], "Transition Dynamics:": [[94, "transition-dynamics"], [95, "transition-dynamics"]], "Reward:": [[94, "reward"]], "Mountain Car Continuous": [[95, "mountain-car-continuous"]], "Reward": [[95, "reward"], [112, "reward"]], "Pendulum": [[96, "pendulum"]], "Episode Truncation": [[96, "episode-truncation"]], "MuJoCo": [[97, "mujoco"]], "Ant": [[98, "ant"]], "Half Cheetah": [[99, "half-cheetah"]], "Hopper": [[100, "hopper"]], "Humanoid": [[101, "humanoid"]], "Humanoid Standup": [[102, "humanoid-standup"]], "Inverted Double Pendulum": [[103, "inverted-double-pendulum"]], "Inverted Pendulum": [[104, "inverted-pendulum"]], "Pusher": [[105, "pusher"]], "Reacher": [[106, "reacher"]], "Swimmer": [[107, "swimmer"]], "Notes": [[107, "notes"]], "Walker2D": [[108, "walker2d"]], "Third-party Environments": [[109, "third-party-environments"]], "Video Game environments": [[109, "video-game-environments"]], "stable-retro: Classic retro games, a maintained version of OpenAI Retro": [[109, "stable-retro-classic-retro-games-a-maintained-version-of-openai-retro"]], "flappy-bird-gym: A Flappy Bird environment for Gym": [[109, "flappy-bird-gym-a-flappy-bird-environment-for-gym"]], "gym-derk: GPU accelerated MOBA environment": [[109, "gym-derk-gpu-accelerated-moba-environment"]], "SlimeVolleyGym: A simple environment for single and multi-agent reinforcement learning": [[109, "slimevolleygym-a-simple-environment-for-single-and-multi-agent-reinforcement-learning"]], "Unity ML Agents: Environments for Unity game engine": [[109, "unity-ml-agents-environments-for-unity-game-engine"]], "PGE: Parallel Game Engine": [[109, "pge-parallel-game-engine"]], "Robotics environments": [[109, "robotics-environments"]], "gym-jiminy: Training Robots in Jiminy": [[109, "gym-jiminy-training-robots-in-jiminy"]], "gym-pybullet-drones: Environments for quadcopter control": [[109, "gym-pybullet-drones-environments-for-quadcopter-control"]], "MarsExplorer: Environments for controlling robot on Mars": [[109, "marsexplorer-environments-for-controlling-robot-on-mars"]], "panda-gym: Robotics environments using the PyBullet physics engine": [[109, "panda-gym-robotics-environments-using-the-pybullet-physics-engine"]], "robo-gym: Real-world and simulation robotics": [[109, "robo-gym-real-world-and-simulation-robotics"]], "Offworld-gym": [[109, "offworld-gym"]], "safe-control-gym": [[109, "safe-control-gym"]], "gym-softrobot: Soft-robotics environments": [[109, "gym-softrobot-soft-robotics-environments"]], "iGibson": [[109, "igibson"]], "DexterousHands: dual dexterous hand manipulation tasks": [[109, "dexteroushands-dual-dexterous-hand-manipulation-tasks"]], "OmniIsaacGymEnvs": [[109, "omniisaacgymenvs"]], "Autonomous Driving environments": [[109, "autonomous-driving-environments"]], "sumo-rl": [[109, "sumo-rl"]], "gym-duckietown": [[109, "gym-duckietown"]], "gym-electric-motor": [[109, "gym-electric-motor"]], "highway-env": [[109, "highway-env"]], "CommonRoad-RL": [[109, "commonroad-rl"]], "tmrl: TrackMania 2020 through RL": [[109, "tmrl-trackmania-2020-through-rl"]], "racing_dreamer": [[109, "racing-dreamer"]], "l2r: Multimodal control environment where agents learn how to race": [[109, "l2r-multimodal-control-environment-where-agents-learn-how-to-race"]], "racecar_gym": [[109, "racecar-gym"]], "Other environments": [[109, "other-environments"]], "CompilerGym: Optimise compiler tasks": [[109, "compilergym-optimise-compiler-tasks"]], "CARL: context adaptive RL": [[109, "carl-context-adaptive-rl"]], "matrix-mdp: Easily create discrete MDPs": [[109, "matrix-mdp-easily-create-discrete-mdps"]], "mo-gym: Multi-objective Reinforcement Learning environments": [[109, "mo-gym-multi-objective-reinforcement-learning-environments"]], "gym-cellular-automata: Cellular Automata environments": [[109, "gym-cellular-automata-cellular-automata-environments"]], "gym-sokoban: 2D Transportation Puzzles": [[109, "gym-sokoban-2d-transportation-puzzles"]], "DACBench: Benchmark Library for Dynamic Algorithm configuration": [[109, "dacbench-benchmark-library-for-dynamic-algorithm-configuration"]], "NLPGym: A toolkit to develop RL agents to solve NLP tasks": [[109, "nlpgym-a-toolkit-to-develop-rl-agents-to-solve-nlp-tasks"]], "gym-saturation: Environments used to prove theorems": [[109, "gym-saturation-environments-used-to-prove-theorems"]], "ShinRL: Environments for evaluating RL algorithms": [[109, "shinrl-environments-for-evaluating-rl-algorithms"]], "gymnax: Hardware Accelerated RL Environments": [[109, "gymnax-hardware-accelerated-rl-environments"]], "gym-anytrading: Financial trading environments for FOREX and STOCKS": [[109, "gym-anytrading-financial-trading-environments-for-forex-and-stocks"]], "gym-mtsim: Financial trading for MetaTrader 5 platform": [[109, "gym-mtsim-financial-trading-for-metatrader-5-platform"]], "openmodelica-microgrid-gym: Environments for controlling power electronic converters in microgrids": [[109, "openmodelica-microgrid-gym-environments-for-controlling-power-electronic-converters-in-microgrids"]], "mobile-env: Environments for coordination of wireless mobile networks": [[109, "mobile-env-environments-for-coordination-of-wireless-mobile-networks"]], "GymFC: A flight control tuning and training framework": [[109, "gymfc-a-flight-control-tuning-and-training-framework"]], "Toy Text": [[110, "toy-text"]], "Blackjack": [[111, "blackjack"]], "Information": [[111, "information"], [112, "information"], [113, "information"], [114, "information"]], "Cliff Walking": [[112, "cliff-walking"]], "Frozen Lake": [[113, "frozen-lake"]], "Taxi": [[114, "taxi"]], "Gymnasium is a standard API for reinforcement learning, and a diverse collection of reference environments": [[115, "gymnasium-is-a-standard-api-for-reinforcement-learning-and-a-diverse-collection-of-reference-environments"]], "Solving Blackjack with Q-Learning": [[116, "solving-blackjack-with-q-learning"]], "Imports and Environment Setup": [[116, "imports-and-environment-setup"]], "Observing the environment": [[116, "observing-the-environment"]], "Executing an action": [[116, "executing-an-action"]], "Building an agent": [[116, "building-an-agent"], [120, "building-an-agent"]], "Visualizing the training": [[116, "visualizing-the-training"]], "Visualising the policy": [[116, "visualising-the-policy"]], "Think you can do better?": [[116, "think-you-can-do-better"]], "Make your own custom environment": [[117, "make-your-own-custom-environment"]], "Subclassing gymnasium.Env": [[117, "subclassing-gymnasium-env"]], "Declaration and Initialization": [[117, "declaration-and-initialization"]], "Constructing Observations From Environment States": [[117, "constructing-observations-from-environment-states"]], "Reset": [[117, "reset"]], "Step": [[117, "step"]], "Rendering": [[117, "rendering"]], "Close": [[117, "close"]], "Registering Envs": [[117, "registering-envs"]], "Creating a Package": [[117, "creating-a-package"]], "Creating Environment Instances": [[117, "creating-environment-instances"]], "Using Wrappers": [[117, "using-wrappers"]], "Handling Time Limits": [[118, "handling-time-limits"]], "Termination": [[118, "termination"]], "Truncation": [[118, "truncation"]], "Importance in learning code": [[118, "importance-in-learning-code"]], "Solution": [[118, "solution"]], "Implementing Custom Wrappers": [[119, "implementing-custom-wrappers"]], "Inheriting from gymnasium.ObservationWrapper": [[119, "inheriting-from-gymnasium-observationwrapper"]], "Inheriting from gymnasium.ActionWrapper": [[119, "inheriting-from-gymnasium-actionwrapper"]], "Inheriting from gymnasium.RewardWrapper": [[119, "inheriting-from-gymnasium-rewardwrapper"]], "Inheriting from gymnasium.Wrapper": [[119, "inheriting-from-gymnasium-wrapper"]], "Training using REINFORCE for Mujoco": [[120, "training-using-reinforce-for-mujoco"]], "Policy Network": [[120, "policy-network"]], "Plot learning curve": [[120, "plot-learning-curve"]]}, "indexentries": {"env (class in gymnasium)": [[2, "gymnasium.Env"]], "action_space (gymnasium.env attribute)": [[2, "gymnasium.Env.action_space"]], "close() (in module gymnasium.env)": [[2, "gymnasium.Env.close"]], "metadata (gymnasium.env attribute)": [[2, "gymnasium.Env.metadata"]], "np_random (gymnasium.env property)": [[2, "gymnasium.Env.np_random"]], "observation_space (gymnasium.env attribute)": [[2, "gymnasium.Env.observation_space"]], "render() (in module gymnasium.env)": [[2, "gymnasium.Env.render"]], "render_mode (gymnasium.env attribute)": [[2, "gymnasium.Env.render_mode"]], "reset() (in module gymnasium.env)": [[2, "gymnasium.Env.reset"]], "reward_range (gymnasium.env attribute)": [[2, "gymnasium.Env.reward_range"]], "spec (gymnasium.env attribute)": [[2, "gymnasium.Env.spec"]], "step() (in module gymnasium.env)": [[2, "gymnasium.Env.step"]], "unwrapped (gymnasium.env property)": [[2, "gymnasium.Env.unwrapped"]], "funcenv (class in gymnasium.experimental)": [[4, "gymnasium.experimental.FuncEnv"]], "initial() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.initial"]], "observation() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.observation"]], "render_close() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.render_close"]], "render_image() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.render_image"]], "render_init() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.render_init"]], "reward() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.reward"]], "state_info() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.state_info"]], "step_info() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.step_info"]], "terminal() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.terminal"]], "transform() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.transform"]], "transition() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.transition"]], "ataripreprocessingv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.AtariPreprocessingV0"]], "autoresetv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.AutoresetV0"]], "clipactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ClipActionV0"]], "cliprewardv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ClipRewardV0"]], "delayobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.DelayObservationV0"]], "dtypeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.DtypeObservationV0"]], "filterobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.FilterObservationV0"]], "flattenobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.FlattenObservationV0"]], "framestackobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.FrameStackObservationV0"]], "grayscaleobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.GrayscaleObservationV0"]], "humanrenderingv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.HumanRenderingV0"]], "jaxtonumpyv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.JaxToNumpyV0"]], "jaxtotorchv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.JaxToTorchV0"]], "lambdaactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.LambdaActionV0"]], "lambdaobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.LambdaObservationV0"]], "lambdarewardv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.LambdaRewardV0"]], "normalizeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.NormalizeObservationV0"]], "normalizerewardv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.NormalizeRewardV0"]], "numpytotorchv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.NumpyToTorchV0"]], "orderenforcingv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.OrderEnforcingV0"]], "passiveenvcheckerv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.PassiveEnvCheckerV0"]], "pixelobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.PixelObservationV0"]], "recordepisodestatisticsv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RecordEpisodeStatisticsV0"]], "recordvideov0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RecordVideoV0"]], "rendercollectionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RenderCollectionV0"]], "rescaleactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RescaleActionV0"]], "rescaleobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RescaleObservationV0"]], "reshapeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ReshapeObservationV0"]], "resizeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ResizeObservationV0"]], "stickyactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.StickyActionV0"]], "timeawareobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.TimeAwareObservationV0"]], "make() (in module gymnasium)": [[8, "gymnasium.make"]], "pprint_registry() (in module gymnasium)": [[8, "gymnasium.pprint_registry"]], "register() (in module gymnasium)": [[8, "gymnasium.register"]], "spec() (in module gymnasium)": [[8, "gymnasium.spec"]], "space (class in gymnasium.spaces)": [[9, "gymnasium.spaces.Space"]], "contains() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.contains"]], "dtype (gymnasium.spaces.space property)": [[9, "gymnasium.spaces.Space.dtype"]], "from_jsonable() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.from_jsonable"]], "gymnasium.spaces": [[9, "module-gymnasium.spaces"]], "is_np_flattenable (gymnasium.spaces.space.space property)": [[9, "gymnasium.spaces.space.Space.is_np_flattenable"]], "module": [[9, "module-gymnasium.spaces"], [16, "module-gymnasium.wrappers"]], "sample() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.sample"]], "seed() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.seed"]], "shape (gymnasium.spaces.space.space property)": [[9, "gymnasium.spaces.space.Space.shape"]], "to_jsonable() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.to_jsonable"]], "dict (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Dict"]], "graph (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Graph"]], "sequence (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Sequence"]], "tuple (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Tuple"]], "sample() (gymnasium.spaces.dict method)": [[10, "gymnasium.spaces.Dict.sample"]], "sample() (gymnasium.spaces.graph method)": [[10, "gymnasium.spaces.Graph.sample"]], "sample() (gymnasium.spaces.sequence method)": [[10, "gymnasium.spaces.Sequence.sample"]], "sample() (gymnasium.spaces.tuple method)": [[10, "gymnasium.spaces.Tuple.sample"]], "seed() (gymnasium.spaces.dict method)": [[10, "gymnasium.spaces.Dict.seed"]], "seed() (gymnasium.spaces.graph method)": [[10, "gymnasium.spaces.Graph.seed"]], "seed() (gymnasium.spaces.sequence method)": [[10, "gymnasium.spaces.Sequence.seed"]], "seed() (gymnasium.spaces.tuple method)": [[10, "gymnasium.spaces.Tuple.seed"]], "box (class in gymnasium.spaces)": [[11, "gymnasium.spaces.Box"]], "discrete (class in gymnasium.spaces)": [[11, "gymnasium.spaces.Discrete"]], "multibinary (class in gymnasium.spaces)": [[11, "gymnasium.spaces.MultiBinary"]], "multidiscrete (class in gymnasium.spaces)": [[11, "gymnasium.spaces.MultiDiscrete"]], "text (class in gymnasium.spaces)": [[11, "gymnasium.spaces.Text"]], "is_bounded() (gymnasium.spaces.box method)": [[11, "gymnasium.spaces.Box.is_bounded"]], "sample() (gymnasium.spaces.box method)": [[11, "gymnasium.spaces.Box.sample"]], "sample() (gymnasium.spaces.discrete method)": [[11, "gymnasium.spaces.Discrete.sample"]], "sample() (gymnasium.spaces.multibinary method)": [[11, "gymnasium.spaces.MultiBinary.sample"]], "sample() (gymnasium.spaces.multidiscrete method)": [[11, "gymnasium.spaces.MultiDiscrete.sample"]], "sample() (gymnasium.spaces.text method)": [[11, "gymnasium.spaces.Text.sample"]], "seed() (gymnasium.spaces.box method)": [[11, "gymnasium.spaces.Box.seed"]], "seed() (gymnasium.spaces.discrete method)": [[11, "gymnasium.spaces.Discrete.seed"]], "seed() (gymnasium.spaces.multibinary method)": [[11, "gymnasium.spaces.MultiBinary.seed"]], "seed() (gymnasium.spaces.multidiscrete method)": [[11, "gymnasium.spaces.MultiDiscrete.seed"]], "seed() (gymnasium.spaces.text method)": [[11, "gymnasium.spaces.Text.seed"]], "flatdim() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.flatdim"]], "flatten() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.flatten"]], "flatten_space() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.flatten_space"]], "unflatten() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.unflatten"]], "batch_space() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.batch_space"]], "concatenate() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.concatenate"]], "create_empty_array() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.create_empty_array"]], "create_shared_memory() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.create_shared_memory"]], "iterate() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.iterate"]], "read_from_shared_memory() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.read_from_shared_memory"]], "write_to_shared_memory() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.write_to_shared_memory"]], "playplot (class in gymnasium.utils.play)": [[14, "gymnasium.utils.play.PlayPlot"]], "playablegame (class in gymnasium.utils.play)": [[14, "gymnasium.utils.play.PlayableGame"]], "callback() (gymnasium.utils.play.playplot method)": [[14, "gymnasium.utils.play.PlayPlot.callback"]], "capped_cubic_video_schedule() (in module gymnasium.utils.save_video)": [[14, "gymnasium.utils.save_video.capped_cubic_video_schedule"]], "check_env() (in module gymnasium.utils.env_checker)": [[14, "gymnasium.utils.env_checker.check_env"]], "convert_to_done_step_api() (in module gymnasium.utils.step_api_compatibility)": [[14, "gymnasium.utils.step_api_compatibility.convert_to_done_step_api"]], "convert_to_terminated_truncated_step_api() (in module gymnasium.utils.step_api_compatibility)": [[14, "gymnasium.utils.step_api_compatibility.convert_to_terminated_truncated_step_api"]], "np_random() (in module gymnasium.utils.seeding)": [[14, "gymnasium.utils.seeding.np_random"]], "play() (in module gymnasium.utils.play)": [[14, "gymnasium.utils.play.play"]], "process_event() (gymnasium.utils.play.playablegame method)": [[14, "gymnasium.utils.play.PlayableGame.process_event"]], "save_video() (in module gymnasium.utils.save_video)": [[14, "gymnasium.utils.save_video.save_video"]], "step_api_compatibility() (in module gymnasium.utils.step_api_compatibility)": [[14, "gymnasium.utils.step_api_compatibility.step_api_compatibility"]], "asyncvectorenv (class in gymnasium.vector)": [[15, "gymnasium.vector.AsyncVectorEnv"]], "syncvectorenv (class in gymnasium.vector)": [[15, "gymnasium.vector.SyncVectorEnv"]], "vectorenv (class in gymnasium.vector)": [[15, "gymnasium.vector.VectorEnv"]], "action_space": [[15, "action_space"]], "close() (gymnasium.vector.vectorenv method)": [[15, "gymnasium.vector.VectorEnv.close"]], "make() (in module gymnasium.vector)": [[15, "gymnasium.vector.make"]], "observation_space": [[15, "observation_space"]], "reset() (gymnasium.vector.vectorenv method)": [[15, "gymnasium.vector.VectorEnv.reset"]], "single_action_space": [[15, "single_action_space"]], "single_observation_space": [[15, "single_observation_space"]], "step() (gymnasium.vector.vectorenv method)": [[15, "gymnasium.vector.VectorEnv.step"]], "wrapper (class in gymnasium)": [[16, "gymnasium.Wrapper"]], "action_space (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.action_space"]], "close() (in module gymnasium.wrapper)": [[16, "gymnasium.Wrapper.close"]], "env (gymnasium.wrappers.gymnasium.wrapper attribute)": [[16, "gymnasium.wrappers.gymnasium.Wrapper.env"]], "gymnasium.wrappers": [[16, "module-gymnasium.wrappers"]], "metadata (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.metadata"]], "np_random (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.np_random"]], "observation_space (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.observation_space"]], "reset() (in module gymnasium.wrapper)": [[16, "gymnasium.Wrapper.reset"]], "reward_range (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.reward_range"]], "spec (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.spec"]], "step() (in module gymnasium.wrapper)": [[16, "gymnasium.Wrapper.step"]], "unwrapped (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.unwrapped"]], "actionwrapper (class in gymnasium)": [[17, "gymnasium.ActionWrapper"]], "clipaction (class in gymnasium.wrappers)": [[17, "gymnasium.wrappers.ClipAction"]], "rescaleaction (class in gymnasium.wrappers)": [[17, "gymnasium.wrappers.RescaleAction"]], "action() (gymnasium.actionwrapper method)": [[17, "gymnasium.ActionWrapper.action"]], "ataripreprocessing (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.AtariPreprocessing"]], "autoresetwrapper (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.AutoResetWrapper"]], "envcompatibility (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.EnvCompatibility"]], "humanrendering (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.HumanRendering"]], "orderenforcing (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.OrderEnforcing"]], "passiveenvchecker (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.PassiveEnvChecker"]], "recordepisodestatistics (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.RecordEpisodeStatistics"]], "recordvideo (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.RecordVideo"]], "rendercollection (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.RenderCollection"]], "stepapicompatibility (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.StepAPICompatibility"]], "timelimit (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.TimeLimit"]], "vectorlistinfo (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.VectorListInfo"]], "filterobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.FilterObservation"]], "flattenobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.FlattenObservation"]], "framestack (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.FrameStack"]], "grayscaleobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.GrayScaleObservation"]], "normalizeobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.NormalizeObservation"]], "observationwrapper (class in gymnasium)": [[19, "gymnasium.ObservationWrapper"]], "pixelobservationwrapper (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.PixelObservationWrapper"]], "resizeobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.ResizeObservation"]], "timeawareobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.TimeAwareObservation"]], "transformobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.TransformObservation"]], "observation() (gymnasium.observationwrapper method)": [[19, "gymnasium.ObservationWrapper.observation"]], "normalizereward (class in gymnasium.wrappers)": [[20, "gymnasium.wrappers.NormalizeReward"]], "rewardwrapper (class in gymnasium)": [[20, "gymnasium.RewardWrapper"]], "transformreward (class in gymnasium.wrappers)": [[20, "gymnasium.wrappers.TransformReward"]], "reward() (gymnasium.rewardwrapper method)": [[20, "gymnasium.RewardWrapper.reward"]]}})
\ No newline at end of file
+Search.setIndex({"docnames": ["404", "README", "api/env", "api/experimental", "api/experimental/functional", "api/experimental/vector", "api/experimental/vector_wrappers", "api/experimental/wrappers", "api/registry", "api/spaces", "api/spaces/composite", "api/spaces/fundamental", "api/spaces/utils", "api/spaces/vector_utils", "api/utils", "api/vector", "api/wrappers", "api/wrappers/action_wrappers", "api/wrappers/misc_wrappers", "api/wrappers/observation_wrappers", "api/wrappers/reward_wrappers", "content/basic_usage", "content/gym_compatibility", "content/migration-guide", "environments/atari", "environments/atari/adventure", "environments/atari/air_raid", "environments/atari/alien", "environments/atari/amidar", "environments/atari/assault", "environments/atari/asterix", "environments/atari/asteroids", "environments/atari/atlantis", "environments/atari/bank_heist", "environments/atari/battle_zone", "environments/atari/beam_rider", "environments/atari/berzerk", "environments/atari/bowling", "environments/atari/boxing", "environments/atari/breakout", "environments/atari/carnival", "environments/atari/centipede", "environments/atari/chopper_command", "environments/atari/complete_list", "environments/atari/crazy_climber", "environments/atari/defender", "environments/atari/demon_attack", "environments/atari/double_dunk", "environments/atari/elevator_action", "environments/atari/enduro", "environments/atari/fishing_derby", "environments/atari/freeway", "environments/atari/frostbite", "environments/atari/gopher", "environments/atari/gravitar", "environments/atari/hero", "environments/atari/ice_hockey", "environments/atari/jamesbond", "environments/atari/journey_escape", "environments/atari/kangaroo", "environments/atari/krull", "environments/atari/kung_fu_master", "environments/atari/montezuma_revenge", "environments/atari/ms_pacman", "environments/atari/name_this_game", "environments/atari/phoenix", "environments/atari/pitfall", "environments/atari/pong", "environments/atari/pooyan", "environments/atari/private_eye", "environments/atari/qbert", "environments/atari/riverraid", "environments/atari/road_runner", "environments/atari/robotank", "environments/atari/seaquest", "environments/atari/skiing", "environments/atari/solaris", "environments/atari/space_invaders", "environments/atari/star_gunner", "environments/atari/tennis", "environments/atari/time_pilot", "environments/atari/tutankham", "environments/atari/up_n_down", "environments/atari/venture", "environments/atari/video_pinball", "environments/atari/wizard_of_wor", "environments/atari/zaxxon", "environments/box2d", "environments/box2d/bipedal_walker", "environments/box2d/car_racing", "environments/box2d/lunar_lander", "environments/classic_control", "environments/classic_control/acrobot", "environments/classic_control/cart_pole", "environments/classic_control/mountain_car", "environments/classic_control/mountain_car_continuous", "environments/classic_control/pendulum", "environments/mujoco", "environments/mujoco/ant", "environments/mujoco/half_cheetah", "environments/mujoco/hopper", "environments/mujoco/humanoid", "environments/mujoco/humanoid_standup", "environments/mujoco/inverted_double_pendulum", "environments/mujoco/inverted_pendulum", "environments/mujoco/pusher", "environments/mujoco/reacher", "environments/mujoco/swimmer", "environments/mujoco/walker2d", "environments/third_party_environments", "environments/toy_text", "environments/toy_text/blackjack", "environments/toy_text/cliff_walking", "environments/toy_text/frozen_lake", "environments/toy_text/taxi", "index", "release_notes/gym_v0.19.0", "release_notes/gym_v0.20.0", "release_notes/gym_v0.21.0", "release_notes/gym_v0.22.0", "release_notes/gym_v0.23.0", "release_notes/gym_v0.23.1", "release_notes/gym_v0.24.0", "release_notes/gym_v0.24.1", "release_notes/gym_v0.25.0", "release_notes/gym_v0.25.1", "release_notes/gym_v0.25.2", "release_notes/gym_v0.26.0", "release_notes/gym_v0.26.1", "release_notes/gym_v0.26.2", "release_notes/index", "tutorials/gymnasium_basics/README", "tutorials/gymnasium_basics/environment_creation", "tutorials/gymnasium_basics/handling_time_limits", "tutorials/gymnasium_basics/implementing_custom_wrappers", "tutorials/gymnasium_basics/index", "tutorials/gymnasium_basics/vector_envs_tutorial", "tutorials/index", "tutorials/training_agents/README", "tutorials/training_agents/blackjack_tutorial", "tutorials/training_agents/index", "tutorials/training_agents/reinforce_invpend_gym_v26"], "filenames": ["404.md", "README.md", "api/env.md", "api/experimental.md", "api/experimental/functional.md", "api/experimental/vector.md", "api/experimental/vector_wrappers.md", "api/experimental/wrappers.md", "api/registry.md", "api/spaces.md", "api/spaces/composite.md", "api/spaces/fundamental.md", "api/spaces/utils.md", "api/spaces/vector_utils.md", "api/utils.md", "api/vector.md", "api/wrappers.md", "api/wrappers/action_wrappers.md", "api/wrappers/misc_wrappers.md", "api/wrappers/observation_wrappers.md", "api/wrappers/reward_wrappers.md", "content/basic_usage.md", "content/gym_compatibility.md", "content/migration-guide.md", "environments/atari.md", "environments/atari/adventure.md", "environments/atari/air_raid.md", "environments/atari/alien.md", "environments/atari/amidar.md", "environments/atari/assault.md", "environments/atari/asterix.md", "environments/atari/asteroids.md", "environments/atari/atlantis.md", "environments/atari/bank_heist.md", "environments/atari/battle_zone.md", "environments/atari/beam_rider.md", "environments/atari/berzerk.md", "environments/atari/bowling.md", "environments/atari/boxing.md", "environments/atari/breakout.md", "environments/atari/carnival.md", "environments/atari/centipede.md", "environments/atari/chopper_command.md", "environments/atari/complete_list.md", "environments/atari/crazy_climber.md", "environments/atari/defender.md", "environments/atari/demon_attack.md", "environments/atari/double_dunk.md", "environments/atari/elevator_action.md", "environments/atari/enduro.md", "environments/atari/fishing_derby.md", "environments/atari/freeway.md", "environments/atari/frostbite.md", "environments/atari/gopher.md", "environments/atari/gravitar.md", "environments/atari/hero.md", "environments/atari/ice_hockey.md", "environments/atari/jamesbond.md", "environments/atari/journey_escape.md", "environments/atari/kangaroo.md", "environments/atari/krull.md", "environments/atari/kung_fu_master.md", "environments/atari/montezuma_revenge.md", "environments/atari/ms_pacman.md", "environments/atari/name_this_game.md", "environments/atari/phoenix.md", "environments/atari/pitfall.md", "environments/atari/pong.md", "environments/atari/pooyan.md", "environments/atari/private_eye.md", "environments/atari/qbert.md", "environments/atari/riverraid.md", "environments/atari/road_runner.md", "environments/atari/robotank.md", "environments/atari/seaquest.md", "environments/atari/skiing.md", "environments/atari/solaris.md", "environments/atari/space_invaders.md", "environments/atari/star_gunner.md", "environments/atari/tennis.md", "environments/atari/time_pilot.md", "environments/atari/tutankham.md", "environments/atari/up_n_down.md", "environments/atari/venture.md", "environments/atari/video_pinball.md", "environments/atari/wizard_of_wor.md", "environments/atari/zaxxon.md", "environments/box2d.md", "environments/box2d/bipedal_walker.md", "environments/box2d/car_racing.md", "environments/box2d/lunar_lander.md", "environments/classic_control.md", "environments/classic_control/acrobot.md", "environments/classic_control/cart_pole.md", "environments/classic_control/mountain_car.md", "environments/classic_control/mountain_car_continuous.md", "environments/classic_control/pendulum.md", "environments/mujoco.md", "environments/mujoco/ant.md", "environments/mujoco/half_cheetah.md", "environments/mujoco/hopper.md", "environments/mujoco/humanoid.md", "environments/mujoco/humanoid_standup.md", "environments/mujoco/inverted_double_pendulum.md", "environments/mujoco/inverted_pendulum.md", "environments/mujoco/pusher.md", "environments/mujoco/reacher.md", "environments/mujoco/swimmer.md", "environments/mujoco/walker2d.md", "environments/third_party_environments.md", "environments/toy_text.md", "environments/toy_text/blackjack.md", "environments/toy_text/cliff_walking.md", "environments/toy_text/frozen_lake.md", "environments/toy_text/taxi.md", "index.md", "release_notes/gym_v0.19.0.md", "release_notes/gym_v0.20.0.md", "release_notes/gym_v0.21.0.md", "release_notes/gym_v0.22.0.md", "release_notes/gym_v0.23.0.md", "release_notes/gym_v0.23.1.md", "release_notes/gym_v0.24.0.md", "release_notes/gym_v0.24.1.md", "release_notes/gym_v0.25.0.md", "release_notes/gym_v0.25.1.md", "release_notes/gym_v0.25.2.md", "release_notes/gym_v0.26.0.md", "release_notes/gym_v0.26.1.md", "release_notes/gym_v0.26.2.md", "release_notes/index.md", "tutorials/gymnasium_basics/README.rst", "tutorials/gymnasium_basics/environment_creation.rst", "tutorials/gymnasium_basics/handling_time_limits.rst", "tutorials/gymnasium_basics/implementing_custom_wrappers.rst", "tutorials/gymnasium_basics/index.rst", "tutorials/gymnasium_basics/vector_envs_tutorial.rst", "tutorials/index.rst", "tutorials/training_agents/README.rst", "tutorials/training_agents/blackjack_tutorial.rst", "tutorials/training_agents/index.rst", "tutorials/training_agents/reinforce_invpend_gym_v26.rst"], "titles": ["404 - Page Not Found", "Gymnasium-docs", "Env", "Experimental", "Functional Environment", "Vectorizing Environment", "Vector Environment Wrappers", "Wrappers", "Registry", "Spaces", "Composite Spaces", "Fundamental Spaces", "Spaces Utils", "Spaces Vector Utils", "Utils", "Vector", "Wrappers", "Action Wrappers", "Misc Wrappers", "Observation Wrappers", "Reward Wrappers", "Basic Usage", "Compatibility with Gym", "v21 to v26 Migration Guide", "Atari", "Adventure", "Air Raid", "Alien", "Amidar", "Assault", "Asterix", "Asteroids", "Atlantis", "Bank Heist", "Battle Zone", "Beam Rider", "Berzerk", "Bowling", "Boxing", "Breakout", "Carnival", "Centipede", "Chopper Command", "Complete List - Atari", "Crazy Climber", "Defender", "Demon Attack", "Double Dunk", "Elevator Action", "Enduro", "FishingDerby", "Freeway", "Frostbite", "Gopher", "Gravitar", "Hero", "IceHockey", "Jamesbond", "JourneyEscape", "Kangaroo", "Krull", "Kung Fu Master", "Montezuma Revenge", "Ms Pacman", "Name This Game", "Phoenix", "Pitfall", "Pong", "Pooyan", "PrivateEye", "Qbert", "Riverraid", "Road Runner", "Robot Tank", "Seaquest", "Skiings", "Solaris", "SpaceInvaders", "StarGunner", "Tennis", "TimePilot", "Tutankham", "Up n\u2019 Down", "Venture", "Video Pinball", "Wizard of Wor", "Zaxxon", "Box2D", "Bipedal Walker", "Car Racing", "Lunar Lander", "Classic Control", "Acrobot", "Cart Pole", "Mountain Car", "Mountain Car Continuous", "Pendulum", "MuJoCo", "Ant", "Half Cheetah", "Hopper", "Humanoid", "Humanoid Standup", "Inverted Double Pendulum", "Inverted Pendulum", "Pusher", "Reacher", "Swimmer", "Walker2D", "Third-party Environments", "Toy Text", "Blackjack", "Cliff Walking", "Frozen Lake", "Taxi", "Gymnasium is a standard API for reinforcement learning, and a diverse collection of reference environments", "v0.19.0 Release Notes", "v0.20.0 Release Notes", "v0.21.0 Release Notes", "v0.22 Release Notes", "v0.23.0 Release Notes", "v0.23.1 Release Notes", "v0.24.0 Release Notes", "v0.24.1 Release Notes", "v0.25.0 Release notes", "0.25.1 Release Notes", "v0.25.2 Release Notes", "v0.26.0 Release Notes", "v0.26.1 Release Notes", "v0.26.2 Release Notes", "Release Notes", "Gymnasium Basics", "Make your own custom environment", "Handling Time Limits", "Implementing Custom Wrappers", "Gymnasium Basics", "Training A2C with Vector Envs and Domain Randomization", "Tutorials", "Training Agents", "Solving Blackjack with Q-Learning", "Training Agents", "Training using REINFORCE for Mujoco"], "terms": {"thi": [1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 116, 117, 119, 120, 121, 122, 123, 124, 126, 127, 128, 129, 130, 132, 133, 134, 136, 139, 141], "folder": [1, 7, 14, 18, 105, 106, 130], "contain": [1, 2, 3, 7, 9, 12, 13, 15, 16, 18, 19, 21, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 105, 106, 107, 108, 109, 111, 114, 117, 120, 121, 129, 130, 132, 136, 139, 141], "If": [1, 2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 30, 33, 38, 41, 50, 51, 52, 53, 54, 68, 75, 79, 88, 89, 90, 95, 98, 100, 101, 102, 106, 108, 111, 112, 113, 122, 127, 130, 132, 134, 136, 139], "you": [1, 2, 7, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 98, 101, 102, 105, 106, 109, 115, 117, 118, 122, 124, 127, 130, 132, 133, 134, 136], "ar": [1, 2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 117, 118, 119, 124, 126, 127, 129, 130, 132, 133, 134, 136, 139, 141], "pleas": [1, 2, 3, 14, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 117, 118, 124, 130], "pr": [1, 23, 109, 118, 122], "repo": [1, 24, 117, 130], "otherwis": [1, 2, 7, 8, 10, 11, 14, 15, 18, 19, 24, 37, 109, 124, 139], "follow": [1, 4, 7, 8, 9, 14, 18, 21, 24, 37, 47, 49, 50, 51, 52, 53, 54, 79, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 127, 130, 132, 133, 134, 139, 141], "below": [1, 11, 16, 21, 50, 88, 90, 96, 105, 108, 130, 133, 136, 139], "directli": [1, 7, 10, 16, 18, 21, 92, 109, 132, 139, 141], "markdown": 1, "file": [1, 92, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 119, 123, 124, 130, 132], "repositori": [1, 97, 122], "fork": [1, 23, 109, 115, 130], "docstr": [1, 123, 124, 125, 130], "": [1, 2, 3, 4, 7, 10, 11, 14, 15, 16, 17, 18, 19, 20, 21, 24, 44, 50, 51, 52, 53, 54, 56, 57, 60, 61, 62, 67, 68, 70, 87, 88, 89, 90, 91, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 118, 119, 122, 127, 129, 130, 132, 133, 134, 136, 139, 141], "python": [1, 7, 8, 13, 18, 21, 88, 89, 90, 115, 119, 122, 124, 130, 132, 133, 134, 136, 137, 139, 141], "Then": [1, 16, 21, 74, 107, 134], "pip": [1, 24, 87, 91, 97, 118, 122, 130, 132], "instal": [1, 7, 14, 18, 32, 87, 91, 97, 116, 117, 118, 122, 128, 130, 132], "your": [1, 2, 3, 7, 9, 15, 16, 17, 18, 19, 20, 21, 27, 28, 30, 32, 33, 35, 37, 38, 39, 41, 47, 49, 50, 51, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 69, 70, 71, 72, 73, 74, 75, 76, 77, 80, 81, 82, 83, 84, 85, 86, 89, 98, 101, 102, 115, 127, 130, 134, 135, 136, 137, 139], "run": [1, 2, 7, 8, 15, 18, 33, 55, 58, 66, 71, 72, 75, 76, 88, 90, 92, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 112, 118, 122, 132, 136], "script": [1, 139], "gen_md": 1, "py": [1, 13, 24, 88, 89, 90, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 116, 117, 118, 121, 122, 123, 124, 125, 127, 128, 130, 132, 133, 134, 136, 139, 141], "automat": [1, 2, 3, 7, 8, 15, 16, 18, 22, 23, 119, 122, 124, 130, 132, 136], "gener": [1, 2, 7, 9, 10, 11, 14, 15, 16, 18, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 116, 119, 127, 130, 132, 136], "For": [1, 2, 4, 7, 11, 12, 14, 18, 19, 21, 22, 23, 24, 25, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 45, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 89, 90, 97, 112, 113, 114, 122, 124, 126, 127, 130, 132, 134, 136, 139], "add": [1, 7, 8, 14, 15, 19, 21, 89, 97, 101, 102, 114, 117, 118, 121, 123, 124, 126, 129, 130, 132, 134, 136, 139], "complet": [1, 7, 18, 21, 23, 24, 44, 57, 69, 70, 89, 90, 130, 133, 134, 139], "ensur": [1, 2, 3, 8, 10, 12, 14, 21, 130, 132, 139], "i": [1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 117, 118, 119, 122, 123, 124, 125, 126, 127, 128, 129, 130, 132, 133, 134, 136, 139, 141], "ha": [1, 2, 3, 7, 9, 11, 12, 14, 16, 18, 19, 20, 21, 23, 24, 54, 74, 88, 89, 90, 91, 97, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 111, 115, 118, 119, 122, 124, 127, 128, 129, 130, 132, 133, 136, 139, 141], "properli": [1, 9, 117, 118], "format": [1, 9, 14, 16, 18, 19, 21, 130], "md": [1, 118, 130, 132], "correspond": [1, 2, 8, 16, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 114, 132, 136, 141], "gif": [1, 92], "_static": 1, "video": [1, 7, 16, 18, 43, 73, 88, 116, 117, 118, 124], "env_typ": 1, "where": [1, 3, 7, 8, 10, 11, 12, 14, 16, 18, 21, 23, 24, 47, 89, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 124, 128, 132, 133, 134, 136, 139, 141], "categori": 1, "e": [1, 2, 3, 7, 9, 11, 12, 13, 15, 16, 18, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 105, 106, 109, 111, 113, 117, 127, 130, 132, 134, 136, 139, 141], "g": [1, 2, 9, 11, 12, 13, 15, 19, 75, 92, 96, 109, 113, 114, 117, 124, 125, 132, 134, 136, 139, 141], "mujoco": [1, 16, 21, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 120, 122, 123, 124, 127, 128, 130, 134, 137, 140], "snake_cas": 1, "name": [1, 3, 4, 8, 16, 43, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 118, 119, 130, 132], "convent": [1, 2, 24], "altern": [1, 24], "gen_gif": 1, "index": [1, 7, 13, 14, 18, 19, 21, 24, 93, 102], "toctre": 1, "requir": [1, 2, 7, 9, 19, 97, 116, 119, 122, 124, 126, 130, 136, 139, 141], "packag": [1, 7, 18, 109, 119], "cd": [1, 132], "r": [1, 7, 11, 18, 20, 73, 92, 96, 109, 111, 112, 114, 134, 141], "txt": [1, 116], "To": [1, 7, 8, 13, 15, 16, 18, 19, 21, 22, 23, 24, 32, 42, 70, 88, 89, 90, 92, 114, 122, 132, 134, 136, 139, 141], "builder": 1, "onc": [1, 7, 18, 31, 41, 48, 55, 103, 106, 114, 127, 132, 136, 139], "make": [1, 2, 7, 10, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 117, 121, 122, 123, 124, 126, 127, 128, 130, 134, 135, 136, 137, 139, 141], "dirhtml": 1, "_build": 1, "rebuild": [1, 32], "everi": [1, 7, 9, 14, 18, 21, 41, 42, 46, 51, 73, 74, 83, 89, 90, 93, 98, 100, 101, 108, 109, 130, 132, 136, 141], "time": [1, 2, 7, 8, 10, 14, 15, 16, 18, 19, 23, 24, 40, 43, 46, 47, 48, 52, 56, 58, 59, 66, 70, 71, 72, 73, 74, 75, 88, 89, 96, 98, 99, 100, 101, 102, 107, 108, 109, 112, 114, 117, 118, 127, 132, 135, 136, 137, 139, 141], "chang": [1, 2, 3, 4, 7, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 52, 60, 70, 89, 92, 93, 94, 95, 96, 97, 105, 106, 109, 114, 116, 118, 121, 130, 132, 134, 139, 141], "made": [1, 3, 105, 106, 117, 118, 130], "sphinx": 1, "autobuild": 1, "b": [1, 11, 105, 106, 114], "we": [1, 2, 3, 4, 7, 11, 18, 19, 21, 23, 24, 98, 101, 102, 118, 122, 124, 126, 127, 130, 132, 133, 134, 136, 139, 141], "us": [1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 113, 114, 116, 117, 118, 119, 122, 124, 125, 126, 127, 128, 130, 133, 134, 137, 139, 140], "galleri": [1, 130], "insid": [1, 10, 11, 18, 132], "directori": 1, "check": [1, 2, 7, 9, 11, 16, 18, 114, 122, 123, 130, 132, 134, 139], "demo": 1, "see": [1, 2, 9, 14, 15, 21, 23, 24, 25, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 45, 55, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 90, 92, 98, 99, 100, 101, 107, 108, 109, 113, 117, 122, 124, 127, 130, 132, 136], "exampl": [1, 2, 7, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 89, 92, 111, 112, 113, 124, 128, 132, 133, 134, 136, 137, 139], "more": [1, 2, 3, 9, 10, 11, 14, 15, 16, 19, 23, 24, 25, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 45, 50, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 88, 90, 97, 98, 101, 102, 103, 104, 105, 106, 107, 109, 122, 124, 127, 128, 130, 132, 133, 134, 136, 139, 141], "inform": [1, 2, 7, 9, 10, 11, 14, 15, 16, 18, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 127, 132, 133, 139, 141], "convert": [1, 7, 8, 9, 10, 16, 18, 19, 22, 23, 89, 127, 130, 139], "jupyt": [1, 121, 132, 133, 134, 136, 137, 139, 141], "notebook": [1, 121, 132, 133, 134, 136, 137, 139, 141], "can": [1, 2, 3, 7, 8, 9, 10, 11, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 117, 118, 119, 122, 124, 126, 127, 129, 130, 132, 134, 136, 141], "want": [1, 2, 7, 10, 15, 16, 18, 21, 33, 55, 127, 130, 132, 134, 136, 139], "execut": [1, 7, 14, 17, 24, 47, 114, 132, 136, 141], "which": [1, 2, 7, 8, 9, 10, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 44, 46, 48, 62, 89, 90, 92, 93, 97, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 109, 112, 113, 114, 119, 121, 124, 127, 130, 132, 133, 134, 136, 141], "output": [1, 7, 13, 14, 18, 21, 127, 136, 139, 141], "plot": [1, 14, 139], "should": [1, 2, 3, 7, 9, 11, 14, 15, 18, 19, 21, 23, 105, 106, 116, 121, 122, 124, 130, 132, 134, 136, 139], "start": [1, 2, 7, 9, 11, 14, 15, 18, 34, 47, 48, 54, 58, 59, 74, 136, 139, 141], "run_": 1, "note": [1, 7, 9, 13, 18, 92, 93, 98, 101, 102, 114, 133, 136, 139, 141], "so": [1, 3, 7, 14, 15, 18, 19, 21, 24, 52, 75, 90, 91, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 113, 118, 127, 130, 132, 133, 134, 136, 139], "sure": [1, 2, 18, 132, 134], "doesn": [1, 2, 7, 8, 18, 24, 90, 133], "t": [1, 2, 3, 7, 8, 10, 12, 14, 16, 18, 19, 20, 23, 24, 40, 50, 51, 52, 53, 54, 73, 74, 89, 90, 94, 95, 114, 124, 127, 130, 132, 133, 136, 139], "take": [1, 2, 4, 7, 8, 11, 12, 14, 15, 17, 18, 21, 23, 24, 77, 91, 93, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 114, 127, 134, 136, 139, 141], "than": [1, 23, 50, 56, 74, 90, 92, 93, 94, 95, 99, 103, 104, 105, 107, 109, 111, 127, 136, 139], "few": [1, 92, 120, 121, 139], "second": [1, 7, 10, 14, 15, 37, 68, 75, 90, 99, 103, 106, 107], "class": [2, 4, 7, 10, 11, 14, 15, 16, 18, 21, 97, 119, 130, 132, 134, 136, 139, 141], "The": [2, 3, 4, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 118, 119, 122, 124, 127, 130, 132, 133, 136, 139, 141], "main": [2, 90, 100, 108, 130], "reinforc": [2, 9, 21, 92, 107, 110, 111, 112, 114, 124, 130, 133, 136, 137, 139, 140], "learn": [2, 7, 9, 10, 18, 19, 20, 21, 24, 50, 51, 52, 53, 54, 88, 89, 90, 92, 93, 94, 95, 99, 102, 103, 104, 107, 110, 111, 112, 114, 119, 124, 132, 136, 137, 140], "agent": [2, 3, 7, 9, 18, 21, 23, 24, 48, 57, 88, 89, 90, 94, 98, 99, 100, 101, 102, 103, 104, 107, 108, 124, 128, 130, 132, 133, 134], "encapsul": 2, "an": [2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 111, 112, 113, 114, 118, 122, 124, 125, 128, 130, 132, 133, 134, 136], "arbitrari": [2, 8, 11, 14, 19, 20, 109, 136], "behind": [2, 68, 130], "scene": [2, 68, 109, 130], "dynam": [2, 24, 39, 50, 51, 52, 53, 54, 92, 96, 97, 103, 124], "through": [2, 7, 8, 9, 15, 22, 23, 24, 39, 55, 58, 61, 62, 66, 75, 76, 101, 102, 116, 122, 124, 127, 130, 133], "step": [2, 7, 8, 15, 16, 17, 18, 19, 20, 21, 24, 88, 90, 92, 93, 96, 112, 113, 114, 115, 118, 119, 122, 123, 124, 126, 127, 129, 130, 133, 134, 136, 139, 141], "reset": [2, 7, 8, 14, 15, 16, 17, 18, 19, 20, 21, 92, 93, 94, 95, 96, 109, 112, 113, 114, 115, 118, 119, 122, 123, 124, 125, 127, 129, 130, 136, 139, 141], "function": [2, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 90, 96, 98, 106, 109, 113, 114, 117, 119, 122, 124, 126, 127, 130, 133, 134, 136, 139, 141], "partial": [2, 124], "fulli": [2, 117], "observ": [2, 4, 9, 10, 13, 14, 15, 16, 17, 18, 20, 23, 109, 115, 117, 122, 123, 124, 125, 127, 128, 129, 130, 133, 134, 141], "singl": [2, 4, 7, 9, 10, 11, 12, 13, 14, 15, 19, 21, 100, 101, 102, 108, 124, 127, 132, 141], "multi": [2, 11, 97, 101, 102, 105], "pettingzoo": [2, 116, 117], "api": [2, 4, 7, 16, 18, 21, 23, 116, 122, 125, 126, 127, 128, 130, 132, 133, 136], "user": [2, 3, 4, 8, 15, 21, 23, 24, 92, 93, 94, 95, 96, 109, 125, 126, 127, 130, 132, 141], "need": [2, 8, 9, 10, 14, 16, 17, 18, 19, 21, 23, 42, 44, 49, 55, 59, 69, 74, 88, 90, 93, 97, 118, 124, 132, 133, 134, 136, 139], "know": [2, 3, 18, 21, 130, 132], "updat": [2, 7, 14, 18, 19, 21, 22, 23, 114, 116, 124, 127, 128, 129, 130, 132, 133, 134, 136, 139, 141], "action": [2, 4, 8, 9, 10, 11, 14, 15, 16, 18, 23, 43, 91, 109, 110, 115, 118, 119, 123, 124, 127, 130, 132, 134, 136, 141], "return": [2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 112, 113, 114, 119, 123, 124, 127, 129, 130, 132, 133, 134, 136, 139, 141], "next": [2, 4, 13, 21, 23, 44, 46, 48, 141], "reward": [2, 4, 8, 14, 15, 16, 18, 21, 23, 97, 109, 115, 116, 124, 127, 130, 132, 133, 134, 136, 139, 141], "termin": [2, 4, 7, 8, 14, 15, 16, 18, 20, 21, 22, 23, 92, 93, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 111, 112, 113, 114, 115, 124, 126, 127, 132, 134, 136, 139, 141], "truncat": [2, 7, 8, 14, 15, 16, 18, 20, 21, 22, 23, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 113, 114, 115, 124, 126, 127, 128, 130, 132, 134, 136, 139, 141], "due": [2, 23, 113, 124, 125, 126, 133, 136], "latest": [2, 141], "from": [2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 26, 40, 47, 52, 53, 56, 58, 67, 68, 80, 86, 88, 89, 90, 92, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 117, 119, 123, 124, 127, 128, 130, 133, 136, 139, 141], "about": [2, 4, 14, 19, 21, 69, 92, 98, 101, 102, 132, 133, 139, 141], "metric": [2, 14, 23, 130, 136], "debug": [2, 9, 14, 110, 126], "info": [2, 4, 7, 14, 15, 16, 18, 20, 21, 23, 98, 99, 100, 101, 105, 106, 107, 108, 114, 115, 119, 122, 124, 126, 127, 129, 130, 132, 133, 134, 136, 139, 141], "initi": [2, 4, 7, 8, 10, 11, 15, 16, 17, 18, 19, 20, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 90, 91, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 124, 127, 134, 136, 139, 141], "state": [2, 4, 7, 16, 18, 19, 21, 23, 24, 91, 97, 109, 110, 124, 126, 133, 136, 139, 141], "befor": [2, 7, 8, 14, 15, 16, 17, 18, 19, 20, 32, 44, 58, 69, 76, 77, 89, 98, 99, 100, 101, 102, 103, 104, 106, 107, 108, 118, 119, 122, 124, 130, 132, 134, 141], "call": [2, 7, 8, 12, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 37, 56, 105, 106, 111, 113, 118, 122, 123, 124, 132, 136, 139], "first": [2, 14, 15, 16, 18, 19, 21, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 116, 118, 123, 130, 132, 139, 141], "episod": [2, 7, 8, 14, 16, 18, 19, 23, 127, 132, 133, 136, 139, 141], "render": [2, 4, 15, 16, 18, 19, 21, 24, 87, 88, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 117, 119, 120, 122, 123, 124, 125, 126, 127, 128, 129, 130, 134, 139], "help": [2, 9, 21, 27, 59, 62, 105, 122, 127, 130, 136, 139], "visualis": [2, 21], "what": [2, 9, 18, 24, 112, 130], "mode": [2, 15, 16, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 124, 127, 130, 132, 136, 139], "human": [2, 7, 10, 16, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 101, 102, 105, 115, 124, 127, 130, 132, 136], "rgb_arrai": [2, 7, 14, 16, 18, 19, 24, 124, 127, 132, 136], "ansi": [2, 127, 132], "text": [2, 9, 10, 12, 21, 111, 112, 113, 114, 117, 124, 127], "close": [2, 4, 11, 14, 15, 16, 21, 23, 44, 69, 98, 100, 101, 106, 108, 115, 117, 123, 127, 130, 134, 136, 139], "import": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 122, 124, 127, 132, 134, 136, 141], "when": [2, 3, 7, 9, 10, 12, 13, 14, 15, 16, 18, 19, 22, 24, 27, 40, 44, 45, 46, 48, 50, 51, 52, 53, 54, 56, 57, 71, 72, 73, 74, 77, 89, 90, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 113, 114, 117, 118, 119, 122, 124, 126, 129, 130, 132, 133, 134, 136, 141], "extern": [2, 23, 98, 101, 102, 127, 133], "softwar": [2, 109], "pygam": [2, 14, 23, 87, 119, 121, 122, 132], "databas": 2, "have": [2, 7, 8, 9, 11, 12, 15, 16, 17, 18, 20, 21, 22, 23, 24, 25, 27, 30, 32, 33, 37, 39, 41, 42, 44, 45, 50, 51, 52, 53, 54, 55, 58, 59, 60, 64, 66, 68, 74, 76, 87, 88, 89, 92, 98, 99, 100, 101, 102, 104, 105, 107, 108, 109, 111, 113, 114, 118, 119, 122, 124, 127, 130, 132, 133, 134, 136, 139, 141], "understand": [2, 12, 15, 105, 109, 141], "action_spac": [2, 7, 9, 14, 15, 16, 17, 19, 20, 21, 23, 114, 115, 132, 134, 139, 141], "space": [2, 3, 7, 14, 15, 16, 17, 18, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 97, 109, 110, 116, 117, 118, 119, 122, 123, 124, 125, 126, 128, 130, 132, 134, 136, 141], "object": [2, 9, 13, 16, 21, 25, 30, 44, 50, 51, 59, 71, 77, 105, 139, 141], "valid": [2, 7, 8, 9, 11, 13, 15, 16, 17, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 136, 139], "all": [2, 3, 7, 9, 10, 11, 15, 16, 18, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 51, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89, 90, 91, 92, 93, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 116, 117, 119, 124, 126, 127, 130, 134, 136, 137, 139, 141], "within": [2, 7, 9, 17, 19, 21, 23, 62, 66, 90, 91, 103, 104, 127, 129, 130], "observation_spac": [2, 7, 9, 10, 11, 15, 16, 19, 21, 132, 134, 141], "reward_rang": [2, 16, 20, 134], "A": [2, 4, 7, 9, 10, 11, 12, 14, 15, 17, 18, 19, 20, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 92, 93, 95, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 112, 113, 118, 124, 125, 127, 130, 132, 133, 134, 136, 139, 141], "tupl": [2, 4, 7, 9, 11, 12, 13, 14, 16, 19, 21, 23, 24, 98, 100, 101, 108, 111, 117, 126, 132, 136, 139, 141], "minimum": [2, 7, 11, 96], "maximum": [2, 7, 8, 11, 14, 18, 21, 73, 74, 90, 92, 96, 103, 132, 136], "possibl": [2, 9, 10, 11, 12, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 94, 95, 99, 101, 103, 104, 107, 108, 109, 112, 113, 114, 127, 132, 139], "over": [2, 7, 13, 18, 27, 33, 56, 66, 71, 99, 101, 111, 113, 119, 126, 127, 134, 136, 139], "default": [2, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 92, 96, 98, 99, 100, 101, 105, 106, 107, 108, 119, 122, 124, 127, 132, 133], "rang": [2, 7, 13, 14, 16, 17, 18, 20, 21, 22, 23, 24, 56, 88, 91, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 132, 134, 136, 139, 141], "set": [2, 7, 9, 10, 11, 15, 17, 18, 19, 23, 24, 25, 26, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 82, 83, 84, 85, 87, 90, 91, 92, 94, 95, 97, 98, 99, 100, 101, 105, 106, 107, 108, 109, 113, 119, 122, 124, 127, 132, 133, 134, 136, 139, 141], "infti": [2, 11], "spec": [2, 15, 16, 18], "metadata": [2, 7, 14, 16, 18, 24, 120, 125, 126, 130, 132, 134], "fp": [2, 7, 14, 24, 132], "np_random": [2, 14, 16, 18, 23, 123, 127, 130, 132], "random": [2, 7, 9, 10, 11, 14, 18, 19, 21, 24, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 106, 107, 108, 109, 113, 114, 119, 124, 127, 130, 132, 135, 137, 139, 141], "number": [2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 24, 26, 27, 28, 29, 30, 31, 32, 35, 36, 37, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 73, 75, 77, 78, 80, 81, 82, 84, 85, 89, 90, 92, 100, 101, 102, 107, 108, 109, 111, 113, 119, 124, 127, 130, 132, 136, 141], "assign": [2, 75, 93, 94, 95, 105, 106, 139], "dure": [2, 7, 10, 11, 15, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 107, 108, 114, 119, 123, 124, 127, 132, 134], "super": [2, 16, 23, 116, 127, 132, 134, 136, 141], "seed": [2, 7, 8, 9, 10, 11, 13, 15, 16, 17, 19, 21, 72, 115, 117, 119, 122, 126, 127, 130, 132, 136, 141], "assess": 2, "self": [2, 4, 7, 9, 10, 16, 17, 18, 19, 20, 95, 109, 132, 134, 136, 139, 141], "modifi": [2, 7, 9, 15, 16, 17, 19, 20, 23, 92, 105, 106, 114, 119, 122, 124, 130, 132, 134], "extend": [2, 98], "wrapper": [2, 8, 10, 11, 14, 15, 21, 22, 105, 106, 109, 113, 114, 115, 117, 118, 120, 122, 123, 124, 125, 126, 127, 129, 130, 135, 136, 137, 139, 141], "acttyp": [2, 4, 7, 14, 16, 17, 19, 20], "obstyp": [2, 4, 7, 14, 16, 17, 19, 20, 23, 141], "supportsfloat": [2, 7, 8, 11, 16, 20, 23, 134, 141], "bool": [2, 7, 8, 9, 11, 14, 15, 16, 18, 19, 23, 24, 90, 98, 99, 100, 101, 107, 108, 132, 139, 141], "dict": [2, 4, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 19, 21, 23, 112, 113, 114, 117, 118, 125, 126, 132, 141], "str": [2, 4, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 19, 23, 24, 98, 99, 100, 101, 107, 108, 139, 141], "ani": [2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 23, 24, 46, 50, 51, 52, 53, 54, 55, 73, 90, 93, 98, 100, 101, 102, 103, 104, 105, 106, 107, 108, 113, 127, 132, 133, 139, 141], "one": [2, 10, 11, 12, 14, 15, 16, 18, 19, 21, 22, 23, 26, 27, 28, 32, 37, 44, 47, 48, 60, 69, 70, 71, 73, 74, 79, 92, 93, 94, 95, 96, 98, 100, 102, 103, 104, 106, 107, 111, 114, 119, 127, 133, 134, 136, 139, 141], "timestep": [2, 7, 18, 19, 21, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 132, 136, 141], "end": [2, 4, 7, 16, 18, 21, 32, 41, 42, 44, 46, 48, 49, 57, 58, 59, 69, 73, 76, 77, 88, 96, 124, 133, 139, 141], "reach": [2, 7, 16, 23, 32, 34, 44, 47, 48, 65, 68, 76, 77, 91, 92, 93, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 112, 113, 114, 132, 133, 139, 141], "necessari": [2, 9, 24, 50, 51, 52, 53, 54, 122, 127, 128], "version": [2, 3, 8, 9, 16, 93, 116, 117, 118, 119, 122, 123, 124, 127, 130, 132, 133, 136, 139], "0": [2, 3, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 119, 121, 123, 126, 128, 129, 132, 133, 134, 136, 139, 141], "26": [2, 22, 98, 101, 102, 132, 133, 141], "wa": [2, 7, 14, 15, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 107, 108, 118, 122, 124, 127, 128, 129, 130, 133, 134, 136, 141], "remov": [2, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 114, 116, 117, 119, 120, 121, 122, 123, 124, 125, 126, 127, 130], "done": [2, 8, 14, 16, 18, 21, 22, 23, 98, 100, 101, 102, 103, 104, 108, 122, 124, 126, 132, 133, 136, 139, 141], "favor": [2, 117, 119, 130], "clearer": [2, 118], "had": [2, 118], "critic": [2, 3, 18, 130], "bootstrap": [2, 23, 133], "algorithm": [2, 9, 97, 110, 114, 117, 124, 133, 136, 139, 141], "paramet": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 107, 108, 122, 124, 127, 132, 134, 136, 141], "provid": [2, 3, 7, 9, 10, 11, 14, 15, 16, 17, 18, 19, 21, 22, 24, 30, 44, 46, 50, 51, 52, 53, 54, 88, 92, 109, 127, 130, 132, 139], "element": [2, 7, 9, 10, 11, 13, 15, 19, 21, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 132, 136], "numpi": [2, 3, 4, 7, 9, 11, 13, 15, 16, 17, 19, 23, 109, 116, 130, 132, 134, 136, 139, 141], "arrai": [2, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 90, 109, 125, 127, 128, 130, 132, 134, 136, 139, 141], "posit": [2, 7, 10, 11, 12, 13, 14, 18, 21, 24, 35, 56, 88, 89, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 113, 114, 132, 134, 139], "veloc": [2, 10, 12, 13, 88, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "pole": [2, 75, 91, 103, 104, 109, 141], "cartpol": [2, 7, 8, 14, 15, 18, 19, 20, 21, 91, 93, 103, 104, 109, 127, 141], "result": [2, 3, 12, 14, 15, 21, 92, 98, 101, 102, 110, 112, 114, 124, 130, 136, 139], "whether": [2, 4, 7, 8, 9, 11, 14, 18, 23, 37, 90, 98, 99, 100, 101, 107, 108, 111, 132, 133, 139, 141], "defin": [2, 4, 7, 9, 12, 17, 18, 19, 20, 21, 96, 101, 132, 133, 134, 136], "under": [2, 7, 19, 20, 139, 141], "mdp": [2, 18, 21, 94, 95], "task": [2, 10, 16, 21, 60, 66, 69, 89, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114, 132, 133, 134, 141], "neg": [2, 11, 14, 21, 75, 95, 98, 99, 101, 102, 103, 105, 106, 114, 141], "goal": [2, 27, 30, 35, 37, 39, 41, 44, 56, 61, 62, 63, 64, 65, 67, 70, 72, 74, 75, 81, 82, 83, 84, 85, 86, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 132, 141], "move": [2, 3, 7, 16, 20, 21, 23, 26, 29, 30, 39, 40, 54, 57, 68, 71, 74, 75, 88, 90, 93, 98, 99, 100, 102, 103, 104, 105, 106, 107, 108, 109, 112, 113, 114, 116, 117, 119, 120, 128, 130, 132, 133, 141], "lava": [2, 21, 55], "sutton": [2, 92, 93, 103, 104, 111, 112, 117, 139], "barton": 2, "gridworld": [2, 109, 112, 132], "true": [2, 7, 8, 12, 14, 15, 17, 18, 19, 21, 22, 23, 24, 50, 51, 52, 53, 54, 88, 89, 90, 98, 99, 100, 101, 107, 108, 111, 113, 119, 122, 124, 126, 127, 130, 132, 136, 139, 141], "condit": [2, 133, 139, 141], "outsid": [2, 7, 89, 90, 117, 133], "scope": [2, 133], "satisfi": 2, "typic": [2, 14, 114], "timelimit": [2, 8, 16, 18, 21, 126, 132, 133], "could": [2, 3, 14, 21, 24, 130, 132, 133, 134, 136, 139, 141], "also": [2, 7, 8, 11, 14, 15, 16, 18, 19, 21, 24, 36, 57, 58, 69, 70, 72, 88, 89, 90, 91, 97, 98, 99, 100, 101, 107, 108, 109, 118, 124, 127, 132, 133, 134, 136, 139, 141], "indic": [2, 7, 14, 18, 23, 89, 92, 93, 98, 101, 102, 111, 112, 113, 114, 122, 133, 139, 141], "physic": [2, 87, 97, 103, 104, 118, 130, 141], "go": [2, 48, 75, 79, 89, 111, 127, 133, 136, 139, 141], "out": [2, 4, 13, 16, 24, 33, 38, 55, 58, 66, 71, 76, 127, 130, 132, 133, 134, 141], "bound": [2, 7, 11, 16, 17, 21, 23, 90, 92, 93, 94, 95, 96, 122, 124, 125, 133], "prematur": 2, "auxiliari": [2, 132], "diagnost": 2, "log": [2, 136, 141], "might": [2, 9, 11, 12, 16, 17, 18, 20, 23, 132, 134, 136, 139, 141], "instanc": [2, 4, 7, 8, 9, 10, 12, 13, 14, 15, 18, 21, 109, 134, 136], "describ": [2, 9, 10, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 93, 107, 111, 118, 130, 134], "perform": [2, 3, 7, 9, 14, 18, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 106, 109, 117, 122, 130, 141], "variabl": [2, 7, 9, 11, 15, 18, 98, 100, 107, 108, 109, 120, 132, 133, 139, 141], "hidden": [2, 97, 141], "individu": [2, 3, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 124, 132, 134, 141], "term": [2, 91, 92, 97, 98, 99, 100, 101, 105, 106, 107, 108, 132, 134, 136], "combin": [2, 9, 14, 16, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 106, 136], "produc": [2, 7, 16, 18, 102, 122, 132], "total": [2, 44, 88, 89, 90, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 114, 136, 141], "In": [2, 3, 7, 11, 14, 15, 16, 17, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 91, 92, 98, 99, 100, 101, 107, 108, 114, 124, 127, 130, 132, 133, 134, 136, 139, 141], "openai": [2, 8, 15, 16, 21, 23, 87, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 133, 139, 141], "gym": [2, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 87, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 132, 133, 134, 136, 139, 141], "v26": [2, 130, 141], "distinguish": [2, 132, 133], "howev": [2, 3, 7, 8, 9, 10, 11, 12, 15, 17, 18, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 105, 106, 122, 130, 132, 133, 134, 136, 139], "deprec": [2, 117, 119, 120, 121, 124, 130], "favour": [2, 22, 23, 124, 126, 130], "boolean": [2, 7, 9, 14, 18, 23, 90, 139, 141], "valu": [2, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 21, 23, 24, 74, 88, 90, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 122, 124, 127, 133, 136, 139, 141], "case": [2, 4, 7, 9, 10, 11, 17, 19, 21, 23, 57, 69, 89, 98, 99, 100, 101, 107, 108, 114, 132, 133, 134, 136, 141], "further": [2, 23, 74, 90, 105, 106, 124, 141], "undefin": 2, "signal": [2, 7, 16, 18, 21, 98, 100, 101, 108, 114, 132, 133], "mai": [2, 7, 9, 11, 12, 14, 15, 16, 17, 18, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 107, 108, 113, 132, 136], "emit": [2, 16, 18], "differ": [2, 9, 10, 14, 15, 17, 18, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 96, 98, 99, 100, 101, 102, 103, 104, 108, 109, 124, 128, 132, 133, 134, 136], "reason": [2, 7, 23, 90, 124], "mayb": 2, "underli": [2, 16, 18, 21, 92], "solv": [2, 88, 91, 93, 97, 103, 104, 132, 137, 140, 141], "successfulli": [2, 114], "certain": [2, 47, 49, 103, 104, 106, 124, 136, 139], "exceed": [2, 16, 18, 21, 98, 100, 101, 108, 139], "simul": [2, 24, 50, 51, 52, 53, 54, 97, 101, 102, 103, 104, 139, 141], "enter": [2, 14, 31, 33, 60, 90, 112], "invalid": [2, 11, 139], "int": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 24, 111, 112, 113, 114, 130, 132, 136, 139, 141], "none": [2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 23, 113, 124, 130, 132, 136, 139], "option": [2, 4, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 23, 89, 92, 93, 94, 95, 96, 119, 122, 124, 132], "intern": [2, 4, 15, 19, 127], "new": [2, 3, 7, 9, 16, 17, 18, 19, 21, 22, 23, 28, 31, 33, 46, 92, 93, 94, 95, 96, 98, 106, 109, 114, 118, 119, 122, 123, 124, 125, 126, 127, 130, 132, 133, 134, 136, 139], "often": [2, 9, 97, 132], "some": [2, 9, 10, 11, 15, 16, 18, 21, 23, 24, 40, 48, 66, 89, 91, 92, 97, 106, 107, 114, 127, 132, 134, 136, 139], "explor": [2, 9, 54, 109, 130, 136, 139], "generalis": 2, "polici": [2, 7, 19, 20, 21, 23, 91, 97, 98, 99, 100, 101, 102, 107, 108, 109, 115, 118, 133, 136], "control": [2, 9, 11, 21, 23, 24, 26, 29, 31, 32, 33, 34, 35, 40, 42, 45, 52, 53, 54, 57, 59, 66, 67, 69, 71, 72, 73, 74, 75, 76, 79, 80, 87, 89, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 117, 119, 122, 124, 130, 134, 136], "alreadi": [2, 3, 16, 19, 21, 130], "rng": [2, 4, 9, 10, 11, 119, 132], "therefor": [2, 3, 7, 9, 15, 23, 122, 126, 127, 130], "right": [2, 11, 24, 26, 28, 29, 30, 31, 35, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 57, 58, 59, 61, 63, 64, 65, 67, 70, 75, 77, 78, 80, 81, 84, 85, 88, 89, 90, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 107, 108, 112, 113, 114, 132, 139, 141], "after": [2, 4, 7, 8, 14, 15, 18, 19, 21, 32, 36, 41, 47, 51, 55, 77, 90, 98, 99, 100, 101, 102, 107, 108, 111, 114, 124, 132, 133, 136, 139, 141], "never": 2, "again": [2, 90, 132], "custom": [2, 7, 9, 14, 15, 16, 19, 21, 23, 24, 50, 51, 52, 53, 54, 107, 109, 113, 116, 126, 135, 137], "line": [2, 9, 23, 47, 92, 114, 117, 132], "correctli": [2, 7, 19, 20, 121, 122, 126, 132], "v0": [2, 3, 8, 14, 16, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 132, 133, 141], "25": [2, 12, 24, 50, 51, 52, 53, 54, 98, 100, 101, 102, 108, 114, 127, 130], "return_info": [2, 119, 124, 127], "now": [2, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 116, 117, 118, 119, 121, 122, 123, 124, 127, 129, 130, 132, 133, 139, 141], "expect": [2, 3, 9, 10, 11, 15, 18, 21, 117, 127, 130, 136, 141], "prng": [2, 9, 10, 11], "doe": [2, 7, 8, 10, 12, 14, 15, 18, 22, 36, 52, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 117, 130, 132, 133, 134, 136], "pass": [2, 4, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 90, 98, 99, 100, 101, 107, 108, 119, 122, 132, 134, 136], "chosen": [2, 11, 24, 136], "sourc": [2, 97, 109, 119, 132, 133, 134, 136, 137, 139, 141], "entropi": [2, 136], "timestamp": 2, "dev": [2, 124, 127, 141], "urandom": 2, "integ": [2, 7, 10, 11, 12, 14, 18, 19, 23, 24, 119, 126, 130, 132], "even": [2, 8, 25, 27, 33, 34, 36, 38, 41, 42, 45, 55, 56, 57, 59, 60, 62, 66, 69, 71, 72, 73, 74, 76, 79, 83, 86, 102, 128, 132, 139], "exist": [2, 8, 9, 15, 16, 21, 130, 136], "usual": [2, 7, 10, 11, 18, 24, 50, 51, 52, 53, 54, 56, 100, 132, 134, 136, 139], "been": [2, 14, 18, 21, 23, 98, 101, 102, 109, 118, 119, 122, 124, 127, 128, 130, 132, 133], "refer": [2, 14, 18, 21, 23, 132, 133], "minim": [2, 132, 133, 136, 141], "abov": [2, 3, 10, 11, 14, 22, 23, 24, 50, 51, 52, 53, 54, 92, 93, 96, 97, 130, 136, 141], "paradigm": 2, "specifi": [2, 7, 9, 10, 11, 14, 16, 17, 18, 19, 20, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 90, 91, 96, 100, 101, 105, 106, 108, 110, 113, 114, 124, 128, 129, 130, 132, 134], "how": [2, 9, 14, 15, 21, 56, 58, 59, 102, 103, 105, 106, 132, 133, 134, 136, 139, 141], "depend": [2, 7, 10, 14, 19, 20, 24, 26, 27, 28, 29, 30, 31, 32, 35, 36, 37, 39, 40, 42, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 59, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 87, 91, 93, 97, 98, 99, 100, 101, 107, 108, 109, 113, 116, 117, 121, 123, 126, 130, 136], "specif": [2, 7, 8, 11, 18, 126, 133, 136, 141], "analog": [2, 24], "dictionari": [2, 7, 9, 10, 14, 15, 16, 18, 19, 21, 23, 122, 126, 132, 134, 139, 141], "complement": 2, "It": [2, 7, 8, 10, 11, 14, 15, 18, 21, 24, 25, 26, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 82, 83, 84, 85, 88, 97, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 127, 130, 132, 139, 141], "renderfram": [2, 14], "list": [2, 3, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 24, 116, 118, 122, 124, 127, 130, 136, 139], "comput": [2, 14, 67, 79, 98, 103, 105, 106, 109, 126, 132, 136, 139], "frame": [2, 7, 14, 18, 19, 24, 37, 89, 90, 102, 105, 124, 127], "render_mod": [2, 7, 14, 15, 16, 18, 19, 21, 23, 24, 115, 124, 126, 127, 128, 129, 130, 132, 136], "wai": [2, 10, 16, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 118, 133, 134, 136, 139], "most": [2, 7, 9, 16, 18, 19, 21, 23, 24, 50, 51, 52, 53, 54, 106, 119, 122, 132, 134, 139], "achiev": [2, 24, 27, 92, 111, 127, 136], "appli": [2, 7, 8, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 88, 90, 91, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 122, 127, 132, 134, 139, 141], "collect": [2, 3, 7, 15, 18, 27, 30, 48, 62, 63, 66, 82, 109, 136, 139], "As": [2, 3, 15, 16, 18, 21, 80, 92, 97, 98, 99, 101, 102, 110, 112, 114, 119, 124, 126, 129, 130, 134, 136, 141], "known": [2, 18, 31], "__init__": [2, 10, 13, 16, 17, 19, 20, 130, 132, 134, 136, 139, 141], "initialis": [2, 7, 18, 23, 111], "By": [2, 7, 8, 18, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 107, 108], "continu": [2, 7, 9, 16, 17, 21, 29, 30, 89, 90, 91, 94, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 119, 122, 124, 133, 136, 139, 141], "current": [2, 3, 7, 11, 14, 16, 18, 19, 24, 89, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 119, 124, 130, 132, 136, 139, 141], "displai": [2, 8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 93, 121, 132], "consumpt": 2, "occur": [2, 7, 18, 92, 93, 132, 136], "repres": [2, 7, 10, 11, 14, 21, 23, 90, 92, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 112, 113, 114, 115, 119, 132, 141], "np": [2, 7, 8, 9, 10, 11, 13, 14, 15, 17, 19, 90, 114, 119, 130, 132, 134, 136, 139, 141], "ndarrai": [2, 4, 7, 8, 10, 11, 12, 13, 14, 17, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114, 136, 141], "shape": [2, 7, 9, 10, 11, 12, 13, 16, 17, 18, 19, 21, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 117, 124, 130, 132, 134, 136, 141], "x": [2, 9, 12, 90, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 109, 112, 130, 132, 136, 141], "y": [2, 24, 90, 96, 98, 99, 101, 102, 103, 105, 106, 107, 114, 141], "3": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 119, 122, 126, 132, 136, 139, 141], "rgb": [2, 7, 16, 18, 19, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 98, 99, 100, 101, 102, 103, 104, 107, 108, 125, 127], "pixel": [2, 7, 16, 19, 89, 109, 132], "imag": [2, 7, 9, 15, 16, 18, 19, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89], "string": [2, 7, 8, 9, 11, 14, 19, 122], "stringio": 2, "style": [2, 15, 18, 109, 122, 124, 141], "represent": [2, 21, 133], "each": [2, 3, 7, 8, 9, 10, 11, 14, 15, 16, 17, 19, 21, 24, 44, 46, 47, 48, 49, 52, 60, 67, 69, 70, 71, 72, 73, 74, 75, 87, 88, 90, 91, 92, 93, 94, 95, 98, 101, 102, 103, 104, 105, 109, 110, 112, 114, 127, 132, 136, 139], "includ": [2, 7, 8, 19, 21, 22, 23, 24, 93, 98, 99, 100, 101, 103, 104, 105, 106, 107, 108, 109, 114, 117, 122, 130, 132, 133], "newlin": 2, "escap": [2, 43, 58, 62], "sequenc": [2, 7, 8, 9, 11, 12, 15, 19, 21, 24, 109, 127, 130], "color": [2, 10, 19, 28, 39, 52, 70, 114], "rgb_array_list": [2, 7, 14, 16, 18, 127], "ansi_list": 2, "base": [2, 3, 4, 7, 10, 15, 16, 18, 21, 48, 57, 74, 87, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 114, 116, 127, 130, 132, 134, 136, 139, 141], "except": [2, 71, 74, 122, 128, 130, 132], "rendercollect": [2, 3, 7, 16, 18, 130], "pop": [2, 106, 124], "kei": [2, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19, 21, 23, 24, 25, 56, 98, 99, 100, 101, 105, 106, 107, 108, 112, 113, 114, 123, 126, 132, 133, 134, 139], "support": [2, 3, 7, 9, 10, 12, 14, 15, 16, 18, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 119, 121, 122, 123, 124, 125, 127, 130, 132], "longer": [2, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 100, 101, 102, 103, 104, 105, 106, 108, 109, 118, 119, 127, 141], "accept": [2, 7, 14, 18, 24, 118, 119, 130, 132], "rather": [2, 23, 89, 109, 136, 139], "v1": [2, 7, 8, 14, 15, 18, 19, 20, 21, 88, 89, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 113, 114, 116, 117, 118, 122, 127, 139], "type": [2, 7, 9, 10, 11, 15, 16, 17, 19, 21, 23, 46, 89, 98, 99, 100, 101, 107, 108, 109, 119, 121, 124, 126, 127, 130, 132, 134, 141], "discret": [2, 7, 9, 10, 12, 13, 15, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 90, 92, 93, 94, 95, 110, 111, 112, 113, 114, 122, 124, 130, 132, 134, 139], "give": [2, 27, 28, 46, 47, 90, 105, 111, 114, 118, 136, 139, 141], "2": [2, 7, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 24, 25, 26, 28, 29, 30, 31, 32, 34, 35, 37, 39, 40, 42, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 79, 80, 81, 82, 84, 85, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 122, 127, 132, 134, 136, 139, 141], "mean": [2, 7, 21, 23, 92, 98, 99, 100, 105, 127, 132, 136, 141], "two": [2, 7, 9, 10, 11, 14, 15, 18, 19, 23, 24, 26, 37, 79, 88, 90, 92, 94, 95, 97, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 117, 127, 132, 136, 139], "1": [2, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 119, 122, 124, 126, 127, 132, 133, 134, 136, 139, 141], "box": [2, 7, 9, 10, 12, 13, 15, 16, 17, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 117, 118, 123, 124, 130, 132, 134], "4028234663852886e": 2, "38": [2, 7, 101, 102], "4": [2, 7, 11, 12, 14, 15, 18, 19, 21, 24, 26, 28, 29, 30, 31, 33, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 58, 61, 63, 64, 65, 67, 68, 69, 70, 77, 78, 80, 81, 82, 84, 85, 88, 89, 90, 92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 127, 132, 134, 139, 141], "float32": [2, 7, 10, 11, 12, 13, 15, 16, 17, 19, 21, 88, 89, 90, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "denot": [2, 93], "well": [2, 9, 31, 74, 92, 98, 101, 102, 109, 126, 132, 136], "high": [2, 3, 10, 11, 12, 13, 15, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 124, 130, 134, 136], "8000002e": [2, 7], "00": [2, 7], "4028235e": [2, 7], "1887903e": [2, 7], "01": [2, 7, 20, 90, 98, 99, 102, 103, 104, 105, 106, 107, 130, 136, 139], "dtype": [2, 7, 9, 10, 11, 12, 13, 15, 19, 90, 117, 130, 132], "low": [2, 10, 11, 12, 13, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 124, 134, 136], "etc": [2, 9, 69, 72, 76, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 117, 130, 132, 133, 134], "determin": [2, 11, 24, 90, 92, 93, 94, 95, 96, 109, 113, 127, 132, 133], "inf": [2, 7, 93, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 134], "envspec": [2, 8, 16], "normal": [2, 7, 11, 16, 18, 19, 20, 88, 89, 96, 98, 99, 103, 127, 130, 136, 141], "finish": [2, 89, 90, 124, 136], "code": [2, 4, 7, 9, 10, 14, 15, 16, 18, 19, 20, 92, 109, 116, 119, 120, 124, 130, 132, 134, 136, 137, 139, 141], "clean": 2, "up": [2, 10, 11, 15, 19, 24, 25, 27, 28, 29, 30, 31, 33, 34, 35, 37, 40, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 57, 58, 61, 63, 68, 70, 71, 72, 74, 75, 78, 79, 80, 81, 84, 85, 88, 90, 97, 98, 101, 102, 106, 111, 112, 113, 114, 130, 132, 136, 139], "window": [2, 14, 44, 89, 121, 127, 130, 132], "http": [2, 14, 24, 89, 92, 111, 112, 113, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 132, 136, 139, 141], "connect": [2, 10, 92, 98, 99, 100, 106, 107, 108], "properti": [2, 7, 9, 16, 117, 118, 123, 130, 133], "unwrap": [2, 16, 21, 92], "non": [2, 7, 9, 12, 14, 113, 122, 132, 133, 141], "wrap": [2, 7, 8, 14, 15, 16, 18, 19, 20, 21, 132, 134, 141], "_np_random": [2, 123], "meth": [2, 7], "much": [2, 24, 50, 51, 52, 53, 54, 58, 59, 102, 118, 127, 130, 132, 133], "creat": [2, 3, 7, 8, 11, 13, 14, 15, 18, 21, 34, 88, 89, 90, 105, 106, 130, 134, 136, 139, 141], "creation": [2, 15, 130, 132], "tutori": [2, 16, 21, 23, 130, 134, 136, 139, 141], "gymnasium": [3, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 130, 133, 136, 139, 141], "env": [3, 4, 7, 8, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 112, 113, 114, 115, 116, 117, 119, 120, 122, 123, 124, 127, 128, 129, 130, 133, 134, 135, 137, 139, 141], "flexibl": [3, 15, 109, 130], "implement": [3, 7, 9, 10, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 96, 103, 109, 110, 114, 124, 126, 130, 132, 135, 136, 137, 141], "complic": [3, 11, 16, 134], "parallel": [3, 15, 136], "propos": [3, 7, 109, 130], "funcenv": [3, 130], "part": [3, 18, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 119, 127, 130, 133], "its": [3, 7, 14, 15, 17, 18, 39, 46, 69, 81, 86, 90, 96, 97, 107, 109, 141], "own": [3, 15, 16, 23, 39, 74, 134, 135, 137], "relat": [3, 141], "larg": [3, 15, 23, 95, 98, 99, 100, 101, 102, 105, 106, 107, 108, 109, 116, 117, 119, 124, 130, 134], "believ": [3, 130], "improv": [3, 15, 117, 119, 122, 127, 130], "work": [3, 9, 14, 16, 18, 19, 24, 89, 90, 92, 99, 100, 103, 104, 108, 116, 118, 124, 130], "progress": [3, 10, 80, 130, 136], "arbitrarili": [3, 130], "complex": [3, 9, 10, 101, 102, 103, 104, 130, 136, 141], "rl": [3, 7, 14, 18, 115, 130, 133, 139, 141], "advanc": [3, 15, 92, 130], "becom": [3, 73, 87, 119, 124, 130, 133, 136], "were": [3, 19, 23, 41, 87, 92, 119, 122, 124, 128, 130, 132, 136], "mind": [3, 130], "jax": [3, 7, 109, 130], "With": [3, 112, 130, 136, 139], "hardwar": [3, 122, 130], "acceler": [3, 89, 94, 95, 96, 122, 130, 136], "brax": [3, 109, 130], "written": [3, 14, 16, 23, 130], "similar": [3, 10, 23, 28, 39, 105, 119, 130, 132], "pytorch": [3, 7, 130, 141], "program": [3, 15, 130], "onli": [3, 7, 8, 9, 10, 11, 12, 15, 16, 18, 19, 23, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 58, 61, 63, 64, 65, 67, 68, 70, 71, 72, 73, 74, 75, 77, 78, 80, 81, 82, 84, 85, 94, 95, 98, 99, 100, 101, 103, 105, 106, 108, 109, 122, 124, 125, 126, 127, 128, 130, 132, 134, 136, 139], "game": [3, 7, 11, 14, 18, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 87, 111, 112, 113, 117, 118, 130, 132, 139], "town": [3, 130], "anymor": [3, 7, 130], "upgrad": [3, 118, 127, 130], "jumpi": [3, 130], "project": [3, 21, 23, 109, 119, 124, 127, 130, 139], "develop": [3, 97, 124, 130, 141], "farama": [3, 14, 109, 130, 132, 136, 139, 141], "foundat": [3, 109, 130, 132], "compat": [3, 16, 19, 21, 109, 115, 119, 121, 122, 124, 125, 126, 130], "futur": [3, 4, 80, 109, 119, 122, 124, 130], "subset": [3, 11, 24, 50, 51, 52, 53, 54, 130, 132, 134], "like": [3, 4, 7, 9, 11, 14, 16, 17, 18, 19, 20, 21, 26, 27, 28, 29, 30, 31, 32, 33, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 66, 67, 68, 69, 70, 75, 77, 78, 80, 81, 82, 84, 85, 88, 90, 98, 99, 101, 102, 104, 105, 106, 108, 114, 119, 124, 127, 130, 132, 134, 136], "supersuit": [3, 130], "aim": [3, 56, 100, 103, 107, 108, 124, 127, 130, 141], "bring": [3, 130], "mani": [3, 7, 11, 18, 19, 21, 24, 30, 33, 37, 56, 59, 74, 84, 117, 120, 130, 132, 139], "awar": [3, 23, 130], "plan": [3, 109, 119, 124, 127, 130], "miss": [3, 21, 75, 92, 114, 118, 130], "issu": [3, 18, 21, 23, 24, 98, 100, 101, 102, 108, 116, 118, 119, 121, 124, 130, 132], "detail": [3, 9, 14, 18, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 109, 122, 124, 127, 130, 132, 136], "caus": [3, 18, 21, 57, 114, 116, 125, 126, 129, 130], "ad": [3, 7, 16, 18, 19, 21, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 52, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 114, 116, 117, 118, 119, 122, 123, 124, 125, 127, 130, 132], "lambaactionv0": [3, 130], "don": [3, 10, 16, 23, 24, 40, 50, 51, 52, 53, 54, 74, 89, 94, 124, 130, 132, 133, 136], "regularli": [3, 130], "signific": [3, 22, 130], "affect": [3, 130], "addition": [3, 101, 102, 127, 130], "hope": [3, 130], "reproduc": [3, 10, 23, 130], "academia": [3, 130], "v28": [3, 130], "rewrit": [3, 123, 130], "vectorenv": [3, 9, 116, 122, 130], "inherit": [3, 9, 16, 17, 19, 20, 21, 117, 130, 132], "replac": [3, 16, 23, 111, 117, 119, 123, 124, 130, 139], "30": [3, 7, 14, 18, 24, 68, 71, 101, 102, 130], "old": [3, 7, 15, 16, 18, 22, 23, 115, 118, 119, 122, 125, 126, 127], "transformobserv": [3, 7, 16, 19], "lambdaobservationv0": [3, 7], "filterobserv": [3, 16, 19], "filterobservationv0": [3, 7], "flattenobserv": [3, 10, 16, 19, 132], "flattenobservationv0": [3, 7], "grayscaleobserv": [3, 16, 19], "grayscaleobservationv0": [3, 7], "resizeobserv": [3, 16, 19, 130], "resizeobservationv0": [3, 7], "reshape_v0": 3, "reshapeobservationv0": [3, 7], "Not": [3, 7, 18], "rescaleobservationv0": [3, 7], "dtype_v0": 3, "dtypeobservationv0": [3, 7], "pixelobservationwrapp": [3, 16, 19, 125, 128], "pixelobservationv0": [3, 7], "normalizeobserv": [3, 16, 19, 117], "normalizeobservationv0": [3, 7], "timeawareobserv": [3, 16, 19, 21], "timeawareobservationv0": [3, 7], "framestack": [3, 7, 16, 18, 19, 117, 125], "framestackobservationv0": [3, 7], "delay_observations_v0": 3, "delayobservationv0": [3, 7], "action_lambda_v1": 3, "lambdaactionv0": [3, 7], "clipact": [3, 16, 17, 21], "clipactionv0": [3, 7], "rescaleact": [3, 7, 16, 17, 21], "rescaleactionv0": [3, 7], "sticky_actions_v0": 3, "stickyactionv0": [3, 7], "transformreward": [3, 16, 20], "lambdarewardv0": [3, 7], "clip_reward_v0": 3, "cliprewardv0": [3, 7], "normalizereward": [3, 16, 20, 117], "normalizerewardv0": [3, 7], "autoresetwrapp": [3, 8, 16, 18, 122, 132], "autoresetv0": [3, 7], "passiveenvcheck": [3, 16, 18, 21, 130], "passiveenvcheckerv0": [3, 7], "orderenforc": [3, 7, 16, 18, 21, 122, 132], "orderenforcingv0": [3, 7], "envcompat": [3, 16, 18, 22, 23, 127], "shimmi": [3, 130], "recordepisodestatist": [3, 16, 18, 23, 117, 136, 139, 141], "recordepisodestatisticsv0": [3, 7], "ataripreprocess": [3, 16, 18], "ataripreprocessingv0": [3, 7], "recordvideo": [3, 16, 18, 23, 117, 124, 127], "recordvideov0": [3, 7, 130], "humanrend": [3, 7, 16, 18, 130], "humanrenderingv0": [3, 7], "rendercollectionv0": [3, 7], "jaxtonumpyv0": [3, 7], "jaxtotorchv0": [3, 7], "numpytotorchv0": [3, 7], "These": [3, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 97, 98, 101, 107, 122, 127, 130, 139, 141], "28": [3, 24, 33, 39, 92, 101, 102], "templat": 4, "meant": 4, "stateless": [4, 130], "manner": [4, 7, 11, 16, 19, 56, 117, 118], "being": [4, 7, 8, 12, 18, 45, 94, 95, 96, 98, 101, 102, 106, 119, 122, 124, 127, 139], "around": [4, 18, 34, 87, 106, 109, 121], "explicitli": [4, 132, 133], "That": [4, 93], "said": [4, 98, 100, 101, 108], "noth": [4, 89, 90, 141], "here": [4, 14, 92, 119, 127, 130, 132, 136], "prevent": [4, 11, 15, 47, 109, 117, 118], "statefulli": 4, "just": [4, 16, 21, 50, 56, 73, 104, 106, 127, 129, 130, 136], "recommend": [4, 7, 8, 15, 18, 23, 24, 50, 51, 52, 53, 54, 90, 98, 101, 102, 117, 124, 126, 132, 136, 139], "consist": [4, 7, 11, 19, 21, 24, 37, 88, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 117, 128, 132, 134, 139, 141], "method": [4, 7, 10, 14, 17, 18, 19, 20, 22, 117, 118, 119, 122, 123, 132, 134, 136, 141], "pomdp": 4, "given": [4, 7, 8, 11, 16, 19, 71, 74, 88, 90, 91, 92, 94, 95, 98, 100, 101, 108, 127, 133, 139], "transit": [4, 14, 21, 24, 50, 51, 52, 53, 54, 101, 102, 112, 113, 114, 133, 136], "next_stat": [4, 139], "state_info": 4, "step_info": 4, "structur": [4, 9, 10, 12, 130, 132], "serv": [4, 9, 98, 99, 100, 101, 107, 108, 141], "purpos": [4, 9, 119, 141], "allow": [4, 7, 8, 9, 10, 14, 16, 21, 22, 23, 24, 55, 73, 92, 93, 94, 95, 96, 103, 104, 105, 106, 109, 122, 124, 126, 127, 128, 130, 132, 134, 136, 141], "constant": [4, 35, 74, 90, 95, 98, 100, 101, 108], "them": [4, 7, 9, 16, 18, 22, 23, 24, 26, 27, 28, 31, 32, 36, 40, 47, 68, 77, 90, 98, 99, 106, 114, 124, 126, 134, 136], "itself": [4, 12, 15, 16, 18, 21, 22, 127, 136], "moment": [4, 74], "predominantli": 4, "intend": [4, 7, 18, 113], "flesh": 4, "offici": 4, "expos": 4, "statetyp": 4, "rewardtyp": 4, "terminaltyp": 4, "full": [4, 21, 24, 50, 51, 52, 53, 54, 89, 90, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 139], "transform": [4, 7, 9, 12, 14, 16, 17, 18, 19, 20, 106, 134], "func": [4, 7], "callabl": [4, 7, 8, 13, 14, 15, 18, 19, 20, 130], "render_imag": 4, "render_st": 4, "renderstatetyp": 4, "show": [4, 19, 23, 24, 34, 40, 111, 122, 134, 136, 139, 141], "render_init": 4, "kwarg": [4, 8, 14, 15, 22, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 132], "render_clos": 4, "experiment": [7, 92, 130], "via": [7, 9, 10, 16, 18, 19, 20, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 91, 97, 109, 110, 132, 141], "lambda": [7, 15, 19, 20, 136, 139], "ob": [7, 14, 17, 18, 19, 114, 119, 124, 127, 129, 130, 134, 139, 141], "42": [7, 10, 11, 13, 15, 17, 19, 93, 101, 102, 115, 130, 136], "06199517": 7, "0511615": 7, "04432538": 7, "02694618": 7, "constructor": [7, 8, 9, 10, 11, 14, 16, 17, 19, 20, 21, 127, 132], "assum": [7, 11, 19, 132], "same": [7, 10, 12, 14, 15, 19, 56, 88, 89, 92, 105, 106, 107, 114, 122, 127, 133, 136, 139], "filter_kei": [7, 19], "filter": [7, 16, 19], "0273956": [7, 15, 19], "00611216": [7, 15, 19], "03585979": [7, 15, 19], "0197368": [7, 15, 19], "fals": [7, 8, 14, 15, 18, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 90, 98, 99, 100, 101, 107, 108, 111, 119, 122, 124, 132, 136, 139, 141], "flatten": [7, 9, 12, 16, 19, 97, 117, 126, 130, 132, 136, 139], "carrac": [7, 8, 14, 19, 89, 122, 124, 126, 128], "v2": [7, 8, 14, 18, 19, 21, 23, 88, 89, 90, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114, 115, 124, 130, 134, 136], "96": [7, 19, 24, 89], "27648": [7, 19], "_": [7, 13, 14, 15, 17, 18, 19, 20, 21, 115, 134, 136], "util": [7, 21, 22, 117, 124, 132, 136], "flatten_spac": [7, 9, 12], "keep_dim": [7, 19], "grayscal": [7, 18, 19, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "keep": [7, 14, 16, 18, 32, 45, 64, 67, 75, 76, 84, 93, 102, 132], "channel": [7, 18], "dimens": [7, 9, 11, 12, 19, 98, 99, 100, 101, 107, 108, 109, 141], "grayscale_env": 7, "resiz": [7, 16, 18, 19, 130], "opencv": [7, 18, 124], "resized_env": 7, "32": [7, 24, 39, 56, 101, 102, 111, 136, 141], "reshap": 7, "reshape_env": 7, "24": [7, 24, 33, 39, 64, 88, 89, 90, 93, 98, 101, 102, 124, 130, 139], "product": [7, 10, 11], "equal": [7, 94, 95, 103, 111, 113], "min_ob": 7, "float": [7, 8, 11, 12, 14, 16, 17, 19, 20, 24, 55, 68, 90, 98, 99, 100, 101, 107, 108, 132, 136, 139, 141], "max_ob": 7, "linearli": [7, 92, 103, 104, 141], "rescal": [7, 16, 17, 21], "between": [7, 10, 11, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 124, 132, 133, 136, 139, 141], "pendulum": [7, 8, 15, 19, 21, 91, 92, 93, 97, 117, 118, 128, 141], "8": [7, 14, 15, 19, 20, 22, 24, 26, 28, 30, 31, 33, 35, 39, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 80, 84, 85, 89, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 113, 127, 136, 141], "10": [7, 10, 11, 24, 31, 37, 50, 51, 52, 53, 54, 61, 68, 71, 74, 88, 90, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 112, 114, 119, 130, 132, 136, 139, 141], "multidiscret": [7, 9, 10, 12, 13, 15, 21, 116, 117, 124, 130, 132], "multibinari": [7, 9, 10, 12, 13, 21], "pixels_onli": [7, 19], "pixels_kei": 7, "obs_kei": 7, "augment": [7, 16, 19], "choos": [7, 19, 132], "origin": [7, 12, 15, 17, 18, 19, 23, 89, 92, 103, 105, 106, 122], "discard": [7, 19], "both": [7, 11, 15, 18, 19, 21, 22, 88, 91, 92, 106, 108, 109, 111, 112, 113, 114, 125, 126, 132, 133, 141], "epsilon": [7, 19, 20, 139], "1e": [7, 19, 20, 101, 107, 108, 141], "coordin": [7, 11, 16, 19, 88, 90, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "center": [7, 16, 19, 89, 90, 93, 96, 98, 99, 101, 102], "unit": [7, 9, 16, 19, 58, 92, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "varianc": [7, 16, 19, 20, 136], "past": [7, 19, 20, 124], "trajectori": [7, 16, 19, 20, 90, 101, 102], "newli": [7, 19, 20], "instanti": [7, 10, 14, 18, 19, 20, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 132], "recent": [7, 18, 19, 20, 22], "stabil": [7, 19, 20, 101, 102, 127, 134, 141], "scale": [7, 16, 18, 19, 20, 24, 90, 98, 99, 100, 101, 107, 108, 109, 136], "normalize_tim": 7, "dict_time_kei": 7, "remain": [7, 55, 58, 59, 74, 112, 116, 117, 132, 133], "final": [7, 14, 15, 18, 57, 124, 129, 130, 132, 133, 139], "sampl": [7, 9, 10, 11, 12, 13, 14, 15, 19, 20, 21, 23, 90, 114, 115, 124, 127, 130, 132, 136, 139, 141], "02727336": [7, 15, 19], "20172954": [7, 19], "03625453": [7, 15, 19], "32351476": [7, 19], "002": [7, 100, 108], "0000000e": 7, "5": [7, 10, 11, 12, 13, 17, 24, 26, 28, 29, 30, 31, 34, 35, 37, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 77, 78, 80, 81, 82, 84, 85, 88, 89, 90, 92, 94, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 111, 114, 122, 127, 132, 136, 139, 141], "stack_siz": 7, "stack": [7, 16, 19, 130], "roll": [7, 16, 19, 37, 105, 139], "process": [7, 13, 14, 15, 19, 21, 92, 105, 132, 133, 136], "buffer": [7, 18, 19, 89], "fill": [7, 19, 33, 132], "num_stack": [7, 19], "ident": [7, 11, 15, 19, 130], "255": [7, 18, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 89, 130, 132], "uint8": [7, 19, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "delai": [7, 132], "delayobserv": 7, "zero": [7, 11, 13, 96, 99, 109, 132, 133, 136, 139, 141], "noop_max": [7, 18], "frame_skip": [7, 18, 98, 99, 100, 101, 107, 108], "screen_siz": [7, 18], "84": [7, 18, 98, 101, 102], "terminal_on_life_loss": [7, 18], "grayscale_ob": [7, 18], "grayscale_newaxi": [7, 18], "scale_ob": [7, 18], "atari": [7, 16, 18, 21, 22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 116, 117, 118, 119, 127, 129, 130, 139], "2600": [7, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "preprocess": [7, 14, 16, 18, 129, 130], "guidelin": [7, 18], "machado": [7, 18, 24], "et": [7, 18, 24], "al": [7, 18, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 116, 117, 118, 122, 127, 130, 139], "2018": [7, 18, 24, 92], "revisit": [7, 18, 24], "arcad": [7, 18, 24, 31, 48, 50, 51, 52, 53, 54, 109], "evalu": [7, 18, 24, 139], "protocol": [7, 18, 24], "open": [7, 18, 24, 37, 97, 109, 127, 130, 132], "problem": [7, 18, 24, 90, 93, 96, 103, 104, 107, 109, 111, 114, 115, 133, 136], "stage": [7, 18, 130], "noop": [7, 11, 14, 18, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 114, 136], "obtain": [7, 16, 18, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 96, 111, 133, 141], "op": [7, 18], "max": [7, 11, 17, 18, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 113, 136, 139, 141], "skip": [7, 14, 18, 24, 125, 132], "pool": [7, 18, 107], "life": [7, 18, 30, 44, 48, 72, 76, 119, 141], "lost": [7, 18, 33, 41, 60, 73, 77], "loss": [7, 18, 111, 136, 141], "turn": [7, 15, 18, 23, 27, 28, 89, 90, 103, 109, 122, 124, 127], "off": [7, 18, 23, 41, 56, 64, 72, 89, 90, 112, 114, 122, 127], "squar": [7, 18, 19, 73, 105, 106, 114, 132, 136], "210x180": [7, 18], "84x84": [7, 18], "colour": [7, 18, 89, 128], "greyscal": [7, 18], "No": [7, 18, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 127, 141], "taken": [7, 15, 18, 48, 54, 91, 93, 132, 136, 141], "effect": [7, 12, 13, 16, 18, 19, 21, 24, 50, 51, 52, 53, 54, 90, 103, 104, 114, 139, 141], "frequenc": [7, 18], "experi": [7, 18, 98, 101, 102, 103, 104, 136, 139, 141], "whenev": [7, 18, 23, 30, 33, 132], "grai": [7, 16, 18, 19], "axi": [7, 18, 94, 95, 99, 101, 102, 107, 139], "dimension": [7, 9, 18, 19, 21, 28, 90, 98, 99, 100, 106, 107, 108, 130, 132, 136], "limit": [7, 8, 18, 21, 23, 40, 48, 55, 56, 69, 74, 103, 104, 130, 135, 136, 137, 139, 141], "memori": [7, 18, 19, 94, 95, 126], "optim": [7, 18, 24, 88, 90, 101, 102, 109, 136, 139, 141], "benefit": [7, 18], "rais": [7, 8, 11, 12, 13, 14, 15, 18, 19, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 125, 128, 130], "dependencynotinstal": [7, 14, 18], "valueerror": [7, 11, 12, 13, 15, 18, 19], "disabl": [7, 8, 18, 113, 122, 123, 124], "wrapperacttyp": [7, 16, 17], "lambdaact": 7, "clip": [7, 16, 17, 21, 94, 95, 98, 101, 132, 134, 136], "hopper": [7, 8, 16, 17, 97, 108], "v4": [7, 8, 16, 17, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 122, 123, 130, 141], "disable_env_check": [7, 8, 15, 122], "min_act": [7, 16, 17, 21], "max_act": [7, 16, 17, 21], "affin": [7, 17, 90], "must": [7, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 21, 23, 24, 25, 26, 29, 31, 32, 33, 34, 36, 40, 41, 42, 45, 48, 49, 58, 62, 64, 65, 70, 89, 100, 108, 126, 132, 133], "match": [7, 12, 15, 17, 79], "75": [7, 17, 98], "wrapped_env": [7, 16, 17, 18, 21, 132, 134, 141], "wrapped_env_ob": [7, 17], "alltru": [7, 17], "min": [7, 11, 17, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111], "scalar": [7, 11, 17, 124], "repeat_action_prob": [7, 24, 50, 51, 52, 53, 54], "probabl": [7, 9, 24, 112, 113, 114, 133, 136, 139, 141], "repeat": [7, 24, 105], "previou": [7, 18, 23, 98, 101, 102, 133, 134], "section": [7, 23, 24, 75, 98, 99, 100, 101, 107, 108, 132, 139], "page": [7, 16, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 130], "12": [7, 12, 24, 31, 33, 37, 39, 50, 51, 52, 53, 54, 58, 61, 73, 90, 92, 93, 98, 99, 100, 101, 102, 105, 108, 111, 112, 130, 136, 139], "stickyact": 7, "rew": [7, 14, 133], "min_reward": [7, 134], "max_reward": [7, 134], "upper": [7, 11, 21, 101, 102, 122], "lower": [7, 11, 21, 98, 99, 100, 101, 102, 105, 106, 107, 108, 122, 126, 136, 139], "cliprewardsv0": 7, "union": [7, 10, 11, 12, 13, 14, 15, 17], "higher": [7, 88, 136], "gamma": [7, 20, 133, 136, 141], "99": [7, 20, 136, 141], "immedi": [7, 14, 16, 20, 111, 132], "exponenti": [7, 11, 16, 20], "averag": [7, 16, 20, 101, 133, 139, 141], "fix": [7, 9, 10, 11, 16, 18, 20, 21, 23, 57, 92, 93, 96, 97, 98, 99, 100, 101, 103, 104, 108, 111, 113, 118, 119, 120, 121, 122, 127, 130, 132, 141], "_update_running_mean": 7, "freez": 7, "calcul": [7, 96, 97, 98, 101, 102, 103, 105, 106, 112, 113, 114, 136], "statist": [7, 14, 18, 141], "runningmeanstd": 7, "get": [7, 16, 21, 24, 44, 47, 48, 49, 56, 66, 67, 68, 69, 71, 72, 73, 78, 88, 90, 98, 100, 101, 103, 111, 124, 130, 133, 136, 139], "discount": [7, 20, 133, 136, 139, 141], "factor": [7, 20, 136, 139, 141], "passiv": [7, 18, 125], "checker": [7, 8, 15, 18, 123, 125, 130], "surround": [7, 18], "thei": [7, 9, 15, 18, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 106, 110, 111, 112, 113, 132, 136], "test": [7, 18, 88, 109, 116, 118, 122, 123, 125, 130, 132, 141], "disable_render_order_enforc": [7, 18, 122], "error": [7, 8, 11, 12, 14, 16, 18, 118, 119, 121, 122, 130, 133, 136, 139], "resetneed": [7, 18], "cannot": [7, 12, 13, 18, 21, 23, 112, 126, 132, 133], "order": [7, 8, 9, 10, 16, 18, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 97, 98, 99, 100, 101, 102, 103, 104, 105, 108, 109, 122, 126, 130, 132], "enforc": [7, 8, 18, 118, 122], "buffer_length": 7, "100": [7, 10, 12, 18, 38, 48, 71, 72, 88, 89, 90, 92, 95, 100, 105, 112, 113], "stats_kei": 7, "track": [7, 14, 16, 18, 76, 89, 93, 98, 99, 100, 101, 102, 103, 104, 107, 108], "cumul": [7, 14, 16, 18], "length": [7, 8, 10, 11, 14, 16, 18, 19, 88, 92, 93, 94, 95, 99, 107, 109, 113, 114, 124, 136, 139], "At": [7, 18, 21, 33, 44, 47, 58], "vector": [7, 11, 14, 16, 18, 88, 90, 102, 103, 104, 105, 106, 119, 122, 127, 129, 130, 135, 137, 141], "_episod": [7, 18], "respect": [7, 11, 15, 16, 18, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 92, 98, 101, 102, 107, 119, 134, 136], "look": [7, 9, 14, 16, 18, 21, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 73, 75, 77, 78, 80, 81, 82, 84, 85, 132, 133, 134, 136], "l": [7, 18, 107], "elaps": [7, 18], "sinc": [7, 18, 22, 24, 50, 51, 52, 53, 54, 87, 93, 97, 106, 109, 114, 116, 124, 132, 133], "begin": [7, 18, 23, 30, 33, 46, 71, 72, 73, 74, 127, 132, 136, 139], "form": [7, 9, 10, 11, 18, 92, 107, 130, 132], "final_observ": [7, 15, 18, 129, 130], "num": [7, 18, 24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "_final_observ": [7, 18], "final_info": [7, 15, 18, 129, 130], "_final_info": [7, 18], "moreov": [7, 9, 16, 18, 24, 27, 37, 132, 134], "store": [7, 14, 15, 18, 141], "access": [7, 16, 18, 23, 55, 123, 130, 134], "return_queu": [7, 18, 136, 139, 141], "length_queu": [7, 18, 139], "episode_reward_buff": 7, "last": [7, 14, 18, 46, 47, 57, 99, 102, 105, 106, 118, 124, 127, 132, 133], "deque_s": [7, 18, 136, 139], "episode_length_buff": 7, "size": [7, 9, 18, 113, 128, 132], "video_fold": [7, 14, 18], "episode_trigg": [7, 14, 18], "step_trigg": [7, 14, 18], "video_length": [7, 14, 18], "name_prefix": [7, 14, 18], "disable_logg": [7, 18], "record": [7, 14, 16, 18, 88, 116, 117, 127, 136, 141], "rollout": [7, 16, 18, 114], "intermitt": [7, 18], "sai": [7, 18, 127, 134], "hundredth": [7, 18], "do": [7, 16, 18, 19, 24, 37, 49, 70, 74, 75, 89, 90, 92, 98, 99, 100, 101, 107, 108, 127, 130, 132, 134, 136, 141], "neither": [7, 11, 18, 111], "nor": [7, 11, 18, 111, 112], "emploi": [7, 18], "capped_cubic_video_schedul": [7, 14], "power": [7, 27, 55, 89, 90, 95, 103, 104, 141], "until": [7, 15, 16, 47, 60, 105, 111, 112, 113, 119, 132, 136, 141], "1000": [7, 14, 18, 21, 72, 74, 89, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 115, 136, 141], "stop": [7, 18, 21, 68, 78, 86, 111, 133, 139], "possibli": [7, 9, 10, 11, 13, 16, 18, 21, 109, 136], "span": [7, 18], "sever": [7, 8, 11, 18, 21, 23, 24, 54, 109, 125, 127, 130], "strictli": [7, 18], "per": [7, 11, 15, 37, 56, 79, 114, 119, 133, 136], "iff": [7, 14, 18, 132], "entir": [7, 14, 18, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 100, 108, 109, 118, 130, 136], "snippet": [7, 14, 18], "captur": [7, 14, 18, 83], "Will": [7, 14, 18], "prepend": [7, 14, 18], "filenam": [7, 14, 18], "moviepi": [7, 14, 18, 124, 127], "logger": [7, 18], "particularli": [7, 14, 18, 93], "haven": [7, 18], "screen": [7, 18, 24, 26, 30, 34, 39, 40, 42, 51, 57, 63, 73, 74, 98, 99, 100, 101, 102, 103, 104, 107, 108, 122], "rememb": [7, 15, 18, 89, 132, 134], "render_fp": [7, 14, 18, 132], "either": [7, 10, 11, 12, 14, 18, 19, 21, 22, 24, 47, 57, 94, 95, 98, 111, 113, 136, 141], "lunarland": [7, 8, 18, 21, 23, 90, 115, 130, 136], "simpli": [7, 17, 18, 19, 20, 24, 132], "nativ": [7, 18], "cartpolejax": [7, 18], "warn": [7, 10, 14, 15, 18, 116, 118, 121, 123, 125, 129, 130], "alwai": [7, 11, 12, 18, 24, 90, 94, 95, 98, 101, 102, 105, 107, 111, 112, 113, 114, 130, 132], "empti": [7, 11, 13, 18, 74, 139, 141], "pop_fram": [7, 18], "reset_clean": [7, 18], "renderedfram": 7, "clear": [7, 18, 78], "interact": [7, 9, 14, 24, 109, 132, 136, 139], "guarante": [7, 10, 24], "roundtrip": 7, "vice": [7, 13, 18], "versa": [7, 13, 18], "int_32": 7, "devicearrai": 7, "jnp": 7, "int23": 7, "input": [7, 14, 15, 19, 21, 136, 139], "devic": [7, 136], "tensor": [7, 136, 141], "torch": [7, 136, 141], "load": [8, 118], "pre": [8, 113, 130], "id": [8, 15, 113, 132], "liter": 8, "mountaincar": [8, 94], "mountaincarcontinu": [8, 95], "acrobot": [8, 91, 117], "lunarlandercontinu": [8, 134], "bipedalwalk": [8, 21, 88, 125, 128], "v3": [8, 21, 88, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 114, 122], "bipedalwalkerhardcor": 8, "blackjack": [8, 21, 110, 117, 119, 124, 130, 137, 140], "frozenlak": [8, 14, 113, 116, 130], "frozenlake8x8": [8, 116], "cliffwalk": [8, 112, 125], "taxi": [8, 110, 124], "reacher": [8, 97, 105, 130, 134], "pusher": [8, 97, 130], "invertedpendulum": [8, 97, 103, 104, 141], "inverteddoublependulum": [8, 103], "halfcheetah": [8, 97, 99], "swimmer": [8, 97, 125, 127], "walker2d": [8, 97], "ant": [8, 97, 100, 108, 122, 130], "humanoidstandup": [8, 97, 102], "humanoid": [8, 45, 97, 98, 130], "accord": [8, 10, 11, 15, 90, 136], "find": [8, 24, 25, 50, 51, 52, 53, 54, 60, 69, 81, 103, 106, 130, 132], "avail": [8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 90, 109, 111, 112, 122, 130, 132], "modul": [8, 9, 13, 16, 122, 128, 132, 134, 136, 141], "eg": [8, 14, 132], "max_episode_step": [8, 18, 132, 136], "autoreset": [8, 15, 132], "apply_api_compat": [8, 22, 127], "stepapicompat": [8, 16, 18, 22, 130], "argument": [8, 9, 10, 11, 12, 14, 15, 19, 21, 22, 87, 91, 110, 116, 119, 123, 127, 132, 139], "addit": [8, 9, 15, 18, 21, 23, 24, 30, 41, 71, 75, 90, 91, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 114, 119, 124, 127, 130, 132, 136, 139, 141], "entry_point": [8, 127, 132], "reward_threshold": [8, 98, 99, 100, 101, 102, 105, 106, 107, 108, 132], "nondeterminist": [8, 132], "order_enforc": [8, 132], "syntax": 8, "namespac": [8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 132], "env_nam": [8, 22], "v": [8, 73, 94, 103, 136], "keyword": [8, 10, 15, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 111, 124, 132, 139], "entri": [8, 23, 118], "point": [8, 12, 14, 16, 24, 27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 45, 46, 47, 48, 49, 51, 55, 56, 59, 66, 67, 68, 69, 70, 71, 72, 74, 76, 77, 78, 80, 81, 84, 88, 89, 90, 92, 93, 96, 106, 111, 118, 124, 133], "threshold": [8, 92, 93, 114, 132], "consid": [8, 89, 90, 91, 97, 98, 100, 101, 108, 132, 134], "learnt": 8, "knowledg": 8, "enabl": [8, 16, 24, 89, 109, 118, 122, 124], "correct": [8, 114, 128, 130, 132, 133], "pprint_registri": [8, 130], "env_id": [8, 22, 130], "retriev": [8, 18, 45, 48, 74], "global": 8, "_registri": 8, "num_col": 8, "exclude_namespac": 8, "disable_print": 8, "column": [8, 141], "arrang": 8, "exclud": [8, 98, 99, 100, 101, 107, 108], "instead": [8, 15, 19, 21, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 92, 108, 119, 122, 127, 132, 136], "consol": [8, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "variou": [9, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 102, 109, 118, 141], "mathemat": [9, 141], "environ": [9, 11, 13, 16, 17, 18, 19, 20, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 110, 111, 112, 113, 114, 117, 118, 122, 123, 124, 125, 126, 127, 129, 130, 133, 134, 135, 137, 141], "three": [9, 16, 19, 24, 27, 30, 32, 45, 53, 59, 66, 71, 72, 73, 74, 98, 100, 101, 102, 107, 108, 119, 132], "cube": [9, 70], "superclass": [9, 17, 19, 20], "npt": [9, 10, 11], "dtypelik": 9, "crucial": 9, "clearli": 9, "u": [9, 24, 127, 132, 134, 139, 141], "highli": [9, 23, 24, 87, 91, 109, 124, 132], "data": [9, 12, 14, 16, 23, 123, 124, 130, 132, 134, 136, 139, 141], "painlessli": 9, "flat": [9, 10, 11, 12, 19], "especi": [9, 132, 134, 136], "hierarch": [9, 114], "build": [9, 26, 44, 48, 103, 108, 130, 134], "express": [9, 14, 98, 101, 102], "cover": [9, 139], "parametr": [9, 141], "distribut": [9, 10, 11, 107, 109, 113, 118, 136, 141], "batch": [9, 13, 15, 119, 122, 136], "handl": [9, 15, 103, 111, 117, 121, 135, 137, 139], "care": [9, 75], "immut": 9, "is_np_flatten": [9, 130], "mask": [9, 10, 11, 23, 114, 124, 127, 136], "t_cov": 9, "randomli": [9, 10, 90, 113, 124, 132, 136], "uniform": [9, 11, 12, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "bounded": 9, "int8": [9, 11], "member": [9, 58], "subspac": [9, 10, 11, 12, 126], "to_json": 9, "sample_n": 9, "jsonabl": 9, "from_json": 9, "matric": [9, 109], "binari": [9, 11, 21, 132], "hold": [9, 111, 139], "down": [9, 11, 24, 28, 30, 31, 37, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 57, 58, 61, 63, 65, 66, 68, 70, 78, 80, 81, 84, 85, 89, 102, 111, 112, 113, 114, 132, 133, 139], "button": [9, 11, 14, 21], "multipl": [9, 10, 11, 13, 15, 122, 136], "ax": [9, 11, 132, 136, 139], "messag": [9, 118, 119, 124, 130], "mission": [9, 57, 59, 60], "join": [9, 130], "togeth": [9, 24, 97], "vectoris": [9, 109], "separ": [9, 10, 23, 130, 132, 136], "readabl": [9, 10, 130], "unord": 9, "entiti": 9, "select": [9, 11, 47, 98, 101, 102, 105, 106, 130, 136], "graph": [9, 12, 21, 124, 127, 136, 139], "node": [9, 10, 12, 124], "edg": [9, 10, 12, 93, 124], "unflatten": [9, 12, 130], "neural": [9, 12, 92, 107, 109, 136, 141], "network": [9, 12, 107, 136], "flatdim": [9, 12, 117], "revers": [9, 12, 111, 136, 141], "batch_spac": [9, 13], "concaten": [9, 12, 13, 97], "iter": [9, 10, 13, 15], "create_empty_arrai": [9, 13], "create_shared_memori": [9, 13], "read_from_shared_memori": [9, 13], "write_to_shared_memori": [9, 13], "spaces_kwarg": 10, "constitu": 10, "usag": [10, 11, 12, 89, 134], "ordereddict": [10, 13, 19], "nest": [10, 13, 109], "ext_control": 10, "inner_st": 10, "charg": 10, "system_check": 10, "job_statu": 10, "conveni": [10, 16, 21, 132], "easili": [10, 16, 127], "deal": [10, 136], "avoid": [10, 13, 16, 21, 24, 27, 28, 30, 35, 36, 44, 46, 62, 63, 65, 70, 72, 73, 78, 112, 114], "independ": [10, 11, 15, 100, 106, 107, 108], "attr": 10, "uniqu": [10, 87, 91, 97, 103, 126], "sub": [10, 15, 74], "precis": [10, 116], "cartesian": [10, 11, 96], "3991573": 10, "21649833": 10, "involv": [10, 87, 103, 104, 109, 112, 113, 114, 124, 130, 132, 141], "draw": [10, 111, 132, 139], "54": 10, "finit": [10, 11, 98, 102, 103, 104, 105, 106, 108, 133, 134, 141], "a_0": 10, "dot": [10, 11, 132], "a_n": 10, "a_i": 10, "belong": [10, 13], "n": [10, 11, 12, 13, 21, 24, 43, 89, 92, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 136, 139, 141], "6369617": 10, "01652764": 10, "8132702": 10, "length_mask": 10, "sample_mask": 10, "drawn": [10, 11, 105, 107, 111], "geometr": 10, "featur": [10, 11, 109, 122, 130, 136], "feature_spac": 10, "node_spac": [10, 12], "edge_spac": [10, 12], "seri": [10, 21], "adjac": 10, "matrix": 10, "edge_link": [10, 12], "num_nod": 10, "num_edg": 10, "graphinst": [10, 12, 129, 130], "edge_mask": 10, "multipli": [10, 95, 136], "attribut": [10, 21, 23, 105, 106, 132, 134], "unbound": 11, "mathbb": 11, "interv": [11, 21, 98, 100, 101, 108], "There": [11, 16, 21, 56, 57, 69, 88, 90, 91, 94, 95, 97, 102, 103, 104, 105, 106, 109, 112, 114, 116, 130, 132], "common": [11, 16, 21, 130, 133, 134, 141], "construct": [11, 24, 50, 51, 52, 53, 54, 92, 98, 99, 100, 101, 107, 108], "across": [11, 13, 30, 40, 51, 52, 55], "infer": [11, 97, 124], "essenti": 11, "shift": [11, 21], "unsupport": 11, "is_bound": 11, "sens": [11, 109], "One": [11, 21, 98, 101, 102], "smallest": 11, "maskndarrai": 11, "uniformli": [11, 24, 92, 93, 105, 106, 132], "infeas": 11, "sort": 11, "fair": 11, "coin": 11, "toss": 11, "nvec": 11, "int64": [11, 12, 130], "keyboard": [11, 14, 122], "nintendo": 11, "conceptu": 11, "arrow": [11, 68], "left": [11, 24, 26, 28, 29, 30, 31, 35, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 56, 57, 58, 61, 63, 64, 65, 67, 70, 74, 75, 77, 78, 80, 81, 84, 85, 88, 89, 90, 93, 94, 98, 101, 102, 103, 104, 108, 112, 113, 114, 132, 139, 141], "param": 11, "press": [11, 14, 21, 89, 92], "although": 11, "rare": [11, 126], "d": [11, 14, 16, 92, 105, 106, 136], "categor": [11, 136], "count": [11, 88, 90, 111, 139], "kind": 11, "unless": [11, 16, 111, 112, 113, 114, 127, 132], "max_length": 11, "min_length": 11, "charset": 11, "alphanumer": 11, "compris": 11, "charact": [11, 14, 58, 69], "b5": 11, "hello": 11, "0123456789abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz": 11, "0123456789": 11, "digit": [11, 109], "inclus": [11, 24, 118], "english": 11, "alphabet": 11, "plu": [11, 40, 112], "latin": 11, "charlist": 11, "matter": [11, 126], "attempt": [12, 90, 102, 103, 114], "compound": 12, "equival": [12, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 124, 136], "oper": [12, 18, 123], "boundari": 12, "while": [12, 23, 24, 27, 28, 44, 59, 62, 63, 64, 65, 68, 70, 72, 74, 78, 81, 83, 90, 92, 93, 96, 98, 99, 101, 102, 105, 106, 107, 111, 112, 127, 132, 133, 136, 139, 141], "exactli": [12, 107], "necessarili": 12, "hot": 12, "encod": [12, 114, 126, 132], "60": [12, 58, 71, 109, 130], "recurs": 12, "6": [12, 24, 28, 29, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 65, 79, 80, 81, 84, 85, 92, 94, 95, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 112, 114, 122, 130, 141], "float64": 12, "notimplementederror": 12, "int32": 12, "15": [12, 24, 47, 50, 51, 52, 53, 54, 58, 77, 90, 98, 99, 101, 102, 105, 108, 109, 113, 136], "datapoint": 12, "k": [12, 14, 90, 107, 136], "m": [12, 24, 43, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 116, 132, 136], "would": [12, 17, 19, 20, 98, 99, 100, 101, 102, 107, 108, 115, 124, 128, 130, 132, 134, 136], "copi": [13, 15, 132], "item": [13, 14, 109, 136, 139], "77395606": 13, "43887845": 13, "85859793": 13, "697368": 13, "09417735": 13, "97562236": 13, "stopiter": 13, "fn": 13, "multi_binari": 13, "built": [13, 23, 109, 119, 130], "multi_discret": 13, "ctx": 13, "mp": 13, "multiprocess": [13, 15], "opt": 13, "hostedtoolcach": 13, "9": [13, 15, 24, 28, 31, 36, 45, 50, 51, 52, 53, 54, 58, 61, 71, 80, 85, 92, 96, 98, 99, 100, 101, 102, 103, 105, 106, 108, 111, 136, 139], "16": [13, 24, 33, 36, 39, 45, 50, 51, 52, 53, 54, 58, 96, 98, 99, 101, 102, 105, 108, 113, 130, 139, 141], "x64": 13, "lib": [13, 112], "python3": 13, "eventu": 13, "shared_memori": [13, 15], "customspaceerror": 13, "read": [13, 23, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 129, 130], "forward": [13, 35, 57, 88, 98, 99, 100, 101, 102, 107, 108, 118, 122, 123, 136, 141], "side": [13, 19, 26, 39, 54, 56, 90], "write": [13, 15, 130, 132], "num_env": [13, 15, 136], "plai": [14, 21, 34, 47, 60, 79, 84, 89, 109, 122, 124, 130, 132, 139], "transpos": [14, 132], "zoom": 14, "callback": [14, 123], "keys_to_act": [14, 124, 136], "w": [14, 98, 101, 102, 136], "7": [14, 24, 28, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 65, 80, 81, 84, 85, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 122, 136], "dw": 14, "verifi": 14, "level": [14, 16, 21, 24, 31, 48, 70, 78, 109], "unplay": 14, "wish": [14, 19, 23, 122], "real": [14, 56, 99], "playplot": 14, "150": 14, "def": [14, 132, 134, 136, 139, 141], "obs_t": 14, "obs_tp1": 14, "plotter": 14, "amount": [14, 44, 46, 47, 49, 55, 74, 88, 93, 103, 104, 141], "receiv": [14, 15, 21, 34, 37, 41, 44, 45, 51, 57, 60, 89, 90, 95, 108, 114, 124, 133, 136, 139, 141], "map": [14, 19, 21, 42, 113, 114, 130, 132, 139, 141], "unicod": 14, "suppos": 14, "trigger": [14, 114], "key_to_act": 14, "ord": [14, 132], "unknown": [14, 109], "horizon_timestep": 14, "plot_nam": 14, "live": [14, 27, 30, 33, 34, 36, 39, 44, 45, 48, 55, 59, 60, 66, 72, 77, 139], "compute_metr": 14, "obs_tp": 14, "cumulative_reward": 14, "linalg": [14, 132], "norm": [14, 105, 106, 132], "along": [14, 16, 21, 22, 58, 59, 61, 72, 89, 93, 94, 95, 99, 103, 104, 107, 112, 124, 130], "conjunct": 14, "evolv": 14, "200": [14, 49, 72, 90, 92, 93, 94, 96, 113, 114], "magnitud": [14, 90, 95, 103, 104, 141], "your_env": 14, "len": [14, 134, 136, 139], "horizon": [14, 74, 100, 108, 133], "titl": [14, 94, 95, 139, 141], "matplotlib": [14, 136, 139, 141], "playablegam": 14, "process_ev": 14, "event": [14, 121, 132], "particular": [14, 16, 19, 21, 22, 90, 130, 133], "exit": [14, 15], "save_video": [14, 127], "episode_index": 14, "step_starting_index": 14, "extract": 14, "compos": 14, "isn": 14, "imagesequenceclip": 14, "durat": [14, 98, 100, 101, 102, 103, 104, 105, 106, 108, 117, 141], "step_index": 14, "199": 14, "episode_id": 14, "27": [14, 92, 98, 101, 102], "729": 14, "2000": [14, 88, 114], "3000": 14, "schedul": [14, 113], "step_api_compat": [14, 22], "step_return": 14, "output_truncation_bool": [14, 18], "is_vector_env": 14, "doc": [14, 18, 90, 118, 130, 134], "interfac": [14, 115], "conflict": [14, 109], "desir": [14, 107, 108, 114, 136], "vec_env": 14, "convert_to_terminated_truncated_step_api": [14, 22], "irrespect": 14, "convert_to_done_step_api": 14, "omit": [14, 97, 98, 99, 100, 101, 102, 107, 108], "env_check": [14, 122], "check_env": [14, 130], "skip_render_check": 14, "invas": [14, 78], "org": [14, 24, 130, 136, 139, 141], "content": 14, "environment_cr": [14, 132], "ignor": [14, 24, 111, 122, 124, 139], "ci": [14, 116, 117, 118, 121, 129, 130], "linear": [15, 90, 92, 103, 104, 107, 136, 141], "speed": [15, 35, 88, 89, 130, 136], "wait": [15, 48, 50, 74, 127, 133], "overwritten": [15, 16, 124, 129, 130], "asyncvectorenv": [15, 130, 136], "syncvectorenv": 15, "single_observation_spac": [15, 136], "single_action_spac": [15, 136], "v25": [15, 23], "vectorlistinfo": [15, 16, 18, 122], "share": [15, 141], "other": [15, 17, 18, 19, 22, 30, 39, 42, 50, 56, 90, 96, 97, 99, 103, 105, 106, 108, 113, 114, 117, 118, 127, 130, 132, 134, 136, 139, 141], "word": [15, 105, 141], "01522993": 15, "04562247": 15, "04799704": 15, "03392126": 15, "03774345": 15, "02418869": 15, "00942293": 15, "0469184": 15, "18847767": 15, "26141977": 15, "01431748": 15, "24002443": 15, "04731862": 15, "3110827": 15, "03822722": 15, "1710671": 15, "00848456": 15, "2487226": 15, "releas": [15, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 132], "resourc": [15, 132, 139], "viewer": [15, 130], "close_extra": 15, "synchron": 15, "asynchron": 15, "garbag": 15, "registri": [15, 21, 122, 130], "env_fn": 15, "context": [15, 141], "daemon": 15, "worker": [15, 68], "pipe": [15, 118], "commun": [15, 23, 124, 127], "81": [15, 96, 130], "62": 15, "14995256": 15, "9886932": 15, "12224312": 15, "5760367": 15, "8174238": 15, "91244936": 15, "back": [15, 16, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 52, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 98, 99], "effici": [15, 19, 94, 95, 103, 130, 136], "subprocess": 15, "flag": [15, 94, 109], "quit": [15, 132], "head": [15, 99], "spawn": [15, 106, 136], "children": 15, "overrid": [15, 134], "inner": 15, "logic": [15, 89, 118, 132], "degre": [15, 101, 102, 103, 134, 136], "chanc": 15, "shoot": [15, 31, 40, 42, 47, 48, 55, 56, 65, 68, 72, 77, 78, 86], "yourself": [15, 48, 68, 89, 132, 139], "foot": [15, 99, 100, 108], "thu": [15, 24, 50, 51, 52, 53, 54, 105, 106, 109, 136], "_worker": 15, "_worker_shared_memori": 15, "runtimeerror": 15, "serial": [15, 136], "without": [16, 21, 23, 46, 101, 111, 113, 119, 125, 129, 130, 132, 139], "alter": [16, 21, 24, 132], "lot": [16, 21, 134, 139, 141], "boilerpl": [16, 21, 132, 134], "modular": [16, 21, 109, 132, 134], "chain": [16, 21, 92, 107], "base_env": [16, 21], "underneath": [16, 21, 93], "anoth": [16, 39, 101, 102, 104, 105, 106, 108, 114, 129, 133, 139, 141], "hopperenv": 16, "layer": [16, 21, 141], "bare": 16, "hopper_v4": 16, "0x7fbb5efd0490": 16, "thing": [16, 24, 119, 127, 134], "Such": [16, 134], "actionwrapp": [16, 17], "observationwrapp": [16, 19, 132], "rewardwrapp": [16, 20], "behavior": [16, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 96, 98, 99, 100, 101, 102, 107, 108, 109, 117, 119, 132, 134], "forget": [16, 132], "wrapperobstyp": [16, 19], "commonli": [16, 21, 127], "found": [16, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 97, 109, 122, 139, 141], "descript": [16, 130, 132], "misc": 16, "tp": 16, "21": [16, 22, 98, 101, 102, 105, 111, 119, 130, 136, 139], "request": [16, 111], "anobserv": 16, "save": [16, 18, 23, 118, 134], "axbxc": [16, 19], "append": [16, 19, 136, 139, 141], "th": 16, "overwrit": [17, 19, 20], "domain": [17, 89, 92, 94, 95, 114, 134, 135, 137], "among": [17, 19, 91, 97], "new_ob": 18, "final_reward": 18, "final_termin": 18, "final_trunc": 18, "final_don": 18, "prior": 18, "alongsid": 18, "re": [18, 56, 71, 74, 118, 122, 130, 132, 141], "old_env": 18, "legacyenv": 18, "retun": 18, "modern": [18, 109], "convers": [18, 22, 109], "cartpoleenv": 18, "place": [18, 28, 93, 94, 95, 132, 136], "\u01f9one": 18, "outermost": 18, "reflect": [19, 93, 124], "f": [19, 20, 136, 139, 141], "ideal": 19, "subclass": 19, "accordingli": [19, 74, 132], "incorrectli": [19, 133], "randn": [19, 119, 127], "20380084": 19, "03390356": 19, "13373359": 19, "24382612": 19, "lz4_compress": 19, "lazyfram": 19, "lz4": 19, "compress": 19, "singleton": 19, "axbx1": 19, "axb": 19, "render_kwarg": 19, "pixel_kei": 19, "odict_kei": 19, "400": [19, 114], "600": [19, 136], "assertionerror": 19, "typeerror": 19, "unexpect": [19, 117], "64": [19, 24, 130], "invari": 20, "incorrect": [20, 114, 124, 125, 133], "mountain": [21, 91, 125], "car": [21, 33, 49, 87, 91, 109, 122, 124, 125], "four": [21, 33, 44, 89, 90, 98, 100, 108, 114, 139], "introduc": [21, 22, 23, 98, 101, 102, 123, 124], "core": [21, 124, 127, 130], "markov": [21, 133], "decis": [21, 23, 109, 127, 133], "theori": [21, 96, 103, 124, 136], "perfect": 21, "reconstruct": 21, "compon": [21, 132], "veri": [21, 90, 105, 116, 127, 128, 129, 130, 132, 136], "easi": [21, 60, 110, 127], "classic": [21, 23, 90, 92, 93, 94, 95, 96, 100, 104, 107, 108, 117, 122, 124, 130, 133], "loop": [21, 23, 109, 136, 139], "pictur": 21, "simplifi": [21, 96, 122, 126, 130], "someth": [21, 127], "spaceship": [21, 31, 45, 76], "land": [21, 38, 72, 90], "safe": [21, 68, 90], "document": [21, 24, 25, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 91, 98, 101, 102, 103, 110, 116, 118, 119, 130, 132, 136, 139], "imagin": [21, 109, 134], "robot": [21, 23, 36, 86, 88, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 119, 133], "destroi": [21, 27, 29, 31, 32, 33, 34, 35, 36, 39, 40, 41, 42, 45, 46, 54, 60, 71, 72, 73, 76, 77, 78, 80, 90], "enemi": [21, 28, 29, 32, 34, 35, 36, 42, 45, 46, 48, 61, 70, 71, 73, 74, 76, 77, 78, 80, 86], "exchang": 21, "crash": [21, 90], "succeed": 21, "similarli": [21, 90, 130], "restart": 21, "train": [21, 23, 118, 122, 130, 135], "major": [21, 124, 130], "our": [21, 130, 132, 136, 139], "simpl": [21, 88, 110, 115, 133, 141], "coupl": [21, 136], "nich": 21, "li": [21, 109], "lie": 21, "manual": [21, 23, 24, 50, 51, 52, 53, 54, 136], "aspect": [21, 127, 133], "box2d": [21, 88, 89, 90, 119, 122, 126], "bipedal_walk": [21, 88], "0x7f87d70712d0": 21, "regist": [22, 24, 127], "sole": [22, 127], "special": [22, 27, 57, 141], "gymv26environ": 22, "relev": [22, 118, 132], "pong": [22, 24, 39, 43], "v5": [22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "introduct": [22, 92, 111, 112], "signatur": 22, "previous": [22, 24, 33, 127, 128], "backward": [22, 99, 119, 121, 124, 125, 128, 136, 141], "compliant": [22, 23, 116, 122], "oldv21env": 22, "break": [23, 31, 39, 89, 116, 118, 121, 130], "briefli": 23, "outlin": [23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 111, 119, 139], "still": [23, 24, 122, 127, 130, 134, 136, 139], "stuck": [23, 27, 36, 55], "123": [23, 130], "becaus": [23, 132, 134, 136, 139], "emul": [23, 24, 118], "randint": [23, 119, 127], "utilis": [23, 127], "contrast": 23, "older": [23, 97], "definit": [23, 98, 100, 101, 125, 133], "expand": [23, 24, 50, 51, 52, 53, 54], "blog": [23, 124, 127], "post": [23, 32, 124, 127], "soon": [23, 124, 127, 132], "extra": [23, 28, 33, 34, 36, 40, 42, 55, 74, 109, 116, 130, 132, 139], "address": 23, "associ": [23, 120], "librari": [23, 115, 130, 132], "primari": 23, "shouldn": [23, 132], "happen": [23, 73, 94, 95, 98, 100, 101, 102, 103, 104, 105, 106, 108, 111, 113, 114, 124, 133], "fly": [23, 26, 31, 55, 71, 90], "shown": [23, 59, 89, 92, 133], "explan": [23, 139, 141], "summari": 23, "goalenv": 23, "reimplement": 23, "classic_control": [23, 119, 122], "system": [23, 54, 92, 96, 103, 104, 109, 118], "monitor": [23, 117, 120], "stella": 24, "adventur": [24, 43], "air": [24, 43], "raid": [24, 43], "alien": [24, 43, 45, 65, 78], "amidar": [24, 32, 43], "assault": [24, 43], "asterix": [24, 43], "asteroid": [24, 43], "atlanti": [24, 43], "licens": [24, 109, 118, 136, 139, 141], "download": [24, 132, 133, 134, 136, 137, 139, 141], "locat": [24, 60, 109, 112, 113, 114, 132], "agre": 24, "todo": [24, 114], "legal": [24, 50, 51, 52, 53, 54], "fire": [24, 26, 28, 29, 31, 32, 35, 37, 39, 40, 50, 51, 52, 53, 54, 57, 64, 65, 67, 68, 70, 71, 73, 74, 77, 78, 80, 82, 84, 85, 90], "upright": [24, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 63, 93, 96, 103, 104], "upleft": [24, 30, 31, 35, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 63], "downright": [24, 30, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63], "downleft": [24, 30, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63], "upfir": [24, 28, 31, 37, 50, 51, 52, 53, 54, 68, 80, 81, 82, 84, 85], "11": [24, 31, 50, 51, 52, 53, 54, 58, 61, 98, 99, 100, 101, 102, 103, 105, 106, 108, 111, 112, 136, 139], "rightfir": [24, 26, 28, 29, 31, 32, 35, 40, 50, 51, 52, 53, 54, 58, 61, 64, 65, 67, 77, 80, 81, 84, 85], "leftfir": [24, 26, 28, 29, 31, 32, 40, 50, 51, 52, 53, 54, 58, 61, 64, 65, 67, 77, 80, 81, 84, 85], "13": [24, 31, 50, 51, 52, 53, 54, 58, 61, 98, 99, 101, 102, 105, 108, 114, 130], "downfir": [24, 28, 31, 37, 50, 51, 52, 53, 54, 58, 61, 65, 68, 80, 82, 85], "14": [24, 50, 51, 52, 53, 54, 58, 88, 90, 98, 99, 101, 102, 105, 108, 139], "uprightfir": [24, 31, 50, 51, 52, 53, 54, 58, 61], "upleftfir": [24, 31, 50, 51, 52, 53, 54, 58, 61], "downrightfir": [24, 50, 51, 52, 53, 54, 58, 61], "17": [24, 36, 50, 51, 52, 53, 54, 58, 98, 99, 101, 102, 105, 108, 111], "downleftfir": [24, 50, 51, 52, 53, 54, 58, 61], "smaller": [24, 29, 31, 50, 51, 52, 53, 54], "enumer": [24, 50, 51, 52, 53, 54], "full_action_spac": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "reduc": [24, 26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 93, 136, 139], "difficulti": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 109, 125], "choic": [24, 136, 141], "player": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 109, 111, 112, 113, 114, 139], "128": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 136], "byte": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "ram": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "exact": [24, 37, 50, 51, 52, 53, 54, 111, 139], "atariag": [24, 25, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 79, 81, 82, 83, 84, 85, 86], "determinist": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 92, 94, 95, 132, 136], "art": 24, "memor": 24, "sticki": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "small": [24, 54, 73, 88, 109, 110, 116, 119, 120, 121, 130, 141], "On": [24, 65, 92, 93, 94, 95, 96, 122], "top": [24, 39, 44, 51, 57, 59, 73, 89, 90, 91, 94, 95, 100, 103, 104, 108, 119, 124, 125, 132, 141], "frameskip": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "exclus": 24, "tabl": [24, 27, 28, 30, 44, 46, 101, 102, 105, 123], "obs_typ": 24, "Its": 24, "stick": [24, 33, 55, 111, 139], "ll": [24, 57, 74, 127, 136, 139], "sound": 24, "lock": 24, "proper": [24, 118, 127], "audio": 24, "framer": [24, 102, 105, 106, 132], "best": [24, 50, 51, 52, 53, 54, 136, 139], "practic": [24, 50, 51, 52, 53, 54, 109, 124, 133, 139], "configur": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 91, 97, 110, 132, 136, 139], "let": [24, 39, 109, 132, 134, 139, 141], "variat": 24, "amidardeterminist": 24, "amidarnoframeskip": 24, "ramdeterminist": 24, "ramnoframeskip": 24, "suffix": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "noframeskip": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "airraid": [24, 26], "31": [24, 31, 101, 102, 130], "bankheist": [24, 33], "20": [24, 33, 39, 66, 74, 90, 98, 101, 102, 105, 114, 118, 130, 136, 139], "battlezon": [24, 34], "beamrid": [24, 35], "berzerk": [24, 43], "18": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 98, 99, 101, 102, 105, 108, 116], "bowl": [24, 43], "breakout": [24, 43], "36": [24, 39, 101, 102, 112], "40": [24, 39, 64, 101, 102], "44": [24, 39, 101, 102], "carniv": [24, 43], "centiped": [24, 43], "22": [24, 41, 98, 101, 102, 105, 120, 122, 127, 130, 139], "86": [24, 41], "choppercommand": [24, 42], "crazyclimb": [24, 44], "defend": [24, 32, 43, 48, 64, 76], "demonattack": [24, 46], "doubledunk": [24, 47], "elevatoract": [24, 48], "enduro": [24, 43], "fishingderbi": 24, "freewai": [24, 43], "frostbit": [24, 43], "gopher": [24, 43], "gravitar": [24, 43], "hero": [24, 43], "icehockei": 24, "jamesbond": [24, 43], "journeyescap": 24, "kangaroo": [24, 43], "krull": [24, 43], "kungfumast": [24, 61], "montezumareveng": [24, 62], "mspacman": [24, 63], "namethisgam": [24, 64], "phoenix": [24, 43], "pitfal": [24, 43, 88], "pooyan": [24, 43], "50": [24, 58, 68, 74, 90, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 141], "70": [24, 68, 130], "privateey": 24, "qbert": [24, 43], "riverraid": [24, 43], "roadrunn": [24, 72], "robotank": [24, 43, 73], "seaquest": [24, 43], "ski": [24, 43], "solari": [24, 43], "spaceinvad": 24, "stargunn": 24, "tenni": [24, 43], "timepilot": 24, "tutankham": [24, 43], "upndown": [24, 82], "ventur": [24, 43], "videopinbal": [24, 84, 85], "wizardofwor": [24, 85], "yarsreveng": 24, "zaxxon": [24, 43], "oppos": 24, "ai": [24, 109], "mg": 24, "bellemar": 24, "naddaf": 24, "j": [24, 136, 141], "veness": 24, "platform": 24, "journal": [24, 114], "artifici": [24, 114], "intellig": [24, 114, 119], "research": [24, 97, 103, 114], "2012": 24, "url": [24, 127, 136], "jair": [24, 114], "php": 24, "articl": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "view": [24, 54], "11182": 24, "250": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 130], "160": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 130], "enchant": [25, 41], "chalic": 25, "golden": 25, "castl": 25, "pick": [25, 48, 59, 72, 114, 136, 139], "sword": 25, "bridg": [25, 71, 109], "magnet": 25, "fight": [25, 38, 61, 64, 109], "outmanoeuvr": 25, "dragon": 25, "flavor": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "switch": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 109, 116, 130], "thorough": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "discuss": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 119, 130], "intric": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "stochast": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 91, 92, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 106, 107, 108, 112, 114, 117], "ship": [26, 27, 29, 35, 45, 54, 76], "sidewai": [26, 29, 35, 71, 75], "protect": [26, 32, 41, 42, 53, 58, 68], "saucer": [26, 31, 78], "try": [26, 27, 28, 37, 39, 44, 67, 68, 122, 129, 130, 139], "drop": [26, 33, 114, 130], "bomb": [26, 45, 57, 65, 78], "those": [26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85, 92, 97, 98, 99, 100, 101, 102, 103, 104, 108, 124, 127, 134, 136], "meaning": [26, 28, 29, 30, 31, 32, 35, 37, 39, 40, 44, 46, 47, 48, 49, 58, 61, 63, 64, 65, 67, 68, 70, 75, 77, 78, 80, 81, 82, 84, 85], "210": [27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 42, 45, 46, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86], "maze": [27, 28, 33, 36, 62, 81], "egg": 27, "scatter": [27, 69], "simultan": [27, 28], "kill": [27, 36, 55, 60], "flamethrow": 27, "awai": [27, 67, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 134], "tricki": 27, "situat": [27, 114], "occasion": [27, 31], "pulsar": 27, "temporari": 27, "abil": [27, 130], "score": [27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 74, 78, 79, 80, 84, 88, 90], "prize": [27, 82], "caught": [27, 33, 125], "lose": [27, 30, 32, 41, 44, 45, 48, 55, 57, 66, 67, 69, 71, 72, 74, 75, 76, 90, 111, 139], "consult": [27, 56, 57, 58, 59, 60], "pac": 28, "man": 28, "visit": [28, 48, 89], "grid": [28, 112, 113, 114, 132, 139], "chicken": [28, 40, 51], "catch": [28, 44, 50, 72], "travers": 28, "vehicl": [29, 34, 49, 58], "big": [29, 50, 132], "mother": [29, 59, 68], "circl": [29, 132], "overhead": [29, 90], "deploi": 29, "drone": 29, "dodg": [29, 35, 74], "attack": [29, 31, 32, 35, 43, 45, 68], "horizont": [30, 40, 88, 90, 92, 132], "vertic": [30, 88, 103, 104, 132], "lyre": 30, "guid": [30, 51, 109, 130], "suffici": [30, 32, 34, 36, 41, 139], "award": [30, 36, 41, 44, 48, 74, 103, 104], "field": 31, "appear": [31, 33, 36, 73, 94, 95], "satellit": 31, "ufo": 31, "job": [32, 132], "submerg": 32, "citi": [32, 33, 69], "slowli": 32, "descend": 32, "toward": [32, 56, 107, 119], "strike": [32, 37], "distanc": [32, 49, 99, 132], "defens": [32, 47], "manag": [32, 37], "seven": 32, "fought": 32, "wave": [32, 42, 46, 58, 65, 78], "outer": [32, 92], "robber": 33, "natur": [33, 111, 113, 139], "rob": 33, "getawai": 33, "navig": [33, 69, 114, 132, 134], "polic": 33, "chase": [33, 36], "dynamit": [33, 55], "ga": [33, 89], "tank": [33, 34, 74], "nine": [33, 98], "leav": [33, 93, 97, 103, 132], "person": [34, 92, 109], "perspect": [34, 109], "3d": [34, 98, 101, 102, 109, 139], "illus": 34, "radar": [34, 85], "gain": [34, 70, 76, 77, 80, 134], "travel": 35, "steer": [35, 82, 89], "debri": 35, "leftifir": 35, "evil": [36, 61, 86], "touch": [36, 69, 132], "wall": [36, 39, 55, 94, 95, 114], "undefeat": 36, "otto": 36, "tri": 37, "knock": [37, 38], "pin": 37, "spar": 37, "spare": 37, "300": [37, 49, 88, 132], "oppon": [38, 50, 56, 67, 82], "ring": 38, "hit": [38, 39, 40, 41, 46, 68, 69, 72, 73, 75, 111, 139], "punch": [38, 59], "famou": 39, "paddl": [39, 67], "ball": [39, 56, 67, 84], "brick": 39, "wreak": 39, "havoc": 39, "five": [39, 69, 75, 91, 124, 141], "214": 40, "em": 40, "target": [40, 92, 105, 106, 132, 133, 134, 136], "gun": [40, 48], "suppli": [40, 64, 133], "ammunit": 40, "steal": [40, 50], "bullet": [40, 109], "subtract": [40, 136], "minu": 40, "sign": [40, 103, 104, 141], "elf": [41, 113], "magic": 41, "wand": 41, "fend": 41, "spider": 41, "flea": 41, "mushroom": 41, "forest": 41, "bitten": 41, "tempor": 41, "paralyz": 41, "scorpion": 41, "round": 41, "helicopt": [42, 44, 71], "truck": [42, 72], "convoi": 42, "aircraft": [42, 80], "mini": 42, "bottom": [42, 57, 89, 94, 95, 100, 108, 112], "plane": 42, "surviv": [42, 46], "bank": [43, 71], "heist": 43, "battl": 43, "zone": 43, "beam": [43, 55], "rider": 43, "chopper": 43, "command": [43, 97], "crazi": 43, "climber": 43, "demon": 43, "doubl": [43, 97], "dunk": 43, "elev": 43, "fish": 43, "derbi": 43, "ic": [43, 46, 52, 56, 113], "hockei": [43, 56], "journei": [43, 58, 127, 139], "kung": 43, "fu": 43, "master": [43, 112, 118, 141], "montezuma": 43, "reveng": 43, "pacman": 43, "privat": [43, 69, 132], "ey": [43, 64, 69], "road": [43, 89], "runner": 43, "invad": [43, 77], "star": 43, "gunner": 43, "pilot": [43, 65, 86], "pinbal": 43, "wizard": [43, 61], "Of": 43, "wor": 43, "yar": 43, "builid": 44, "obstacl": [44, 58, 59], "fall": [44, 46, 66, 72, 88, 101, 112, 113, 133], "damag": [44, 73], "climb": [44, 59, 91], "fast": [44, 89, 97, 99, 101, 103, 107, 109], "least": [44, 79, 90], "row": [44, 77, 112, 113], "earth": [45, 77], "rescu": [45, 55, 59, 60, 61, 74, 76], "smart": 45, "shot": [45, 47, 48, 56, 57, 72, 109], "abduct": 45, "unlimit": 45, "laser": [45, 55, 77, 85], "missil": [45, 71], "aliv": [45, 101, 108], "face": [46, 98, 101, 102, 111, 139], "planet": [46, 54, 76], "krybor": 46, "accumul": 46, "reserv": [46, 71, 73, 74], "bunker": [46, 54], "increas": [46, 74, 90, 92, 93, 100, 107, 108, 136, 139], "grant": [46, 90], "slai": 46, "2v2": 47, "basketbal": 47, "possess": 47, "rival": 47, "team": [47, 117], "rule": [47, 79, 109, 111, 139], "foul": 47, "success": [47, 92, 114, 130, 133], "secret": [48, 69], "ground": [48, 68, 88, 90, 98, 102, 109], "stair": 48, "equip": 48, "against": [48, 67, 79, 109, 123], "floor": 48, "gather": [48, 69, 132], "apart": [48, 132], "mark": 48, "red": [48, 54, 114, 122, 132], "door": 48, "unreleas": 48, "prototyp": 48, "500": [48, 71, 92, 93, 114, 127, 136, 139], "racer": 49, "nation": 49, "long": [49, 84, 93, 103, 104, 117, 130], "endur": 49, "race": [49, 72, 75, 87, 122, 124], "overtak": 49, "dai": [49, 87], "stai": [49, 136], "meet": 49, "quota": 49, "sunfish": 50, "But": [50, 75, 98], "fisherman": 50, "black": [50, 60, 92, 116, 122, 139], "shark": [50, 64, 74], "lurk": [50, 70], "surfac": [50, 74, 103, 104, 132], "lane": [51, 109], "busi": 51, "rush": 51, "hour": 51, "traffic": [51, 109], "cross": [51, 112, 113], "bailei": 52, "who": [52, 53, 58, 59, 75, 130], "hop": [52, 70, 100, 108], "forth": 52, "arctic": 52, "river": [52, 71], "block": 52, "white": [52, 122], "blue": [52, 54, 79, 92, 114, 132], "he": [52, 54], "hi": [52, 54, 69], "igloo": 52, "shovel": 53, "wield": 53, "farmer": 53, "crop": 53, "carrot": 53, "spacecraft": 54, "fiction": 54, "solar": 54, "landscap": 54, "miner": 55, "mine": [55, 72], "shaft": 55, "tool": [55, 62], "propel": 55, "backpack": 55, "wherev": 55, "blast": [55, 74, 76], "vermin": 55, "raft": 55, "stretch": 55, "critter": 55, "standard": [56, 98, 99, 103, 109, 117, 118, 119, 120, 136, 141], "minut": [56, 66], "period": [56, 100, 108], "puck": 56, "angl": [56, 57, 88, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 128], "extrem": [56, 109, 110], "rink": 56, "realli": [56, 139], "mr": 57, "bond": 57, "design": [57, 92, 98, 101, 102, 109, 110, 114, 132], "multipurpos": 57, "craft": [57, 90], "varieti": [57, 109], "motion": [57, 109], "slightli": [57, 88, 98], "jump": [57, 72, 75], "dive": 57, "lob": 57, "rate": [57, 74, 136, 139, 141], "highest": 57, "novic": 57, "006": 57, "007": 57, "lead": [58, 136], "peski": 58, "backstag": 58, "scarab": 58, "000": [58, 71, 74], "concert": 58, "cash": 58, "grasp": 58, "groupi": 58, "photograph": 58, "promot": 58, "encount": [58, 59, 80], "her": [59, 68], "preciou": 59, "babi": 59, "ladder": [59, 88], "bonu": [59, 73, 74, 78, 108, 136], "fruit": 59, "throw": [59, 134], "monkei": 59, "corner": 59, "beast": 60, "fortress": [60, 62], "princess": [60, 61], "lyssa": 60, "sunris": 60, "monster": [60, 83], "templ": 61, "victoria": 61, "defeat": [61, 70], "acquir": [62, 97], "treasur": [62, 64, 66, 81, 83], "chamber": [62, 83], "emperor": 62, "deadli": 62, "creatur": [62, 70, 78, 109], "valuabl": 62, "pellet": 63, "ghost": 63, "discov": 64, "octopu": 64, "oxygen": [64, 74], "elimin": [65, 81, 82, 83], "war": 65, "bird": 65, "harri": 66, "jungl": 66, "die": [66, 68, 89], "misfortun": 66, "hole": [66, 113], "compet": 67, "deflect": 67, "pig": 68, "piglet": 68, "wolv": 68, "rope": 68, "balloon": 68, "guard": 68, "eat": [68, 72], "bait": 68, "wolf": 68, "stone": 68, "rock": [68, 72], "french": 69, "pierr": 69, "street": 69, "park": 69, "passag": 69, "dead": [69, 116, 120, 130], "search": 69, "ringlead": 69, "henri": 69, "le": 69, "fiend": 69, "gang": 69, "evid": 69, "stolen": 69, "good": [69, 136, 139], "statut": 69, "expir": 69, "nab": 69, "question": [69, 109], "auto": [69, 130], "pothol": 69, "q": [70, 114, 133, 136, 137, 140], "bert": 70, "pyramid": 70, "destin": [70, 114], "nasti": 70, "jet": 71, "fli": 71, "fuel": [71, 76, 90], "depot": 71, "collid": [71, 74, 90], "squadron": [71, 73], "tanker": 71, "80": [71, 113], "tm": 72, "direct": [72, 74, 92, 93, 94, 95, 98, 100, 101, 103, 104, 108, 112, 113, 114, 132, 141], "outrun": 72, "wile": 72, "coyot": 72, "hazard": 72, "desert": 72, "rocket": [72, 73, 90], "cannon": [72, 77], "cliff": [72, 110], "steel": 72, "pile": 72, "birdse": 72, "cannonbal": 72, "activ": [73, 132], "scrambl": 73, "static": 73, "interfer": 73, "sensor": [73, 89], "report": [73, 88, 98, 101, 102, 124, 130], "flash": 73, "panel": 73, "c": [73, 90, 92, 130, 141], "earn": 73, "twelv": 73, "abl": [74, 130, 133], "torpedo": 74, "diver": 74, "killer": 74, "deliv": [74, 114], "six": [74, 108], "explod": 74, "anyth": 74, "decreas": [74, 90], "almost": [74, 127], "yoursub": 74, "blow": 74, "forc": [74, 76, 90, 93, 94, 95, 98, 101, 102, 103, 104, 117, 141], "less": [74, 88, 90, 103, 107], "worth": [74, 77], "90": [74, 139], "skier": 75, "gate": 75, "fastest": 75, "penal": [75, 134], "tree": 75, "slalom": 75, "penalti": [75, 114], "warp": 76, "sector": 76, "feder": 76, "zylon": 76, "refuel": 76, "cadet": 76, "corridor": 76, "mania": [78, 80], "orang": 79, "win": [79, 111, 139], "margin": 79, "ti": 79, "sport": 79, "till": [79, 136, 139], "technologi": 80, "increasingli": 80, "difficult": [80, 93, 97, 103, 104], "rack": 81, "tomb": 81, "guardian": 81, "baja": 82, "bugger": 82, "dungeon": 83, "beat": [85, 111], "scanner": 85, "armi": 86, "enslav": 86, "galaxi": 86, "fighter": 86, "biped": [87, 101, 102, 117, 119, 122], "walker": [87, 97, 106, 108, 117, 119, 122], "lunar": [87, 119, 122, 125], "lander": [87, 119, 122, 125], "toi": [87, 111, 112, 113, 114, 117], "contribut": [87, 130], "earli": [87, 139, 141], "oleg": [87, 88, 89, 90], "klimov": [87, 88, 89, 90], "popular": [87, 109, 139], "benchmark": 87, "ever": [87, 119], "joint": [88, 90, 92, 93, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "uneven": 88, "terrain": [88, 109], "hardcor": 88, "stump": 88, "1600": 88, "heurist": [88, 90], "demonstr": [88, 109], "motor": [88, 107, 127], "hip": [88, 98, 101, 102], "knee": [88, 101, 102], "hull": 88, "angular": [88, 90, 92, 93, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "leg": [88, 90, 98, 100, 101, 102, 108], "contact": [88, 90, 97, 98, 101, 102, 103, 122], "lidar": 88, "rangefind": 88, "measur": [88, 96, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 117], "far": [88, 89, 103, 105, 106, 113], "torqu": [88, 92, 96, 98, 99, 100, 101, 102, 105, 106, 107, 108], "cost": [88, 99, 100, 107, 108, 109], "better": [88, 117, 130, 136], "stand": [88, 97, 98, 101, 102, 103, 104, 130, 139], "slight": 88, "exce": [88, 111], "closest": 88, "trace": 88, "furthest": 88, "faster": [88, 90, 102, 136], "energi": [88, 90, 93, 109, 134], "spent": [88, 90], "friction": [88, 107, 109], "nervous": 88, "easiest": [89, 118], "ab": [89, 136], "wheel": [89, 128], "gyroscop": 89, "car_rac": [89, 119], "rear": 89, "drive": 89, "brake": 89, "96x96": 89, "tile": 89, "732": 89, "926": 89, "rest": [89, 90, 100, 108, 136], "playfield": 89, "lap_complete_perc": 89, "dictat": [89, 90], "percentag": [89, 139], "lap": [89, 124], "domain_random": 89, "variant": [89, 132], "scenario": [89, 109], "background": 89, "demand": 89, "correspondingli": 89, "init": [89, 132, 136], "scheme": [89, 109], "chri": 89, "campbel": 89, "2014": 89, "www": [89, 111, 112, 113, 119, 127], "iforce2d": 89, "net": [89, 111, 112, 136, 141], "b2dtut": 89, "pontryagin": 90, "principl": 90, "engin": [90, 97, 130], "throttl": 90, "why": 90, "pad": 90, "infinit": [90, 100, 108, 111, 122, 133, 139], "lunar_land": 90, "orient": [90, 98, 101, 102], "sum": [90, 98, 99, 100, 101, 102, 105, 106, 107, 108, 111, 139], "closer": [90, 111], "slower": [90, 136], "tilt": 90, "03": 90, "solut": [90, 139], "viewport": 90, "mass": [90, 98, 99, 101, 102, 107], "bodi": [90, 97, 98, 99, 100, 101, 102, 103, 104, 107, 108], "moon": 90, "greater": [90, 92, 93, 94, 95, 99, 104, 105, 107, 111, 139], "awak": 90, "group": [90, 123], "come": [90, 98, 99, 100, 101, 102, 103, 104, 107, 108], "sleep": 90, "littl": [90, 102, 130], "cpu": [90, 136], "wake": 90, "attach": [90, 93, 96, 98, 108], "graviti": [90, 93, 94, 96, 103, 104, 136, 141], "enable_wind": [90, 136], "wind_pow": [90, 122, 136], "turbulence_pow": [90, 136], "later": [90, 136, 139, 141], "booster": 90, "gravit": 90, "wind": [90, 122], "tanh": [90, 141], "sin": [90, 96, 103, 106], "pi": [90, 92, 96], "9999": 90, "rotat": [90, 92, 98, 101, 102, 105, 125, 139], "turbul": [90, 122], "renorm": 90, "harder": 90, "push": [90, 93, 103, 104, 109, 141], "cart": [91, 103, 104, 141], "nois": [91, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "regard": 91, "underpow": 91, "effort": [91, 130], "easier": [91, 118, 126, 136], "ones": [91, 97, 118, 136, 139], "57": [92, 136], "spars": [92, 132], "coars": 92, "barto": [92, 93, 103, 104, 111, 112, 117, 139], "book": [92, 111, 112, 117, 139], "link": [92, 98, 99, 100, 101, 102, 106, 107, 130], "actuat": [92, 93, 101, 102], "swing": [92, 96], "free": [92, 96, 97, 98, 101, 102, 103, 104, 109, 139, 141], "height": [92, 100, 103, 108, 127], "hang": [92, 126], "downward": 92, "seen": [92, 99], "green": [92, 114], "cosin": [92, 103, 106], "theta1": 92, "sine": [92, 103, 106], "theta2": 92, "567": 92, "274": [92, 130], "rel": [92, 101, 102, 132, 134, 136], "4\u03c0": 92, "9\u03c0": 92, "rad": [92, 93, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "incur": [92, 112, 114], "co": [92, 94, 95, 96, 103, 106, 109], "book_or_nip": 92, "neurip": 92, "paper": [92, 103, 114], "nip": 92, "equat": [92, 96], "present": [92, 109, 124, 132, 133, 139], "confirm": [92, 118], "radian": [92, 93, 96, 104], "1996": 92, "touretzki": 92, "mozer": 92, "hasselmo": 92, "ed": 92, "vol": [92, 114], "mit": [92, 136, 139, 141], "proceed": 92, "cc": 92, "1995": 92, "8f1d43620bc6bb580df6e80b0dc05c48": 92, "pdf": [92, 111, 112], "anderson": [93, 103, 104], "neuronlik": [93, 103, 104], "adapt": [93, 103, 104, 112], "un": 93, "frictionless": 93, "balanc": [93, 103, 104, 136, 141], "vari": [93, 103, 104, 141], "418": 93, "untermin": 93, "2095": 93, "allot": 93, "475": 93, "05": [93, 98, 99, 105, 130], "07": [94, 95], "sinusoid": [94, 95], "vallei": [94, 95], "strateg": [94, 95], "hill": [94, 95], "andrew": [94, 95], "moor": [94, 95], "phd": [94, 95, 107], "thesi": [94, 95, 107], "1990": [94, 95], "techreport": [94, 95], "moore90efficientmemori": [94, 95], "author": [94, 95, 109, 136, 139, 141], "william": [94, 95, 141], "institut": [94, 95], "univers": [94, 95], "cambridg": [94, 95], "year": [94, 95, 124], "001": [94, 96, 100, 103, 108, 136, 139], "0025": [94, 95], "collis": [94, 95, 126], "inelast": [94, 95], "upon": [94, 95], "quickli": [94, 139], "penalis": [94, 95, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "0015": 95, "45": [95, 102, 105], "999": [95, 136], "invert": [96, 97, 141], "swingup": 96, "diagram": 96, "meter": 96, "theta": [96, 128], "tau": 96, "counter": 96, "clockwis": 96, "theta_dt": 96, "heta": 96, "2736044": 96, "math": [96, 123], "half": 97, "cheetah": 97, "standup": 97, "facilit": [97, 109], "biomechan": 97, "graphic": [97, 109, 119], "anim": 97, "area": 97, "accur": 97, "octob": 97, "2021": [97, 109], "deepmind": [97, 122], "2022": [97, 124, 130], "everyon": 97, "instruct": [97, 132], "websit": [97, 109, 118, 119, 127, 130], "github": [97, 112, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 132], "framework": 97, "ten": [97, 111, 139], "iverteddoublependulum": 97, "gaussian": 97, "mjsim": 97, "qpo": 97, "qvel": [97, 101, 102], "indirectli": 97, "xml": [97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 123, 130], "tweak": [97, 107], "schulman": [98, 136], "moritz": [98, 136], "levin": [98, 136], "jordan": [98, 136], "abbeel": [98, 136], "advantag": [98, 109, 132], "estim": [98, 133, 136, 141], "torso": [98, 99, 100, 101, 102, 108], "eight": 98, "hing": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "rotor": [98, 99, 100, 101, 102, 107, 108], "hip_4": 98, "right_back_leg": 98, "angle_4": 98, "front": [98, 99, 107], "hip_1": 98, "front_left_leg": 98, "angle_1": 98, "hip_2": 98, "front_right_leg": 98, "angle_2": 98, "hip_3": 98, "back_leg": 98, "angle_3": 98, "deriv": [98, 99, 100, 101, 102, 103, 104, 107, 108, 141], "exclude_current_positions_from_observ": [98, 99, 100, 101, 107, 108], "29": [98, 101, 102], "regardless": [98, 99, 100, 101, 107, 108], "x_posit": [98, 99, 100, 101, 107, 108], "y_posit": [98, 101, 107], "z": [98, 99, 100, 101, 102, 105, 106, 108], "centr": [98, 101, 102], "ankle_1": 98, "ankle_2": 98, "ankle_3": 98, "ankle_4": 98, "19": [98, 101, 102, 105, 117, 130, 136], "23": [98, 101, 102, 105, 109, 122, 130], "use_contact_forc": [98, 130], "translat": [98, 101, 102, 130, 132], "dof": [98, 101, 102], "quaternion": [98, 101, 102], "healthy_reward": [98, 100, 101, 108], "healthi": [98, 100, 101, 108], "forward_reward": [98, 99, 100, 101, 107, 108], "dt": [98, 99, 100, 101, 105, 106, 107, 108], "frametim": [98, 99, 100, 101, 107, 108], "ctrl_cost": [98, 99, 100, 101, 107, 108], "too": [98, 99, 100, 101, 102, 103, 105, 106, 107, 108], "ctrl_cost_weight": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "ctr_cost_weight": 98, "contact_cost": [98, 101], "contact_cost_weight": [98, 101], "contact_force_rang": 98, "reset_noise_scal": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "deviat": [98, 99, 103, 107, 136, 141], "intention": [98, 101, 102], "therebi": [98, 101, 102], "unhealthi": [98, 100, 101, 108], "healthy_z_rang": [98, 100, 101, 108], "terminate_when_unhealthi": [98, 100, 101, 108], "xml_file": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "path": [98, 99, 100, 101, 105, 106, 107, 108, 113, 136], "model": [98, 99, 100, 101, 107, 108, 109, 130, 139], "weight": [98, 99, 100, 101, 105, 106, 107, 108, 109, 127, 133, 134, 141], "5e": [98, 100, 101, 108], "perturb": [98, 99, 100, 101, 107, 108], "induct": [98, 99, 100, 101, 107, 108], "bia": [98, 99, 100, 101, 107, 108, 136], "induc": [98, 99, 100, 101, 107, 108], "agnost": [98, 99, 100, 101, 102, 107, 108], "bind": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 122], "restor": 98, "camera": [98, 99, 100, 101, 102, 103, 104, 107, 108], "max_time_step": [98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108], "p": [99, 109, 112, 113, 114, 136], "wawrzy\u0144ski": 99, "cat": 99, "paw": 99, "alloc": 99, "thigh": [99, 100, 101, 102, 108], "shin": [99, 101, 102], "feet": [99, 108], "bthigh": 99, "bshin": 99, "bfoot": 99, "fthigh": 99, "fshin": 99, "ffoot": 99, "tip": [99, 103, 107], "rootz": [99, 100, 108], "slide": [99, 100, 103, 104, 105, 106, 107, 108], "rooti": [99, 100, 108], "rootx": [99, 100, 108], "forward_reward_weight": [99, 100, 101, 107, 108], "half_cheetah": 99, "erez": [100, 101, 102, 108], "tassa": [100, 101, 102, 108], "todorov": [100, 101, 102, 108], "predict": [100, 108, 136, 141], "nonlinear": [100, 108], "compar": [100, 107, 108, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 136, 139], "figur": [100, 108, 139, 141], "middl": [100, 108], "thigh_joint": [100, 108], "leg_joint": [100, 108], "foot_joint": [100, 108], "008": [100, 108], "els": [100, 108, 113, 132, 136, 139], "healthy_state_rang": 100, "fallen": 100, "healthy_angle_rang": [100, 108], "376": [101, 102], "synthesi": [101, 102], "onlin": [101, 102, 109, 111, 112], "abdomen": [101, 102], "pair": [101, 102], "arm": [101, 102, 105, 106, 109], "elbow": [101, 102, 105], "walk": [101, 108, 110, 113, 132], "abdomen_i": [101, 102], "abdomen_z": [101, 102], "abdomen_x": [101, 102], "right_hip_x": [101, 102], "right_thigh": [101, 102], "right_hip_z": [101, 102], "right_hip_i": [101, 102], "right_kne": [101, 102], "left_hip_x": [101, 102], "left_thigh": [101, 102], "left_hip_z": [101, 102], "left_hip_i": [101, 102], "left_kne": [101, 102], "right_shoulder1": [101, 102], "right_shoulder2": [101, 102], "right_elbow": [101, 102], "left_shoulder1": [101, 102], "left_shoulder2": [101, 102], "left_elbow": [101, 102], "378": 101, "root": [101, 102, 130], "lower_waist": [101, 102], "pelvi": [101, 102], "right_upper_arm": [101, 102], "right_lower_arm": [101, 102], "left_upper_arm": [101, 102], "left_lower_arm": [101, 102], "anglular": [101, 102, 104], "aanglular": [101, 102], "33": [101, 102], "34": 101, "35": [101, 102, 130], "37": [101, 102, 130], "39": [101, 102, 130], "41": [101, 102, 130], "43": [101, 102], "velocitti": 101, "cinert": [101, 102], "inertia": [101, 102], "rigid": [101, 102], "intermedi": [101, 102], "nbodi": [101, 102], "henc": [101, 102, 110, 141], "140": [101, 102], "cvel": [101, 102], "qfrc_actuat": [101, 102], "constraint": [101, 102, 103, 109], "nv": [101, 102], "cfrc_ext": [101, 102], "freedom": [101, 102, 103, 134], "dim": [101, 102], "003": 101, "015": 101, "nu": [101, 102], "contact_cost_rang": 101, "lai": 102, "numer": [102, 103, 104, 111, 116, 130, 134, 141], "state_spac": 102, "uph_cost": 102, "upward": 102, "absolut": [102, 104, 108], "overal": 102, "atom": 102, "movement": [102, 132], "though": [102, 117, 119], "inflat": 102, "quad_ctrl_cost": 102, "quad_impact_cost": 102, "000001": 102, "105": [102, 130], "unlik": [102, 103, 104, 105, 106, 141], "beyond": [102, 103, 104, 105, 106, 108, 130], "slider": [103, 104], "unitless": [103, 106], "hinge2": 103, "realist": [103, 109], "accuraci": 103, "approach": [103, 117, 132, 139, 141], "analyt": 103, "alive_bonu": 103, "distance_penalti": 103, "velocity_penalti": 103, "005": [103, 105, 106, 136], "y_coordin": 103, "196": 103, "perpendicularli": 103, "cylind": 105, "effector": [105, 106], "fingertip": [105, 106], "shoulder": 105, "forearm": 105, "wrist": 105, "pan": 105, "r_shoulder_pan_joint": 105, "lift": 105, "r_shoulder_lift_joint": 105, "r_upper_arm_roll_joint": 105, "flex": 105, "r_elbow_flex_joint": 105, "r_forearm_roll_joint": 105, "r_wrist_flex_joint": 105, "r_wrist_roll_joint": 105, "analogi": 105, "tips_arm": 105, "obj_slidex": 105, "obj_slidei": 105, "goal_slidex": 105, "goal_slidei": 105, "sphere": 105, "reward_near": 105, "unattach": [105, 106], "reward_dist": [105, 106, 134], "reward_control": [105, 106], "euclidean": [105, 106], "reward_ctrl": [105, 106, 134], "perman": 105, "323": 105, "rise": 105, "modif": [105, 106, 134], "asset": [105, 106, 109, 113], "gymnasmium": 105, "fixtur": 106, "joint0": 106, "joint1": 106, "target_x": 106, "target_i": 106, "position_fingertip": 106, "position_target": 106, "na": 106, "constantli": 106, "2d": [106, 134], "disk": 106, "radiu": 106, "everyth": 106, "02": 106, "r\u00e9mi": 107, "coulom": 107, "applic": [107, 109], "segment": 107, "articul": [107, 109], "suspend": 107, "subject": 107, "fluid": 107, "viscou": 107, "coeffici": [107, 136], "motor1_rot": 107, "motor2_rot": 107, "\u03b8": 107, "free_body_rot": 107, "slider1": 107, "slider2": 107, "04": 107, "swim": 107, "split": [108, 130], "thigh_left_joint": 108, "leg_left_joint": 108, "foot_left_joint": 108, "ctr_cost": 108, "procedur": [109, 141], "clone": [109, 132], "huge": [109, 119, 127], "3v3": 109, "hundr": 109, "roughli": 109, "tweakabl": 109, "monei": 109, "slime": 109, "volleybal": 109, "motiv": 109, "premad": 109, "foss": 109, "interoper": 109, "extens": [109, 141], "light": 109, "poli": 109, "pinocchio": 109, "meshcat": 109, "web": 109, "bitcraz": 109, "crazyfli": 109, "nanoquadrotor": 109, "endeavor": 109, "gap": 109, "deep": [109, 124, 133, 136, 141], "methodologi": 109, "coverag": 109, "laboratori": 109, "internet": 109, "quadrotor": 109, "casadi": 109, "symbol": 109, "priori": 109, "isaac": 109, "omnivers": 109, "cours": 109, "wide": 109, "account": [109, 126], "yield": 109, "voltag": 109, "duti": 109, "cycl": 109, "emphasi": 109, "percept": 109, "social": 109, "driver": 109, "whose": [109, 111], "uncertain": 109, "merg": [109, 130], "intersect": 109, "roundabout": 109, "latent": 109, "transfer": 109, "miniatur": 109, "racecar": 109, "llvm": 109, "phase": [109, 136], "gcc": 109, "cuda": [109, 136], "p_0": 109, "morl": 109, "cell": [109, 112, 132], "warehous": 109, "storag": 109, "overfit": [109, 136], "predefin": [109, 132], "hyperparamet": [109, 136, 139, 141], "cheap": 109, "surrog": 109, "evolutionari": 109, "tag": [109, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129], "answer": 109, "classif": 109, "dataset": 109, "autom": [109, 118], "prover": 109, "vampir": 109, "theoret": [109, 124], "workshop": 109, "minatari": 109, "bsuit": 109, "great": [109, 134, 139], "focu": [109, 136], "simplic": 109, "comprehens": [109, 119], "crypto": 109, "omg": 109, "toolbox": 109, "minimalist": 109, "synthes": 109, "neuro": 109, "architectur": 109, "integr": 109, "twin": 109, "concept": 109, "seamless": 109, "firmwar": 109, "neuroflight": 109, "frozen": [110, 119, 130], "lake": [110, 119, 130], "suitabl": 110, "card": [111, 139], "dealer": [111, 124, 139], "deck": [111, 139], "jack": 111, "queen": 111, "king": 111, "ac": [111, 139], "usabl": [111, 119, 139], "held": 111, "decid": 111, "bust": [111, 139], "reveal": 111, "facedown": 111, "goe": [111, 127], "outcom": 111, "hand": [111, 141], "sab": [111, 139], "2020": [111, 112], "incompleteidea": [111, 112], "rlbook2020": [111, 112], "48": 112, "4x12": 112, "world": [112, 113, 114], "132": [112, 130], "inspir": 112, "com": [112, 113, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 132, 141], "dennybritz": 112, "blob": [112, 118], "cliff_walk": 112, "latter": 112, "current_row": [112, 113], "nrow": [112, 113, 136], "current_col": [112, 113], "col": [112, 113], "47": 112, "proabil": [112, 114], "slipperi": 113, "extent": 113, "4x4": 113, "perpendicular": 113, "sometim": [113, 132, 134], "is_slipperi": 113, "stool": 113, "franuka": 113, "itch": 113, "io": 113, "rpg": 113, "snow": 113, "tileset": 113, "mel": 113, "tilleri": 113, "cyaneu": 113, "ncol": [113, 136, 139], "time_limit": [113, 114], "8x8": 113, "desc": 113, "map_nam": 113, "preload": 113, "sfff": 113, "fhfh": 113, "fffh": 113, "hffg": 113, "generate_random_map": 113, "toy_text": [113, 116, 119, 127, 139], "frozen_lak": 113, "sfffffff": 113, "ffffffff": 113, "fffhffff": 113, "fffffhff": 113, "fhhfffhf": 113, "fhffhfhf": 113, "fffhfffg": 113, "bug": [113, 119, 120, 121, 122, 130], "passeng": 114, "yellow": 114, "5x5": 114, "maxq": 114, "decomposit": 114, "tom": 114, "dietterich": 114, "pickup": 114, "south": 114, "north": 114, "east": 114, "west": 114, "letter": [114, 122], "taxi_row": 114, "taxi_col": 114, "passenger_loc": 114, "actual": [114, 132, 136], "404": 114, "reachabl": 114, "illeg": 114, "action_mask": 114, "fickl": 114, "Or": 114, "argmax": [114, 139], "q_valu": [114, 139], "pp": 114, "227": 114, "303": 114, "nov": 114, "doi": 114, "1613": 114, "639": 114, "cleaner": 114, "disallow": 114, "loc": [114, 136], "passidx": 114, "maintain": [115, 116, 130], "capabl": [115, 127], "insert": [115, 141], "mainten": [116, 127], "jkterry1": [116, 117], "becam": 116, "datatyp": 116, "orsini": 116, "complianc": 116, "sb3": 116, "amtamasi": 116, "recordepisodestat": 116, "vwxyzjn": [116, 117, 119, 120, 121], "unus": [116, 117, 120], "unnescesari": 116, "newer": 116, "machin": [116, 127, 141], "notabl": [116, 130, 133], "peg": 116, "pend": 116, "bump": [116, 117, 118, 119, 122, 124, 127], "zhiqingxiao": [116, 121, 125], "depreci": 116, "pirata": 116, "mahiuchun": 116, "zlig": [116, 118], "ez": 116, "pickl": [116, 126, 129, 130], "zzyunzhi": 116, "jamesborg46": 116, "minor": [116, 117, 118, 120, 124, 126, 128, 129, 130], "nonbreak": 116, "bit": 116, "typo": [116, 128, 129, 130], "mostli": 116, "cclauss": 116, "readm": [116, 119, 128, 130, 132], "third": [116, 117, 118, 130, 132], "parti": [116, 117, 118, 130, 132], "flake8": 116, "changelog": [116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130], "massiv": [117, 124], "explain": [117, 124, 127], "jessefarebro": [117, 118, 119], "rom": [117, 118], "bash": 117, "autorom": [117, 118], "pip3": 117, "guess": 117, "hotter": 117, "colder": 117, "nchain": 117, "roulett": 117, "kellycoinflip": 117, "rohan138": [117, 118, 119], "tristandeleu": [117, 118, 119, 120], "__setitem__": 117, "jfpettit": 117, "firefoxmetzg": 117, "compli": 117, "redtachyon": [117, 118, 119, 120, 122, 123, 124, 126, 127, 130], "__getitem__": 117, "__len__": 117, "xuehaipan": [117, 118, 119, 126, 130], "gracefulli": 117, "unnecessari": [117, 122], "setup": [117, 123, 132], "resolv": 117, "tuckerbmorgan": 117, "sent": 117, "olimoyo": 117, "registr": [117, 119, 123, 132], "obswrapp": 117, "behav": 117, "raghuspacerajan": 117, "perf_count": 117, "short": 117, "zuoxingdong": 117, "broken": 118, "plugin": [118, 130], "clobber": 118, "hook": [118, 130], "suit": 118, "shorthand": 118, "accident": [118, 122, 124], "refactor": [118, 119, 130], "orderli": 118, "helper": 118, "ahmedo42": 118, "stderr": 118, "compil": 118, "submit": 118, "third_party_environ": 118, "largest": 119, "2524": 119, "site": 119, "gymlibrari": [119, 127], "ml": 119, "curriculum": 119, "reiniti": [119, 141], "expens": 119, "suggest": [119, 136], "kir0ul": [119, 126, 130], "terminal_observ": 119, "nice": 119, "1b15": [119, 125], "seungjaeryanle": 119, "2456": 119, "comment": [119, 122, 130], "consolid": 119, "andrewtanj": [119, 120, 121, 122], "carloslui": 119, "araffin": [119, 124], "pyglet": 119, "frequent": [119, 133], "discreteenv": 119, "hint": [119, 124, 127, 130], "ikamensh": [119, 128, 130], "ton": 119, "cleanup": 119, "markus28": [119, 120, 122, 124, 130], "rushivarora": [119, 120, 121], "jjshoot": [119, 122, 124, 130], "ericonaldo": 119, "adilzouitin": 119, "dramat": 119, "pull": 119, "randomst": 119, "rand": [119, 127], "standard_norm": 119, "consequ": 119, "mt19937": 119, "pcg64": 119, "ahead": 120, "trigaten": 120, "gianlucadecola": [120, 122, 123, 130], "striker": [120, 121], "thrower": [120, 121], "archiv": 120, "rusu24edward": [120, 130], "dsctt": 120, "duburcqa": 120, "videorecord": [121, 130], "2678": 121, "2684": 121, "package_data": 121, "micim": 121, "2683": 121, "1500": 121, "pseudo": [121, 122, 123, 124, 125, 126, 127, 129, 130, 136], "rnd": [121, 122, 123, 124, 125, 126, 127, 129, 130], "thought": [121, 122, 123, 124, 125, 126, 127, 129, 130], "curv": 122, "significantli": [122, 136], "wors": 122, "rodrigodelazcano": [122, 123, 130], "arjun": [122, 124, 126, 127], "kg": [122, 124, 126, 127], "virgilt": 122, "ascii": 122, "googl": 122, "pydoc": 122, "balisujohn": [122, 124, 127, 130], "polygon": 122, "vecenvwrapp": 122, "backend": 122, "backport": 122, "v24": 123, "mujoco_rend": 123, "redund": 123, "quagla": 123, "unpickl": 123, "legaci": 123, "meta": 123, "rescale_act": 123, "subsequ": 124, "smoother": [124, 139], "discourag": 124, "put": [124, 132], "discord": [124, 130], "youni": 124, "2671": 124, "single_rgb_arrai": [124, 127], "2906": 124, "ryanrud": 124, "revert": [124, 125, 127, 130], "kexianshen": 124, "whole": [124, 130], "pyright": 124, "hinter": 124, "strict": [124, 130], "identifi": 124, "psc": [124, 125], "textencod": 124, "johnminelli": 124, "dockerfil": 124, "optimis": 124, "2gb": 124, "5gb": 124, "theden": 124, "younik": [125, 127, 128, 129, 130], "vmoen": [125, 128, 130], "edit": [125, 132], "rephras": 125, "evanmath": 125, "ice1187": 125, "humanoidstandup_v4": 125, "mujoco_env": 125, "vector_list_info": 125, "timgates42": 125, "lin826": [125, 127], "fairli": [126, 141], "_timelimit": 126, "envpool": 126, "__eq__": 126, "8gb": 126, "subse": 126, "olipinski": [126, 130], "metaclass": 126, "youjiacheng": [126, 127, 128], "swig": 126, "annot": [126, 130, 136, 139, 141], "bkrl": 126, "aren": 127, "semant": 127, "terribli": 127, "wrong": 127, "patch": [127, 139], "ve": 127, "stabl": [127, 132], "tremend": 127, "appreci": 127, "patienc": 127, "gone": 127, "hopefulli": [127, 139], "v21": [127, 130], "3040": 127, "3016": 127, "get_stat": 127, "set_stat": 127, "hash_se": 127, "create_se": 127, "_bigint_from_byt": 127, "_int_list_from_bigint": 127, "envapicompat": 127, "3041": 127, "fieteo": 127, "offscreen": [127, 130], "3044": 127, "headless": 127, "3037": 127, "renam": [127, 141], "3036": 127, "3072": 128, "3076": 128, "3080": 128, "foxik": 128, "3083": 128, "rdnfn": 128, "vermouth1992": [129, 130], "harden": [129, 130], "permiss": [129, 130], "sashashura": [129, 130], "clarifi": [129, 130], "ekalosak": [129, 130], "2023": 130, "pypi": 130, "bool8": 130, "bool_": 130, "221": 130, "272": 130, "silent": 130, "ianyfan": 130, "230": 130, "254": 130, "assert": [130, 132], "215": 130, "216": 130, "prompt": 130, "252": 130, "259": 130, "rafaelcp": 130, "277": 130, "gt": 130, "5x": 130, "paulmest": 130, "235": 130, "nidhish": 130, "238": 130, "241": 130, "225": 130, "mgoulao": 130, "257": 130, "246": 130, "explicit": 130, "fail": 130, "pierremardon": 130, "267": 130, "doctest": 130, "valentin": 130, "cnt": 130, "a2c": [130, 135, 137], "till2": 130, "234": 130, "kallinteri": 130, "andrea": 130, "206": 130, "218": 130, "219": 130, "mention": [130, 141], "migrat": 130, "lpizzinidev": 130, "244": 130, "hooper": 130, "247": 130, "convolv": [130, 136, 139], "264": 130, "270": 130, "258": 130, "thank": 130, "contributor": 130, "server": 130, "qualiti": 130, "backlog": 130, "mujocorend": 130, "roadmap": 130, "publicli": 130, "overhaul": 130, "feedback": 130, "interest": 130, "act": [130, 134, 139], "compartment": 130, "funcjaxenv": 130, "potenti": 130, "mechan": 130, "thread": 130, "opengl": 130, "mujocorender": 130, "112": 130, "apply_env_compat": 130, "gymv22environ": 130, "125": 130, "pretti": 130, "print": [130, 132, 134, 136, 141], "kad99kev": 130, "124": 130, "141": 130, "72": 130, "190": 130, "siddarth": [130, 141], "155": 130, "theo": 130, "brown": 130, "74": 130, "mujocoenv": 130, "83": 130, "tobirohr": 130, "89": 130, "117": 130, "thytu": 130, "__all__": 130, "130": 130, "is_slippi": 130, "marionj": 130, "136": 130, "143": 130, "172": 130, "173": 130, "174": 130, "139": 130, "164": 130, "axb2035": 130, "171": 130, "step_wait": 130, "aaronwalsman": 130, "178": 130, "183": 130, "cool": 130, "rr": 130, "184": 130, "186": 130, "208": 130, "codebas": [130, 132], "205": 130, "73": 130, "78": 130, "workflow": 130, "110": 130, "138": 130, "146": 130, "rerun": 130, "commit": 130, "179": 130, "yet": 130, "meantim": 130, "61": 130, "episodestatisticsrecord": 130, "davidslayback": 130, "overflow": 130, "55": 130, "vairodp": 130, "willdudlei": 130, "51": 130, "58": 130, "upstream": 130, "09": 130, "conduct": 130, "overview": [132, 141], "virtual": 132, "git": 132, "venv": 132, "bin": 132, "concern": [132, 141], "gym_exampl": 132, "grid_world": 132, "relative_posit": 132, "reacher_weighted_reward": 132, "discrete_act": 132, "clip_reward": 132, "illustr": [132, 133], "simplist": 132, "gridworldenv": 132, "piec": 132, "abstract": 132, "window_s": 132, "512": 132, "_action_to_direct": 132, "clock": 132, "_get_ob": 132, "mandatori": 132, "_agent_loc": 132, "_target_loc": 132, "manhattan": 132, "_get_info": 132, "oftentim": 132, "worri": 132, "coincid": 132, "earlier": 132, "array_equ": 132, "_render_fram": 132, "trivial": 132, "skeleton": 132, "set_mod": 132, "canva": 132, "pix_square_s": 132, "rect": 132, "gridlin": 132, "width": 132, "visibl": 132, "blit": 132, "get_rect": 132, "pump": 132, "tick": 132, "surfarrai": 132, "pixels3d": 132, "bother": 132, "detect": 132, "appropri": [132, 141], "entrypoint": 132, "mere": 132, "setuptool": 132, "install_requir": 132, "local": 132, "perfectli": 132, "fine": [132, 136], "relativeposit": [132, 134], "ipynb": [132, 133, 134, 136, 139, 141], "did": 133, "failur": 133, "inher": 133, "preserv": 133, "obviou": 133, "forev": 133, "forcibli": 133, "halt": 133, "idea": [133, 139], "tell": [133, 139], "formal": 133, "q_": 133, "o_t": 133, "a_t": 133, "r_t": 133, "max_a": 133, "o_": 133, "a_": [133, 136], "q_target": 133, "distinct": 133, "differenti": 133, "vf_target": 133, "vf_next_stat": 133, "onward": 133, "handling_time_limit": 133, "agent_posit": 134, "target_posit": 134, "discreteact": 134, "disc_to_cont": 134, "__name__": [134, 136], "__main__": 134, "intrins": 134, "clipreward": 134, "nevertheless": 134, "reacherrewardwrapp": 134, "reward_dist_weight": 134, "reward_ctrl_weight": 134, "implementing_custom_wrapp": 134, "a3c": 136, "quicker": 136, "robust": 136, "scratch": [136, 141], "feed": 136, "minibatch": 136, "n_steps_per_upd": 136, "n_env": 136, "gradient": [136, 141], "gae": 136, "tradeoff": 136, "logit": 136, "select_act": 136, "prob": [136, 141], "ent_coef": 136, "encourag": 136, "get_loss": 136, "update_paramet": 136, "zemann": [136, 139], "__future__": [136, 139, 141], "o": [136, 141], "pyplot": [136, 139, 141], "plt": [136, 139, 141], "nn": [136, 141], "tqdm": [136, 139], "maxim": [136, 141], "arg": [136, 139, 141], "n_featur": 136, "n_action": 136, "gpu": 136, "larger": 136, "critic_lr": 136, "actor_lr": 136, "critic_lay": 136, "relu": 136, "actor_lay": 136, "fed": 136, "softmax": 136, "sequenti": [136, 141], "critic_optim": 136, "rmsprop": 136, "lr": [136, 139, 141], "actor_optim": 136, "state_valu": [136, 139], "action_logits_vec": 136, "action_log_prob": 136, "action_logit": 136, "action_pd": 136, "implicitli": 136, "log_prob": [136, 141], "value_pr": 136, "lam": 136, "arxiv": 136, "1506": 136, "02438": 136, "mont": [136, 139, 141], "carlo": [136, 139, 141], "td": 136, "bias": 136, "critic_loss": 136, "actor_loss": 136, "td_error": 136, "pow": 136, "detach": 136, "zero_grad": [136, 141], "epoch": 136, "speedup": 136, "theirfor": 136, "simplest": 136, "parameter": [136, 141], "a_min": 136, "a_max": 136, "proport": 136, "mitig": 136, "remak": 136, "hyperparam": 136, "n_updat": 136, "randomize_domain": 136, "95": [136, 139], "stationari": 136, "obs_shap": 136, "action_shap": 136, "use_cuda": 136, "is_avail": 136, "notic": 136, "envs_wrapp": 136, "bar": 136, "sample_phas": 136, "ep_value_pr": 136, "ep_reward": 136, "ep_action_log_prob": 136, "s_": 136, "state_value_pr": 136, "r_": 136, "squeez": 136, "ongo": 136, "inlin": 136, "rolling_length": [136, 139], "fig": [136, 139], "subplot": [136, 139], "figsiz": [136, 139, 141], "suptitl": [136, 139], "__class__": 136, "set_titl": [136, 139], "episode_returns_moving_averag": 136, "arang": [136, 139], "set_xlabel": [136, 139], "entropy_moving_averag": 136, "critic_losses_moving_averag": 136, "actor_losses_moving_averag": 136, "tight_layout": [136, 139], "karp": 136, "flatt": 136, "120": 136, "enough": 136, "runtim": 136, "save_weight": 136, "load_weight": 136, "actor_weights_path": 136, "actor_weight": 136, "h5": 136, "critic_weights_path": 136, "critic_weight": 136, "mkdir": 136, "state_dict": 136, "load_state_dict": 136, "eval": 136, "n_showcase_episod": 136, "no_grad": 136, "mnih": 136, "badia": 136, "mirza": 136, "grave": 136, "lillicrap": 136, "harlei": 136, "silver": 136, "kavukcuoglu": 136, "icml": 136, "2016": 136, "iclr": 136, "vector_envs_tutori": 136, "tutorials_python": 137, "zip": [137, 141], "tutorials_jupyt": 137, "casino": 139, "infam": 139, "beatabl": 139, "won": 139, "viabl": 139, "strategi": 139, "favorit": 139, "defaultdict": 139, "seaborn": [139, 141], "sn": [139, 141], "explanatori": 139, "slow": 139, "showcas": 139, "respond": 139, "harm": 139, "greedi": 139, "blackjackag": 139, "learning_r": [139, 141], "initial_epsilon": 139, "epsilon_decai": 139, "final_epsilon": 139, "discount_factor": 139, "decai": 139, "training_error": 139, "get_act": 139, "greedili": 139, "exploit": 139, "next_ob": [139, 141], "future_q_valu": 139, "temporal_differ": 139, "decay_epsilon": 139, "readi": 139, "n_episod": 139, "100_000": 139, "start_epsilon": 139, "decent": [139, 141], "converg": 139, "10x": 139, "reward_moving_averag": 139, "length_moving_averag": 139, "training_error_moving_averag": 139, "create_grid": 139, "usable_ac": 139, "action_valu": 139, "player_count": 139, "dealer_count": 139, "meshgrid": 139, "apply_along_axi": 139, "arr": 139, "dstack": 139, "value_grid": 139, "policy_grid": 139, "create_plot": 139, "figaspect": 139, "fontsiz": 139, "ax1": 139, "add_subplot": 139, "plot_surfac": 139, "rstride": 139, "cstride": 139, "cmap": 139, "viridi": 139, "edgecolor": 139, "xtick": 139, "ytick": 139, "set_ylabel": 139, "zaxi": 139, "set_rotate_label": 139, "set_zlabel": 139, "view_init": 139, "220": 139, "ax2": 139, "heatmap": 139, "linewidth": 139, "accent_r": 139, "cbar": 139, "set_xticklabel": 139, "set_yticklabel": 139, "legend": 139, "legend_el": 139, "facecolor": 139, "lightgreen": 139, "label": 139, "grei": 139, "bbox_to_anchor": 139, "fig1": 139, "fig2": 139, "grip": 139, "challeng": 139, "fun": [139, 141], "blackjack_tutori": 139, "earliest": 141, "burden": 141, "inverted_pendulum": 141, "1d": 141, "panda": 141, "pd": 141, "rcparam": 141, "policy_network": 141, "obs_space_dim": 141, "action_space_dim": 141, "hidden_space1": 141, "feel": 141, "hidden_space2": 141, "shared_net": 141, "policy_mean_net": 141, "std": 141, "policy_stddev_net": 141, "action_mean": 141, "action_stddev": 141, "shared_featur": 141, "exp": 141, "fact": 141, "reinfroc": 141, "acronym": 141, "ward": 141, "ncrement": 141, "actor": 141, "ffset": 141, "einforc": 141, "haracterist": 141, "ligibl": 141, "tune": 141, "ep": 141, "adamw": 141, "sample_act": 141, "distrib": 141, "running_g": 141, "delta": 141, "centric": 141, "brittl": 141, "spinningup": 141, "en": 141, "html": 141, "total_num_episod": 141, "5e3": 141, "rewards_over_se": 141, "fibonacci": 141, "manual_se": 141, "reward_over_episod": 141, "avg_reward": 141, "rewards_to_plot": 141, "df1": 141, "datafram": 141, "melt": 141, "inplac": 141, "darkgrid": 141, "talk": 141, "palett": 141, "rainbow": 141, "lineplot": 141, "chandrasekar": 141, "ronald": 141, "connectionist": 141, "2004": 141, "229": 141, "256": 141, "reinforce_invpend_gym_v26": 141}, "objects": {"": [[15, 0, 1, "", "action_space"], [15, 0, 1, "", "observation_space"], [15, 0, 1, "", "single_action_space"], [15, 0, 1, "", "single_observation_space"]], "gymnasium": [[17, 1, 1, "", "ActionWrapper"], [2, 1, 1, "", "Env"], [19, 1, 1, "", "ObservationWrapper"], [20, 1, 1, "", "RewardWrapper"], [16, 1, 1, "", "Wrapper"], [8, 3, 1, "", "make"], [8, 3, 1, "", "pprint_registry"], [8, 3, 1, "", "register"], [9, 5, 0, "-", "spaces"], [8, 3, 1, "", "spec"], [16, 5, 0, "-", "wrappers"]], "gymnasium.ActionWrapper": [[17, 2, 1, "", "action"]], "gymnasium.Env": [[2, 0, 1, "", "action_space"], [2, 3, 1, "", "close"], [2, 0, 1, "", "metadata"], [2, 4, 1, "", "np_random"], [2, 0, 1, "", "observation_space"], [2, 3, 1, "", "render"], [2, 0, 1, "", "render_mode"], [2, 3, 1, "", "reset"], [2, 0, 1, "", "reward_range"], [2, 0, 1, "", "spec"], [2, 3, 1, "", "step"], [2, 4, 1, "", "unwrapped"]], "gymnasium.ObservationWrapper": [[19, 2, 1, "", "observation"]], "gymnasium.RewardWrapper": [[20, 2, 1, "", "reward"]], "gymnasium.Wrapper": [[16, 4, 1, "", "action_space"], [16, 3, 1, "", "close"], [16, 4, 1, "", "metadata"], [16, 4, 1, "", "np_random"], [16, 4, 1, "", "observation_space"], [16, 3, 1, "", "reset"], [16, 4, 1, "", "reward_range"], [16, 4, 1, "", "spec"], [16, 3, 1, "", "step"], [16, 4, 1, "", "unwrapped"]], "gymnasium.experimental": [[4, 1, 1, "", "FuncEnv"]], "gymnasium.experimental.FuncEnv": [[4, 3, 1, "", "initial"], [4, 3, 1, "", "observation"], [4, 3, 1, "", "render_close"], [4, 3, 1, "", "render_image"], [4, 3, 1, "", "render_init"], [4, 3, 1, "", "reward"], [4, 3, 1, "", "state_info"], [4, 3, 1, "", "step_info"], [4, 3, 1, "", "terminal"], [4, 3, 1, "", "transform"], [4, 3, 1, "", "transition"]], "gymnasium.experimental.wrappers": [[7, 1, 1, "", "AtariPreprocessingV0"], [7, 1, 1, "", "AutoresetV0"], [7, 1, 1, "", "ClipActionV0"], [7, 1, 1, "", "ClipRewardV0"], [7, 1, 1, "", "DelayObservationV0"], [7, 1, 1, "", "DtypeObservationV0"], [7, 1, 1, "", "FilterObservationV0"], [7, 1, 1, "", "FlattenObservationV0"], [7, 1, 1, "", "FrameStackObservationV0"], [7, 1, 1, "", "GrayscaleObservationV0"], [7, 1, 1, "", "HumanRenderingV0"], [7, 1, 1, "", "JaxToNumpyV0"], [7, 1, 1, "", "JaxToTorchV0"], [7, 1, 1, "", "LambdaActionV0"], [7, 1, 1, "", "LambdaObservationV0"], [7, 1, 1, "", "LambdaRewardV0"], [7, 1, 1, "", "NormalizeObservationV0"], [7, 1, 1, "", "NormalizeRewardV0"], [7, 1, 1, "", "NumpyToTorchV0"], [7, 1, 1, "", "OrderEnforcingV0"], [7, 1, 1, "", "PassiveEnvCheckerV0"], [7, 1, 1, "", "PixelObservationV0"], [7, 1, 1, "", "RecordEpisodeStatisticsV0"], [7, 1, 1, "", "RecordVideoV0"], [7, 1, 1, "", "RenderCollectionV0"], [7, 1, 1, "", "RescaleActionV0"], [7, 1, 1, "", "RescaleObservationV0"], [7, 1, 1, "", "ReshapeObservationV0"], [7, 1, 1, "", "ResizeObservationV0"], [7, 1, 1, "", "StickyActionV0"], [7, 1, 1, "", "TimeAwareObservationV0"]], "gymnasium.spaces": [[11, 1, 1, "", "Box"], [10, 1, 1, "", "Dict"], [11, 1, 1, "", "Discrete"], [10, 1, 1, "", "Graph"], [11, 1, 1, "", "MultiBinary"], [11, 1, 1, "", "MultiDiscrete"], [10, 1, 1, "", "Sequence"], [9, 1, 1, "", "Space"], [11, 1, 1, "", "Text"], [10, 1, 1, "", "Tuple"]], "gymnasium.spaces.Box": [[11, 2, 1, "", "is_bounded"], [11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Dict": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.Discrete": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Graph": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.MultiBinary": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.MultiDiscrete": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Sequence": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.Space": [[9, 4, 1, "", "dtype"]], "gymnasium.spaces.Text": [[11, 2, 1, "", "sample"], [11, 2, 1, "", "seed"]], "gymnasium.spaces.Tuple": [[10, 2, 1, "", "sample"], [10, 2, 1, "", "seed"]], "gymnasium.spaces.space.Space": [[9, 3, 1, "", "contains"], [9, 3, 1, "", "from_jsonable"], [9, 4, 1, "", "is_np_flattenable"], [9, 3, 1, "", "sample"], [9, 3, 1, "", "seed"], [9, 4, 1, "", "shape"], [9, 3, 1, "", "to_jsonable"]], "gymnasium.spaces.utils": [[12, 3, 1, "", "flatdim"], [12, 3, 1, "", "flatten"], [12, 3, 1, "", "flatten_space"], [12, 3, 1, "", "unflatten"]], "gymnasium.utils.env_checker": [[14, 3, 1, "", "check_env"]], "gymnasium.utils.play": [[14, 1, 1, "", "PlayPlot"], [14, 1, 1, "", "PlayableGame"], [14, 3, 1, "", "play"]], "gymnasium.utils.play.PlayPlot": [[14, 2, 1, "", "callback"]], "gymnasium.utils.play.PlayableGame": [[14, 2, 1, "", "process_event"]], "gymnasium.utils.save_video": [[14, 3, 1, "", "capped_cubic_video_schedule"], [14, 3, 1, "", "save_video"]], "gymnasium.utils.seeding": [[14, 3, 1, "", "np_random"]], "gymnasium.utils.step_api_compatibility": [[14, 3, 1, "", "convert_to_done_step_api"], [14, 3, 1, "", "convert_to_terminated_truncated_step_api"], [14, 3, 1, "", "step_api_compatibility"]], "gymnasium.vector": [[15, 1, 1, "", "AsyncVectorEnv"], [15, 1, 1, "", "SyncVectorEnv"], [15, 1, 1, "", "VectorEnv"], [15, 3, 1, "", "make"]], "gymnasium.vector.VectorEnv": [[15, 2, 1, "", "close"], [15, 2, 1, "", "reset"], [15, 2, 1, "", "step"]], "gymnasium.vector.utils": [[13, 3, 1, "", "batch_space"], [13, 3, 1, "", "concatenate"], [13, 3, 1, "", "create_empty_array"], [13, 3, 1, "", "create_shared_memory"], [13, 3, 1, "", "iterate"], [13, 3, 1, "", "read_from_shared_memory"], [13, 3, 1, "", "write_to_shared_memory"]], "gymnasium.wrappers": [[18, 1, 1, "", "AtariPreprocessing"], [18, 1, 1, "", "AutoResetWrapper"], [17, 1, 1, "", "ClipAction"], [18, 1, 1, "", "EnvCompatibility"], [19, 1, 1, "", "FilterObservation"], [19, 1, 1, "", "FlattenObservation"], [19, 1, 1, "", "FrameStack"], [19, 1, 1, "", "GrayScaleObservation"], [18, 1, 1, "", "HumanRendering"], [19, 1, 1, "", "NormalizeObservation"], [20, 1, 1, "", "NormalizeReward"], [18, 1, 1, "", "OrderEnforcing"], [18, 1, 1, "", "PassiveEnvChecker"], [19, 1, 1, "", "PixelObservationWrapper"], [18, 1, 1, "", "RecordEpisodeStatistics"], [18, 1, 1, "", "RecordVideo"], [18, 1, 1, "", "RenderCollection"], [17, 1, 1, "", "RescaleAction"], [19, 1, 1, "", "ResizeObservation"], [18, 1, 1, "", "StepAPICompatibility"], [19, 1, 1, "", "TimeAwareObservation"], [18, 1, 1, "", "TimeLimit"], [19, 1, 1, "", "TransformObservation"], [20, 1, 1, "", "TransformReward"], [18, 1, 1, "", "VectorListInfo"]], "gymnasium.wrappers.gymnasium.Wrapper": [[16, 0, 1, "", "env"]]}, "objtypes": {"0": "py:attribute", "1": "py:class", "2": "py:method", "3": "py:function", "4": "py:property", "5": "py:module"}, "objnames": {"0": ["py", "attribute", "Python attribute"], "1": ["py", "class", "Python class"], "2": ["py", "method", "Python method"], "3": ["py", "function", "Python function"], "4": ["py", "property", "Python property"], "5": ["py", "module", "Python module"]}, "titleterms": {"404": 0, "page": [0, 1], "Not": 0, "found": 0, "The": [0, 9], "request": 0, "could": 0, "gymnasium": [1, 2, 4, 5, 15, 16, 115, 131, 132, 134, 135, 137], "doc": 1, "instruct": 1, "modifi": [1, 21], "environ": [1, 2, 3, 4, 5, 6, 7, 8, 14, 15, 21, 22, 23, 109, 115, 119, 132, 136, 139], "edit": 1, "an": [1, 139, 141], "ad": 1, "new": [1, 14, 116, 117], "atari": [1, 24, 43], "env": [1, 2, 15, 109, 132, 136], "non": 1, "other": [1, 7, 109, 116, 119], "step": [1, 14, 22, 23, 132], "build": [1, 139, 141], "document": 1, "write": 1, "tutori": [1, 137], "method": [2, 9, 15, 16], "attribut": [2, 9, 15, 16], "addit": 2, "implement": [2, 134], "experiment": [3, 4, 5], "function": [3, 4], "wrapper": [3, 6, 7, 16, 17, 18, 19, 20, 23, 132, 134], "observ": [3, 6, 7, 19, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 132, 139], "action": [3, 6, 7, 17, 21, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 139], "reward": [3, 6, 7, 20, 24, 27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "common": [3, 6, 24], "render": [3, 7, 14, 23, 132], "data": [3, 7], "convers": [3, 7], "vector": [3, 5, 6, 9, 13, 15, 136], "funcenv": 4, "func2env": 4, "functionaljaxcompatibilityenv": 4, "vectorenv": [5, 15], "asyncvectorenv": 5, "syncvectorenv": 5, "custom": [5, 132, 134], "envpool": 5, "lambda": 6, "onli": 6, "registri": 8, "make": [8, 15, 132], "regist": [8, 132], "all": 8, "spec": 8, "pretti": 8, "print": 8, "space": [9, 10, 11, 12, 13, 21, 24, 50, 51, 52, 53, 54, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "base": [9, 17, 19, 20], "class": [9, 17, 19, 20], "fundament": [9, 11], "composit": [9, 10], "util": [9, 12, 13, 14], "dict": 10, "tupl": 10, "sequenc": 10, "graph": 10, "box": [11, 38], "discret": [11, 109], "multibinari": 11, "multidiscret": 11, "text": [11, 110], "share": 13, "memori": 13, "visual": [14, 139], "save": [14, 136], "video": [14, 84, 109], "old": 14, "api": [14, 22, 115, 119, 124], "compat": [14, 22, 127], "seed": [14, 23], "check": 14, "async": 15, "sync": 15, "avail": [17, 19, 20], "misc": [18, 124], "basic": [21, 131, 135, 137], "usag": 21, "initi": [21, 130, 132], "interact": 21, "explain": 21, "code": [21, 23, 133], "more": 21, "inform": [21, 111, 112, 113, 114], "gym": [22, 109, 130], "load": [22, 136], "openai": [22, 109], "v0": [22, 116, 117, 118, 119, 120, 121, 122, 123, 124, 126, 127, 128, 129, 130], "22": [22, 119], "v21": 23, "v26": 23, "migrat": 23, "guid": 23, "exampl": 23, "random": [23, 136], "number": 23, "gener": 23, "reset": [23, 89, 132], "timelimit": 23, "remov": 23, "autorom": 24, "instal": 24, "rom": 24, "stochast": 24, "argument": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "version": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114], "histori": [24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "name": [24, 64], "scheme": 24, "flavor": 24, "refer": [24, 89, 92, 111, 112, 114, 115, 136, 141], "adventur": 25, "descript": [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "air": 26, "raid": 26, "alien": 27, "amidar": 28, "assault": 29, "asterix": 30, "asteroid": 31, "atlanti": 32, "bank": 33, "heist": 33, "battl": 34, "zone": 34, "beam": 35, "rider": 35, "berzerk": 36, "bowl": 37, "breakout": 39, "carniv": 40, "centiped": 41, "chopper": 42, "command": 42, "complet": 43, "list": 43, "crazi": 44, "climber": 44, "defend": 45, "demon": 46, "attack": 46, "doubl": [47, 103], "dunk": 47, "elev": 48, "enduro": 49, "fishingderbi": 50, "freewai": 51, "frostbit": 52, "gopher": 53, "gravitar": 54, "hero": 55, "icehockei": 56, "jamesbond": 57, "journeyescap": 58, "kangaroo": 59, "krull": 60, "kung": 61, "fu": 61, "master": 61, "montezuma": 62, "reveng": 62, "m": 63, "pacman": 63, "thi": 64, "game": [64, 109], "phoenix": 65, "pitfal": 66, "pong": 67, "pooyan": 68, "privateey": 69, "qbert": 70, "riverraid": 71, "road": 72, "runner": 72, "robot": [73, 109], "tank": 73, "seaquest": 74, "ski": 75, "solari": 76, "spaceinvad": 77, "stargunn": 78, "tenni": 79, "timepilot": 80, "tutankham": 81, "up": 82, "n": 82, "down": 82, "ventur": 83, "pinbal": 84, "wizard": 85, "wor": 85, "zaxxon": 86, "box2d": 87, "biped": 88, "walker": 88, "start": [88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "state": [88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 132], "episod": [88, 89, 90, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "termin": [88, 89, 90, 133], "credit": [88, 89, 90], "car": [89, 94, 95], "race": [89, 109], "lunar": 90, "lander": 90, "classic": [91, 109], "control": [91, 109], "acrobot": 92, "end": [92, 93, 94, 95, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114], "cart": 93, "pole": 93, "mountain": [94, 95], "transit": [94, 95], "dynam": [94, 95, 109], "continu": 95, "pendulum": [96, 103, 104], "truncat": [96, 133], "mujoco": [97, 141], "ant": 98, "half": 99, "cheetah": 99, "hopper": 100, "humanoid": [101, 102], "standup": 102, "invert": [103, 104], "pusher": 105, "reacher": 106, "swimmer": 107, "note": [107, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130], "walker2d": 108, "third": 109, "parti": 109, "stabl": 109, "retro": 109, "maintain": 109, "flappi": 109, "bird": 109, "A": 109, "derk": 109, "gpu": 109, "acceler": 109, "moba": 109, "slimevolleygym": 109, "simpl": 109, "singl": 109, "multi": 109, "agent": [109, 136, 137, 138, 139, 140, 141], "reinforc": [109, 115, 141], "learn": [109, 115, 133, 139, 141], "uniti": 109, "ml": 109, "engin": 109, "pge": 109, "parallel": 109, "jimini": 109, "train": [109, 136, 137, 138, 139, 140, 141], "pybullet": 109, "drone": 109, "quadcopt": 109, "marsexplor": 109, "mar": 109, "panda": 109, "us": [109, 132, 136, 141], "physic": 109, "robo": 109, "real": 109, "world": 109, "simul": 109, "offworld": 109, "safe": 109, "softrobot": 109, "soft": 109, "igibson": 109, "dexteroushand": 109, "dual": 109, "dexter": 109, "hand": 109, "manipul": 109, "task": 109, "omniisaacgymenv": 109, "autonom": 109, "drive": 109, "sumo": 109, "rl": 109, "duckietown": 109, "electr": 109, "motor": 109, "highwai": 109, "commonroad": 109, "tmrl": 109, "trackmania": 109, "2020": 109, "through": 109, "racing_dream": 109, "l2r": 109, "multimod": 109, "where": 109, "how": 109, "racecar_gym": 109, "compilergym": 109, "optimis": 109, "compil": 109, "carl": 109, "context": 109, "adapt": 109, "matrix": 109, "mdp": 109, "easili": 109, "creat": [109, 132], "mo": 109, "object": 109, "cellular": 109, "automata": 109, "sokoban": 109, "2d": 109, "transport": 109, "puzzl": 109, "dacbench": 109, "benchmark": 109, "librari": 109, "algorithm": 109, "configur": 109, "nlpgym": 109, "toolkit": 109, "develop": 109, "solv": [109, 139], "nlp": 109, "satur": 109, "prove": 109, "theorem": 109, "shinrl": 109, "evalu": 109, "gymnax": 109, "hardwar": 109, "anytrad": 109, "financi": 109, "trade": 109, "forex": 109, "stock": 109, "mtsim": 109, "metatrad": 109, "5": 109, "platform": 109, "openmodelica": 109, "microgrid": 109, "power": 109, "electron": 109, "convert": 109, "mobil": 109, "coordin": 109, "wireless": 109, "network": [109, 141], "gymfc": 109, "flight": 109, "tune": 109, "framework": 109, "toi": 110, "blackjack": [111, 139], "cliff": 112, "walk": 112, "frozen": 113, "lake": 113, "taxi": 114, "i": 115, "standard": 115, "divers": 115, "collect": 115, "19": 116, "0": [116, 117, 118, 120, 122, 124, 125, 127, 130], "releas": [116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130], "featur": [116, 117], "bug": [116, 117, 123, 124, 125, 126, 128, 129], "fix": [116, 117, 123, 124, 125, 126, 128, 129], "20": 117, "major": [117, 122, 127], "chang": [117, 119, 120, 122, 124, 127], "break": [117, 120, 127], "upgrad": 117, "21": 118, "23": [120, 121], "1": [121, 123, 125, 128, 130], "24": [122, 123], "minor": [122, 127], "25": [124, 125, 126], "typo": 125, "2": [126, 129, 130], "updat": 126, "26": [127, 128, 129, 130], "backward": 127, "spell": 128, "27": 130, "3": 130, "your": 132, "own": 132, "subclass": 132, "declar": 132, "construct": 132, "from": [132, 134], "close": 132, "packag": 132, "instanc": 132, "handl": 133, "time": 133, "limit": 133, "import": [133, 139], "solut": 133, "inherit": 134, "observationwrapp": 134, "actionwrapp": 134, "rewardwrapp": 134, "a2c": 136, "domain": 136, "introduct": 136, "advantag": 136, "actor": 136, "critic": 136, "setup": [136, 139], "plot": [136, 141], "perform": 136, "analysi": 136, "synchron": 136, "asynchron": 136, "weight": 136, "showcas": 136, "try": 136, "plai": 136, "yourself": 136, "q": 139, "execut": 139, "visualis": 139, "polici": [139, 141], "think": 139, "you": 139, "can": 139, "do": 139, "better": 139, "curv": 141}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 8, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx": 57}, "alltitles": {"404 - Page Not Found": [[0, "page-not-found"]], "The requested page could not be found.": [[0, "the-requested-page-could-not-be-found"]], "Gymnasium-docs": [[1, "gymnasium-docs"]], "Instructions for modifying environment pages": [[1, "instructions-for-modifying-environment-pages"]], "Editing an environment page": [[1, "editing-an-environment-page"]], "Adding a new environment": [[1, "adding-a-new-environment"]], "Atari env": [[1, "atari-env"]], "Non-Atari env": [[1, "non-atari-env"]], "Other steps": [[1, "other-steps"]], "Build the Documentation": [[1, "build-the-documentation"]], "Writing Tutorials": [[1, "writing-tutorials"]], "Env": [[2, "env"]], "gymnasium.Env": [[2, "gymnasium-env"]], "Methods": [[2, "methods"], [9, "methods"], [15, "methods"], [16, "methods"]], "Attributes": [[2, "attributes"], [9, "attributes"], [15, "attributes"], [16, "attributes"]], "Additional Methods": [[2, "additional-methods"]], "Implementing environments": [[2, "implementing-environments"]], "Experimental": [[3, "experimental"]], "Functional Environments": [[3, "functional-environments"]], "Wrappers": [[3, "wrappers"], [7, "wrappers"], [16, "wrappers"]], "Observation Wrappers": [[3, "observation-wrappers"], [7, "observation-wrappers"], [19, "observation-wrappers"]], "Action Wrappers": [[3, "action-wrappers"], [7, "action-wrappers"], [17, "action-wrappers"]], "Reward Wrappers": [[3, "reward-wrappers"], [7, "reward-wrappers"], [20, "reward-wrappers"]], "Common Wrappers": [[3, "common-wrappers"]], "Rendering Wrappers": [[3, "rendering-wrappers"], [7, "rendering-wrappers"]], "Environment data conversion": [[3, "environment-data-conversion"], [7, "environment-data-conversion"]], "Vector Environment": [[3, "vector-environment"]], "Wrappers for Vector Environments": [[3, "wrappers-for-vector-environments"]], "Functional Environment": [[4, "functional-environment"]], "gymnasium.experimental.FuncEnv": [[4, "gymnasium-experimental-funcenv"]], "gymnasium.experimental.func2env.FunctionalJaxCompatibilityEnv": [[4, "gymnasium-experimental-func2env-functionaljaxcompatibilityenv"]], "Vectorizing Environment": [[5, "vectorizing-environment"]], "gymnasium.experimental.VectorEnv": [[5, "gymnasium-experimental-vectorenv"]], "gymnasium.experimental.vector.AsyncVectorEnv": [[5, "gymnasium-experimental-vector-asyncvectorenv"]], "gymnasium.experimental.vector.SyncVectorEnv": [[5, "gymnasium-experimental-vector-syncvectorenv"]], "Custom Vector environments": [[5, "custom-vector-environments"]], "EnvPool": [[5, "envpool"]], "Vector Environment Wrappers": [[6, "vector-environment-wrappers"]], "Vector Lambda Observation Wrappers": [[6, "vector-lambda-observation-wrappers"]], "Vector Lambda Action Wrappers": [[6, "vector-lambda-action-wrappers"]], "Vector Lambda Reward Wrappers": [[6, "vector-lambda-reward-wrappers"]], "Vector Common Wrappers": [[6, "vector-common-wrappers"]], "Vector Only Wrappers": [[6, "vector-only-wrappers"]], "Other Wrappers": [[7, "other-wrappers"]], "Registry": [[8, "registry"]], "Make": [[8, "make"]], "Register": [[8, "register"]], "All registered environments": [[8, "all-registered-environments"]], "Spec": [[8, "spec"]], "Pretty print registry": [[8, "pretty-print-registry"]], "Spaces": [[9, "spaces"]], "The Base Class": [[9, "the-base-class"]], "Fundamental Spaces": [[9, "fundamental-spaces"], [11, "fundamental-spaces"]], "Composite Spaces": [[9, "composite-spaces"], [10, "composite-spaces"]], "Utils": [[9, "utils"], [14, "utils"]], "Vector Utils": [[9, "vector-utils"]], "Dict": [[10, "dict"]], "Tuple": [[10, "tuple"]], "Sequence": [[10, "sequence"]], "Graph": [[10, "graph"]], "Box": [[11, "box"]], "Discrete": [[11, "discrete"]], "MultiBinary": [[11, "multibinary"]], "MultiDiscrete": [[11, "multidiscrete"]], "Text": [[11, "text"]], "Spaces Utils": [[12, "spaces-utils"]], "Spaces Vector Utils": [[13, "spaces-vector-utils"]], "Shared Memory Utils": [[13, "shared-memory-utils"]], "Visualization": [[14, "visualization"]], "Save Rendering Videos": [[14, "save-rendering-videos"]], "Old to New Step API Compatibility": [[14, "old-to-new-step-api-compatibility"]], "Seeding": [[14, "seeding"]], "Environment Checking": [[14, "environment-checking"]], "Vector": [[15, "vector"]], "Gymnasium.vector.VectorEnv": [[15, "gymnasium-vector-vectorenv"]], "Making Vector Environments": [[15, "making-vector-environments"]], "Async Vector Env": [[15, "async-vector-env"]], "Sync Vector Env": [[15, "sync-vector-env"]], "gymnasium.Wrapper": [[16, "gymnasium-wrapper"]], "Gymnasium Wrappers": [[16, "gymnasium-wrappers"]], "Base Class": [[17, "base-class"], [19, "base-class"], [20, "base-class"]], "Available Action Wrappers": [[17, "available-action-wrappers"]], "Misc Wrappers": [[18, "misc-wrappers"]], "Available Observation Wrappers": [[19, "available-observation-wrappers"]], "Available Reward Wrappers": [[20, "available-reward-wrappers"]], "Basic Usage": [[21, "basic-usage"]], "Initializing Environments": [[21, "initializing-environments"]], "Interacting with the Environment": [[21, "interacting-with-the-environment"]], "Explaining the code": [[21, "explaining-the-code"]], "Action and observation spaces": [[21, "action-and-observation-spaces"]], "Modifying the environment": [[21, "modifying-the-environment"]], "More information": [[21, "more-information"]], "Compatibility with Gym": [[22, "compatibility-with-gym"]], "Loading OpenAI Gym environments": [[22, "loading-openai-gym-environments"]], "Gym v0.22 Environment Compatibility": [[22, "gym-v0-22-environment-compatibility"]], "Step API Compatibility": [[22, "step-api-compatibility"]], "v21 to v26 Migration Guide": [[23, "v21-to-v26-migration-guide"]], "Example code for v21": [[23, "example-code-for-v21"]], "Example code for v26": [[23, "example-code-for-v26"]], "Seed and random number generator": [[23, "seed-and-random-number-generator"]], "Environment Reset": [[23, "environment-reset"]], "Environment Step": [[23, "environment-step"]], "TimeLimit Wrapper": [[23, "timelimit-wrapper"]], "Environment Render": [[23, "environment-render"]], "Removed code": [[23, "removed-code"]], "Atari": [[24, "atari"]], "AutoROM (installing the ROMs)": [[24, "autorom-installing-the-roms"]], "Action Space": [[24, "action-space"], [50, "action-space"], [51, "action-space"], [52, "action-space"], [53, "action-space"], [54, "action-space"], [88, "action-space"], [89, "action-space"], [90, "action-space"], [92, "action-space"], [93, "action-space"], [94, "action-space"], [95, "action-space"], [96, "action-space"], [98, "action-space"], [99, "action-space"], [100, "action-space"], [101, "action-space"], [102, "action-space"], [103, "action-space"], [104, "action-space"], [105, "action-space"], [106, "action-space"], [107, "action-space"], [108, "action-space"], [111, "action-space"], [112, "action-space"], [113, "action-space"], [114, "action-space"]], "Observation Space": [[24, "observation-space"], [88, "observation-space"], [89, "observation-space"], [90, "observation-space"], [92, "observation-space"], [93, "observation-space"], [94, "observation-space"], [95, "observation-space"], [96, "observation-space"], [98, "observation-space"], [99, "observation-space"], [100, "observation-space"], [101, "observation-space"], [102, "observation-space"], [103, "observation-space"], [104, "observation-space"], [105, "observation-space"], [106, "observation-space"], [107, "observation-space"], [108, "observation-space"], [111, "observation-space"], [112, "observation-space"], [113, "observation-space"], [114, "observation-space"]], "Rewards": [[24, "rewards"], [27, "rewards"], [28, "rewards"], [30, "rewards"], [31, "rewards"], [32, "rewards"], [33, "rewards"], [34, "rewards"], [35, "rewards"], [36, "rewards"], [37, "rewards"], [38, "rewards"], [39, "rewards"], [40, "rewards"], [41, "rewards"], [42, "rewards"], [44, "rewards"], [45, "rewards"], [46, "rewards"], [47, "rewards"], [48, "rewards"], [49, "rewards"], [50, "rewards"], [51, "rewards"], [52, "rewards"], [53, "rewards"], [54, "rewards"], [55, "rewards"], [56, "rewards"], [57, "rewards"], [58, "rewards"], [59, "rewards"], [60, "rewards"], [66, "rewards"], [67, "rewards"], [68, "rewards"], [69, "rewards"], [70, "rewards"], [71, "rewards"], [72, "rewards"], [73, "rewards"], [74, "rewards"], [75, "rewards"], [76, "rewards"], [77, "rewards"], [78, "rewards"], [79, "rewards"], [80, "rewards"], [88, "rewards"], [89, "rewards"], [90, "rewards"], [92, "rewards"], [93, "rewards"], [96, "rewards"], [98, "rewards"], [99, "rewards"], [100, "rewards"], [101, "rewards"], [102, "rewards"], [103, "rewards"], [104, "rewards"], [105, "rewards"], [106, "rewards"], [107, "rewards"], [108, "rewards"], [111, "rewards"], [113, "rewards"], [114, "rewards"]], "Stochasticity": [[24, "stochasticity"]], "Common Arguments": [[24, "common-arguments"]], "Version History and Naming Schemes": [[24, "version-history-and-naming-schemes"]], "Flavors": [[24, "flavors"]], "References": [[24, "references"], [89, "references"], [92, "references"], [111, "references"], [112, "references"], [114, "references"], [136, "references"], [141, "references"]], "Adventure": [[25, "adventure"]], "Description": [[25, "description"], [26, "description"], [27, "description"], [28, "description"], [29, "description"], [30, "description"], [31, "description"], [32, "description"], [33, "description"], [34, "description"], [35, "description"], [36, "description"], [37, "description"], [38, "description"], [39, "description"], [40, "description"], [41, "description"], [42, "description"], [44, "description"], [45, "description"], [46, "description"], [47, "description"], [48, "description"], [49, "description"], [50, "description"], [51, "description"], [52, "description"], [53, "description"], [54, "description"], [55, "description"], [56, "description"], [57, "description"], [58, "description"], [59, "description"], [60, "description"], [61, "description"], [62, "description"], [63, "description"], [64, "description"], [65, "description"], [66, "description"], [67, "description"], [68, "description"], [69, "description"], [70, "description"], [71, "description"], [72, "description"], [73, "description"], [74, "description"], [75, "description"], [76, "description"], [77, "description"], [78, "description"], [79, "description"], [80, "description"], [81, "description"], [82, "description"], [83, "description"], [84, "description"], [85, "description"], [86, "description"], [88, "description"], [89, "description"], [90, "description"], [92, "description"], [93, "description"], [94, "description"], [95, "description"], [96, "description"], [98, "description"], [99, "description"], [100, "description"], [101, "description"], [102, "description"], [103, "description"], [104, "description"], [105, "description"], [106, "description"], [107, "description"], [108, "description"], [111, "description"], [112, "description"], [113, "description"], [114, "description"]], "Actions": [[25, "actions"], [26, "actions"], [27, "actions"], [28, "actions"], [29, "actions"], [30, "actions"], [31, "actions"], [32, "actions"], [33, "actions"], [34, "actions"], [35, "actions"], [36, "actions"], [37, "actions"], [38, "actions"], [39, "actions"], [40, "actions"], [41, "actions"], [42, "actions"], [44, "actions"], [45, "actions"], [46, "actions"], [47, "actions"], [48, "actions"], [49, "actions"], [55, "actions"], [56, "actions"], [57, "actions"], [58, "actions"], [59, "actions"], [60, "actions"], [61, "actions"], [62, "actions"], [63, "actions"], [64, "actions"], [65, "actions"], [66, "actions"], [67, "actions"], [68, "actions"], [69, "actions"], [70, "actions"], [71, "actions"], [72, "actions"], [73, "actions"], [74, "actions"], [75, "actions"], [76, "actions"], [77, "actions"], [78, "actions"], [79, "actions"], [80, "actions"], [81, "actions"], [82, "actions"], [83, "actions"], [84, "actions"], [85, "actions"], [86, "actions"]], "Observations": [[25, "observations"], [26, "observations"], [27, "observations"], [28, "observations"], [29, "observations"], [30, "observations"], [31, "observations"], [32, "observations"], [33, "observations"], [34, "observations"], [35, "observations"], [36, "observations"], [37, "observations"], [38, "observations"], [39, "observations"], [40, "observations"], [41, "observations"], [42, "observations"], [44, "observations"], [45, "observations"], [46, "observations"], [47, "observations"], [48, "observations"], [49, "observations"], [50, "observations"], [51, "observations"], [52, "observations"], [53, "observations"], [54, "observations"], [55, "observations"], [56, "observations"], [57, "observations"], [58, "observations"], [59, "observations"], [60, "observations"], [61, "observations"], [62, "observations"], [63, "observations"], [64, "observations"], [65, "observations"], [66, "observations"], [67, "observations"], [68, "observations"], [69, "observations"], [70, "observations"], [71, "observations"], [72, "observations"], [73, "observations"], [74, "observations"], [75, "observations"], [76, "observations"], [77, "observations"], [78, "observations"], [79, "observations"], [80, "observations"], [81, "observations"], [82, "observations"], [83, "observations"], [84, "observations"], [85, "observations"], [86, "observations"]], "Arguments": [[25, "arguments"], [26, "arguments"], [27, "arguments"], [28, "arguments"], [29, "arguments"], [30, "arguments"], [31, "arguments"], [32, "arguments"], [33, "arguments"], [34, "arguments"], [35, "arguments"], [36, "arguments"], [37, "arguments"], [38, "arguments"], [39, "arguments"], [40, "arguments"], [41, "arguments"], [42, "arguments"], [44, "arguments"], [45, "arguments"], [46, "arguments"], [47, "arguments"], [48, "arguments"], [49, "arguments"], [50, "arguments"], [51, "arguments"], [52, "arguments"], [53, "arguments"], [54, "arguments"], [55, "arguments"], [56, "arguments"], [57, "arguments"], [58, "arguments"], [59, "arguments"], [60, "arguments"], [61, "arguments"], [62, "arguments"], [63, "arguments"], [64, "arguments"], [65, "arguments"], [66, "arguments"], [67, "arguments"], [68, "arguments"], [69, "arguments"], [70, "arguments"], [71, "arguments"], [72, "arguments"], [73, "arguments"], [74, "arguments"], [75, "arguments"], [76, "arguments"], [77, "arguments"], [78, "arguments"], [79, "arguments"], [80, "arguments"], [81, "arguments"], [82, "arguments"], [83, "arguments"], [84, "arguments"], [85, "arguments"], [86, "arguments"], [88, "arguments"], [89, "arguments"], [90, "arguments"], [92, "arguments"], [93, "arguments"], [94, "arguments"], [95, "arguments"], [96, "arguments"], [98, "arguments"], [99, "arguments"], [100, "arguments"], [101, "arguments"], [102, "arguments"], [103, "arguments"], [104, "arguments"], [105, "arguments"], [106, "arguments"], [107, "arguments"], [108, "arguments"], [111, "arguments"], [112, "arguments"], [113, "arguments"], [114, "arguments"]], "Version History": [[25, "version-history"], [26, "version-history"], [27, "version-history"], [28, "version-history"], [29, "version-history"], [30, "version-history"], [31, "version-history"], [32, "version-history"], [33, "version-history"], [34, "version-history"], [35, "version-history"], [36, "version-history"], [37, "version-history"], [38, "version-history"], [39, "version-history"], [40, "version-history"], [41, "version-history"], [42, "version-history"], [44, "version-history"], [45, "version-history"], [46, "version-history"], [47, "version-history"], [48, "version-history"], [49, "version-history"], [50, "version-history"], [51, "version-history"], [52, "version-history"], [53, "version-history"], [54, "version-history"], [55, "version-history"], [56, "version-history"], [57, "version-history"], [58, "version-history"], [59, "version-history"], [60, "version-history"], [61, "version-history"], [62, "version-history"], [63, "version-history"], [64, "version-history"], [65, "version-history"], [66, "version-history"], [67, "version-history"], [68, "version-history"], [69, "version-history"], [70, "version-history"], [71, "version-history"], [72, "version-history"], [73, "version-history"], [74, "version-history"], [75, "version-history"], [76, "version-history"], [77, "version-history"], [78, "version-history"], [79, "version-history"], [80, "version-history"], [81, "version-history"], [82, "version-history"], [83, "version-history"], [84, "version-history"], [85, "version-history"], [86, "version-history"], [88, "version-history"], [89, "version-history"], [90, "version-history"], [92, "version-history"], [94, "version-history"], [95, "version-history"], [96, "version-history"], [98, "version-history"], [99, "version-history"], [100, "version-history"], [101, "version-history"], [102, "version-history"], [103, "version-history"], [104, "version-history"], [105, "version-history"], [106, "version-history"], [107, "version-history"], [108, "version-history"], [111, "version-history"], [112, "version-history"], [113, "version-history"], [114, "version-history"]], "Air Raid": [[26, "air-raid"]], "Alien": [[27, "alien"]], "Amidar": [[28, "amidar"]], "Assault": [[29, "assault"]], "Asterix": [[30, "asterix"]], "Asteroids": [[31, "asteroids"]], "Atlantis": [[32, "atlantis"]], "Bank Heist": [[33, "bank-heist"]], "Battle Zone": [[34, "battle-zone"]], "Beam Rider": [[35, "beam-rider"]], "Berzerk": [[36, "berzerk"]], "Bowling": [[37, "bowling"]], "Boxing": [[38, "boxing"]], "Breakout": [[39, "breakout"]], "Carnival": [[40, "carnival"]], "Centipede": [[41, "centipede"]], "Chopper Command": [[42, "chopper-command"]], "Complete List - Atari": [[43, "complete-list-atari"]], "Crazy Climber": [[44, "crazy-climber"]], "Defender": [[45, "defender"]], "Demon Attack": [[46, "demon-attack"]], "Double Dunk": [[47, "double-dunk"]], "Elevator Action": [[48, "elevator-action"]], "Enduro": [[49, "enduro"]], "FishingDerby": [[50, "fishingderby"]], "Freeway": [[51, "freeway"]], "Frostbite": [[52, "frostbite"]], "Gopher": [[53, "gopher"]], "Gravitar": [[54, "gravitar"]], "Hero": [[55, "hero"]], "IceHockey": [[56, "icehockey"]], "Jamesbond": [[57, "jamesbond"]], "JourneyEscape": [[58, "journeyescape"]], "Kangaroo": [[59, "kangaroo"]], "Krull": [[60, "krull"]], "Kung Fu Master": [[61, "kung-fu-master"]], "Montezuma Revenge": [[62, "montezuma-revenge"]], "Ms Pacman": [[63, "ms-pacman"]], "Name This Game": [[64, "name-this-game"]], "Phoenix": [[65, "phoenix"]], "Pitfall": [[66, "pitfall"]], "Pong": [[67, "pong"]], "Pooyan": [[68, "pooyan"]], "PrivateEye": [[69, "privateeye"]], "Qbert": [[70, "qbert"]], "Riverraid": [[71, "riverraid"]], "Road Runner": [[72, "road-runner"]], "Robot Tank": [[73, "robot-tank"]], "Seaquest": [[74, "seaquest"]], "Skiings": [[75, "skiings"]], "Solaris": [[76, "solaris"]], "SpaceInvaders": [[77, "spaceinvaders"]], "StarGunner": [[78, "stargunner"]], "Tennis": [[79, "tennis"]], "TimePilot": [[80, "timepilot"]], "Tutankham": [[81, "tutankham"]], "Up n\u2019 Down": [[82, "up-n-down"]], "Venture": [[83, "venture"]], "Video Pinball": [[84, "video-pinball"]], "Wizard of Wor": [[85, "wizard-of-wor"]], "Zaxxon": [[86, "zaxxon"]], "Box2D": [[87, "box2d"]], "Bipedal Walker": [[88, "bipedal-walker"]], "Starting State": [[88, "starting-state"], [89, "starting-state"], [90, "starting-state"], [92, "starting-state"], [93, "starting-state"], [94, "starting-state"], [95, "starting-state"], [96, "starting-state"], [98, "starting-state"], [99, "starting-state"], [100, "starting-state"], [101, "starting-state"], [102, "starting-state"], [103, "starting-state"], [104, "starting-state"], [105, "starting-state"], [106, "starting-state"], [107, "starting-state"], [108, "starting-state"], [111, "starting-state"], [112, "starting-state"], [113, "starting-state"], [114, "starting-state"]], "Episode Termination": [[88, "episode-termination"], [89, "episode-termination"], [90, "episode-termination"]], "Credits": [[88, "credits"], [89, "credits"], [90, "credits"]], "Car Racing": [[89, "car-racing"]], "Reset Arguments": [[89, "reset-arguments"]], "Lunar Lander": [[90, "lunar-lander"]], "Classic Control": [[91, "classic-control"]], "Acrobot": [[92, "acrobot"]], "Episode End": [[92, "episode-end"], [93, "episode-end"], [94, "episode-end"], [95, "episode-end"], [98, "episode-end"], [99, "episode-end"], [100, "episode-end"], [101, "episode-end"], [102, "episode-end"], [103, "episode-end"], [104, "episode-end"], [105, "episode-end"], [106, "episode-end"], [107, "episode-end"], [108, "episode-end"], [111, "episode-end"], [112, "episode-end"], [113, "episode-end"], [114, "episode-end"]], "Cart Pole": [[93, "cart-pole"]], "Mountain Car": [[94, "mountain-car"]], "Transition Dynamics:": [[94, "transition-dynamics"], [95, "transition-dynamics"]], "Reward:": [[94, "reward"]], "Mountain Car Continuous": [[95, "mountain-car-continuous"]], "Reward": [[95, "reward"], [112, "reward"]], "Pendulum": [[96, "pendulum"]], "Episode Truncation": [[96, "episode-truncation"]], "MuJoCo": [[97, "mujoco"]], "Ant": [[98, "ant"]], "Half Cheetah": [[99, "half-cheetah"]], "Hopper": [[100, "hopper"]], "Humanoid": [[101, "humanoid"]], "Humanoid Standup": [[102, "humanoid-standup"]], "Inverted Double Pendulum": [[103, "inverted-double-pendulum"]], "Inverted Pendulum": [[104, "inverted-pendulum"]], "Pusher": [[105, "pusher"]], "Reacher": [[106, "reacher"]], "Swimmer": [[107, "swimmer"]], "Notes": [[107, "notes"]], "Walker2D": [[108, "walker2d"]], "Third-party Environments": [[109, "third-party-environments"]], "Video Game environments": [[109, "video-game-environments"]], "stable-retro: Classic retro games, a maintained version of OpenAI Retro": [[109, "stable-retro-classic-retro-games-a-maintained-version-of-openai-retro"]], "flappy-bird-gym: A Flappy Bird environment for Gym": [[109, "flappy-bird-gym-a-flappy-bird-environment-for-gym"]], "gym-derk: GPU accelerated MOBA environment": [[109, "gym-derk-gpu-accelerated-moba-environment"]], "SlimeVolleyGym: A simple environment for single and multi-agent reinforcement learning": [[109, "slimevolleygym-a-simple-environment-for-single-and-multi-agent-reinforcement-learning"]], "Unity ML Agents: Environments for Unity game engine": [[109, "unity-ml-agents-environments-for-unity-game-engine"]], "PGE: Parallel Game Engine": [[109, "pge-parallel-game-engine"]], "Robotics environments": [[109, "robotics-environments"]], "gym-jiminy: Training Robots in Jiminy": [[109, "gym-jiminy-training-robots-in-jiminy"]], "gym-pybullet-drones: Environments for quadcopter control": [[109, "gym-pybullet-drones-environments-for-quadcopter-control"]], "MarsExplorer: Environments for controlling robot on Mars": [[109, "marsexplorer-environments-for-controlling-robot-on-mars"]], "panda-gym: Robotics environments using the PyBullet physics engine": [[109, "panda-gym-robotics-environments-using-the-pybullet-physics-engine"]], "robo-gym: Real-world and simulation robotics": [[109, "robo-gym-real-world-and-simulation-robotics"]], "Offworld-gym": [[109, "offworld-gym"]], "safe-control-gym": [[109, "safe-control-gym"]], "gym-softrobot: Soft-robotics environments": [[109, "gym-softrobot-soft-robotics-environments"]], "iGibson": [[109, "igibson"]], "DexterousHands: dual dexterous hand manipulation tasks": [[109, "dexteroushands-dual-dexterous-hand-manipulation-tasks"]], "OmniIsaacGymEnvs": [[109, "omniisaacgymenvs"]], "Autonomous Driving environments": [[109, "autonomous-driving-environments"]], "sumo-rl": [[109, "sumo-rl"]], "gym-duckietown": [[109, "gym-duckietown"]], "gym-electric-motor": [[109, "gym-electric-motor"]], "highway-env": [[109, "highway-env"]], "CommonRoad-RL": [[109, "commonroad-rl"]], "tmrl: TrackMania 2020 through RL": [[109, "tmrl-trackmania-2020-through-rl"]], "racing_dreamer": [[109, "racing-dreamer"]], "l2r: Multimodal control environment where agents learn how to race": [[109, "l2r-multimodal-control-environment-where-agents-learn-how-to-race"]], "racecar_gym": [[109, "racecar-gym"]], "Other environments": [[109, "other-environments"]], "CompilerGym: Optimise compiler tasks": [[109, "compilergym-optimise-compiler-tasks"]], "CARL: context adaptive RL": [[109, "carl-context-adaptive-rl"]], "matrix-mdp: Easily create discrete MDPs": [[109, "matrix-mdp-easily-create-discrete-mdps"]], "mo-gym: Multi-objective Reinforcement Learning environments": [[109, "mo-gym-multi-objective-reinforcement-learning-environments"]], "gym-cellular-automata: Cellular Automata environments": [[109, "gym-cellular-automata-cellular-automata-environments"]], "gym-sokoban: 2D Transportation Puzzles": [[109, "gym-sokoban-2d-transportation-puzzles"]], "DACBench: Benchmark Library for Dynamic Algorithm configuration": [[109, "dacbench-benchmark-library-for-dynamic-algorithm-configuration"]], "NLPGym: A toolkit to develop RL agents to solve NLP tasks": [[109, "nlpgym-a-toolkit-to-develop-rl-agents-to-solve-nlp-tasks"]], "gym-saturation: Environments used to prove theorems": [[109, "gym-saturation-environments-used-to-prove-theorems"]], "ShinRL: Environments for evaluating RL algorithms": [[109, "shinrl-environments-for-evaluating-rl-algorithms"]], "gymnax: Hardware Accelerated RL Environments": [[109, "gymnax-hardware-accelerated-rl-environments"]], "gym-anytrading: Financial trading environments for FOREX and STOCKS": [[109, "gym-anytrading-financial-trading-environments-for-forex-and-stocks"]], "gym-mtsim: Financial trading for MetaTrader 5 platform": [[109, "gym-mtsim-financial-trading-for-metatrader-5-platform"]], "openmodelica-microgrid-gym: Environments for controlling power electronic converters in microgrids": [[109, "openmodelica-microgrid-gym-environments-for-controlling-power-electronic-converters-in-microgrids"]], "mobile-env: Environments for coordination of wireless mobile networks": [[109, "mobile-env-environments-for-coordination-of-wireless-mobile-networks"]], "GymFC: A flight control tuning and training framework": [[109, "gymfc-a-flight-control-tuning-and-training-framework"]], "Toy Text": [[110, "toy-text"]], "Blackjack": [[111, "blackjack"]], "Information": [[111, "information"], [112, "information"], [113, "information"], [114, "information"]], "Cliff Walking": [[112, "cliff-walking"]], "Frozen Lake": [[113, "frozen-lake"]], "Taxi": [[114, "taxi"]], "Gymnasium is a standard API for reinforcement learning, and a diverse collection of reference environments": [[115, "gymnasium-is-a-standard-api-for-reinforcement-learning-and-a-diverse-collection-of-reference-environments"]], "v0.19.0 Release Notes": [[116, "v0-19-0-release-notes"]], "New features": [[116, "new-features"]], "Bug fixes": [[116, "bug-fixes"], [124, "bug-fixes"]], "Other": [[116, "other"]], "v0.20.0 Release Notes": [[117, "v0-20-0-release-notes"]], "Major Change": [[117, "major-change"]], "Breaking changes and new features:": [[117, "breaking-changes-and-new-features"]], "Bug fixes and upgrades": [[117, "bug-fixes-and-upgrades"]], "v0.21.0 Release Notes": [[118, "v0-21-0-release-notes"]], "v0.22 Release Notes": [[119, "v0-22-release-notes"]], "API changes": [[119, "api-changes"]], "Environment changes": [[119, "environment-changes"]], "Other changes": [[119, "other-changes"]], "v0.23.0 Release Notes": [[120, "v0-23-0-release-notes"]], "Breaking changes": [[120, "breaking-changes"]], "v0.23.1 Release Notes": [[121, "v0-23-1-release-notes"]], "v0.24.0 Release Notes": [[122, "v0-24-0-release-notes"]], "Major changes": [[122, "major-changes"], [127, "major-changes"]], "Minor changes": [[122, "minor-changes"], [127, "minor-changes"]], "v0.24.1 Release Notes": [[123, "v0-24-1-release-notes"]], "Bugs fixed": [[123, "bugs-fixed"]], "v0.25.0 Release notes": [[124, "v0-25-0-release-notes"]], "API Changes": [[124, "api-changes"]], "Misc": [[124, "misc"]], "0.25.1 Release Notes": [[125, "release-notes"]], "Bug fix": [[125, "bug-fix"]], "Typos": [[125, "typos"]], "v0.25.2 Release Notes": [[126, "v0-25-2-release-notes"]], "Bug Fixes": [[126, "bug-fixes"], [128, "bug-fixes"]], "Updates": [[126, "updates"]], "v0.26.0 Release Notes": [[127, "v0-26-0-release-notes"]], "Breaking backward compatibility": [[127, "breaking-backward-compatibility"]], "v0.26.1 Release Notes": [[128, "v0-26-1-release-notes"]], "Spelling": [[128, "spelling"]], "v0.26.2 Release Notes": [[129, "v0-26-2-release-notes"]], "Bugs Fixes": [[129, "bugs-fixes"]], "Release Notes": [[130, "release-notes"]], "v0.27.1": [[130, "release-v0-27-1"]], "v0.27.0": [[130, "release-v0-27-0"]], "v0.26.3": [[130, "release-v0-26-3"]], "v0.26.2: 0.26.2 ": [[130, "release-v0-26-2"]], "v0.26.1: 0.26.1": [[130, "release-v0-26-1"]], "v0.26.0: Initial Release": [[130, "release-v0-26-0"]], "Gym Release Notes": [[130, "gym-release-notes"]], "Gymnasium Basics": [[131, "gymnasium-basics"], [135, "gymnasium-basics"], [137, "gymnasium-basics"]], "Make your own custom environment": [[132, "make-your-own-custom-environment"]], "Subclassing gymnasium.Env": [[132, "subclassing-gymnasium-env"]], "Declaration and Initialization": [[132, "declaration-and-initialization"]], "Constructing Observations From Environment States": [[132, "constructing-observations-from-environment-states"]], "Reset": [[132, "reset"]], "Step": [[132, "step"]], "Rendering": [[132, "rendering"]], "Close": [[132, "close"]], "Registering Envs": [[132, "registering-envs"]], "Creating a Package": [[132, "creating-a-package"]], "Creating Environment Instances": [[132, "creating-environment-instances"]], "Using Wrappers": [[132, "using-wrappers"]], "Handling Time Limits": [[133, "handling-time-limits"]], "Termination": [[133, "termination"]], "Truncation": [[133, "truncation"]], "Importance in learning code": [[133, "importance-in-learning-code"]], "Solution": [[133, "solution"]], "Implementing Custom Wrappers": [[134, "implementing-custom-wrappers"]], "Inheriting from gymnasium.ObservationWrapper": [[134, "inheriting-from-gymnasium-observationwrapper"]], "Inheriting from gymnasium.ActionWrapper": [[134, "inheriting-from-gymnasium-actionwrapper"]], "Inheriting from gymnasium.RewardWrapper": [[134, "inheriting-from-gymnasium-rewardwrapper"]], "Inheriting from gymnasium.Wrapper": [[134, "inheriting-from-gymnasium-wrapper"]], "Training A2C with Vector Envs and Domain Randomization": [[136, "training-a2c-with-vector-envs-and-domain-randomization"]], "Introduction": [[136, "introduction"]], "Advantage Actor-Critic (A2C)": [[136, "advantage-actor-critic-a2c"]], "Using Vectorized Environments": [[136, "using-vectorized-environments"]], "Domain Randomization": [[136, "domain-randomization"]], "Setup": [[136, "setup"]], "Training the A2C Agent": [[136, "training-the-a2c-agent"]], "Plotting": [[136, "plotting"]], "Performance Analysis of Synchronous and Asynchronous Vectorized Environments": [[136, "performance-analysis-of-synchronous-and-asynchronous-vectorized-environments"]], "Saving/ Loading Weights": [[136, "saving-loading-weights"]], "Showcase the Agent": [[136, "showcase-the-agent"]], "Try playing the environment yourself": [[136, "try-playing-the-environment-yourself"]], "Tutorials": [[137, "tutorials"]], "Training Agents": [[137, "training-agents"], [138, "training-agents"], [140, "training-agents"]], "Solving Blackjack with Q-Learning": [[139, "solving-blackjack-with-q-learning"]], "Imports and Environment Setup": [[139, "imports-and-environment-setup"]], "Observing the environment": [[139, "observing-the-environment"]], "Executing an action": [[139, "executing-an-action"]], "Building an agent": [[139, "building-an-agent"], [141, "building-an-agent"]], "Visualizing the training": [[139, "visualizing-the-training"]], "Visualising the policy": [[139, "visualising-the-policy"]], "Think you can do better?": [[139, "think-you-can-do-better"]], "Training using REINFORCE for Mujoco": [[141, "training-using-reinforce-for-mujoco"]], "Policy Network": [[141, "policy-network"]], "Plot learning curve": [[141, "plot-learning-curve"]]}, "indexentries": {"env (class in gymnasium)": [[2, "gymnasium.Env"]], "action_space (gymnasium.env attribute)": [[2, "gymnasium.Env.action_space"]], "close() (in module gymnasium.env)": [[2, "gymnasium.Env.close"]], "metadata (gymnasium.env attribute)": [[2, "gymnasium.Env.metadata"]], "np_random (gymnasium.env property)": [[2, "gymnasium.Env.np_random"]], "observation_space (gymnasium.env attribute)": [[2, "gymnasium.Env.observation_space"]], "render() (in module gymnasium.env)": [[2, "gymnasium.Env.render"]], "render_mode (gymnasium.env attribute)": [[2, "gymnasium.Env.render_mode"]], "reset() (in module gymnasium.env)": [[2, "gymnasium.Env.reset"]], "reward_range (gymnasium.env attribute)": [[2, "gymnasium.Env.reward_range"]], "spec (gymnasium.env attribute)": [[2, "gymnasium.Env.spec"]], "step() (in module gymnasium.env)": [[2, "gymnasium.Env.step"]], "unwrapped (gymnasium.env property)": [[2, "gymnasium.Env.unwrapped"]], "funcenv (class in gymnasium.experimental)": [[4, "gymnasium.experimental.FuncEnv"]], "initial() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.initial"]], "observation() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.observation"]], "render_close() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.render_close"]], "render_image() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.render_image"]], "render_init() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.render_init"]], "reward() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.reward"]], "state_info() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.state_info"]], "step_info() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.step_info"]], "terminal() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.terminal"]], "transform() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.transform"]], "transition() (in module gymnasium.experimental.funcenv)": [[4, "gymnasium.experimental.FuncEnv.transition"]], "ataripreprocessingv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.AtariPreprocessingV0"]], "autoresetv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.AutoresetV0"]], "clipactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ClipActionV0"]], "cliprewardv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ClipRewardV0"]], "delayobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.DelayObservationV0"]], "dtypeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.DtypeObservationV0"]], "filterobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.FilterObservationV0"]], "flattenobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.FlattenObservationV0"]], "framestackobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.FrameStackObservationV0"]], "grayscaleobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.GrayscaleObservationV0"]], "humanrenderingv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.HumanRenderingV0"]], "jaxtonumpyv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.JaxToNumpyV0"]], "jaxtotorchv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.JaxToTorchV0"]], "lambdaactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.LambdaActionV0"]], "lambdaobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.LambdaObservationV0"]], "lambdarewardv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.LambdaRewardV0"]], "normalizeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.NormalizeObservationV0"]], "normalizerewardv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.NormalizeRewardV0"]], "numpytotorchv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.NumpyToTorchV0"]], "orderenforcingv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.OrderEnforcingV0"]], "passiveenvcheckerv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.PassiveEnvCheckerV0"]], "pixelobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.PixelObservationV0"]], "recordepisodestatisticsv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RecordEpisodeStatisticsV0"]], "recordvideov0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RecordVideoV0"]], "rendercollectionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RenderCollectionV0"]], "rescaleactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RescaleActionV0"]], "rescaleobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.RescaleObservationV0"]], "reshapeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ReshapeObservationV0"]], "resizeobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.ResizeObservationV0"]], "stickyactionv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.StickyActionV0"]], "timeawareobservationv0 (class in gymnasium.experimental.wrappers)": [[7, "gymnasium.experimental.wrappers.TimeAwareObservationV0"]], "make() (in module gymnasium)": [[8, "gymnasium.make"]], "pprint_registry() (in module gymnasium)": [[8, "gymnasium.pprint_registry"]], "register() (in module gymnasium)": [[8, "gymnasium.register"]], "spec() (in module gymnasium)": [[8, "gymnasium.spec"]], "space (class in gymnasium.spaces)": [[9, "gymnasium.spaces.Space"]], "contains() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.contains"]], "dtype (gymnasium.spaces.space property)": [[9, "gymnasium.spaces.Space.dtype"]], "from_jsonable() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.from_jsonable"]], "gymnasium.spaces": [[9, "module-gymnasium.spaces"]], "is_np_flattenable (gymnasium.spaces.space.space property)": [[9, "gymnasium.spaces.space.Space.is_np_flattenable"]], "module": [[9, "module-gymnasium.spaces"], [16, "module-gymnasium.wrappers"]], "sample() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.sample"]], "seed() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.seed"]], "shape (gymnasium.spaces.space.space property)": [[9, "gymnasium.spaces.space.Space.shape"]], "to_jsonable() (in module gymnasium.spaces.space.space)": [[9, "gymnasium.spaces.space.Space.to_jsonable"]], "dict (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Dict"]], "graph (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Graph"]], "sequence (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Sequence"]], "tuple (class in gymnasium.spaces)": [[10, "gymnasium.spaces.Tuple"]], "sample() (gymnasium.spaces.dict method)": [[10, "gymnasium.spaces.Dict.sample"]], "sample() (gymnasium.spaces.graph method)": [[10, "gymnasium.spaces.Graph.sample"]], "sample() (gymnasium.spaces.sequence method)": [[10, "gymnasium.spaces.Sequence.sample"]], "sample() (gymnasium.spaces.tuple method)": [[10, "gymnasium.spaces.Tuple.sample"]], "seed() (gymnasium.spaces.dict method)": [[10, "gymnasium.spaces.Dict.seed"]], "seed() (gymnasium.spaces.graph method)": [[10, "gymnasium.spaces.Graph.seed"]], "seed() (gymnasium.spaces.sequence method)": [[10, "gymnasium.spaces.Sequence.seed"]], "seed() (gymnasium.spaces.tuple method)": [[10, "gymnasium.spaces.Tuple.seed"]], "box (class in gymnasium.spaces)": [[11, "gymnasium.spaces.Box"]], "discrete (class in gymnasium.spaces)": [[11, "gymnasium.spaces.Discrete"]], "multibinary (class in gymnasium.spaces)": [[11, "gymnasium.spaces.MultiBinary"]], "multidiscrete (class in gymnasium.spaces)": [[11, "gymnasium.spaces.MultiDiscrete"]], "text (class in gymnasium.spaces)": [[11, "gymnasium.spaces.Text"]], "is_bounded() (gymnasium.spaces.box method)": [[11, "gymnasium.spaces.Box.is_bounded"]], "sample() (gymnasium.spaces.box method)": [[11, "gymnasium.spaces.Box.sample"]], "sample() (gymnasium.spaces.discrete method)": [[11, "gymnasium.spaces.Discrete.sample"]], "sample() (gymnasium.spaces.multibinary method)": [[11, "gymnasium.spaces.MultiBinary.sample"]], "sample() (gymnasium.spaces.multidiscrete method)": [[11, "gymnasium.spaces.MultiDiscrete.sample"]], "sample() (gymnasium.spaces.text method)": [[11, "gymnasium.spaces.Text.sample"]], "seed() (gymnasium.spaces.box method)": [[11, "gymnasium.spaces.Box.seed"]], "seed() (gymnasium.spaces.discrete method)": [[11, "gymnasium.spaces.Discrete.seed"]], "seed() (gymnasium.spaces.multibinary method)": [[11, "gymnasium.spaces.MultiBinary.seed"]], "seed() (gymnasium.spaces.multidiscrete method)": [[11, "gymnasium.spaces.MultiDiscrete.seed"]], "seed() (gymnasium.spaces.text method)": [[11, "gymnasium.spaces.Text.seed"]], "flatdim() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.flatdim"]], "flatten() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.flatten"]], "flatten_space() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.flatten_space"]], "unflatten() (in module gymnasium.spaces.utils)": [[12, "gymnasium.spaces.utils.unflatten"]], "batch_space() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.batch_space"]], "concatenate() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.concatenate"]], "create_empty_array() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.create_empty_array"]], "create_shared_memory() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.create_shared_memory"]], "iterate() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.iterate"]], "read_from_shared_memory() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.read_from_shared_memory"]], "write_to_shared_memory() (in module gymnasium.vector.utils)": [[13, "gymnasium.vector.utils.write_to_shared_memory"]], "playplot (class in gymnasium.utils.play)": [[14, "gymnasium.utils.play.PlayPlot"]], "playablegame (class in gymnasium.utils.play)": [[14, "gymnasium.utils.play.PlayableGame"]], "callback() (gymnasium.utils.play.playplot method)": [[14, "gymnasium.utils.play.PlayPlot.callback"]], "capped_cubic_video_schedule() (in module gymnasium.utils.save_video)": [[14, "gymnasium.utils.save_video.capped_cubic_video_schedule"]], "check_env() (in module gymnasium.utils.env_checker)": [[14, "gymnasium.utils.env_checker.check_env"]], "convert_to_done_step_api() (in module gymnasium.utils.step_api_compatibility)": [[14, "gymnasium.utils.step_api_compatibility.convert_to_done_step_api"]], "convert_to_terminated_truncated_step_api() (in module gymnasium.utils.step_api_compatibility)": [[14, "gymnasium.utils.step_api_compatibility.convert_to_terminated_truncated_step_api"]], "np_random() (in module gymnasium.utils.seeding)": [[14, "gymnasium.utils.seeding.np_random"]], "play() (in module gymnasium.utils.play)": [[14, "gymnasium.utils.play.play"]], "process_event() (gymnasium.utils.play.playablegame method)": [[14, "gymnasium.utils.play.PlayableGame.process_event"]], "save_video() (in module gymnasium.utils.save_video)": [[14, "gymnasium.utils.save_video.save_video"]], "step_api_compatibility() (in module gymnasium.utils.step_api_compatibility)": [[14, "gymnasium.utils.step_api_compatibility.step_api_compatibility"]], "asyncvectorenv (class in gymnasium.vector)": [[15, "gymnasium.vector.AsyncVectorEnv"]], "syncvectorenv (class in gymnasium.vector)": [[15, "gymnasium.vector.SyncVectorEnv"]], "vectorenv (class in gymnasium.vector)": [[15, "gymnasium.vector.VectorEnv"]], "action_space": [[15, "action_space"]], "close() (gymnasium.vector.vectorenv method)": [[15, "gymnasium.vector.VectorEnv.close"]], "make() (in module gymnasium.vector)": [[15, "gymnasium.vector.make"]], "observation_space": [[15, "observation_space"]], "reset() (gymnasium.vector.vectorenv method)": [[15, "gymnasium.vector.VectorEnv.reset"]], "single_action_space": [[15, "single_action_space"]], "single_observation_space": [[15, "single_observation_space"]], "step() (gymnasium.vector.vectorenv method)": [[15, "gymnasium.vector.VectorEnv.step"]], "wrapper (class in gymnasium)": [[16, "gymnasium.Wrapper"]], "action_space (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.action_space"]], "close() (in module gymnasium.wrapper)": [[16, "gymnasium.Wrapper.close"]], "env (gymnasium.wrappers.gymnasium.wrapper attribute)": [[16, "gymnasium.wrappers.gymnasium.Wrapper.env"]], "gymnasium.wrappers": [[16, "module-gymnasium.wrappers"]], "metadata (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.metadata"]], "np_random (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.np_random"]], "observation_space (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.observation_space"]], "reset() (in module gymnasium.wrapper)": [[16, "gymnasium.Wrapper.reset"]], "reward_range (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.reward_range"]], "spec (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.spec"]], "step() (in module gymnasium.wrapper)": [[16, "gymnasium.Wrapper.step"]], "unwrapped (gymnasium.wrapper property)": [[16, "gymnasium.Wrapper.unwrapped"]], "actionwrapper (class in gymnasium)": [[17, "gymnasium.ActionWrapper"]], "clipaction (class in gymnasium.wrappers)": [[17, "gymnasium.wrappers.ClipAction"]], "rescaleaction (class in gymnasium.wrappers)": [[17, "gymnasium.wrappers.RescaleAction"]], "action() (gymnasium.actionwrapper method)": [[17, "gymnasium.ActionWrapper.action"]], "ataripreprocessing (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.AtariPreprocessing"]], "autoresetwrapper (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.AutoResetWrapper"]], "envcompatibility (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.EnvCompatibility"]], "humanrendering (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.HumanRendering"]], "orderenforcing (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.OrderEnforcing"]], "passiveenvchecker (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.PassiveEnvChecker"]], "recordepisodestatistics (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.RecordEpisodeStatistics"]], "recordvideo (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.RecordVideo"]], "rendercollection (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.RenderCollection"]], "stepapicompatibility (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.StepAPICompatibility"]], "timelimit (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.TimeLimit"]], "vectorlistinfo (class in gymnasium.wrappers)": [[18, "gymnasium.wrappers.VectorListInfo"]], "filterobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.FilterObservation"]], "flattenobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.FlattenObservation"]], "framestack (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.FrameStack"]], "grayscaleobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.GrayScaleObservation"]], "normalizeobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.NormalizeObservation"]], "observationwrapper (class in gymnasium)": [[19, "gymnasium.ObservationWrapper"]], "pixelobservationwrapper (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.PixelObservationWrapper"]], "resizeobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.ResizeObservation"]], "timeawareobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.TimeAwareObservation"]], "transformobservation (class in gymnasium.wrappers)": [[19, "gymnasium.wrappers.TransformObservation"]], "observation() (gymnasium.observationwrapper method)": [[19, "gymnasium.ObservationWrapper.observation"]], "normalizereward (class in gymnasium.wrappers)": [[20, "gymnasium.wrappers.NormalizeReward"]], "rewardwrapper (class in gymnasium)": [[20, "gymnasium.RewardWrapper"]], "transformreward (class in gymnasium.wrappers)": [[20, "gymnasium.wrappers.TransformReward"]], "reward() (gymnasium.rewardwrapper method)": [[20, "gymnasium.RewardWrapper.reward"]]}})
\ No newline at end of file
diff --git a/tutorials/gymnasium_basics/README/index.html b/tutorials/gymnasium_basics/README/index.html
new file mode 100644
index 000000000..d53f64259
--- /dev/null
+++ b/tutorials/gymnasium_basics/README/index.html
@@ -0,0 +1,670 @@
+
+
+  
+    
+    
+    
+    
+    
+    
+    
+
+        
+
+    
+        Gymnasium Basics - Gymnasium Documentation
+      
+    
+    
+    
+    
+    
+    
+    
+    
+
+
+
+  
+    
    
+      
    
+        
    
+          
+        
    
+        
    
+          
+              
+              
+            Gymnasium Documentation
+          
+        
    
+        
    
+          
    
+            
+            
+          
    
+        
    
+      
    
+    
    
+
+    
+    
+    
+
+
+  
+    Contents
+    
+      
+    
+  
+  
+    Menu
+    
+      
+      
+      
+    
+  
+  
+    Expand
+    
+      
+    
+  
+  
+    Light mode
+    
+      
+      
+      
+      
+      
+      
+      
+      
+      
+    
+  
+  
+    Dark mode
+    
+      
+      
+    
+  
+  
+    Auto light/dark mode
+    
+      
+      
+      
+      
+      
+      
+      
+    
+  
+
+
+
+
+
+
+
+
+
+
    
+   
+  
+  
    
+    
    
+      
    
+        
+          
+            
+          
+          Back to top
+        
+        
    
+          
+    
+    
    
+  
+    
+    
+  
+
    
+            
+          
    
+          
+        
    
+        
    
+          
+          
    
+
    Gymnasium Basics#
    
+
    
+
+        
    
+      
    
+      
    
+        
+        
    
+          
+            
+        
    
+        
    
+          
    
+            
    
+                Copyright © 2022 Farama Foundation
+            
    
+            
+          
    
+          
    
+            
    
+              
+            
    
+          
    
+        
    
+        
+      
    
+    
    
+    
+  
    
+
    
+    
+
+    
+    
+
+      
+      
+
+    
+    
+    
+    
+    
+    
+    
+    
+    
+
+    
+    
+
\ No newline at end of file
diff --git a/tutorials/environment_creation/index.html b/tutorials/gymnasium_basics/environment_creation/index.html
similarity index 83%
rename from tutorials/environment_creation/index.html
rename to tutorials/gymnasium_basics/environment_creation/index.html
index a6e36ccb1..7dd65ef4d 100644
--- a/tutorials/environment_creation/index.html
+++ b/tutorials/gymnasium_basics/environment_creation/index.html
@@ -7,15 +7,19 @@
     
     
     
-    
-
-        
+    
+
+        
 
-    
+    
         Make your own custom environment - Gymnasium Documentation
-      
-    
-    
+      
+    
+    
+    
+    
+    
+    
     
     
 
@@ -68,16 +72,16 @@
           
         
    
         
    
-          
-              
-              
+          
+              
+              
             Gymnasium Documentation
           
         
    
         
    
           
    
             
+          
    
+          
+        
    
+        
    
+          
+          
    
+
    Handling Time Limits#
    
+
    In using Gymnasium environments with reinforcement learning code, a common problem observed is how time limits are incorrectly handled. The done signal received (in previous versions of OpenAI Gym < 0.26) from env.step indicated whether an episode has ended. However, this signal did not distinguish whether the episode ended due to termination or truncation.
    
+
    
+
    Termination#
    
+
    Termination refers to the episode ending after reaching a terminal state that is defined as part of the environment
+definition. Examples are - task success, task failure, robot falling down etc. Notably, this also includes episodes
+ending in finite-horizon environments due to a time-limit inherent to the environment. Note that to preserve Markov
+property, a representation of the remaining time must be present in the agent’s observation in finite-horizon environments.
+(Reference)
    
+
    
+
    
+
    Truncation#
    
+
    Truncation refers to the episode ending after an externally defined condition (that is outside the scope of the Markov
+Decision Process). This could be a time-limit, a robot going out of bounds etc.
    
+
    An infinite-horizon environment is an obvious example of where this is needed. We cannot wait forever for the episode
+to complete, so we set a practical time-limit after which we forcibly halt the episode. The last state in this case is
+not a terminal state since it has a non-zero transition probability of moving to another state as per the Markov
+Decision Process that defines the RL problem. This is also different from time-limits in finite horizon environments
+as the agent in this case has no idea about this time-limit.
    
+
    
+
    
+
    Importance in learning code#
    
+
    Bootstrapping (using one or more estimated values of a variable to update estimates of the same variable) is a key
+aspect of Reinforcement Learning. A value function will tell you how much discounted reward you will get from a
+particular state if you follow a given policy. When an episode stops at any given point, by looking at the value of
+the final state, the agent is able to estimate how much discounted reward could have been obtained if the episode has
+continued. This is an example of handling truncation.
    
+
    More formally, a common example of bootstrapping in RL is updating the estimate of the Q-value function,
    
+
    
+
    
+\[Q_{target}(o_t, a_t) = r_t + \gamma . \max_a(Q(o_{t+1}, a_{t+1}))\]
    
+
    
+
    In classical RL, the new Q estimate is a weighted average of the previous Q estimate and Q_target while in Deep
+Q-Learning, the error between Q_target and the previous Q estimate is minimized.
    
+
    However, at the terminal state, bootstrapping is not done,
    
+
    
+
    
+\[Q_{target}(o_t, a_t) = r_t\]
    
+
    
+
    This is where the distinction between termination and truncation becomes important. When an episode ends due to
+termination we don’t bootstrap, when it ends due to truncation, we bootstrap.
    
+
    While using gymnasium environments, the done signal (default for < v0.26) is frequently used to determine whether to
+bootstrap or not. However, this is incorrect since it does not differentiate between termination and truncation.
    
+
    A simple example of value functions is shown below. This is an illustrative example and not part of any specific algorithm.
    
+
    # INCORRECT
+vf_target = rew + gamma * (1 - done) * vf_next_state
+
    
+
    
+
    This is incorrect in the case of episode ending due to a truncation, where bootstrapping needs to happen but it doesn’t.
    
+
    
+
    
+
    Solution#
    
+
    From v0.26 onwards, Gymnasium’s env.step API returns both termination and truncation information explicitly.
+In the previous version truncation information was supplied through the info key TimeLimit.truncated.
+The correct way to handle terminations and truncations now is,
    
+
    # terminated = done and 'TimeLimit.truncated' not in info
+# This was needed in previous versions.
+
+vf_target = rew + gamma * (1 - terminated) * vf_next_state
+
    
+
    
+
+
    
+
    
+
+        
    
+      
+      
+    
+    
+  
+
+    
+
+    
+    
+
+      
+      
+
+    
+    
+    
+    
+    
+    
+    
+    
+    
+    
+
+    
+    
+
\ No newline at end of file
diff --git a/tutorials/implementing_custom_wrappers/index.html b/tutorials/gymnasium_basics/implementing_custom_wrappers/index.html
similarity index 70%
rename from tutorials/implementing_custom_wrappers/index.html
rename to tutorials/gymnasium_basics/implementing_custom_wrappers/index.html
index 4edd2fc45..11fce19d7 100644
--- a/tutorials/implementing_custom_wrappers/index.html
+++ b/tutorials/gymnasium_basics/implementing_custom_wrappers/index.html
@@ -7,15 +7,19 @@
     
     
     
-    
-
-        
+    
+
+        
 
-    
+    
         Implementing Custom Wrappers - Gymnasium Documentation
-      
-    
-    
+      
+    
+    
+    
+    
+    
+    
     
     
 
@@ -68,16 +72,16 @@
           
         
         
    
-          
-              
-              
+          
+              
+              
             Gymnasium Documentation
           
         
    
         
    
           
    
             
+          
    
+          
+        
    
+        
+      
+      
+    
+    
+  
+
+    
+
+    
+    
+
+      
+      
+
+    
+    
+    
+    
+    
+    
+    
+    
+    
+
+    
+    
+
\ No newline at end of file
diff --git a/tutorials/gymnasium_basics/vector_envs_tutorial/index.html b/tutorials/gymnasium_basics/vector_envs_tutorial/index.html
new file mode 100644
index 000000000..176fada23
--- /dev/null
+++ b/tutorials/gymnasium_basics/vector_envs_tutorial/index.html
@@ -0,0 +1,1334 @@
+
+
+  
+    
+    
+    
+    
+    
+    
+    
+
+        
+
+    
+        Training A2C with Vector Envs and Domain Randomization - Gymnasium Documentation
+      
+    
+    
+    
+    
+    
+    
+    
+    
+
+
+
+  
+    
    
+      
    
+        
    
+          
+        
    
+        
    
+          
+              
+              
+            Gymnasium Documentation
+          
+        
    
+        
    
+          
    
+            
+            
+          
    
+        
    
+      
    
+    
    
+
+    
+    
+    
+
+
+  
+    Contents
+    
+      
+    
+  
+  
+    Menu
+    
+      
+      
+      
+    
+  
+  
+    Expand
+    
+      
+    
+  
+  
+    Light mode
+    
+      
+      
+      
+      
+      
+      
+      
+      
+      
+    
+  
+  
+    Dark mode
+    
+      
+      
+    
+  
+  
+    Auto light/dark mode
+    
+      
+      
+      
+      
+      
+      
+      
+    
+  
+
+
+
+
+
+
+
+
+
+
    
+   
+  
+  
    
+    
    
+      
    
+        
+          
+            
+          
+          Back to top
+        
+        
    
+          
+    
+    
    
+  
+    
+    
+  
+
    
+            
+          
    
+          
+        
    
+        
    
+          
+          
    
+
    Training A2C with Vector Envs and Domain Randomization#
    
+
    
+
    Introduction#
    
+
    In this tutorial, you’ll learn how to use vectorized environments to train an Advantage Actor-Critic agent.
+We are going to use A2C, which is the synchronous version of the A3C algorithm [1].
    
+
    Vectorized environments [3] can help to achieve quicker and more robust training by allowing multiple instances
+of the same environment to run in parallel (on multiple CPUs). This can significantly reduce the variance and thus speeds up the training.
    
+
    We will implement an Advantage Actor-Critic from scratch to look at how you can feed batched states into your networks to get a vector of actions
+(one action per environment) and calculate the losses for actor and critic on minibatches of transitions.
+Each minibatch contains the transitions of one sampling phase: n_steps_per_update steps are executed in n_envs environments in parallel
+(multiply the two to get the number of transitions in a minibatch). After each sampling phase,  the losses are calculated and one gradient step is executed.
+To calculate the advantages, we are going to use the Generalized Advantage Estimation (GAE) method [2], which balances the tradeoff
+between variance and bias of the advantage estimates.
    
+
    The A2C agent class is initialized with the number of features of the input state, the number of actions the agent can take,
+the learning rates and the number of environments that run in parallel to collect experiences. The actor and critic networks are defined
+and their respective optimizers are initialized. The forward pass of the networks takes in a batched vector of states and returns a tensor of state values
+and a tensor of action logits. The select_action method returns a tuple of the chosen actions, the log-probs of those actions, and the state values for each action.
+In addition, it also returns the entropy of the policy distribution, which is subtracted from the loss later (with a weighting factor ent_coef) to encourage exploration.
    
+
    The get_losses function calculates the losses for the actor and critic networks (using GAE), which are then updated using the update_parameters function.
    
+
    
+
    # Author: Till Zemann
+# License: MIT License
+
+from __future__ import annotations
+
+import os
+
+import matplotlib.pyplot as plt
+import numpy as np
+import torch
+import torch.nn as nn
+from torch import optim
+from tqdm import tqdm
+
+import gymnasium as gym
+
    
+
    
+
    
+
    
+
    Advantage Actor-Critic (A2C)#
    
+
    The Actor-Critic combines elements of value-based and policy-based methods. In A2C, the agent has two separate neural networks:
+a critic network that estimates the state-value function, and an actor network that outputs logits for a categorical probability distribution over all actions.
+The critic network is trained to minimize the mean squared error between the predicted state values and the actual returns received by the agent
+(this is equivalent to minimizing the squared advantages, because the advantage of an action is as the difference between the return and the state-value: A(s,a) = Q(s,a) - V(s).
+The actor network is trained to maximize the expected return by selecting actions that have high expected values according to the critic network.
    
+
    The focus of this tutorial will not be on the details of A2C itself. Instead, the tutorial will focus on how to use vectorized environments
+and domain randomization to accelerate the training process for A2C (and other reinforcement learning algorithms).
    
+
    
+
    class A2C(nn.Module):
+    """
+    (Synchronous) Advantage Actor-Critic agent class
+
+    Args:
+        n_features: The number of features of the input state.
+        n_actions: The number of actions the agent can take.
+        device: The device to run the computations on (running on a GPU might be quicker for larger Neural Nets,
+                for this code CPU is totally fine).
+        critic_lr: The learning rate for the critic network (should usually be larger than the actor_lr).
+        actor_lr: The learning rate for the actor network.
+        n_envs: The number of environments that run in parallel (on multiple CPUs) to collect experiences.
+    """
+
+    def __init__(
+        self,
+        n_features: int,
+        n_actions: int,
+        device: torch.device,
+        critic_lr: float,
+        actor_lr: float,
+        n_envs: int,
+    ) -> None:
+        """Initializes the actor and critic networks and their respective optimizers."""
+        super().__init__()
+        self.device = device
+        self.n_envs = n_envs
+
+        critic_layers = [
+            nn.Linear(n_features, 32),
+            nn.ReLU(),
+            nn.Linear(32, 32),
+            nn.ReLU(),
+            nn.Linear(32, 1),  # estimate V(s)
+        ]
+
+        actor_layers = [
+            nn.Linear(n_features, 32),
+            nn.ReLU(),
+            nn.Linear(32, 32),
+            nn.ReLU(),
+            nn.Linear(
+                32, n_actions
+            ),  # estimate action logits (will be fed into a softmax later)
+        ]
+
+        # define actor and critic networks
+        self.critic = nn.Sequential(*critic_layers).to(self.device)
+        self.actor = nn.Sequential(*actor_layers).to(self.device)
+
+        # define optimizers for actor and critic
+        self.critic_optim = optim.RMSprop(self.critic.parameters(), lr=critic_lr)
+        self.actor_optim = optim.RMSprop(self.actor.parameters(), lr=actor_lr)
+
+    def forward(self, x: np.ndarray) -> tuple[torch.Tensor, torch.Tensor]:
+        """
+        Forward pass of the networks.
+
+        Args:
+            x: A batched vector of states.
+
+        Returns:
+            state_values: A tensor with the state values, with shape [n_envs,].
+            action_logits_vec: A tensor with the action logits, with shape [n_envs, n_actions].
+        """
+        x = torch.Tensor(x).to(self.device)
+        state_values = self.critic(x)  # shape: [n_envs,]
+        action_logits_vec = self.actor(x)  # shape: [n_envs, n_actions]
+        return (state_values, action_logits_vec)
+
+    def select_action(
+        self, x: np.ndarray
+    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
+        """
+        Returns a tuple of the chosen actions and the log-probs of those actions.
+
+        Args:
+            x: A batched vector of states.
+
+        Returns:
+            actions: A tensor with the actions, with shape [n_steps_per_update, n_envs].
+            action_log_probs: A tensor with the log-probs of the actions, with shape [n_steps_per_update, n_envs].
+            state_values: A tensor with the state values, with shape [n_steps_per_update, n_envs].
+        """
+        state_values, action_logits = self.forward(x)
+        action_pd = torch.distributions.Categorical(
+            logits=action_logits
+        )  # implicitly uses softmax
+        actions = action_pd.sample()
+        action_log_probs = action_pd.log_prob(actions)
+        entropy = action_pd.entropy()
+        return (actions, action_log_probs, state_values, entropy)
+
+    def get_losses(
+        self,
+        rewards: torch.Tensor,
+        action_log_probs: torch.Tensor,
+        value_preds: torch.Tensor,
+        entropy: torch.Tensor,
+        masks: torch.Tensor,
+        gamma: float,
+        lam: float,
+        ent_coef: float,
+        device: torch.device,
+    ) -> tuple[torch.Tensor, torch.Tensor]:
+        """
+        Computes the loss of a minibatch (transitions collected in one sampling phase) for actor and critic
+        using Generalized Advantage Estimation (GAE) to compute the advantages (https://arxiv.org/abs/1506.02438).
+
+        Args:
+            rewards: A tensor with the rewards for each time step in the episode, with shape [n_steps_per_update, n_envs].
+            action_log_probs: A tensor with the log-probs of the actions taken at each time step in the episode, with shape [n_steps_per_update, n_envs].
+            value_preds: A tensor with the state value predictions for each time step in the episode, with shape [n_steps_per_update, n_envs].
+            masks: A tensor with the masks for each time step in the episode, with shape [n_steps_per_update, n_envs].
+            gamma: The discount factor.
+            lam: The GAE hyperparameter. (lam=1 corresponds to Monte-Carlo sampling with high variance and no bias,
+                                          and lam=0 corresponds to normal TD-Learning that has a low variance but is biased
+                                          because the estimates are generated by a Neural Net).
+            device: The device to run the computations on (e.g. CPU or GPU).
+
+        Returns:
+            critic_loss: The critic loss for the minibatch.
+            actor_loss: The actor loss for the minibatch.
+        """
+        T = len(rewards)
+        advantages = torch.zeros(T, self.n_envs, device=device)
+
+        # compute the advantages using GAE
+        gae = 0.0
+        for t in reversed(range(T - 1)):
+            td_error = (
+                rewards[t] + gamma * masks[t] * value_preds[t + 1] - value_preds[t]
+            )
+            gae = td_error + gamma * lam * masks[t] * gae
+            advantages[t] = gae
+
+        # calculate the loss of the minibatch for actor and critic
+        critic_loss = advantages.pow(2).mean()
+
+        # give a bonus for higher entropy to encourage exploration
+        actor_loss = (
+            -(advantages.detach() * action_log_probs).mean() - ent_coef * entropy.mean()
+        )
+        return (critic_loss, actor_loss)
+
+    def update_parameters(
+        self, critic_loss: torch.Tensor, actor_loss: torch.Tensor
+    ) -> None:
+        """
+        Updates the parameters of the actor and critic networks.
+
+        Args:
+            critic_loss: The critic loss.
+            actor_loss: The actor loss.
+        """
+        self.critic_optim.zero_grad()
+        critic_loss.backward()
+        self.critic_optim.step()
+
+        self.actor_optim.zero_grad()
+        actor_loss.backward()
+        self.actor_optim.step()
+
    
+
    
+
    
+
    
+
    Using Vectorized Environments#
    
+
    When you calculate the losses for the two Neural Networks over only one epoch, it might have a high variance. With vectorized environments,
+we can play with n_envs in parallel and thus get up to a linear speedup (meaning that in theory, we collect samples n_envs times quicker)
+that we can use to calculate the loss for the current policy and critic network. When we are using more samples to calculate the loss,
+it will have a lower variance and theirfore leads to quicker learning.
    
+
    A2C is a synchronous method, meaning that the parameter updates to Networks take place deterministically (after each sampling phase),
+but we can still make use of asynchronous vector envs to spawn multiple processes for parallel environment execution.
    
+
    The simplest way to create vector environments is by calling gym.vector.make, which creates multiple instances of the same environment:
    
+
    envs = gym.vector.make("LunarLander-v2", num_envs=3, max_episode_steps=600)
+
    
+
    
+
    
+
    
+
    Domain Randomization#
    
+
    If we want to randomize the environment for training to get more robust agents (that can deal with different parameterizations of an environment
+and theirfore might have a higher degree of generalization), we can set the desired parameters manually or use a pseudo-random number generator to generate them.
    
+
    Manually setting up 3 parallel ‘LunarLander-v2’ envs with different parameters:
    
+
    envs = gym.vector.AsyncVectorEnv(
+    [
+        lambda: gym.make(
+            "LunarLander-v2",
+            gravity=-10.0,
+            enable_wind=True,
+            wind_power=15.0,
+            turbulence_power=1.5,
+            max_episode_steps=600,
+        ),
+        lambda: gym.make(
+            "LunarLander-v2",
+            gravity=-9.8,
+            enable_wind=True,
+            wind_power=10.0,
+            turbulence_power=1.3,
+            max_episode_steps=600,
+        ),
+        lambda: gym.make(
+            "LunarLander-v2", gravity=-7.0, enable_wind=False, max_episode_steps=600
+        ),
+    ]
+)
+
    
+
    
+
    
+
    Randomly generating the parameters for 3 parallel ‘LunarLander-v2’ envs, using np.clip to stay in the recommended parameter space:
    
+
    envs = gym.vector.AsyncVectorEnv(
+    [
+        lambda: gym.make(
+            "LunarLander-v2",
+            gravity=np.clip(
+                np.random.normal(loc=-10.0, scale=1.0), a_min=-11.99, a_max=-0.01
+            ),
+            enable_wind=np.random.choice([True, False]),
+            wind_power=np.clip(
+                np.random.normal(loc=15.0, scale=1.0), a_min=0.01, a_max=19.99
+            ),
+            turbulence_power=np.clip(
+                np.random.normal(loc=1.5, scale=0.5), a_min=0.01, a_max=1.99
+            ),
+            max_episode_steps=600,
+        )
+        for i in range(3)
+    ]
+)
+
    
+
    
+
    
+
    Here we are using normal distributions with the standard parameterization of the environment as the mean and an arbitrary standard deviation (scale).
+Depending on the problem, you can experiment with higher variance and use different distributions as well.
    
+
    If you are training on the same n_envs environments for the entire training time, and n_envs is a relatively low number
+(in proportion to how complex the environment is), you might still get some overfitting to the specific parameterizations that you picked.
+To mitigate this, you can either pick a high number of randomly parameterized environments or remake your environments every couple of sampling phases
+to generate a new set of pseudo-random parameters.
    
+
    
+
    
+
    Setup#
    
+
    # environment hyperparams
+n_envs = 10
+n_updates = 1000
+n_steps_per_update = 128
+randomize_domain = False
+
+# agent hyperparams
+gamma = 0.999
+lam = 0.95  # hyperparameter for GAE
+ent_coef = 0.01  # coefficient for the entropy bonus (to encourage exploration)
+actor_lr = 0.001
+critic_lr = 0.005
+
+# Note: the actor has a slower learning rate so that the value targets become
+# more stationary and are theirfore easier to estimate for the critic
+
+# environment setup
+if randomize_domain:
+    envs = gym.vector.AsyncVectorEnv(
+        [
+            lambda: gym.make(
+                "LunarLander-v2",
+                gravity=np.clip(
+                    np.random.normal(loc=-10.0, scale=1.0), a_min=-11.99, a_max=-0.01
+                ),
+                enable_wind=np.random.choice([True, False]),
+                wind_power=np.clip(
+                    np.random.normal(loc=15.0, scale=1.0), a_min=0.01, a_max=19.99
+                ),
+                turbulence_power=np.clip(
+                    np.random.normal(loc=1.5, scale=0.5), a_min=0.01, a_max=1.99
+                ),
+                max_episode_steps=600,
+            )
+            for i in range(n_envs)
+        ]
+    )
+
+else:
+    envs = gym.vector.make("LunarLander-v2", num_envs=n_envs, max_episode_steps=600)
+
+
+obs_shape = envs.single_observation_space.shape[0]
+action_shape = envs.single_action_space.n
+
+# set the device
+use_cuda = False
+if use_cuda:
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+else:
+    device = torch.device("cpu")
+
+# init the agent
+agent = A2C(obs_shape, action_shape, device, critic_lr, actor_lr, n_envs)
+
    
+
    
+
    
+
    
+
    Training the A2C Agent#
    
+
    For our training loop, we are using the RecordEpisodeStatistics wrapper to record the episode lengths and returns and we are also saving
+the losses and entropies to plot them after the agent finished training.
    
+
    You may notice that the don’t reset the vectorized envs at the start of each episode like we would usually do.
+This is because each environment resets automatically once the episode finishes (each environment takes a different number of timesteps to finish
+an episode because of the random seeds). As a result, we are also not collecting data in episodes, but rather just play a certain number of steps
+(n_steps_per_update) in each environment (as an example, this could mean that we play 20 timesteps to finish an episode and then
+use the rest of the timesteps to begin a new one).
    
+
    # create a wrapper environment to save episode returns and episode lengths
+envs_wrapper = gym.wrappers.RecordEpisodeStatistics(envs, deque_size=n_envs * n_updates)
+
+critic_losses = []
+actor_losses = []
+entropies = []
+
+# use tqdm to get a progress bar for training
+for sample_phase in tqdm(range(n_updates)):
+
+    # we don't have to reset the envs, they just continue playing
+    # until the episode is over and then reset automatically
+
+    # reset lists that collect experiences of an episode (sample phase)
+    ep_value_preds = torch.zeros(n_steps_per_update, n_envs, device=device)
+    ep_rewards = torch.zeros(n_steps_per_update, n_envs, device=device)
+    ep_action_log_probs = torch.zeros(n_steps_per_update, n_envs, device=device)
+    masks = torch.zeros(n_steps_per_update, n_envs, device=device)
+
+    # at the start of training reset all envs to get an initial state
+    if sample_phase == 0:
+        states, info = envs_wrapper.reset(seed=42)
+
+    # play n steps in our parallel environments to collect data
+    for step in range(n_steps_per_update):
+
+        # select an action A_{t} using S_{t} as input for the agent
+        actions, action_log_probs, state_value_preds, entropy = agent.select_action(
+            states
+        )
+
+        # perform the action A_{t} in the environment to get S_{t+1} and R_{t+1}
+        states, rewards, terminated, truncated, infos = envs_wrapper.step(
+            actions.numpy()
+        )
+
+        ep_value_preds[step] = torch.squeeze(state_value_preds)
+        ep_rewards[step] = torch.tensor(rewards, device=device)
+        ep_action_log_probs[step] = action_log_probs
+
+        # add a mask (for the return calculation later);
+        # for each env the mask is 1 if the episode is ongoing and 0 if it is terminated (not by truncation!)
+        masks[step] = torch.tensor([not term for term in terminated])
+
+    # calculate the losses for actor and critic
+    critic_loss, actor_loss = agent.get_losses(
+        ep_rewards,
+        ep_action_log_probs,
+        ep_value_preds,
+        entropy,
+        masks,
+        gamma,
+        lam,
+        ent_coef,
+        device,
+    )
+
+    # update the actor and critic networks
+    agent.update_parameters(critic_loss, actor_loss)
+
+    # log the losses and entropy
+    critic_losses.append(critic_loss.detach().cpu().numpy())
+    actor_losses.append(actor_loss.detach().cpu().numpy())
+    entropies.append(entropy.detach().mean().cpu().numpy())
+
    
+
    
+
    
+
    
+
    Plotting#
    
+
    """ plot the results """
+
+# %matplotlib inline
+
+rolling_length = 20
+fig, axs = plt.subplots(nrows=2, ncols=2, figsize=(12, 5))
+fig.suptitle(
+    f"Training plots for {agent.__class__.__name__} in the LunarLander-v2 environment \n \
+             (n_envs={n_envs}, n_steps_per_update={n_steps_per_update}, randomize_domain={randomize_domain})"
+)
+
+# episode return
+axs[0][0].set_title("Episode Returns")
+episode_returns_moving_average = (
+    np.convolve(
+        np.array(envs_wrapper.return_queue).flatten(),
+        np.ones(rolling_length),
+        mode="valid",
+    )
+    / rolling_length
+)
+axs[0][0].plot(
+    np.arange(len(episode_returns_moving_average)) / n_envs,
+    episode_returns_moving_average,
+)
+axs[0][0].set_xlabel("Number of episodes")
+
+# entropy
+axs[1][0].set_title("Entropy")
+entropy_moving_average = (
+    np.convolve(np.array(entropies), np.ones(rolling_length), mode="valid")
+    / rolling_length
+)
+axs[1][0].plot(entropy_moving_average)
+axs[1][0].set_xlabel("Number of updates")
+
+
+# critic loss
+axs[0][1].set_title("Critic Loss")
+critic_losses_moving_average = (
+    np.convolve(
+        np.array(critic_losses).flatten(), np.ones(rolling_length), mode="valid"
+    )
+    / rolling_length
+)
+axs[0][1].plot(critic_losses_moving_average)
+axs[0][1].set_xlabel("Number of updates")
+
+
+# actor loss
+axs[1][1].set_title("Actor Loss")
+actor_losses_moving_average = (
+    np.convolve(np.array(actor_losses).flatten(), np.ones(rolling_length), mode="valid")
+    / rolling_length
+)
+axs[1][1].plot(actor_losses_moving_average)
+axs[1][1].set_xlabel("Number of updates")
+
+plt.tight_layout()
+plt.show()
+
    
+
    
+training_plots
+
    
+
    
+
    Performance Analysis of Synchronous and Asynchronous Vectorized Environments#
    
+
    
+
    Asynchronous environments can lead to quicker training times and a higher speedup
+for data collection compared to synchronous environments. This is because asynchronous environments
+allow multiple agents to interact with their environments in parallel,
+while synchronous environments run multiple environments serially.
+This results in better efficiency and faster training times for asynchronous environments.
    
+performance_plots
+
    
+
    According to the Karp-Flatt metric (a metric used in parallel computing to estimate the limit for the
+speedup when scaling up the number of parallel processes, here the number of environments),
+the estimated max. speedup for asynchronous environments is 57, while the estimated maximum speedup
+for synchronous environments is 21. This suggests that asynchronous environments have significantly
+faster training times compared to synchronous environments (see graphs).
    
+karp_flatt_metric
+
    
+
    However, it is important to note that increasing the number of parallel vector environments
+can lead to slower training times after a certain number of environments (see plot below, where the
+agent was trained until the mean training returns were above -120). The slower training times might occur
+because the gradients of the environments are good enough after a relatively low number of environments
+(especially if the environment is not very complex). In this case, increasing the number of environments
+does not increase the learning speed, and actually increases the runtime, possibly due to the additional time
+needed to calculate the gradients. For LunarLander-v2, the best performing configuration used a AsyncVectorEnv
+with 10 parallel environments, but environments with a higher complexity may require more
+parallel environments to achieve optimal performance.
    
+runtime_until_threshold_plot
+
    
+
    
+
    Saving/ Loading Weights#
    
+
    save_weights = False
+load_weights = False
+
+actor_weights_path = "weights/actor_weights.h5"
+critic_weights_path = "weights/critic_weights.h5"
+
+if not os.path.exists("weights"):
+    os.mkdir("weights")
+
+""" save network weights """
+if save_weights:
+    torch.save(agent.actor.state_dict(), actor_weights_path)
+    torch.save(agent.critic.state_dict(), critic_weights_path)
+
+
+""" load network weights """
+if load_weights:
+    agent = A2C(obs_shape, action_shape, device, critic_lr, actor_lr)
+
+    agent.actor.load_state_dict(torch.load(actor_weights_path))
+    agent.critic.load_state_dict(torch.load(critic_weights_path))
+    agent.actor.eval()
+    agent.critic.eval()
+
    
+
    
+
    
+
    
+
    Showcase the Agent#
    
+
    """ play a couple of showcase episodes """
+
+n_showcase_episodes = 3
+
+for episode in range(n_showcase_episodes):
+    print(f"starting episode {episode}...")
+
+    # create a new sample environment to get new random parameters
+    if randomize_domain:
+        env = gym.make(
+            "LunarLander-v2",
+            render_mode="human",
+            gravity=np.clip(
+                np.random.normal(loc=-10.0, scale=2.0), a_min=-11.99, a_max=-0.01
+            ),
+            enable_wind=np.random.choice([True, False]),
+            wind_power=np.clip(
+                np.random.normal(loc=15.0, scale=2.0), a_min=0.01, a_max=19.99
+            ),
+            turbulence_power=np.clip(
+                np.random.normal(loc=1.5, scale=1.0), a_min=0.01, a_max=1.99
+            ),
+            max_episode_steps=500,
+        )
+    else:
+        env = gym.make("LunarLander-v2", render_mode="human", max_episode_steps=500)
+
+    # get an initial state
+    state, info = env.reset()
+
+    # play one episode
+    done = False
+    while not done:
+
+        # select an action A_{t} using S_{t} as input for the agent
+        with torch.no_grad():
+            action, _, _, _ = agent.select_action(state[None, :])
+
+        # perform the action A_{t} in the environment to get S_{t+1} and R_{t+1}
+        state, reward, terminated, truncated, info = env.step(action.item())
+
+        # update if the environment is done
+        done = terminated or truncated
+
+env.close()
+
    
+
    
+
    
+
    
+
    Try playing the environment yourself#
    
+
    # from gymnasium.utils.play import play
+#
+# play(gym.make('LunarLander-v2', render_mode='rgb_array'),
+#     keys_to_action={'w': 2, 'a': 1, 'd': 3}, noop=0)
+
    
+
    
+
    
+
    
+
    References#
    
+
    [1] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. P. Lillicrap, T. Harley, D. Silver, K. Kavukcuoglu. “Asynchronous Methods for Deep Reinforcement Learning” ICML (2016).
    
+
    [2] J. Schulman, P. Moritz, S. Levine, M. Jordan and P. Abbeel. “High-dimensional continuous control using generalized advantage estimation.” ICLR (2016).
    
+
    [3] Gymnasium Documentation: Vector environments. (URL: https://gymnasium.farama.org/api/vector/)
    
+
+
    
+
    
+
+        
    
+      
    
+      
+    
    
+    
+  
    
+
    
+    
+
+    
+    
+
+      
+      
+
+    
+    
+    
+    
+    
+    
+    
+    
+    
+
+    
+    
+
\ No newline at end of file
diff --git a/tutorials/index.html b/tutorials/index.html
new file mode 100644
index 000000000..ab735ca13
--- /dev/null
+++ b/tutorials/index.html
@@ -0,0 +1,724 @@
+
+
+  
+    
+    
+    
+    
+    
+    
+    
+
+        
+
+    
+        Tutorials - Gymnasium Documentation
+      
+    
+    
+    
+    
+    
+    
+    
+    
+
+
+
+  
+    
    
+      
    
+        
    
+          
+        
    
+        
    
+          
+              
+              
+            Gymnasium Documentation
+          
+        
    
+        
    
+          
    
+            
+            
+          
    
+        
    
+      
    
+    
    
+
+    
+    
+    
+
+
+  
+    Contents
+    
+      
+    
+  
+  
+    Menu
+    
+      
+      
+      
+    
+  
+  
+    Expand
+    
+      
+    
+  
+  
+    Light mode
+    
+      
+      
+      
+      
+      
+      
+      
+      
+      
+    
+  
+  
+    Dark mode
+    
+      
+      
+    
+  
+  
+    Auto light/dark mode
+    
+      
+      
+      
+      
+      
+      
+      
+    
+  
+
+
+
+
+
+
+
+
+
+
    
+   
+  
+  
    
+    
    
+      
    
+        
+          
+            
+          
+          Back to top
+        
+        
    
+          
+    
+    
    
+  
+    
+    
+  
+
    
+            
+          
    
+          
+        
    
+        
    
+          
+          
    
+
    Tutorials#
    
+
    
+
    Gymnasium Basics#
    
+
    Handling Time Limits
+
    Handling Time Limits
    
+  
    Handling Time Limits
    
+
    Implementing Custom Wrappers
+
    Implementing Custom Wrappers
    
+  
    Implementing Custom Wrappers
    
+
    Make your own custom environment
+
    Make your own custom environment
    
+  
    Make your own custom environment
    
+
    Training A2C with Vector Envs and Domain Randomization
+
    Training A2C with Vector Envs and Domain Randomization
    
+  
    Training A2C with Vector Envs and Domain Randomization
    
+
    
+
    
+
    Training Agents#
    
+
    Training using REINFORCE for Mujoco
+
    Training using REINFORCE for Mujoco
    
+  
    Training using REINFORCE for Mujoco
    
+
    Solving Blackjack with Q-Learning
+
    Solving Blackjack with Q-Learning
    
+  
    Solving Blackjack with Q-Learning
    
+
    
+
    
+
    
+
    
+
    Download all examples in Python source code: tutorials_python.zip
    
+
    
+
    
+
    Download all examples in Jupyter notebooks: tutorials_jupyter.zip
    
+
    
+
    
+
    
+
    
+
+        
    
+      
    
+      
    
+        
+        
    
+          
+            
+        
    
+        
    
+          
    
+            
    
+                Copyright © 2022 Farama Foundation
+            
    
+            
+          
    
+          
    
+            
    
+              
+            
    
+          
    
+        
    
+        
+      
    
+    
    
+    
+  
    
+
    
+    
+
+    
+    
+
+      
+      
+
+    
+    
+    
+    
+    
+    
+    
+    
+    
+
+    
+    
+
\ No newline at end of file
diff --git a/tutorials/training_agents/README/index.html b/tutorials/training_agents/README/index.html
new file mode 100644
index 000000000..8d3f47c7b
--- /dev/null
+++ b/tutorials/training_agents/README/index.html
@@ -0,0 +1,670 @@
+
+
+  
+    
+    
+    
+    
+    
+    
+    
+
+        
+
+    
+        Training Agents - Gymnasium Documentation
+      
+    
+    
+    
+    
+    
+    
+    
+    
+
+
+
+  
+    
    
+      
    
+        
    
+          
+        
    
+        
    
+          
+              
+              
+            Gymnasium Documentation
+          
+        
    
+        
    
+          
    
+            
+            
+          
    
+        
    
+      
    
+    
    
+
+    
+    
+    
+
+
+  
+    Contents
+    
+      
+    
+  
+  
+    Menu
+    
+      
+      
+      
+    
+  
+  
+    Expand
+    
+      
+    
+  
+  
+    Light mode
+    
+      
+      
+      
+      
+      
+      
+      
+      
+      
+    
+  
+  
+    Dark mode
+    
+      
+      
+    
+  
+  
+    Auto light/dark mode
+    
+      
+      
+      
+      
+      
+      
+      
+    
+  
+
+
+
+
+
+
+
+
+
+
    
+   
+  
+  
    
+    
    
+      
    
+        
+          
+            
+          
+          Back to top
+        
+        
    
+          
+    
+    
    
+  
+    
+    
+  
+
    
+            
+          
    
+          
+        
    
+        
    
+          
+          
    
+
    Training Agents#
    
+
    
+
+        
    
+      
    
+      
    
+        
+        
    
+          
+            
+        
    
+        
    
+          
    
+            
    
+                Copyright © 2022 Farama Foundation
+            
    
+            
+          
    
+          
    
+            
    
+              
+            
    
+          
    
+        
    
+        
+      
    
+    
    
+    
+  
    
+
    
+    
+
+    
+    
+
+      
+      
+
+    
+    
+    
+    
+    
+    
+    
+    
+    
+
+    
+    
+
\ No newline at end of file
diff --git a/tutorials/blackjack_tutorial/index.html b/tutorials/training_agents/blackjack_tutorial/index.html
similarity index 81%
rename from tutorials/blackjack_tutorial/index.html
rename to tutorials/training_agents/blackjack_tutorial/index.html
index 320790199..479e2ca17 100644
--- a/tutorials/blackjack_tutorial/index.html
+++ b/tutorials/training_agents/blackjack_tutorial/index.html
@@ -7,15 +7,19 @@
     
     
     
-    
-
-        
+    
+
+        
 
-    
+    
         Solving Blackjack with Q-Learning - Gymnasium Documentation
-      
-    
-    
+      
+    
+    
+    
+    
+    
+    
     
     
 
@@ -68,16 +72,16 @@
           
         
         
    
-          
-              
-              
+          
+              
+              
             Gymnasium Documentation
           
         
    
         
    
           
    
             
    use_contact_forces
    		
 
    bool
    		
 
    False
    If true, it extends the observation space by adding contact forces (see Observation Space section)
    If true, it extends the observation space by adding contact forces (see Observation Space section) and includes contact_cost to the reward function (see Rewards section)
    		
 
    
 
    contact_cost_weight
    		
 
    float
    hinge
    		
 
    torque (N m)
    		
 
    3
    2
    		
 
    Torque applied on the foot rotor
    		
 
    -1
    		
 
    1
    Torque applied on the hinge in the y-coordinate of the abdomen
    		
 
    -0.4
    		
 
    0.4
    hip_1 (front_left_leg)
    abdomen_y
    		
 
    hinge
    		
 
    torque (N m)
    		
 
    Torque applied on the hinge in the z-coordinate of the abdomen
    		
 
    -0.4
    		
 
    0.4
    angle_1 (front_left_leg)
    abdomen_z
    		
 
    hinge
    		
 
    torque (N m)
    		
 
    Torque applied on the hinge in the x-coordinate of the abdomen
    		
 
    -0.4
    		
 
    0.4
    hip_2 (front_right_leg)
    abdomen_x
    		
 
    hinge
    		
 
    torque (N m)
    		
 
    Torque applied on the hinge in the y-coordinate of the abdomen
    		
 
    -0.4
    		
 
    0.4
    hip_1 (front_left_leg)
    abdomen_y
    		
 
    hinge
    		
 
    torque (N m)
    		
 
    Torque applied on the hinge in the z-coordinate of the abdomen
    		
 
    -0.4
    		
 
    0.4
    angle_1 (front_left_leg)
    abdomen_z
    		
 
    hinge
    		
 
    torque (N m)
    		
 
    Torque applied on the hinge in the x-coordinate of the abdomen
    		
 
    -0.4
    		
 
    0.4
    hip_2 (front_right_leg)
    abdomen_x
    		
 
    hinge
    		
 
    torque (N m)
    		
 
    sine of the angle of the first arm
    		
 
    -Inf
    		
 
    Inf
    cos(joint0)
    sin(joint0)
    		
 
    hinge
    		
 
    unitless
    		
 
    sine of the angle of the second arm
    		
 
    -Inf
    		
 
    Inf
    cos(joint1)
    sin(joint1)
    		
 
    hinge
    		
 
    unitless
    		
 
    position (m)
    		
 
    
 
    10
    z-value of position_fingertip - position_target (0 since reacher is 2d and z is same for both)
    z-value of position_fingertip - position_target (constantly 0 since reacher is 2d and z is same for both)
    		
 
    -Inf
    		
 
    Inf
    		
 
    NA