Commit Graph

33 Commits

Author SHA1 Message Date
Gianluca De Cola
49d8299a1e New info API for vectorized environments #2657 (#2773)
* WIP refactor info API sync vector.

* Add missing untracked file.

* Add info strategy to reset_wait.

* Add interface and docstring.

* info with strategy pattern on async vector env.

* Add default to async vecenv.

* episode statistics for asyncvecnev.

* Add tests info strategy format.

* Add info strategy to reset_wait.

* refactor and cleanup.

* Code cleanup. Add tests.

* Add tests for video recording with new info format.

* fix test case.

* fix camelcase.

* rename enum.

* update tests, docstrings, cleanup.

* Changes brax strategy to numpy. add_strategy method in StrategyFactory. Add tests.

* fix docstring and logging format.

* Set Brax info format as default. Remove classic info format. Update tests.

* breaking the wrong loop.

* WIP: wrapper.

* Add wrapper for brax to classic info.

* WIP: wrapper with nested RecordEpisodeStatistic.

* Add tests. Refactor docstrings. Cleanup.

* cleanup.

* patch conflicts.

* rebase and conflicts.

* new pre-commit conventions.

* docstring.

* renaming.

* incorporate info_processor in vecEnv.

* renaming. Create info dict only if needed.

* remove all brax references. update docstring. Update duplicate test.

* reviews.

* pre-commit.

* reviews.

* docstring.

* cleanup blank lines.

* add support for numpy dtypes.

* docstring fix.

* formatting.

* naming.

* assert correct info from wrappers chaining. Test correct wrappers chaining. naming.

* simplify episode_statistics.

* change args orer.

* update tests.

* wip: refactor episode_statistics.

* Add test for add_vecore_episode_statistics.
2022-05-24 10:36:35 -04:00
Mark Towers
e2266025e6 Pydocstyle utils vector docstring (#2788)
* Added pydocstyle to pre-commit

* Added docstrings for tests and updated the tests for autoreset

* Add pydocstyle exclude folder to allow slowly adding new docstrings

* Add docstrings for setup.py and gym/__init__.py, core.py, error.py and logger.py

* Check that all unwrapped environment are of a particular wrapper type

* Reverted back to import gym.spaces.Space to gym.spaces

* Fixed the __init__.py docstring

* Fixed autoreset autoreset test

* Updated gym __init__.py top docstring

* Fix examples in docstrings

* Add docstrings and type hints where known to all functions and classes in gym/utils and gym/vector

* Remove unnecessary import

* Removed "unused error" and make APIerror deprecated at gym 1.0

* Add pydocstyle description to CONTRIBUTING.md

* Added docstrings section to CONTRIBUTING.md

* Added :meth: and :attr: keywords to docstrings

* Added :meth: and :attr: keywords to docstrings

* Imported annotations from __future__ to fix python 3.7

* Add __future__ import annotations for python 3.7

* isort

* Remove utils and vectors for this PR and spaces for previous PR

* Update gym/envs/classic_control/acrobot.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/envs/classic_control/acrobot.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/envs/classic_control/acrobot.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/spaces/dict.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/env_checker.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/env_checker.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/env_checker.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/env_checker.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/env_checker.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/ezpickle.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/ezpickle.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Update gym/utils/play.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Pre-commit

* Updated docstrings with :meth:

* Updated docstrings with :meth:

* Update gym/utils/play.py

* Update gym/utils/play.py

* Update gym/utils/play.py

* Apply suggestions from code review

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* pre-commit

* Update gym/utils/play.py

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Updated fps and zoom parameter docstring

* Update play docstring

* Apply suggestions from code review

Added suggested corrections from @markus28

Co-authored-by: Markus Krimmel <montcyril@gmail.com>

* Pre-commit magic

* Update the `gym.make` docstring with a warning for `env_checker`

* Updated and fixed vector docstrings

* Update test names for reflect the project filename style

Co-authored-by: Markus Krimmel <montcyril@gmail.com>
2022-05-20 09:49:30 -04:00
Mark Towers
bf093c6890 Update the flake8 pre-commit ignores (#2778)
* Remove additional ignores from flake8

* Remove all unused imports

* Remove all unused imports

* Update flake8 and pyupgrade

* F841, removed unused variables

* E731, removed lambda assignment to variables

* Remove E731, F403, F405, F524

* Remove E722, bare exceptions

* Remove E712, compare variable == True or == False to is True or is False

* Remove E402, module level import not at top of file

* Added --pre-file-ignores

* Add --per-file-ignores removing E741, E302 and E704

* Add E741, do not use variables named ‘l’, ‘O’, or ‘I’ to ignore issues in classic control

* Fixed issues for pytest==6.2

* Remove unnecessary # noqa

* Edit comment with the removal of E302

* Added warnings and declared module, attr for pyright type hinting

* Remove unused import

* Removed flake8 E302

* Updated flake8 from 3.9.2 to 4.0.1

* Remove unused variable
2022-04-26 11:18:37 -04:00
Andrea PIERRÉ
e913bc81b8 Improve pre-commit workflow (#2602)
* feat: add `isort` to `pre-commit`

* ci: skip `__init__.py` file for `isort`

* ci: make `isort` mandatory in lint pipeline

* docs: add a section on Git hooks

* ci: check isort diff

* fix: isort from master branch

* docs: add pre-commit badge

* ci: update black + bandit versions

* feat: add PR template

* refactor: PR template

* ci: remove bandit

* docs: add Black badge

* ci: try to remove all `|| true` statements

* ci: remove lint_python job

- Remove `lint_python` CI job
- Move `pyupgrade` job to `pre-commit` workflow

* fix: avoid messing with typing

* docs: add a note on running `pre-cpmmit` manually

* ci: apply `pre-commit` to the whole codebase
2022-03-31 15:50:38 -04:00
Mark Towers
850247f888 Reduces warnings produced by pytest from ~1500 to 13 (#2660)
* Updated cartpole-v0 to v1 to prevent warning and added pytest.mark.filterwarnings for tests where warnings are unavoidable

* Change np.bool to bool as numpy raises a warning and bool is the suggested solution

* Seeding randint is deprecated in the future, integers is new solution

* Fixed errors thrown when the video recorder is deleted but not closed

* spaces.Box expects a floating array, updated all cases where this was not true and modified float32 to float64 as float array default to float64. Otherwise space.Box raises warning that dtype precision (float32) is lower than array precision (float64).

* Added pytest.mark.filterwarnings to preventing the raising of an intended warning

* Added comment to explain why a warning is raised that can't be prevented without version update to the environment

* Added comment to explain why warning is raised

* Changed values to float as expected by the box which default to float64

* Removed --forked from pytest as the pytest-forked project is no being maintained and was not raising warnings as expected

* When AsyncVectorEnv has shared_memory=True then a ValueError is raised before _state is initialised. Therefore, on the destruction on the env an error is thrown in .close_extra as _state does not exist

* Possible fix that was causing an error in test_call_async_vector_env by ensuring that pygame resources are released

* Pygame throws an error with ALSA when closed, using a fix from PettingZoo (https://github.com/Farama-Foundation/PettingZoo/blob/master/pettingzoo/__init__.py). We use the dsp audiodriver to prevent this issue

* Modification due to running pre-commit locally

* Updated cartpole-v0 to v1 to prevent warning and added pytest.mark.filterwarnings for tests where warnings are unavoidable

* Change np.bool to bool as numpy raises a warning and bool is the suggested solution

* Seeding randint is deprecated in the future, integers is new solution

* Fixed errors thrown when the video recorder is deleted but not closed

* spaces.Box expects a floating array, updated all cases where this was not true and modified float32 to float64 as float array default to float64. Otherwise space.Box raises warning that dtype precision (float32) is lower than array precision (float64).

* Added pytest.mark.filterwarnings to preventing the raising of an intended warning

* Added comment to explain why a warning is raised that can't be prevented without version update to the environment

* Added comment to explain why warning is raised

* Changed values to float as expected by the box which default to float64

* Removed --forked from pytest as the pytest-forked project is no being maintained and was not raising warnings as expected

* When AsyncVectorEnv has shared_memory=True then a ValueError is raised before _state is initialised. Therefore, on the destruction on the env an error is thrown in .close_extra as _state does not exist

* Possible fix that was causing an error in test_call_async_vector_env by ensuring that pygame resources are released

* Pygame throws an error with ALSA when closed, using a fix from PettingZoo (https://github.com/Farama-Foundation/PettingZoo/blob/master/pettingzoo/__init__.py). We use the dsp audiodriver to prevent this issue

* Modification due to running pre-commit locally
2022-03-14 10:27:03 -04:00
John Balis
15049e22d7 Adding return_info argument to reset to allow for optional info dict as a second return value (#2546)
* initial draft of optional info dict in reset function, implemented for cartpole, tests seem to be passing

* merged core.py

* updated return type annotation for reset function in core.py

* optional metadata with return_info from reset added for all first party environments, with corresponding tests. Incomplete implementation for wrappers and vector wrappers

* removed Optional type for return_info arguments

* added tests for return_info to normalize wrapper and sync_vector_env

* autoformatted using black

* added optional reset metadata tests to several wrappers

* added return_info capability to async_vector_env.py and test to verify functionality

* added optional return_info test for record_video.py

* removed tests for mujoco environments

* autoformatted

* improved test coverage for optional reset return_info

* re-removed unit test envs accidentally reintroduced in merge

* removed unnecessary import

* changes based on code-review

* small fix to core wrapper typing and autoformatted record_epsisode_stats

* small change to pass flake8 style
2022-02-06 18:28:27 -05:00
Tristan Deleu
081c5c1e80 Add call, get_attr and set_attr methods to VectorEnv (#1600)
* Add call, get_attr and set_attr methods

* Use f-strings and remove assert

* Allow tuples in set_attr and move docstrings

* Replace CubeCrash by CartPole in tests
2022-01-29 12:32:35 -05:00
Rushiv Arora
fcbff7de12 Added singledispatch utility to vector.utils & changed order of space argument. (#2536)
* Fixed ordering of space. Added singledispatch utility.

* Added singledispatch utility to vector.utils & changed order of space argument

* Fixed Error from _BaseGymSpaces

* Minor adjustment for Discrete Spaces

* Fixed Tests/ to reflect changes

* Fixed precommit error - custom namespaces

* Concrete Implementations start with _
2022-01-21 11:28:34 -05:00
Ariel Kwiatkowski
925823661d Add options to the signature of env.reset (#2515)
* First find/replace, now tests

* Fixes to the vector env

* Make seed keyword only in wrappers

* (try to) fix the bug with old environments using new wrappers (with the seed keyword)

* black

* Change **kwargs to options, try to make it work; black

* Add OrderEnforcing wrapper to wrapper exports
Add a test for compatibility with old (pybullet-like) envs

* Add OrderEnforcing wrapper to wrapper exports
Add a test for compatibility with old (pybullet-like) envs
black

* Update the env checker

* Update the env checker

* Update the env checker to use inspect (might fail tests, let's see)

* Allow the signature to include kwargs in env_checker

* Minor fix
2022-01-19 17:28:59 -05:00
Tristan Deleu
fbe3631aa9 Batch action_space in VectorEnv (#2280)
* Batch the action space in VectorEnv and add iterate utility function

* Add tests for iterate

* Add tests for action spaces in SyncVectorEnv and AsyncVectorEnv

* Black formatting

* Use singledispatch for iterate utility function

* Update the ordering of the arguments in the docstring

* Fix ordering in docstring example of iterate

* Check for same action spaces in vectorized environments

* Separate Discrete from other space types in iterate singledispatch
2021-12-08 21:31:41 -05:00
Tristan Deleu
cdb72ea552 Fix close_extras not implemented by default in VectorEnv (#2507) 2021-12-08 19:55:09 -05:00
Ariel Kwiatkowski
c364506710 Seeding update (#2422)
* Ditch most of the seeding.py and replace np_random with the numpy default_rng. Let's see if tests pass

* Updated a bunch of RNG calls from the RandomState API to Generator API

* black; didn't expect that, did ya?

* Undo a typo

* blaaack

* More typo fixes

* Fixed setting/getting state in multidiscrete spaces

* Fix typo, fix a test to work with the new sampling

* Correctly (?) pass the randomly generated seed if np_random is called with None as seed

* Convert the Discrete sample to a python int (as opposed to np.int64)

* Remove some redundant imports

* First version of the compatibility layer for old-style RNG. Mainly to trigger tests.

* Removed redundant f-strings

* Style fixes, removing unused imports

* Try to make tests pass by removing atari from the dockerfile

* Try to make tests pass by removing atari from the setup

* Try to make tests pass by removing atari from the setup

* Try to make tests pass by removing atari from the setup

* First attempt at deprecating `env.seed` and supporting `env.reset(seed=seed)` instead. Tests should hopefully pass but throw up a million warnings.

* black; didn't expect that, didya?

* Rename the reset parameter in VecEnvs back to `seed`

* Updated tests to use the new seeding method

* Removed a bunch of old `seed` calls.

Fixed a bug in AsyncVectorEnv

* Stop Discrete envs from doing part of the setup (and using the randomness) in init (as opposed to reset)

* Add explicit seed to wrappers reset

* Remove an accidental return

* Re-add some legacy functions with a warning.

* Use deprecation instead of regular warnings for the newly deprecated methods/functions
2021-12-08 16:14:15 -05:00
Tristan Deleu
01cc8a3a16 Add documentation for Vectorized environments (#2327) 2021-11-14 08:59:04 -05:00
Costa Huang
778d3c2473 [gym.vector]: add terminal_observation to info (#2484)
* [gym.vector]: add `terminal_observation` to `info`

* add test cases

* quick format fix
2021-11-14 08:57:44 -05:00
Elijah K
cb79c763bb Py36+ syntax in gym/vector: derived by running pyupgrade --py36-plus gym/vector/**.py and flynt gym --ll 120 (#2474)
Co-authored-by: Ilya Kamen <ilya.kamenshchikov@bosch.com>
2021-11-14 08:51:32 -05:00
Xingdong Zuo
263a3419ef [Python 3]: replace time() with perf_counter() for better measurements of short duration. (#2385)
* Update record_episode_statistics.py

* Update stats_recorder.py

* Update async_vector_env.py
2021-09-11 13:03:54 -04:00
Costa Huang
2853ce4797 Add RecordVideo wrapper (#2300)
* Add RecordVideo wrapper

* bug fix

* don't change gym's core API

* add test cases

* reformat
2021-08-18 16:36:40 -04:00
J K Terry
78d2b512d8 redo black (#2272) 2021-07-29 15:39:42 -04:00
Justin Terry
e9d2c41f2b redo black 2021-07-29 12:42:48 -04:00
Christian Clauss
bb81e141ea Blacken the codebase (#2265) 2021-07-28 20:26:34 -04:00
Tristan Deleu
58401db97e Allow custom spaces in VectorEnv (#2038)
* Allow custom observation spaces in VectorEnv

* Replace np.copy by deepcopy in reset of SyncVectorEnv

* Add tests for VectorEnv with custom spaces

* Add tests for shared memory and batches of custom spaces

* Remove unused import in VectorEnv test

* Add warning note in the Space class for custom spaces
2020-09-21 13:38:51 -07:00
Tristan Deleu
1a92bc6c83 Remove Python 2 compatibility in VectorEnv (#2034) 2020-09-11 15:07:27 -07:00
Christian Clauss
b7bd6bd1cc Typo: enviroment --> environment (#1828)
* Typo: enviroment --> environment

* Typo: information does not have a plural form in English
2020-03-06 16:37:23 -05:00
Xingdong Zuo
fe244fadf6 [gym.vector]: a genericclose with user-defined close_extras (#1631)
* Update vector_env.py

* Update sync_vector_env.py

* Update async_vector_env.py

* Update vector_env.py

* Update async_vector_env.py
2019-10-25 15:18:54 -07:00
pzhokhov
51136b1290 vector env updates (#1706)
* make daemon=True an option of async_vector_env

* custom worker in async_vector_env

* add compatibility methods to SyncVectorEnv

* fix name in sync_vector_env

* vectorenv api cleanup

* add docstrings for daemon and worker options in AsyncVectorEnv
2019-10-09 15:08:10 -07:00
Tristan Deleu
50ef7bf199 Fix AsyncVectorEnv with spawn context and shared memory (Fixes #1542) 2019-07-12 14:44:25 -07:00
Tristan Deleu
6ffcd212df Check for closed pipes in close for AsyncVectorEnv 2019-06-28 16:30:53 -07:00
Tristan Deleu
316156903d Check if the pipe is not None is _poll (Fixes #1542) 2019-06-28 16:30:53 -07:00
Tristan Deleu
cab1d0ec38 Add success flag in results from workers in AsyncVectorEnv 2019-06-28 16:30:53 -07:00
Tristan Deleu
ad8b3b20e6 Move _raise_if_errors after recv in AsyncVectorEnv (Fixes #1546) 2019-06-28 16:30:53 -07:00
Tristan Deleu
2e56028e9a Handle KeyboardInterrupt exception in AsyncVectorEnv (Fixes #1543) 2019-06-28 15:00:49 -07:00
Tristan Deleu
02cff040ee Fix typo in AsyncVectorEnv (Fixes #1544) 2019-06-28 14:42:11 -07:00
Tristan Deleu
c6a97e17ee Vectorized environments (#1513)
* Initial version of vectorized environments

* Raise an exception in the main process if child process raises an exception

* Add list of exposed functions in vector module

* Use deepcopy instead of np.copy

* Add documentation for vector utils

* Add tests for copy in AsyncVectorEnv

* Add example in documentation for batch_space

* Add cloudpickle dependency in setup.py

* Fix __del__ in VectorEnv

* Check if all observation spaces are equal in AsyncVectorEnv

* Check if all observation spaces are equal in SyncVectorEnv

* Fix spaces non equality in SyncVectorEnv for Python 2

* Handle None parameter in create_empty_array

* Fix check_observation_space with spaces equality

* Raise an exception when operations are out of order in AsyncVectorEnv

* Add version requirement for cloudpickle

* Use a state instead of binary flags in AsyncVectorEnv

* Use numpy.zeros when initializing observations in vectorized environments

* Remove poll from public API in AsyncVectorEnv

* Remove close_extras from VectorEnv

* Add test between AsyncVectorEnv and SyncVectorEnv

* Remove close in check_observation_space

* Add documentation for seed and close

* Refactor exceptions for AsyncVectorEnv

* Close pipes if the environment raises an error

* Add tests for out of order operations

* Change default argument in create_empty_array to np.zeros

* Add get_attr and set_attr methods to VectorEnv

* Improve consistency in SyncVectorEnv
2019-06-21 14:29:44 -07:00