Files
Gymnasium/gym/spaces/space.py
Mark Towers 3354451300 Fixed batch spaces where the original space's seed was ignored. Issue 2680 (#2727)
* Add a case for the Box shape where the low and high values are both scalars

* Add seeding.RandomNumberGenerator parameter to Dict seed. Modify __repr__ for the dictionary space string looks similar to an actual dictionary

* Add seeding.RandomNumberGenerator parameter to Multi Binary seed

* Add seeding.RandomNumberGenerator parameter to Multi Binary seed. Modify nvec typing to include np.ndarray

* Space seed typing can be a seeding.RandomNumberGenerator. If a seeding.RNG is provided then it is assigned to _np_random and .seed is not run

* Fixed the tuple seeding type as List[int] is not a valid Space seed type

* Added typing to batch_space. The batch_space seed is equal to the space's seeding

* Fixed the seeding type

* Add test for batch space seeds are identical to the original space's seeding

* Add equivalence function for RandomNumberGenerator comparing the bit_generator.state

* The batch_space functions uses a copy of the seed for the original space

* Set the action space seed for sync_vector_env seed testing

* Add test for the seeding of the sync vector environment

* Update the test_batch_space_seed to check the resulting sampling are equivalent for testing

* Revert representation back to the original version

* Remove additional Box shape initialisation

* Remove additional typing of MultiDiscrete

* Fixed bug of Space batch space where the original space's np_random is not a complete copy of the original space

* Add CustomSpace to the batched space seed test

* Modify the CustomSpace sample to produce a random number not a static value

* Fix CustomSpace to reflect the sample function

* Copy the space.np_random for the batched_space seed to ensure that the original space doesn't sampling doesn't effect the batched_space

* Parameterized the batch_space_seed, added testing for rng_different_at_each_index and test_deterministic

* Black and isort pre-commit changes

* Pre-commit fix

* MacOS, test_read_from_shared_memory throws an error that the inner _process_write function was unpicklable. Making the function a top-level function solves this error

* Fixed typing of seed where a space's seed function differs from Space.seed's typing

* Added check that the sample lengths are equal and explicitly provided the number of batched spaces n=1

* Removed relative imports for absolute imports

* Use deepcopy instead of copy

* Replaces `from numpy.testing._private.utils import assert_array_equal` with `from numpy.testing import assert_array_equal`

* Using the seeding `__eq__` function, replace `np_random.bit_generator.state` with `np_random`

* Added docstrings and comments to the tests to explain their purpose

* Remove __eq__ from RandomNumberGenerator and add to tests/vector/utils

* Add sync vector determinism test for issue #2680

* Fixed bug for 462101d384 (r850740825)

* Made the new seeds a list of integers
2022-04-24 12:14:33 -04:00

105 lines
3.7 KiB
Python

from __future__ import annotations
from typing import Generic, Iterable, Mapping, Optional, Sequence, Type, TypeVar
import numpy as np
from gym.utils import seeding
T_cov = TypeVar("T_cov", covariant=True)
class Space(Generic[T_cov]):
"""Defines the observation and action spaces, so you can write generic
code that applies to any Env. For example, you can choose a random
action.
WARNING - Custom observation & action spaces can inherit from the `Space`
class. However, most use-cases should be covered by the existing space
classes (e.g. `Box`, `Discrete`, etc...), and container classes (`Tuple` &
`Dict`). Note that parametrized probability distributions (through the
`sample()` method), and batching functions (in `gym.vector.VectorEnv`), are
only well-defined for instances of spaces provided in gym by default.
Moreover, some implementations of Reinforcement Learning algorithms might
not handle custom spaces properly. Use custom spaces with care.
"""
def __init__(
self,
shape: Optional[Sequence[int]] = None,
dtype: Optional[Type | str] = None,
seed: Optional[int | seeding.RandomNumberGenerator] = None,
):
self._shape = None if shape is None else tuple(shape)
self.dtype = None if dtype is None else np.dtype(dtype)
self._np_random = None
if seed is not None:
if isinstance(seed, seeding.RandomNumberGenerator):
self._np_random = seed
else:
self.seed(seed)
@property
def np_random(self) -> seeding.RandomNumberGenerator:
"""Lazily seed the rng since this is expensive and only needed if
sampling from this space.
"""
if self._np_random is None:
self.seed()
return self._np_random # type: ignore ## self.seed() call guarantees right type.
@property
def shape(self) -> Optional[tuple[int, ...]]:
"""Return the shape of the space as an immutable property"""
return self._shape
def sample(self) -> T_cov:
"""Randomly sample an element of this space. Can be
uniform or non-uniform sampling based on boundedness of space."""
raise NotImplementedError
def seed(self, seed: Optional[int] = None) -> list:
"""Seed the PRNG of this space."""
self._np_random, seed = seeding.np_random(seed)
return [seed]
def contains(self, x) -> bool:
"""
Return boolean specifying if x is a valid
member of this space
"""
raise NotImplementedError
def __contains__(self, x) -> bool:
return self.contains(x)
def __setstate__(self, state: Iterable | Mapping):
# Don't mutate the original state
state = dict(state)
# Allow for loading of legacy states.
# See:
# https://github.com/openai/gym/pull/2397 -- shape
# https://github.com/openai/gym/pull/1913 -- np_random
#
if "shape" in state:
state["_shape"] = state["shape"]
del state["shape"]
if "np_random" in state:
state["_np_random"] = state["np_random"]
del state["np_random"]
# Update our state
self.__dict__.update(state)
def to_jsonable(self, sample_n: Sequence[T_cov]) -> list:
"""Convert a batch of samples from this space to a JSONable data type."""
# By default, assume identity is JSONable
return list(sample_n)
def from_jsonable(self, sample_n: list) -> list[T_cov]:
"""Convert a JSONable data type to a batch of samples from this space."""
# By default, assume identity is JSONable
return sample_n