Seeding update (#2422)

* Ditch most of the seeding.py and replace np_random with the numpy default_rng. Let's see if tests pass

* Updated a bunch of RNG calls from the RandomState API to Generator API

* black; didn't expect that, did ya?

* Undo a typo

* blaaack

* More typo fixes

* Fixed setting/getting state in multidiscrete spaces

* Fix typo, fix a test to work with the new sampling

* Correctly (?) pass the randomly generated seed if np_random is called with None as seed

* Convert the Discrete sample to a python int (as opposed to np.int64)

* Remove some redundant imports

* First version of the compatibility layer for old-style RNG. Mainly to trigger tests.

* Removed redundant f-strings

* Style fixes, removing unused imports

* Try to make tests pass by removing atari from the dockerfile

* Try to make tests pass by removing atari from the setup

* Try to make tests pass by removing atari from the setup

* Try to make tests pass by removing atari from the setup

* First attempt at deprecating `env.seed` and supporting `env.reset(seed=seed)` instead. Tests should hopefully pass but throw up a million warnings.

* black; didn't expect that, didya?

* Rename the reset parameter in VecEnvs back to `seed`

* Updated tests to use the new seeding method

* Removed a bunch of old `seed` calls.

Fixed a bug in AsyncVectorEnv

* Stop Discrete envs from doing part of the setup (and using the randomness) in init (as opposed to reset)

* Add explicit seed to wrappers reset

* Remove an accidental return

* Re-add some legacy functions with a warning.

* Use deprecation instead of regular warnings for the newly deprecated methods/functions

tests/vector/utils.py

@@ -1,3 +1,5 @@
+from typing import Optional
+
 import numpy as np
 import gym
 import time
@@ -55,7 +57,8 @@ class UnittestSlowEnv(gym.Env):
         )
         self.action_space = Box(low=0.0, high=1.0, shape=(), dtype=np.float32)
 
-    def reset(self):
+    def reset(self, seed: Optional[int] = None):
+        super().reset(seed=seed)
         if self.slow_reset > 0:
             time.sleep(self.slow_reset)
         return self.observation_space.sample()
@@ -86,7 +89,8 @@ class CustomSpaceEnv(gym.Env):
         self.observation_space = CustomSpace()
         self.action_space = CustomSpace()
 
-    def reset(self):
+    def reset(self, seed: Optional[int] = None):
+        super().reset(seed=seed)
         return "reset"
 
     def step(self, action):
@@ -98,7 +102,7 @@ class CustomSpaceEnv(gym.Env):
 def make_env(env_name, seed):
     def _make():
         env = gym.make(env_name)
-        env.seed(seed)
+        env.reset(seed=seed)
         return env
 
     return _make
@@ -107,7 +111,7 @@ def make_env(env_name, seed):
 def make_slow_env(slow_reset, seed):
     def _make():
         env = UnittestSlowEnv(slow_reset=slow_reset)
-        env.seed(seed)
+        env.reset(seed=seed)
         return env
 
     return _make
@@ -116,7 +120,7 @@ def make_slow_env(slow_reset, seed):
 def make_custom_space_env(seed):
     def _make():
         env = CustomSpaceEnv()
-        env.seed(seed)
+        env.reset(seed=seed)
         return env
 
     return _make