353bb15e90
* move vec_env * cleaning up rl_common * tests are passing (but mosts tests are deleted as moved to baselines) * add benchmark runner for smoke tests * removed duplicated algos * route references to rl_algs.a2c to baselines.a2c * route references to rl_algs.a2c to baselines.a2c * unify conftest.py * removing references to duplicated algs from codegen * removing references to duplicated algs from codegen * alex's changes to dummy_vec_env * fixed test_carpole[deepq] testcase by decreasing number of training steps... alex's changes seemed to have fixed the bug and make it train better, but at seed=0 there is a dip in the training curve at 30k steps that fails the test * codegen tests with atol=1e-6 seem to be unstable * rl_common.vec_env -> baselines.common.vec_env mass replace * fixed reference in trpo_mpi * a2c.util references * restored rl_algs.bench in sonic_prob * fix reference in ci/runtests.sh * simplifed expression in baselines/common/cmd_util * further increased rtol to 1e-3 in codegen tests * switched vecenvs to use SimpleImageViewer from gym instead of cv2 * make run.py --play option work with num_envs > 1 * make rosenbrock test reproducible * git subrepo pull (merge) baselines subrepo: subdir: "baselines" merged: "e23524a5" upstream: origin: "git@github.com:openai/baselines.git" branch: "master" commit: "bcde04e7" git-subrepo: version: "0.4.0" origin: "git@github.com:ingydotnet/git-subrepo.git" commit: "74339e8" * updated baselines README (num-timesteps --> num_timesteps) * typo in deepq/README.md
34 lines
1.2 KiB
Python
34 lines
1.2 KiB
Python
from . import VecEnvWrapper
|
|
import numpy as np
|
|
from gym import spaces
|
|
|
|
|
|
class VecFrameStack(VecEnvWrapper):
|
|
def __init__(self, venv, nstack):
|
|
self.venv = venv
|
|
self.nstack = nstack
|
|
wos = venv.observation_space # wrapped ob space
|
|
low = np.repeat(wos.low, self.nstack, axis=-1)
|
|
high = np.repeat(wos.high, self.nstack, axis=-1)
|
|
self.stackedobs = np.zeros((venv.num_envs,) + low.shape, low.dtype)
|
|
observation_space = spaces.Box(low=low, high=high, dtype=venv.observation_space.dtype)
|
|
VecEnvWrapper.__init__(self, venv, observation_space=observation_space)
|
|
|
|
def step_wait(self):
|
|
obs, rews, news, infos = self.venv.step_wait()
|
|
self.stackedobs = np.roll(self.stackedobs, shift=-1, axis=-1)
|
|
for (i, new) in enumerate(news):
|
|
if new:
|
|
self.stackedobs[i] = 0
|
|
self.stackedobs[..., -obs.shape[-1]:] = obs
|
|
return self.stackedobs, rews, news, infos
|
|
|
|
def reset(self):
|
|
obs = self.venv.reset()
|
|
self.stackedobs[...] = 0
|
|
self.stackedobs[..., -obs.shape[-1]:] = obs
|
|
return self.stackedobs
|
|
|
|
def close(self):
|
|
self.venv.close()
|