* I investigated a bit about running a training in a vectorized monitored mujoco env and found out that the 0.monitor.csv file could not be plotted using baselines.results_plotter.py functions. Moreover the seed is the same in every parallel environments due to the particular behaviour of lambda. this fixes both issues without breaking the function in other files (baselines.acktr.run_mujoco still works)
* unifies make_atari_env and make_mujoco_env
* redefine make_mujoco_env because of run_mujoco in acktr not compatible with DummyVecEnv and SubprocVecEnv
* fix if else
* Update run.py
* rig buildkite pipeline to run benchmarks when commit ends with RUN BENCHMARKS
* fix the buildkite pipeline file
* fix the buildkite pipeline file
* fix the buildkite pipeline file
* fix the buildkite pipeline file
* fix the buildkite pipeline file
* fix the buildkite pipeline file
* fix the buildkite pipeline file - merge test and benchmark steps
* fix the buildkite pipeline file - merge test and benchmark steps
* fix buildkite pipeline file
* fix buildkite pipeline file
* dry RUN BENCHMARKS
* dry RUN BENCHMARKS
* dry not run BENCHMARKS
* not run benchmarks
* not running benchmarks
* no running benchmarks
* no running benchmarks
* still not running benchmarks
* dummy commit to RUN BENCHMARKS
* trigger benchmarks from buildkite RUN BENCHMARKS
* specifying RCALL_KUBE_CLUSTER RUN BENCHMARKS
* remove rl-algs/run-benchmarks-new.py (moved to ci), merged baselines/common/console_util and baselines/common/util.py
* added missing imports in console_util
* clone subrepo over https
* Reduce duplication in VecEnv subclasses.
Now VecEnv base class handles rendering and closing; subclasses should provide get_images and (optionally) close_extras.
* fix tests
* minor docstring change
* raise NotImplementedError
* copied parts of util.py to baselines.common from rcall
* merged fix for baselines.logger, resolved conflicts
* copied ccap to baselines/baselines/common/util.py
* make nminibatces = min(nminibatches, nenv)
* clarify the usage of lstm policy, add an example and a test
* cleaned up example, added assert to the test
* remove nminibatches -> min(nminibatches, num_env)
* removed code snippet from the docstring, pointing to the file
* add _mujoco_present flag to skip the tests that require mujoco if mujoco is not present
* re-format skip message in test_doc_examples
* flake8 complaints
* make nminibatces = min(nminibatches, nenv)
* clarify the usage of lstm policy, add an example and a test
* cleaned up example, added assert to the test
* remove nminibatches -> min(nminibatches, num_env)
* removed code snippet from the docstring, pointing to the file
* restore conftest.py in subrepos
* remove conftest files from subrepos in the docker image
* remove runslow flag from baselines .travis.yml and rl-algs ci/runtests.sh
* move import of rendering module into the code to fix tests that don't require a display
* restore the dockerfile
* move vec_env
* cleaning up rl_common
* tests are passing (but mosts tests are deleted as moved to baselines)
* add benchmark runner for smoke tests
* removed duplicated algos
* route references to rl_algs.a2c to baselines.a2c
* route references to rl_algs.a2c to baselines.a2c
* unify conftest.py
* removing references to duplicated algs from codegen
* removing references to duplicated algs from codegen
* alex's changes to dummy_vec_env
* fixed test_carpole[deepq] testcase by decreasing number of training steps... alex's changes seemed to have fixed the bug and make it train better, but at seed=0 there is a dip in the training curve at 30k steps that fails the test
* codegen tests with atol=1e-6 seem to be unstable
* rl_common.vec_env -> baselines.common.vec_env mass replace
* fixed reference in trpo_mpi
* a2c.util references
* restored rl_algs.bench in sonic_prob
* fix reference in ci/runtests.sh
* simplifed expression in baselines/common/cmd_util
* further increased rtol to 1e-3 in codegen tests
* switched vecenvs to use SimpleImageViewer from gym instead of cv2
* make run.py --play option work with num_envs > 1
* make rosenbrock test reproducible
* git subrepo pull (merge) baselines
subrepo:
subdir: "baselines"
merged: "e23524a5"
upstream:
origin: "git@github.com:openai/baselines.git"
branch: "master"
commit: "bcde04e7"
git-subrepo:
version: "0.4.0"
origin: "git@github.com:ingydotnet/git-subrepo.git"
commit: "74339e8"
* updated baselines README (num-timesteps --> num_timesteps)
* typo in deepq/README.md
* re-setting up travis
* re-setting up travis
* resolved merge conflicts, added missing dependency for codegen
* removed parallel tests (workers are failing for some reason)
* try test baselines only
* added language options - some weirdness in rcall image that requires them?
* added verbosity to tests
* try tests in baselines only
* ci/runtests.sh tests codegen (some failure on baselines specifically on travis, trying to narrow down the problem)
* removed render from codegen test - maybe that's the problem?
* trying even simpler command within the image to figure out the problem
* print out system info in ci/runtests.sh
* print system info outside of docker as well
* trying single test file in codegen
* install graphviz in the docker image
* git subrepo pull baselines
subrepo:
subdir: "baselines"
merged: "8c2aea2"
upstream:
origin: "git@github.com:openai/baselines.git"
branch: "master"
commit: "8c2aea2"
git-subrepo:
version: "0.4.0"
origin: "git@github.com:ingydotnet/git-subrepo.git"
commit: "74339e8"
* added graphviz to the dockerfile (need both graphviz-dev and graphviz)
* only tests in codegen/algo/test_algo_builder.py
* run baselines tests only. still no clue why collection of codegen tests fails
* update baselines setup to install filelock for tests
* run slow tests
* skip slow tests in baselines
* single test file in baselines
* try reinstalling tensorflow
* running slow tests
* try full baselines and codegen test suite
* in the test Dockerfile, reinstall tensorflow
* using fake display for codegen render tests
* fixed display-related failures by adding a custom entrpoint to the docker image
* set LC_ALL and LANG env variables in docker image
* try sequential tests
* include psutil in requirements; increase relative tolerance in test_low_level_algo_distr
* trying to fix codegen failures on travis
* git subrepo commit (merge) baselines
subrepo:
subdir: "baselines"
merged: "9ce84da"
upstream:
origin: "git@github.com:openai/baselines.git"
branch: "master"
commit: "b222dd0"
git-subrepo:
version: "0.4.0"
origin: "git@github.com:ingydotnet/git-subrepo.git"
commit: "74339e8"
* syntax in install.py
* changing the order of package installation
* removed supervised-reptile from installation list
* cron uses the full games repo in rcall
* flake8 complaints
* rewrite all extras logic in baselines, install.py always uses [all]
* re-setting up travis
* re-setting up travis
* resolved merge conflicts, added missing dependency for codegen
* removed parallel tests (workers are failing for some reason)
* try test baselines only
* added language options - some weirdness in rcall image that requires them?
* added verbosity to tests
* try tests in baselines only
* ci/runtests.sh tests codegen (some failure on baselines specifically on travis, trying to narrow down the problem)
* removed render from codegen test - maybe that's the problem?
* trying even simpler command within the image to figure out the problem
* print out system info in ci/runtests.sh
* print system info outside of docker as well
* trying single test file in codegen
* install graphviz in the docker image
* git subrepo pull baselines
subrepo:
subdir: "baselines"
merged: "8c2aea2"
upstream:
origin: "git@github.com:openai/baselines.git"
branch: "master"
commit: "8c2aea2"
git-subrepo:
version: "0.4.0"
origin: "git@github.com:ingydotnet/git-subrepo.git"
commit: "74339e8"
* added graphviz to the dockerfile (need both graphviz-dev and graphviz)
* only tests in codegen/algo/test_algo_builder.py
* run baselines tests only. still no clue why collection of codegen tests fails
* update baselines setup to install filelock for tests
* run slow tests
* skip slow tests in baselines
* single test file in baselines
* try reinstalling tensorflow
* running slow tests
* try full baselines and codegen test suite
* in the test Dockerfile, reinstall tensorflow
* using fake display for codegen render tests
* fixed display-related failures by adding a custom entrpoint to the docker image
* set LC_ALL and LANG env variables in docker image
* try sequential tests
* include psutil in requirements; increase relative tolerance in test_low_level_algo_distr
* trying to fix codegen failures on travis
* git subrepo commit (merge) baselines
subrepo:
subdir: "baselines"
merged: "9ce84da"
upstream:
origin: "git@github.com:openai/baselines.git"
branch: "master"
commit: "b222dd0"
git-subrepo:
version: "0.4.0"
origin: "git@github.com:ingydotnet/git-subrepo.git"
commit: "74339e8"
* syntax in install.py
* changing the order of package installation
* removed supervised-reptile from installation list
* cron uses the full games repo in rcall
* flake8 complaints
* rewrite all extras logic in baselines, install.py always uses [all]
* update per-algorithm READMEs to reflect new way of running algorithms
* adding a link to repo-wide README
* updated README files and deepq.train_cartpole example
* update per-algorithm READMEs to reflect new way of running algorithms
* adding a link to repo-wide README
* updated README files and deepq.train_cartpole example
* updated benchmark pages with final rewards
* use htmlpreview to render pages
* use htmlpreview to render pages
* use htmlpreview to render pages
* updated README to reflect ppo1 being obsolete
* removed navbars from published benchmark pages
* fixed link in README
* exported rl-algs
* more stuff from rl-algs
* run slow tests
* re-exported rl_algs
* re-exported rl_algs - fixed problems with serialization test and test_cartpole
* replaced atari_arg_parser with common_arg_parser
* run.py can run algos from both baselines and rl_algs
* added approximate humanoid reward with ppo2 into the README for reference
* dummy commit to RUN BENCHMARKS
* dummy commit to RUN BENCHMARKS
* dummy commit to RUN BENCHMARKS
* dummy commit to RUN BENCHMARKS
* very dummy commit to RUN BENCHMARKS
* serialize variables as a dict, not as a list
* running_mean_std uses tensorflow variables
* fixed import in vec_normalize
* dummy commit to RUN BENCHMARKS
* dummy commit to RUN BENCHMARKS
* flake8 complaints
* save all variables to make sure we save the vec_normalize normalization
* benchmarks on ppo2 only RUN BENCHMARKS
* make_atari_env compatible with mpi
* run ppo_mpi benchmarks only RUN BENCHMARKS
* hardcode names of retro environments
* add defaults
* changed default ppo2 lr schedule to linear RUN BENCHMARKS
* non-tf normalization benchmark RUN BENCHMARKS
* use ncpu=1 for mujoco sessions - gives a bit of a performance speedup
* reverted running_mean_std to user property decorators for mean, var, count
* reverted VecNormalize to use RunningMeanStd (no tf)
* reverted VecNormalize to use RunningMeanStd (no tf)
* profiling wip
* use VecNormalize with regular RunningMeanStd
* added acer runner (missing import)
* flake8 complaints
* added a note in README about TfRunningMeanStd and serialization of VecNormalize
* dummy commit to RUN BENCHMARKS
* merged benchmarks branch
* import rl-algs from 2e3a166 commit
* extra import of the baselines badge
* exported commit with identity test
* proper rng seeding in the test_identity
* import internal
* adding missing tile_images.py
* import rl-algs from 2e3a166 commit
* extra import of the baselines badge
* exported commit with identity test
* proper rng seeding in the test_identity
* import internal
* import rl-algs from 2e3a166 commit
* extra import of the baselines badge
* exported commit with identity test
* proper rng seeding in the test_identity
* simple .travis.yml file
* added static syntax checks of common to .travis.yml
* dockerizing the build
* fix Dockerfile, adding build shield
* cleaning up workdir in Dockerfile and .travis.yml
* .travis.yml fixed common -> baselines/common for style check
* changes to README.md files with more detailed installation instructions
* md-fying the changes better
* link on the word homebrew in readme.md
* typos in README.md
* README.md
* removed extra comma sign
* removed sudo from brew command
