Commit Graph

198 Commits

Author SHA1 Message Date
Peter Zhokhov
1f99a562e3 autopep8 2018-09-11 13:21:52 -07:00
Peter Zhokhov
4e2a888273 Merge commit 'refs/subrepo/baselines/fetch' into subrepo/baselines 2018-09-11 13:19:39 -07:00
Peter Zhokhov
c5b2918607 git subrepo pull (merge) baselines
subrepo:
  subdir:   "baselines"
  merged:   "2742f819"
upstream:
  origin:   "git@github.com:openai/baselines.git"
  branch:   "master"
  commit:   "5c5a9f4b"
git-subrepo:
  version:  "0.4.0"
  origin:   "git@github.com:ingydotnet/git-subrepo.git"
  commit:   "74339e8"
2018-09-11 13:18:43 -07:00
Peter Zhokhov
3bf31a4330 git subrepo commit (merge) baselines
subrepo:
  subdir:   "baselines"
  merged:   "0846932a"
upstream:
  origin:   "git@github.com:openai/baselines.git"
  branch:   "master"
  commit:   "c5d6f299"
git-subrepo:
  version:  "0.4.0"
  origin:   "git@github.com:ingydotnet/git-subrepo.git"
  commit:   "74339e8"
2018-09-11 13:18:43 -07:00
pzhokhov
9070ee7ef3 tighten flake8, autopep8 to fix trailing whitespaces and blank lines with whitespaces (#87) 2018-09-11 13:18:43 -07:00
Peter Zhokhov
e56803491f git subrepo pull (merge) baselines
subrepo:
  subdir:   "baselines"
  merged:   "5c6a1fd9"
upstream:
  origin:   "git@github.com:openai/baselines.git"
  branch:   "master"
  commit:   "23b23332"
git-subrepo:
  version:  "0.4.0"
  origin:   "git@github.com:ingydotnet/git-subrepo.git"
  commit:   "74339e8"
2018-09-11 13:18:42 -07:00
pzhokhov
b3bc25d99a add fast failure when calling methods on a closed subprocvecenv (#84) 2018-09-11 13:18:42 -07:00
Peter Zhokhov
5c5a9f4b31 autopep8 on deepq/experiments 2018-09-11 12:47:50 -07:00
Peter Zhokhov
5183fa9f29 autopep8 on deepq/experiments 2018-09-11 12:47:50 -07:00
Peter Zhokhov
3bf35cb468 added peterz to baselines authorlist 2018-09-11 12:44:51 -07:00
Peter Zhokhov
5c62f5c7dd added peterz to baselines authorlist 2018-09-11 12:44:51 -07:00
Peter Zhokhov
29bf587d15 Merge branch 'master' of github.com:openai/baselines 2018-09-11 12:40:29 -07:00
Peter Zhokhov
c5d6f2996c Merge branch 'master' of github.com:openai/baselines 2018-09-11 12:40:29 -07:00
Peter Zhokhov
06bdc2860c docstrings about vecenvs 2018-09-11 12:40:23 -07:00
pzhokhov
adaa8aefa8 baselines issue #564 (#574)
* fixes to enjoy_cartpole, enjoy_mountaincar.py

* fixed {train,enjoy}_pong, removed enjoy_retro

* set number of timesteps to 1e7 in train_pong

* flake8 complaints

* use synchronous version fo acktr in test_env_after_learn

* flake8
2018-09-10 11:50:59 -07:00
pzhokhov
23b2333238 baselines issue #564 (#574)
* fixes to enjoy_cartpole, enjoy_mountaincar.py

* fixed {train,enjoy}_pong, removed enjoy_retro

* set number of timesteps to 1e7 in train_pong

* flake8 complaints

* use synchronous version fo acktr in test_env_after_learn

* flake8
2018-09-10 11:50:59 -07:00
Peter Zhokhov
8614c4ddbf flake8 2018-09-10 10:41:29 -07:00
Peter Zhokhov
59a7ffb84d fixe tests of test_env_after_learn 2018-09-10 10:32:42 -07:00
Daniel Angelov
58b1021b28 Add tensorboard start command for convenience (#569) 2018-09-07 17:04:02 -07:00
Peter Zhokhov
a60e88bff9 git subrepo pull (merge) baselines
subrepo:
  subdir:   "baselines"
  merged:   "8785db28"
upstream:
  origin:   "git@github.com:openai/baselines.git"
  branch:   "master"
  commit:   "35e95ee8"
git-subrepo:
  version:  "0.4.0"
  origin:   "git@github.com:ingydotnet/git-subrepo.git"
  commit:   "74339e8"
2018-09-07 16:35:00 -07:00
pzhokhov
75b93b890e implement pdfromlatent in BernoulliPdType (#81)
* implement pdfromlatent in BernoulliPdType

* remove env.close() at the end of algorithms

* test case for environment after learn

* closing env in run.py

* fixes for acktr and trpo_mpi

* add make_session with new graph for every call in test_env_after_learn

* remove extra prints from test_env_after_learn
2018-09-07 16:35:00 -07:00
John Schulman
565b2153d7 Add lots of docstrings (#76)
* Add lots of docstrings
Change hyperparameter transformations for slightly better efficiency and to avoid circular dependency.
Now all parameters are stored in a “human-readable” form.

* improve pretty-print of nodes and trees

* newlines at end-of-file, return graph in render(), assert_valid() fix

* split run_algo_search.py into several simpler scripts

* add joint_train option to get_prob

* minor changes to soln_db and embedding script

* Arguments: -> Args:

* fix replay, part 1

* fix behavior when using unpickled algos

* re-add retrieve_weights

* make training scripts more consistent

* lint

* lint

* lint + remove rendering some rendering functionality from trex env as it’s also elsewhere

* get rid of warnings

* refactor functionality for getting final q-function and losses. revive code for removing useless terms & tests for simplification.

* fix vecenv closing

* finish removing algo folder (most useful functionality has been moved out of it)

* control verbosity of trex

* fix tests

* rename spec => choice_spec, some comments, asserts, debug prints

* fix some tests
2018-09-07 16:34:59 -07:00
Peter Zhokhov
35e95ee85a fix python 3.5 string format compatibility 2018-09-06 12:00:19 -07:00
Isaac Lascasas
ad219e205d VecNormalize: set env. returns to zero on resets. (#556)
* VecNormalize: set env. returns to zero on resets.

* VecNormalize: returns reset in step_wait after ret_rms.update.
2018-09-06 10:21:50 -07:00
Peter Zhokhov
be9118bcd8 git subrepo pull (merge) baselines
subrepo:
  subdir:   "baselines"
  merged:   "f2a9b8f2"
upstream:
  origin:   "git@github.com:openai/baselines.git"
  branch:   "master"
  commit:   "cc4215ef"
git-subrepo:
  version:  "0.4.0"
  origin:   "git@github.com:ingydotnet/git-subrepo.git"
  commit:   "74339e8"
2018-09-06 10:18:13 -07:00
pzhokhov
02a5e7aed5 fixes to readme and baselines/run.py (#80)
* fixes to readme and baselines/run.py

* polish installation section of baselines README

* polish installation section of baselines README
2018-09-06 10:18:13 -07:00
pzhokhov
87ac8bc317 install roboschool in install.py (#55)
* putting instructions from README.md into a script

* install roboschool as a part of setup.py

* install roboschool from install.py

* export pkg_config_path

* remove compilation step from roboschool/setup.py

* removed roboschool install from games install due to extra compilation step

* removed unused import from roboschool/setup.py
2018-09-06 10:18:13 -07:00
Tom
cc4215ef4b refactor common.models via registering reflection (#565) 2018-09-06 10:16:06 -07:00
Clayton Thorrez
1e9051e87e fixed warning (#464) 2018-09-05 15:12:01 -07:00
uronce-cc
43ed76944b Fix mean reward per episode after training Pong. (#562)
* Fix mean reward per episode after training Pong.

* Fix typo.
2018-09-05 15:06:29 -07:00
Peter Zhokhov
7f08c675bb git subrepo pull (merge) baselines
subrepo:
  subdir:   "baselines"
  merged:   "39f8be8f"
upstream:
  origin:   "git@github.com:openai/baselines.git"
  branch:   "master"
  commit:   "0a40206c"
git-subrepo:
  version:  "0.4.0"
  origin:   "git@github.com:ingydotnet/git-subrepo.git"
  commit:   "74339e8"
2018-09-04 10:23:40 -07:00
pzhokhov
b3f966aa02 use env.render in dummy_vec_env.render when num_envs == 1 (#74)
* use env.render in dummy_vec_env.render when num_envs == 1

* use shorter super() syntax per Alex's suggestion
2018-09-04 10:23:40 -07:00
pzhokhov
51cefc933b make load_variables compatible with old list format (#71)
* make load_variables compatible with old list format

* cosmetic fixes
2018-09-04 10:23:39 -07:00
Christopher Hesse
7bccb2969f baselines: default logger similar to configure() logger, rcall: don't call logger.configure() for new rl_algs
* error if logger looks wrong

* check version of logger, call logger.configure() on import

* remove changes entry

* add version to rl-algs

* fix typo

* add comment

* switch version to string

* set logger env variable
2018-09-04 10:23:39 -07:00
uronce-cc
0a40206c6c ncpu needs to be an integer. (#558) 2018-08-31 09:02:18 -07:00
Alfredo Canziani
1937826784 Fix alien syntax and apply PEP 8 style (#554) 2018-08-30 17:21:25 -07:00
pzhokhov
b29c8020d7 remove saving model as a pickle file in ppo2 (tries to pull environment in; bad idea - may need to use constructor argument pickling or somesuch if at all necessary) (#69) 2018-08-30 13:41:38 -07:00
Peter Zhokhov
4ec308aaa4 fixed syntax 2018-08-30 13:41:38 -07:00
Peter Zhokhov
3bbf3f3511 allow_early_resets=True in create_vec_env 2018-08-30 13:41:38 -07:00
Joshua Meier
e5de29a954 instructions for tensorboard (#61) 2018-08-30 13:41:37 -07:00
Joshua Meier
2507d335f9 Tensorboard util (#60)
* separate_validation_set was not imported

* launching tensorboard automatically
2018-08-30 13:41:37 -07:00
Damien Lancry
bdd4d385a6 Fix result_plotters in vectorized mujoco environments (#533)
* I investigated a bit about running a training in a vectorized monitored mujoco env and found out that the 0.monitor.csv file could not be plotted using baselines.results_plotter.py functions. Moreover the seed is the same in every parallel environments due to the particular behaviour of lambda. this fixes both issues without breaking the function in other files (baselines.acktr.run_mujoco still works)

* unifies make_atari_env and make_mujoco_env

* redefine make_mujoco_env because of run_mujoco in acktr not compatible with DummyVecEnv and SubprocVecEnv

* fix if else

* Update run.py
2018-08-28 17:48:56 -07:00
Peter Zhokhov
0961f5dd94 git subrepo pull (merge) baselines
subrepo:
  subdir:   "baselines"
  merged:   "95a81e86"
upstream:
  origin:   "git@github.com:openai/baselines.git"
  branch:   "master"
  commit:   "c6c0f45c"
git-subrepo:
  version:  "0.4.0"
  origin:   "git@github.com:ingydotnet/git-subrepo.git"
  commit:   "74339e8"
2018-08-27 16:40:14 -07:00
Christopher Hesse
337d913a8f remove reset_task from subproc vec env (#45) 2018-08-27 16:40:14 -07:00
Karl Cobbe
34af61a132 baselines: fix dummy vec env render mode (#42) 2018-08-27 16:40:14 -07:00
Christopher Hesse
1ea5ec647c export SimpleEnv and assert_envs_equal, fix minor bug in action space (#46) 2018-08-27 16:40:14 -07:00
pzhokhov
2fc7a1cbee Trigger benchmarks from buildkite (#40)
* rig buildkite pipeline to run benchmarks when commit ends with RUN BENCHMARKS

* fix the buildkite pipeline file

* fix the buildkite pipeline file

* fix the buildkite pipeline file

* fix the buildkite pipeline file

* fix the buildkite pipeline file

* fix the buildkite pipeline file

* fix the buildkite pipeline file - merge test and benchmark steps

* fix the buildkite pipeline file - merge test and benchmark steps

* fix buildkite pipeline file

* fix buildkite pipeline file

* dry RUN BENCHMARKS

* dry RUN BENCHMARKS

* dry not run BENCHMARKS

* not run benchmarks

* not running benchmarks

* no running benchmarks

* no running benchmarks

* still not running benchmarks

* dummy commit to RUN BENCHMARKS

* trigger benchmarks from buildkite RUN BENCHMARKS

* specifying RCALL_KUBE_CLUSTER RUN BENCHMARKS

* remove rl-algs/run-benchmarks-new.py (moved to ci), merged baselines/common/console_util and baselines/common/util.py

* added missing imports in console_util

* clone subrepo over https
2018-08-27 16:40:14 -07:00
John Schulman
14c1d69ef4 Reduce duplication in VecEnv subclasses. (#38)
* Reduce duplication in VecEnv subclasses.
Now VecEnv base class handles rendering and closing; subclasses should provide get_images and (optionally) close_extras.

* fix tests

* minor docstring change

* raise NotImplementedError
2018-08-27 16:40:13 -07:00
pzhokhov
c8f6d8bac7 address rl-algs issue #169 (missing util functions from rcall) (#30)
* copied parts of util.py to baselines.common from rcall

* merged fix for baselines.logger, resolved conflicts

* copied ccap to baselines/baselines/common/util.py
2018-08-27 16:40:13 -07:00
pzhokhov
3a006ba50e flake8 fixes (#35)
* flake8 fixes

* added baselines/setup.cfg

* style checks using setup.cfg in baselines
2018-08-27 16:40:13 -07:00