Commit Graph

36 Commits

Author SHA1 Message Date
Jie Tang
6e0f83e9d5 Validate multiple seeds for benchmark 2016-10-17 17:56:14 -07:00
Jie Tang
422882913b Separate benchmark and evaluation control flow in upload(), add client side validation of benchmark contents 2016-10-09 20:49:49 -07:00
Greg Brockman
934b2acbb7 Add benchmark support (#338)
* Warn if seed doesn't return a list

* Add preliminary BenchmarkRun support

* Add experimental benchmark registration

* Flesh out interface

* Add preliminary BenchmarkRun support

* Warn if seed doesn't return a list

* Add experimental benchmark registration

* Flesh out interface

* Make benchmarkrun upload recursive

* Add evaluation episodes

* Add benchmark scoring

* Tweak reward locations

* Tweak scoring

* Clear default metadata in Wrapper

* Improve scoring

* Expose registry; fix test

* Add initial_reset_timestamp

* Add back algorithm; fix tests
2016-09-23 01:04:26 -07:00
Greg Brockman
bdba063624 Warn if seed doesn't return a list 2016-09-09 10:13:41 -07:00
Trevor Blackwell
17bde31080 Merge branch 'master' into trace_recording 2016-09-07 19:41:46 -07:00
Trevor Blackwell
0f02e3a817 Change trace recording from a monitor to a wrapper. 2016-09-07 19:38:24 -07:00
Greg Brockman
9b8472726c Check for hasattr _monitor before invoking monitor 2016-09-04 19:41:11 -07:00
Trevor Blackwell
9b882e7c7b Recording traces
With monitor is started with `.start(…save_trace=True)`, record
complete action/observation/reward tuples for any environment. Does it
efficiently in picked numpy arrays in the same directly as the other
monitoring files.
2016-09-02 11:22:37 -07:00
Jesse Cooper
932ed300f4 Fix error when running min on empty list (#314)
* Fix error when running min on empty list

Running min on an empty list (which occurs on first episode from `if len(content['timestamps'])==0: continue`) causes it to throw an error

* Update monitor.py

Return 0 for initial_reset_timestamp when no initial_reset_timestamps
2016-08-23 22:55:42 -07:00
Greg Brockman
fd051e0895 Drop 'ending episode' message to debug 2016-08-18 20:02:04 -07:00
John Schulman
75b6d9993f minor fix 2016-08-18 09:26:18 -07:00
Greg Brockman
b3ef84839e Add training directory to logging 2016-08-17 14:00:53 -07:00
John Schulman
c6a3f66537 log error in load_results 2016-08-17 12:51:29 -07:00
Greg Brockman
8a9bcdf061 Don't write stats file unless requested either 2016-08-17 11:59:29 -07:00
Greg Brockman
922fc56d2f Improve performance for short episodes by only writing upon reset when requested 2016-08-17 11:26:55 -07:00
Greg Brockman
36b2a58b3f Fix monitored env memory leak (#277) 2016-08-06 14:53:52 -07:00
Greg Brockman
d257dac86e Go back to lazily creating the monitor
The refresh case is complicated enough that special handling is needed anyway
2016-06-09 11:29:34 -07:00
Greg Brockman
e759526268 Handle monitor close better 2016-06-09 11:27:27 -07:00
Greg Brockman
3328144465 Support disabling videos by passing video_callable=False (#140)
* Support disabling videos by passing video_callable=False

* monitor.py: Flush after closing video recorder

This ensures that the last video recorded ends up
in the manifest
2016-05-29 13:56:38 -07:00
Greg Brockman
58e6aa95e5 [WIP] add support for seeding environments (#135)
* Make environments seedable

* Fix monitor bugs

- Set monitor_id before setting the infix. This was a bug that would yield incorrect results with multiple monitors.
- Remove extra pid from stats recorder filename. This should be purely cosmetic.

* Start uploading seeds in episode_batch

* Fix _bigint_from_bytes for python3

* Set seed explicitly in random_agent

* Pass through seed argument

* Also pass through random state to spaces

* Pass random state into the observation/action spaces

* Make all _seed methods return the list of used seeds

* Switch over to np.random where possible

* Start hashing seeds, and also seed doom engine

* Fixup seeding determinism in many cases

* Seed before loading the ROM

* Make seeding more Python3 friendly

* Make the MuJoCo skipping a bit more forgiving

* Remove debugging PDB calls

* Make setInt argument into raw bytes

* Validate and upload seeds

* Skip box2d

* Make seeds smaller, and change representation of seeds in upload

* Handle long seeds

* Fix RandomAgent example to be deterministic

* Handle integer types correctly in Python2 and Python3

* Try caching pip

* Try adding swap

* Add df and free calls

* Bump swap

* Bump swap size

* Try setting overcommit

* Try other sysctls

* Try fixing overcommit

* Try just setting overcommit_memory=1

* Add explanatory comment

* Add what's new section to readme

* BUG: Mark ElevatorAction-ram-v0 as non-deterministic for now

* Document seed

* Move nondetermistic check into spec
2016-05-29 09:07:09 -07:00
Greg Brockman
56ac98a841 Update name in _open_monitors 2016-05-27 20:00:57 -07:00
Greg Brockman
8376580c85 Improve closer implementation and docstrings (#126)
* Improve auto close implementation

- Register all envs at initialization time, not just ones created via make
- Simplify names and add more documentation on interface
- Move closer instances into the relevant modules

review-requested: @jietang

* Close environments in the tests

This isn't strictly needed, but means there are fewer Doom
subprocesses hanging around while the tests run.

* Use 4 space indent in comment

* Improve docstrings in core

* Don't pass through args to __new__

The __init__ method gets called once __new__ returns,
so these arguments are either ignored (Python 2) or
result in an error (Python 3). The __init__ method
automatically gets called with the correct arguments.

* Fixup comments
2016-05-27 12:16:35 -07:00
Jie Tang
b92796ae89 Renaming close_registry to closer 2016-05-18 01:27:58 -07:00
Jie Tang
b578a22bef Refactor monitor to use CloseRegistry 2016-05-16 23:45:56 -07:00
Greg Brockman
064f7377d9 Allow environment to create a new monitor after closing 2016-05-15 16:43:53 -07:00
Trevor Blackwell
d845b79a07 Allow resuming monitoring 2016-05-10 17:25:23 -07:00
Greg Brockman
8781f6d623 Fix flushing of final episode in monitor
Thanks @JKCooper2.

We also change to getting the PID at monitor construction time, rather
than at runtime. This means you need to start your environments
post-fork, which should be fine.
2016-05-06 22:00:29 -07:00
Greg Brockman
21169ef620 Add TODO 2016-05-06 18:23:08 -07:00
Greg Brockman
9984589731 Improve score_from_local implementation (#66)
* Make sure that a callable is passed to start

* Improve autoflushing for score calculation

* Write stats and manifests using proper atomic_writes
2016-05-06 18:19:16 -07:00
JKCooper2
95a0151f13 Issue 45 - Easily return environment score values (#57)
* Set restriction on selected actions

* Used self.action_space instead of custom set

* Move action validation to core.py

* Added env.score() function to return current score

* Set up multi monitor scoring

* Removed unneccesary package inclusions

* Monitor writes scores automatically. Video callable can now use monitor_id

* Monitor ID not necessary for video filtering

* Fix so final stats get written when env.reset() doesn't happen

* gitignore PyCharm Project Files
2016-05-06 17:38:42 -07:00
Jonas Schneider
d22f96525f Safely iterate over monitor weakrefs when closing 2016-05-01 23:17:47 -04:00
Dominik Mueller
d3d23f3455 Fix readme example, documentation and error messages 2016-04-28 14:01:24 +02:00
Greg Brockman
b742eaca13 Fix warning 2016-04-27 19:48:34 -07:00
Greg Brockman
dbde599c17 monitor.py: Still write manifest if close errors
Before, on a headless server I was losing manifests with:

NullFunctionError: Attempt to call an undefined function glIsFramebuffer, check for bool(glIsFramebuffer) before calling (when closing Ant-v0)
2016-04-27 19:35:33 -07:00
Greg Brockman
bd36a91603 Record initial reset timestamp, and use it for duration calculations 2016-04-27 09:17:05 -07:00
Greg Brockman
e8f2980603 Initial release. Hello world :). 2016-04-27 08:00:58 -07:00