Jie Tang
a8e4734cbc
Fix evaluation scoring bug (numpy casts generator to a single-element array containing the generator, which is a truthy object)
2016-10-20 19:39:20 -07:00
Greg Brockman
934b2acbb7
Add benchmark support ( #338 )
...
* Warn if seed doesn't return a list
* Add preliminary BenchmarkRun support
* Add experimental benchmark registration
* Flesh out interface
* Add preliminary BenchmarkRun support
* Warn if seed doesn't return a list
* Add experimental benchmark registration
* Flesh out interface
* Make benchmarkrun upload recursive
* Add evaluation episodes
* Add benchmark scoring
* Tweak reward locations
* Tweak scoring
* Clear default metadata in Wrapper
* Improve scoring
* Expose registry; fix test
* Add initial_reset_timestamp
* Add back algorithm; fix tests
2016-09-23 01:04:26 -07:00
Greg Brockman
a7e5a581dc
scoring.py: Avoid NaN when computing standard error
2016-06-17 18:45:19 -07:00
JKCooper2
7c5b962110
scoring.compute_graph_stats Default return and fix import statement ( #90 )
2016-05-11 08:38:08 -07:00
Greg Brockman
9984589731
Improve score_from_local implementation ( #66 )
...
* Make sure that a callable is passed to start
* Improve autoflushing for score calculation
* Write stats and manifests using proper atomic_writes
2016-05-06 18:19:16 -07:00
JKCooper2
95a0151f13
Issue 45 - Easily return environment score values ( #57 )
...
* Set restriction on selected actions
* Used self.action_space instead of custom set
* Move action validation to core.py
* Added env.score() function to return current score
* Set up multi monitor scoring
* Removed unneccesary package inclusions
* Monitor writes scores automatically. Video callable can now use monitor_id
* Monitor ID not necessary for video filtering
* Fix so final stats get written when env.reset() doesn't happen
* gitignore PyCharm Project Files
2016-05-06 17:38:42 -07:00
Greg Brockman
a5d71bb554
scoring.py: Make the success threshold non-strict
2016-04-28 07:17:16 -07:00
Greg Brockman
0b34c296b1
scoring.py: Change the graph wallclock time format
2016-04-27 22:52:23 -07:00
Greg Brockman
8302c188de
scoring.py: Correct stats key
2016-04-27 22:46:01 -07:00
Greg Brockman
c02a213f8e
scoring.py: Handle entries without initial_reset_timestamp
2016-04-27 09:32:31 -07:00
Greg Brockman
bd36a91603
Record initial reset timestamp, and use it for duration calculations
2016-04-27 09:17:05 -07:00
Greg Brockman
6ff2b059fe
scoring.py: Include more timestamp metrics
2016-04-27 09:10:19 -07:00
Greg Brockman
e8f2980603
Initial release. Hello world :).
2016-04-27 08:00:58 -07:00
