[](https://travis-ci.org/openai/baselines) # Baselines OpenAI Baselines is a set of high-quality implementations of reinforcement learning algorithms. These algorithms will make it easier for the research community to replicate, refine, and identify new ideas, and will create good baselines to build research on top of. Our DQN implementation and its variants are roughly on par with the scores in published papers. We expect they will be used as a base around which new ideas can be added, and as a tool for comparing a new approach against existing ones. ## Prerequisites Baselines requires python3 (>=3.5) with the development headers. You'll also need system packages CMake, OpenMPI and zlib. Those can be installed as follows ### Ubuntu ```bash sudo apt-get update && sudo apt-get install cmake libopenmpi-dev python3-dev zlib1g-dev ``` ### Mac OS X Installation of system packages on Mac requires [Homebrew](https://brew.sh). With Homebrew installed, run the follwing: ```bash brew install cmake openmpi ``` ## Virtual environment From the general python package sanity perspective, it is a good idea to use virtual environments (virtualenvs) to make sure packages from different projects do not interfere with each other. You can install virtualenv (which is itself a pip package) via ```bash pip install virtualenv ``` Virtualenvs are essentially folders that have copies of python executable and all python packages. To create a virtualenv called venv with python3, one runs ```bash virtualenv /path/to/venv --python=python3 ``` To activate a virtualenv: ``` . /path/to/venv/bin/activate ``` More thorough tutorial on virtualenvs and options can be found [here](https://virtualenv.pypa.io/en/stable/) ## Installation Clone the repo and cd into it: ```bash git clone https://github.com/openai/baselines.git cd baselines ``` If using virtualenv, create a new virtualenv and activate it ```bash virtualenv env --python=python3 . env/bin/activate ``` Install baselines package ```bash pip install -e . ``` ### MuJoCo Some of the baselines examples use [MuJoCo](http://www.mujoco.org) (multi-joint dynamics in contact) physics simulator, which is proprietary and requires binaries and a license (temporary 30-day license can be obtained from [www.mujoco.org](http://www.mujoco.org)). Instructions on setting up MuJoCo can be found [here](https://github.com/openai/mujoco-py) ## Testing the installation All unit tests in baselines can be run using pytest runner: ``` pip install pytest pytest ``` ## Subpackages - [A2C](baselines/a2c) - [ACER](baselines/acer) - [ACKTR](baselines/acktr) - [DDPG](baselines/ddpg) - [DQN](baselines/deepq) - [GAIL](baselines/gail) - [HER](baselines/her) - [PPO1](baselines/ppo1) (Multi-CPU using MPI) - [PPO2](baselines/ppo2) (Optimized for GPU) - [TRPO](baselines/trpo_mpi) To cite this repository in publications: @misc{baselines, author = {Dhariwal, Prafulla and Hesse, Christopher and Klimov, Oleg and Nichol, Alex and Plappert, Matthias and Radford, Alec and Schulman, John and Sidor, Szymon and Wu, Yuhuai}, title = {OpenAI Baselines}, year = {2017}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/openai/baselines}}, }