[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-9725":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":25,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":43,"readmeContent":44,"aiSummary":45,"trendingCount":16,"starSnapshotCount":16,"syncStatus":46,"lastSyncTime":47,"discoverSource":48},9725,"cleanrl","vwxyzjn\u002Fcleanrl","vwxyzjn","High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)","http:\u002F\u002Fdocs.cleanrl.dev",null,"Python",9942,1099,47,67,0,4,41,223,29,103.12,"Other",false,"master",true,[27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"a2c","actor-critic","advantage-actor-critic","ale","atari","deep-learning","deep-reinforcement-learning","gym","machine-learning","phasic-policy-gradient","ppo","proximal-policy-optimization","python","pytorch","reinforcement-learning","wandb","2026-06-12 04:00:46","# CleanRL (Clean Implementation of RL Algorithms)\n\n\n[\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-blue\">](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl)\n[![tests](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Factions\u002Fworkflows\u002Ftests.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Factions\u002Fworkflows\u002Ftests.yaml)\n[![docs](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fdeployments\u002Fvwxyzjn\u002Fcleanrl\u002FProduction?label=docs&logo=vercel)](https:\u002F\u002Fdocs.cleanrl.dev\u002F)\n[\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F767863440248143916?label=discord\">](https:\u002F\u002Fdiscord.gg\u002FD6RCjA6sVT)\n[\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fyoutube\u002Fchannel\u002Fviews\u002FUCDdC6BIFRI0jvcwuhi3aI6w?style=social\">](https:\u002F\u002Fwww.youtube.com\u002Fchannel\u002FUCDdC6BIFRI0jvcwuhi3aI6w\u002Fvideos)\n[![Code style: black](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcode%20style-black-000000.svg)](https:\u002F\u002Fgithub.com\u002Fpsf\u002Fblack)\n[![Imports: isort](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https:\u002F\u002Fpycqa.github.io\u002Fisort\u002F)\n[\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Models-Huggingface-F8D521\">](https:\u002F\u002Fhuggingface.co\u002Fcleanrl)\n[![Open In Colab](https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg)](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fdocs\u002Fget-started\u002FCleanRL_Huggingface_Integration_Demo.ipynb)\n\n\nCleanRL is a Deep Reinforcement Learning library that provides high-quality single-file implementation with research-friendly features. The implementation is clean and simple, yet we can scale it to run thousands of experiments using AWS Batch. The highlight features of CleanRL are:\n\n\n\n* 📜 Single-file implementation\n   * *Every detail about an algorithm variant is put into a single standalone file.* \n   * For example, our `ppo_atari.py` only has 340 lines of code but contains all implementation details on how PPO works with Atari games, **so it is a great reference implementation to read for folks who do not wish to read an entire modular library**.\n* 📊 Benchmarked Implementation (7+ algorithms and 34+ games at https:\u002F\u002Fbenchmark.cleanrl.dev)\n* 📈 Tensorboard Logging\n* 🪛 Local Reproducibility via Seeding\n* 🎮 Videos of Gameplay Capturing\n* 🧫 Experiment Management with [Weights and Biases](https:\u002F\u002Fwandb.ai\u002Fsite)\n* 💸 Cloud Integration with docker and AWS \n\nYou can read more about CleanRL in our [JMLR paper](https:\u002F\u002Fwww.jmlr.org\u002Fpapers\u002Fvolume23\u002F21-1342\u002F21-1342.pdf) and [documentation](https:\u002F\u002Fdocs.cleanrl.dev\u002F).\n\nNotable CleanRL-related projects:\n\n* [corl-team\u002FCORL](https:\u002F\u002Fgithub.com\u002Fcorl-team\u002FCORL): Offline RL algorithm implemented in CleanRL style\n* [pytorch-labs\u002FLeanRL](https:\u002F\u002Fgithub.com\u002Fpytorch-labs\u002FLeanRL): Fast optimized PyTorch implementation of CleanRL RL algorithms using CUDAGraphs.\n\n\n> ℹ️ **Support for Gymnasium**: [Farama-Foundation\u002FGymnasium](https:\u002F\u002Fgithub.com\u002FFarama-Foundation\u002FGymnasium) is the next generation of [`openai\u002Fgym`](https:\u002F\u002Fgithub.com\u002Fopenai\u002Fgym) that will continue to be maintained and introduce new features. Please see their [announcement](https:\u002F\u002Ffarama.org\u002FAnnouncing-The-Farama-Foundation) for further detail. We are migrating to `gymnasium` and the progress can be tracked in [vwxyzjn\u002Fcleanrl#277](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fpull\u002F277). \n\n\n> ⚠️ **NOTE**: CleanRL is *not* a modular library and therefore it is not meant to be imported. At the cost of duplicate code, we make all implementation details of a DRL algorithm variant easy to understand, so CleanRL comes with its own pros and cons. You should consider using CleanRL if you want to 1) understand all implementation details of an algorithm's variant or 2) prototype advanced features that other modular DRL libraries do not support (CleanRL has minimal lines of code so it gives you great debugging experience and you don't have do a lot of subclassing like sometimes in modular DRL libraries).\n\n## Get started\n\nPrerequisites:\n* Python >=3.7.1,\u003C3.11\n* [uv 0.7.9+](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F)\n\nTo run experiments locally, give the following a try:\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl.git && cd cleanrl\nuv pip install .\n\n# alternatively, you could use `uv venv` and do\n# `python run cleanrl\u002Fppo.py`\nuv run python cleanrl\u002Fppo.py \\\n    --seed 1 \\\n    --env-id CartPole-v0 \\\n    --total-timesteps 50000\n\n# open another terminal and enter `cd cleanrl\u002Fcleanrl`\ntensorboard --logdir runs\n```\n\nTo use experiment tracking with wandb, run\n```bash\nwandb login # only required for the first time\nuv run python cleanrl\u002Fppo.py \\\n    --seed 1 \\\n    --env-id CartPole-v0 \\\n    --total-timesteps 50000 \\\n    --track \\\n    --wandb-project-name cleanrltest\n```\n\nIf you are not using `uv`, you can install CleanRL with `requirements.txt`:\n\n```bash\n# core dependencies\npip install -r requirements\u002Frequirements.txt\n\n# optional dependencies\npip install -r requirements\u002Frequirements-atari.txt\npip install -r requirements\u002Frequirements-mujoco.txt\npip install -r requirements\u002Frequirements-mujoco_py.txt\npip install -r requirements\u002Frequirements-procgen.txt\npip install -r requirements\u002Frequirements-envpool.txt\npip install -r requirements\u002Frequirements-pettingzoo.txt\npip install -r requirements\u002Frequirements-jax.txt\npip install -r requirements\u002Frequirements-docs.txt\npip install -r requirements\u002Frequirements-cloud.txt\npip install -r requirements\u002Frequirements-memory_gym.txt\n```\n\nTo run training scripts in other games:\n```\nuv venv\n\n# classic control\npython cleanrl\u002Fdqn.py --env-id CartPole-v1\npython cleanrl\u002Fppo.py --env-id CartPole-v1\npython cleanrl\u002Fc51.py --env-id CartPole-v1\n\n# atari\nuv pip install \".[atari]\"\npython cleanrl\u002Fdqn_atari.py --env-id BreakoutNoFrameskip-v4\npython cleanrl\u002Fc51_atari.py --env-id BreakoutNoFrameskip-v4\npython cleanrl\u002Fppo_atari.py --env-id BreakoutNoFrameskip-v4\npython cleanrl\u002Fsac_atari.py --env-id BreakoutNoFrameskip-v4\n\n# NEW: 3-4x side-effects free speed up with envpool's atari (only available to linux)\nuv pip install \".[envpool]\"\npython cleanrl\u002Fppo_atari_envpool.py --env-id BreakoutNoFrameskip-v4\n# Learn Pong-v5 in ~5-10 mins\n# Side effects such as lower sample efficiency might occur\nuv run python ppo_atari_envpool.py --clip-coef=0.2 --num-envs=16 --num-minibatches=8 --num-steps=128 --update-epochs=3\n\n# procgen\nuv pip install \".[procgen]\"\npython cleanrl\u002Fppo_procgen.py --env-id starpilot\npython cleanrl\u002Fppg_procgen.py --env-id starpilot\n\n# ppo + lstm\nuv pip install \".[atari]\"\npython cleanrl\u002Fppo_atari_lstm.py --env-id BreakoutNoFrameskip-v4\n```\n\nYou may also use a prebuilt development environment hosted in Gitpod:\n\n[![Open in Gitpod](https:\u002F\u002Fgitpod.io\u002Fbutton\u002Fopen-in-gitpod.svg)](https:\u002F\u002Fgitpod.io\u002F#https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl)\n\n## Algorithms Implemented\n\n\n| Algorithm      | Variants Implemented |\n| ----------- | ----------- |\n| ✅ [Proximal Policy Optimization (PPO)](https:\u002F\u002Farxiv.org\u002Fpdf\u002F1707.06347.pdf)  |  [`ppo.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo.py),   [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppopy) |\n| |  [`ppo_atari.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_atari.py),   [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_ataripy)\n| |  [`ppo_continuous_action.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_continuous_action.py),   [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_continuous_actionpy)\n| |  [`ppo_atari_lstm.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_atari_lstm.py),   [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_atari_lstmpy)\n| |  [`ppo_atari_envpool.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_atari_envpool.py),   [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_atari_envpoolpy)\n| | [`ppo_atari_envpool_xla_jax.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_atari_envpool_xla_jax.py), [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_atari_envpool_xla_jaxpy)\n| | [`ppo_atari_envpool_xla_jax_scan.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_atari_envpool_xla_jax_scan.py), [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_atari_envpool_xla_jax_scanpy))\n| |  [`ppo_procgen.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_procgen.py),   [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_procgenpy)\n| |  [`ppo_atari_multigpu.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_atari_multigpu.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_atari_multigpupy)\n| | [`ppo_pettingzoo_ma_atari.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_pettingzoo_ma_atari.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_pettingzoo_ma_ataripy)\n| | [`ppo_continuous_action_isaacgym.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_continuous_action_isaacgym\u002Fppo_continuous_action_isaacgym.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo\u002F#ppo_continuous_action_isaacgympy)\n| | [`ppo_trxl.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_trxl\u002Fppo_trxl.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppo-trxl\u002F)\n| ✅ [Deep Q-Learning (DQN)](https:\u002F\u002Fweb.stanford.edu\u002Fclass\u002Fpsych209\u002FReadings\u002FMnihEtAlHassibis15NatureControlDeepRL.pdf) |  [`dqn.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fdqn.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fdqn\u002F#dqnpy) |\n| | [`dqn_atari.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fdqn_atari.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fdqn\u002F#dqn_ataripy) |\n| | [`dqn_jax.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fdqn_jax.py), [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fdqn\u002F#dqn_jaxpy) |\n| | [`dqn_atari_jax.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fdqn_atari_jax.py), [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fdqn\u002F#dqn_atari_jaxpy) |\n| ✅ [Categorical DQN (C51)](https:\u002F\u002Farxiv.org\u002Fpdf\u002F1707.06887.pdf) |  [`c51.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fc51.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fc51\u002F#c51py) |\n| |  [`c51_atari.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fc51_atari.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fc51\u002F#c51_ataripy) |\n| | [`c51_jax.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fc51_jax.py), [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fc51\u002F#c51_jaxpy) |\n| | [`c51_atari_jax.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fc51_atari_jax.py), [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fc51\u002F#c51_atari_jaxpy) |\n| ✅ [Soft Actor-Critic (SAC)](https:\u002F\u002Farxiv.org\u002Fpdf\u002F1812.05905.pdf) |  [`sac_continuous_action.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fsac_continuous_action.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fsac\u002F#sac_continuous_actionpy) |\n| |  [`sac_atari.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fsac_atari.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fsac\u002F#sac_atarinpy) |\n| ✅ [Deep Deterministic Policy Gradient (DDPG)](https:\u002F\u002Farxiv.org\u002Fpdf\u002F1509.02971.pdf) |  [`ddpg_continuous_action.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fddpg_continuous_action.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fddpg\u002F#ddpg_continuous_actionpy) |\n| | [`ddpg_continuous_action_jax.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fddpg_continuous_action_jax.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fddpg\u002F#ddpg_continuous_action_jaxpy)\n| ✅ [Twin Delayed Deep Deterministic Policy Gradient (TD3)](https:\u002F\u002Farxiv.org\u002Fpdf\u002F1802.09477.pdf) |  [`td3_continuous_action.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Ftd3_continuous_action.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Ftd3\u002F#td3_continuous_actionpy) |\n|  | [`td3_continuous_action_jax.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Ftd3_continuous_action_jax.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Ftd3\u002F#td3_continuous_action_jaxpy) |\n| ✅ [Phasic Policy Gradient (PPG)](https:\u002F\u002Farxiv.org\u002Fabs\u002F2009.04416) |  [`ppg_procgen.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppg_procgen.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fppg\u002F#ppg_procgenpy) |\n| ✅ [Random Network Distillation (RND)](https:\u002F\u002Farxiv.org\u002Fabs\u002F1810.12894) |  [`ppo_rnd_envpool.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fppo_rnd_envpool.py),  [docs](\u002Frl-algorithms\u002Fppo-rnd\u002F#ppo_rnd_envpoolpy) |\n| ✅ [Qdagger](https:\u002F\u002Farxiv.org\u002Fabs\u002F2206.01626)  |  [`qdagger_dqn_atari_impalacnn.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fqdagger_dqn_atari_impalacnn.py),  [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fqdagger\u002F#qdagger_dqn_atari_impalacnnpy) |\n|  | [`qdagger_dqn_atari_jax_impalacnn.py`](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fblob\u002Fmaster\u002Fcleanrl\u002Fqdagger_dqn_atari_jax_impalacnn.py), [docs](https:\u002F\u002Fdocs.cleanrl.dev\u002Frl-algorithms\u002Fqdagger\u002F#qdagger_dqn_atari_jax_impalacnnpy) |\n\n\n## Open RL Benchmark\n\nTo make our experimental data transparent, CleanRL participates in a related project called [Open RL Benchmark](https:\u002F\u002Fgithub.com\u002Fopenrlbenchmark\u002Fopenrlbenchmark), which contains tracked experiments from popular DRL libraries such as ours, [Stable-baselines3](https:\u002F\u002Fgithub.com\u002FDLR-RM\u002Fstable-baselines3), [openai\u002Fbaselines](https:\u002F\u002Fgithub.com\u002Fopenai\u002Fbaselines), [jaxrl](https:\u002F\u002Fgithub.com\u002Fikostrikov\u002Fjaxrl), and others. \n\nCheck out https:\u002F\u002Fbenchmark.cleanrl.dev\u002F for a collection of Weights and Biases reports showcasing tracked DRL experiments. The reports are interactive, and researchers can easily query information such as GPU utilization and videos of an agent's gameplay that are normally hard to acquire in other RL benchmarks. In the future, Open RL Benchmark will likely provide an dataset API for researchers to easily access the data (see [repo](https:\u002F\u002Fgithub.com\u002Fopenrlbenchmark\u002Fopenrlbenchmark)).\n\n![](docs\u002Fstatic\u002Fo1.png)\n![](docs\u002Fstatic\u002Fo2.png)\n![](docs\u002Fstatic\u002Fo3.png)\n\n\n## Support and get involved\n\nWe have a [Discord Community](https:\u002F\u002Fdiscord.gg\u002FD6RCjA6sVT) for support. Feel free to ask questions. Posting in [Github Issues](https:\u002F\u002Fgithub.com\u002Fvwxyzjn\u002Fcleanrl\u002Fissues) and PRs are also welcome. Also our past video recordings are available at [YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=dm4HdGujpPs&list=PLQpKd36nzSuMynZLU2soIpNSMeXMplnKP&index=2)\n\n## Citing CleanRL\n\nIf you use CleanRL in your work, please cite our technical [paper](https:\u002F\u002Fwww.jmlr.org\u002Fpapers\u002Fv23\u002F21-1342.html):\n\n```bibtex\n@article{huang2022cleanrl,\n  author  = {Shengyi Huang and Rousslan Fernand Julien Dossa and Chang Ye and Jeff Braga and Dipam Chakraborty and Kinal Mehta and João G.M. Araújo},\n  title   = {CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms},\n  journal = {Journal of Machine Learning Research},\n  year    = {2022},\n  volume  = {23},\n  number  = {274},\n  pages   = {1--18},\n  url     = {http:\u002F\u002Fjmlr.org\u002Fpapers\u002Fv23\u002F21-1342.html}\n}\n```\n\n\n## Acknowledgement\n\nCleanRL is a community-powered by project and our contributors run experiments on a variety of hardware.\n\n* We thank many contributors for using their own computers to run experiments\n* We thank Google's [TPU research cloud](https:\u002F\u002Fsites.research.google\u002Ftrc\u002Fabout\u002F) for providing TPU resources.\n* We thank [Hugging Face](https:\u002F\u002Fhuggingface.co\u002F)'s cluster for providing GPU resources. \n","CleanRL 是一个深度强化学习库，提供高质量的单文件实现，并具备研究友好的特性。其核心功能包括单文件算法实现、基准测试、Tensorboard 日志记录、本地可重复性通过种子设置、游戏过程视频捕获以及使用 Weights and Biases 进行实验管理等。技术上采用 Python 和 PyTorch 开发，支持与 AWS 等云服务集成，适用于需要快速原型设计和大规模实验的研究场景或教育用途。",2,"2026-06-11 03:24:24","top_topic"]