At Hugging Face, we’re contributing to the ecosystem for Deep Reinforcement Learning researchers and enthusiasts. That’s why we’re completely satisfied to announce that we integrated Stable-Baselines3 to the Hugging Face Hub.
Stable-Baselines3 is probably the most popular PyTorch Deep Reinforcement Learning library that makes it easy to coach and test your agents in quite a lot of environments (Gym, Atari, MuJoco, Procgen…).
With this integration, you may now host your saved models 💾 and cargo powerful models from the community.
In this text, we’re going to indicate how you may do it.
Installation
To make use of stable-baselines3 with Hugging Face Hub, you simply need to put in these 2 libraries:
pip install huggingface_hub
pip install huggingface_sb3
Finding Models
We’re currently uploading saved models of agents playing Space Invaders, Breakout, LunarLander and more. On top of this, you will discover all stable-baselines-3 models from the community here
Whenever you found the model you would like, you simply need to copy the repository id:
Download a model from the Hub
The good feature of this integration is that you may now very easily load a saved model from Hub to Stable-baselines3.
With a purpose to do that you just just must copy the repo-id that accommodates your saved model and the name of the saved model zip file within the repo.
As an illustrationsb3/demo-hf-CartPole-v1:
import gym
from huggingface_sb3 import load_from_hub
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy
checkpoint = load_from_hub(
repo_id="sb3/demo-hf-CartPole-v1",
filename="ppo-CartPole-v1.zip",
)
model = PPO.load(checkpoint)
eval_env = gym.make("CartPole-v1")
mean_reward, std_reward = evaluate_policy(
model, eval_env, render=True, n_eval_episodes=5, deterministic=True, warn=False
)
print(f"mean_reward={mean_reward:.2f} +/- {std_reward}")
Sharing a model to the Hub
In only a minute, you may get your saved model within the Hub.
First, it’s good to be logged in to Hugging Face to upload a model:
- Should you’re using Colab/Jupyter Notebooks:
from huggingface_hub import notebook_login
notebook_login()
huggingface-cli login
Then, in this instance, we train a PPO agent to play CartPole-v1 and push it to a brand new repo ThomasSimonini/demo-hf-CartPole-v1
`
from huggingface_sb3 import push_to_hub
from stable_baselines3 import PPO
model = PPO("MlpPolicy", "CartPole-v1", verbose=1)
model.learn(total_timesteps=10_000)
model.save("ppo-CartPole-v1")
push_to_hub(
repo_id="ThomasSimonini/demo-hf-CartPole-v1",
filename="ppo-CartPole-v1.zip",
commit_message="Added Cartpole-v1 model trained with PPO",
)
Try it out and share your models with the community!
What’s next?
In the approaching weeks and months, we shall be extending the ecosystem by:
- Integrating RL-baselines3-zoo
- Uploading RL-trained-agents models into the Hub: a giant collection of pre-trained Reinforcement Learning agents using stable-baselines3
- Integrating other Deep Reinforcement Learning libraries
- Implementing Decision Transformers 🔥
- And more to return 🥳
One of the best solution to communicate is to join our discord server to exchange with us and with the community.
And if you ought to dive deeper, we wrote a tutorial where you’ll learn:
- How you can train a Deep Reinforcement Learning lander agent to land accurately on the Moon 🌕
- How you can upload it to the Hub 🚀
- How you can download and use a saved model from the Hub that plays Space Invaders 👾.
Conclusion
We’re excited to see what you are working on with Stable-baselines3 and check out your models within the Hub 😍.
And we’d love to listen to your feedback 💖. 📧 Be happy to reach us.
Finally, we would really like to thank the SB3 team and specifically Antonin Raffin for his or her precious help for the combination of the library 🤗.
Would you wish to integrate your library to the Hub?
This integration is feasible because of the huggingface_hub library which has all our widgets and the API for all our supported libraries. Should you would really like to integrate your library to the Hub, now we have a guide for you!
