We’re pleased to announce that we’re partnering with Wiz with the goal of improving security across our platform and the AI/ML ecosystem at large.
Wiz researchers collaborated with Hugging Face on the safety of our platform and shared their findings. Wiz is a cloud security company that helps their customers construct and maintain software in a secure manner. Together with the publication of this research, we’re taking the chance to spotlight some related Hugging Face security improvements.
Hugging Face has recently integrated Wiz for Vulnerability Management, a continuous and proactive process to maintain our platform freed from security vulnerabilities. As well as, we’re using Wiz for Cloud Security Posture Management (CSPM), which allows us to configure our cloud environment securely, and monitor to make sure it stays secure.
Certainly one of our favourite Wiz features is a holistic view of Vulnerabilities, from storage to compute to network. We run multiple Kubernetes (k8s) clusters and have resources across multiple regions and cloud providers, so it is amazingly helpful to have a central report in a single location with the complete context graph for every vulnerability. We’ve also built on top of their tooling, to routinely remediate detected issues in our products, most notably in Spaces.
As a part of the joint work, Wiz’s security research team identified shortcomings of our sandboxed compute environments by running arbitrary code throughout the system because of pickle. As you read this blog and the Wiz security research paper, it’s important to keep in mind that we now have resolved all issues related to the exploit and proceed to stay diligent in our Threat Detection and Incident Response process.
Hugging Face Security
At Hugging Face we take security seriously, as AI rapidly evolves, latest threat vectors seemingly pop up on daily basis. At the same time as Hugging Face declares multiple partnerships and business relationships with the most important names in tech, we remain committed to permit our users and the AI community to responsibly experiment with and operationalize AI/ML systems and technologies. We’re dedicated to securing our platform in addition to democratizing AI/ML, such that the community can contribute to and be an element of this paradigm shifting event that may impact us all. We’re writing this blog to reaffirm our commitment to protecting our users and customers from security threats. Below we may even discuss Hugging Face’s philosophy regarding our support of the controversial pickle files in addition to discuss the shared responsibility of moving away from the pickle format.
There are a lot of other exciting security improvements and announcements coming within the near future. The publications is not going to only discuss the safety risks to the Hugging Face platform community, but in addition cover systemic security risks of AI in addition to best practices for mitigation. We remain committed to creating our products, our infrastructure, and the AI community secure, stay tuned for followup security blog posts and whitepapers.
Open Source Security Collaboration and Tools for the Community
We highly value transparency and collaboration with the community and this includes participation within the identification and disclosure of vulnerabilities, collaborating on resolving security issues, and security tooling. Below are examples of our security wins born from collaboration, which help all the AI community lower their security risk:
- Picklescan was in-built partnership with Microsoft; Matthieu Maitre began the project and given we had our own internal version of the identical tool, we joined forces and contributed to picklescan. Check with the next documentation page should you are curious to know more on how it really works:
https://huggingface.co/docs/hub/en/security-pickle - Safetensors, which was developed by Nicolas Patry, is a secure alternative to pickle files. Safetensors has been audited by Trail of Bits on a collaborative initiative with EuletherAI & Stability AI.
https://huggingface.co/docs/safetensors/en/index - Now we have a strong bug bounty program, with many amazing researchers from all around the globe. Researchers who’ve identified a security vuln may inquire about joining our program through security@huggingface.co
- Malware Scanning: https://huggingface.co/docs/hub/en/security-malware
- Secrets Scanning: https://huggingface.co/docs/hub/security-secrets
- As previously mentioned, we’re also collaborating with Wiz to lower Platform security risks
- We’re starting a series of security publications which address security issues facing the AI/ML community.
Security Best Practices for Open Source AI/ML users
AI/ML has introduced latest vectors of attack, but for lots of these attacks mitigants are long standing and well-known. Security professionals should be sure that they apply relevant security controls to AI resources and models. As well as, below are some resources and best practices when working with open source software and models:
Pickle Files – The Insecure Elephant within the Room
Pickle files have been on the core of a lot of the research done by Wiz and other recent publications by security researchers about Hugging Face. Pickle files have long been considered to have security risks related to them, see our doc files for more information: https://huggingface.co/docs/hub/en/security-pickle
Despite these known security flaws, the AI/ML community still continuously uses pickles (or similarly trivially exploitable formats). A lot of these use cases are low risk or for test purposes making the familiarity and ease of use of pickle files more attractive than the secure alternative.
Because the open source AI platform, we’re left with the next options:
- Ban pickle files entirely
- Do nothing about pickle files
- Finding a middle ground that each allows for pickle use in addition to reasonably and practicably mitigating the risks related to pickle files
Now we have chosen option 3, the center ground for now. This selection is a burden on our engineering and security teams and we now have put in significant effort to mitigate the risks while allowing the AI community to make use of tools they select. A few of the key mitigants we now have implemented to the risks related to pickle include:
- Creating clear documentation outlining the risks
- Developing automated scanning tools
- Using scanning tools and labeling models with security vulnerabilities with clear warnings
- Now we have even provided a secure solution to make use of in lieu of pickle (Safetensors)
- Now we have also made Safetensors a first-class citizen on our platform to guard the community members who may not understand the risks
- Along with the above, we now have also needed to significantly segment and enhance security of the areas through which models are used to account for potential vulnerabilities inside them
We intend to proceed to be the leader in protecting and securing the AI Community. A part of this can be monitoring and addressing risks related to pickle files. Sunsetting support of pickle can be not out of the query either, nonetheless, we do our greatest to balance the impact on the community as a part of a choice like this.
A vital note that the upstream open source communities in addition to large tech and security firms, have been largely silent on contributing to solutions here and left Hugging Face to each define philosophy and invest heavily in developing and implementing mitigating controls to make sure the solution is each acceptable and practicable.
Closing remarks
I spoke extensively to Nicolas Patry, the creator of Safetensors in writing this blog post and he requested that I add a call to motion to the AI open source community and AI enthusiasts:
- Pro-actively start replacing your pickle files with Safetensors. As mentioned earlier, pickle accommodates inherent security flaws and will be unsupported within the near future.
- Keep opening issues/PRs upstream about security to your favorite libraries to push secure defaults as much as possible upstream.
The AI industry is rapidly changing and latest attack vectors / exploits are being identified on a regular basis. Huggingface has a one in all a form community and we partner heavily with you to assist us maintain a secure platform.
Please remember to responsibly disclose security vulns/bugs through the suitable channels to avoid potential legal liability and violation of laws.
Want to affix the discussion? Reach out to us as security@huggingface.co or follow us on Linkedin/Twitter.
