Today we’re excited to announce SafeCoder – a code assistant solution built for the enterprise.
The goal of SafeCoder is to unlock software development productivity for the enterprise, with a totally compliant and self-hosted pair programmer. In marketing speak: “your individual on-prem GitHub copilot”.
Before we dive deeper, here’s what that you must know:
- SafeCoder isn’t a model, but a whole end-to-end business solution
- SafeCoder is built with security and privacy as core principles – code never leaves the VPC during training or inference
- SafeCoder is designed for self-hosting by the client on their very own infrastructure
- SafeCoder is designed for patrons to own their very own Code Large Language Model
Why SafeCoder?
Code assistant solutions built upon LLMs, comparable to GitHub Copilot, are delivering strong productivity boosts. For the enterprise, the flexibility to tune Code LLMs on the corporate code base to create proprietary Code LLMs improves reliability and relevance of completions to create one other level of productivity boost. For example, Google internal LLM code assistant reports a completion acceptance rate of 25-34% by being trained on an internal code base.
Nonetheless, counting on closed-source Code LLMs to create internal code assistants exposes corporations to compliance and security issues. First during training, as fine-tuning a closed-source Code LLM on an internal codebase requires exposing this codebase to a 3rd party. After which during inference, as fine-tuned Code LLMs are prone to “leak” code from their training dataset during inference. To satisfy compliance requirements, enterprises have to deploy fine-tuned Code LLMs inside their very own infrastructure – which isn’t possible with closed source LLMs.
With SafeCoder, Hugging Face will help customers construct their very own Code LLMs, fine-tuned on their proprietary codebase, using state-of-the-art open models and libraries, without sharing their code with Hugging Face or some other third party. With SafeCoder, Hugging Face delivers a containerized, hardware-accelerated Code LLM inference solution, to be deployed by the client directly inside the Customer secure infrastructure, without code inputs and completions leaving their secure IT environment.
From StarCoder to SafeCoder
On the core of the SafeCoder solution is the StarCoder family of Code LLMs, created by the BigCode project, a collaboration between Hugging Face, ServiceNow and the open source community.
The StarCoder models offer unique characteristics ideally suited to enterprise self-hosted solution:
- Cutting-edge code completion results – see benchmarks within the paper and multilingual code evaluation leaderboard
- Designed for inference performance: a 15B parameters model with code optimizations, Multi-Query Attention for reduced memory footprint, and Flash Attention to scale to eight,192 tokens context.
- Trained on the Stack, an ethically sourced, open source code dataset containing only commercially permissible licensed code, with a developer opt-out mechanism from the get-go, refined through intensive PII removal and deduplication efforts.
Note: While StarCoder is the inspiration and model powering the initial version of SafeCoder, a very important good thing about constructing a LLM solution upon open source models is that it will probably adapt to the newest and biggest open source models available. In the long run, SafeCoder may offer other similarly commercially permissible open source models built upon ethically sourced and transparent datasets as the bottom LLM available for fine-tuning.
Privacy and Security as a Core Principle
For any company, the inner codebase is a few of its most significant and priceless mental property. A core principle of SafeCoder is that the client internal codebase won’t ever be accessible to any third party (including Hugging Face) during training or inference.
Within the initial arrange phase of SafeCoder, the Hugging Face team provides containers, scripts and examples to work hand in hand with the client to pick, extract, prepare, duplicate, deidentify internal codebase data right into a training dataset to be utilized in a Hugging Face provided training container configured to the hardware infrastructure available to the client.
Within the deployment phase of SafeCoder, the client deploys containers provided by Hugging Face on their very own infrastructure to show internal private endpoints inside their VPC. These containers are configured to the precise hardware configuration available to the client, including NVIDIA GPUs, AMD Instinct GPUs, Intel Xeon CPUs, AWS Inferentia2 or Habana Gaudi accelerators.
Compliance as a Core Principle
Because the regulation framework around machine learning models and datasets remains to be being written internationally, global corporations have to ensure that the solutions they use minimize legal risks.
Data sources, data governance, management of copyrighted data are only just a few of a very powerful compliance areas to think about. BigScience, the older cousin and inspiration for BigCode, addressed these areas in working groups before they were broadly recognized by the draft AI EU Act, and because of this was graded as most compliant amongst Foundational Model Providers in a Stanford CRFM study.
BigCode expanded upon this work by implementing novel techniques for the code domain and constructing The Stack with compliance as a core principle, comparable to commercially permissible license filtering, consent mechanisms (developers can easily discover if their code is present and request to be opted out of the dataset), and extensive documentation and tools to examine the source data, and dataset improvements (comparable to deduplication and PII removal).
All these efforts translate into legal risk minimization for users of the StarCoder models, and customers of SafeCoder. And for SafeCoder users, these efforts translate into compliance features: when software developers get code completions these suggestions are checked against The Stack, so users know if the suggested code matches existing code within the source dataset, and what the license is. Customers can specify which licenses are preferred and surface those preferences to their users.
How does it work?
SafeCoder is a whole business solution, including service, software and support.
Training your individual SafeCoder model
StarCoder was trained in greater than 80 programming languages and offers state-of-the-art performance on multiple benchmarks. To supply higher code suggestions specifically for a SafeCoder customer, we start the engagement with an optional training phase, where the Hugging Face team works directly with the client team to guide them through the steps to arrange and construct a training code dataset, and to create their very own code generation model through fine-tuning, without ever exposing their codebase to 3rd parties or the web.
The top result’s a model that is customized to the code languages, standards and practices of the client. Through this process, SafeCoder customers learn the method and construct a pipeline for creating and updating their very own models, ensuring no vendor lock-in, and keeping control of their AI capabilities.
Deploying SafeCoder
Through the setup phase, SafeCoder customers and Hugging Face design and provision the optimal infrastructure to support the required concurrency to supply an ideal developer experience. Hugging Face then builds SafeCoder inference containers which are hardware-accelerated and optimized for throughput, to be deployed by the client on their very own infrastructure.
SafeCoder inference supports various hardware to present customers a big selection of options: NVIDIA Ampere GPUs, AMD Instinct GPUs, Habana Gaudi2, AWS Inferentia 2, Intel Xeon Sapphire Rapids CPUs and more.
Using SafeCoder
Once SafeCoder is deployed and its endpoints are live inside the customer VPC, developers can install compatible SafeCoder IDE plugins to get code suggestions as they work. Today, SafeCoder supports popular IDEs, including VSCode, IntelliJ and with more plugins coming from our partners.
How can I get SafeCoder?
Today, we’re announcing SafeCoder in collaboration with VMware on the VMware Explore conference and making SafeCoder available to VMware enterprise customers. Working with VMware helps make sure the deployment of SafeCoder on customers’ VMware Cloud infrastructure is successful – whichever cloud, on-premises or hybrid infrastructure scenario is preferred by the client. Along with utilizing SafeCoder, VMware has published a reference architecture with code samples to enable the fastest possible time-to-value when deploying and operating SafeCoder on VMware infrastructure. VMware’s Private AI Reference Architecture makes it easy for organizations to quickly leverage popular open source projects comparable to ray and kubeflow to deploy AI services adjoining to their private datasets, while working with Hugging Face to make sure that organizations maintain the pliability to make the most of the newest and biggest in open-source models. That is all without tradeoffs in total cost of ownership or performance.
“Our collaboration with Hugging Face around SafeCoder fully aligns to VMware’s goal of enabling customer alternative of solutions while maintaining privacy and control of their business data. In actual fact, we’ve got been running SafeCoder internally for months and have seen excellent results. Better of all, our collaboration with Hugging Face is just getting began, and I’m excited to take our solution to our lots of of hundreds of consumers worldwide,” says Chris Wolf, Vice President of VMware AI Labs. Learn more about private AI and VMware’s differentiation on this emerging space here.
If you happen to’re fascinated with SafeCoder to your company, please contact us here – our team will contact you to debate your requirements!
