Hugging Face has turn out to be synonymous with advancing AI at scale. With over 4 million builders deploying models on the Hub, the rapid growth of the platform necessitated a rethinking of how sensitive configuration data —secrets— are managed.
Last yr, the engineering teams got down to improve the handling of their secrets and credentials. After evaluating tools like HashiCorp Vault, they ultimately selected Infisical.
This case study details their migration to Infisical, explains how they integrated its powerful features, and highlights the way it enabled engineers to work more efficiently and securely.
Background
As Hugging Face’s infrastructure evolved from an AWS-only setup to a multi-cloud environment that features Azure and GCP, the engineering team needed a more agile, secure, and centralized method to manage secrets. As an alternative of remodeling legacy systems or adopting heavyweight solutions like HashiCorp Vault, they turned to Infisical on account of its developer-friendly workflows, multi-cloud abstraction, and robust security capabilities.
The important thing challenges they faced were:
- An increased risk of “secret sprawl” on account of inconsistent management across environments.
- Complex permission management because the team scaled, requiring tight, role-based access controls (RBAC) integrated with the organization’s SSO (Okta).
- Difficulties with local development where traditional .env files compromised each security and developer productivity.
- The burden of manual secret rotation, which became painfully evident after a security incident that involved exposed credentials.
As well as, the team needed an answer that adhered to infrastructure-as-code practices, supported project-by-project secret management, and provided a smooth balance between automation and manual control during deployments.
Implementation
Infisical’s flexible architecture was a super solution. The engineering team seized the chance to re-examine their internal project structure, splitting projects into distinct infrastructure and application domains. This allowed them to implement a clearer separation of concerns and standardize secret rotation practices—a priority within the wake of a recent security incident.
By leveraging Terraform, which was previously used to create Kubernetes secrets from AWS configurations, they found the transition to the Infisical Kubernetes Operator exceptionally smooth. This integration enabled security improvements while standardizing secrets management across all environments.
Kubernetes Integration
Kubernetes is at the center of Hugging Face’s production environment, and Infisical’s Kubernetes Operator has been instrumental in automating secret updates. The Operator constantly monitors for changes to any secret in Infisical and ensures that these updates are propagated to the corresponding Kubernetes objects. At any time when a change is detected, it could actually mechanically reload dependent Deployments, ensuring that containers at all times run with essentially the most recent secrets.
Example:
A brand new secret is required by an application running in Kubernetes. The key may be created via the Infisical’s CLI or the net UI, then the developer creates an InfisicalSecret resource in Kubernetes that specifies which secret from Infisical must be synced:
apiVersion: infisical.com/v1alpha1
kind: InfisicalSecret
metadata:
name: my-app-secret
namespace: production
spec:
infisicalSecretId: "123e4567-e89b-12d3-a456-426614174000"
targetSecretName: "my-app-k8s-secret"
Once the CRD is applied, the Infisical Operator constantly watches for updates. When changes are detected in Infisical, the Operator mechanically updates the Kubernetes secret (my-app-k8s-secret).
Higher yet, since the appliance’s Deployment references my-app-k8s-secret as an environment variable source or mounted volume, the Operator can mechanically trigger a container reload when the key changes.
In practice, Hugging Face engineers favor waiting for manual redeployments despite the Operator’s ability to mechanically trigger container restarts. This decision was driven by the necessity for precise control over deployments, particularly when high traffic (over 10 million requests per minute) and various replicas are involved.
Local Development
For local development, Infisical’s CLI streamlines workflows by injecting secrets directly into development environments. This removes the necessity for insecure local .env files, aligning local configurations with production standards and reducing onboarding friction.
Security and Access Management
Security improvements form the backbone of this migration. By integrating Infisical with existing identity providers similar to Okta, Hugging Face established a fine-grained RBAC system. Permissions are mechanically mapped from Okta groups, ensuring that developers retain administrative rights over their projects, while frontend and backend teams receive appropriately restricted read or write access.
Moreover, the secret sharing functionality allows secure credentials sharing amongst ML/AI researchers at Hugging Face. The centralized Infisical platform also simplifies auditing and managing secret rotations—a necessity highlighted by previous security incidents.
CI/CD and Infrastructure Integration
Seamless integration with CI/CD pipelines further enhanced the general security posture. Infisical was embedded into the deployment pipeline via GitHub Actions using OIDC authentication and Terraform integration. By operating self-hosted runners inside a secure environment, every deployment adhered to production-grade security standards. This integrated approach minimized risks and ensured a uniform experience from local development to cloud deployment.
Technical Outcomes & Insights
Centralizing secrets management with Infisical brought tangible improvements:
- Engineers now not must spend helpful time manually configuring environment secrets. Self-serve workflows accelerated onboarding and every day development cycles.
- Automated audits and fine-grained access controls enabled rapid incident response and promoted a “shift left” approach to security.
- Consistent integration across cloud providers, Kubernetes clusters, and CI/CD pipelines eliminated discrepancies in secret management, thus reinforcing the infrastructure’s security and reliability.
As noted by Adrien Carreira, Head of Infrastructure at Hugging Face,
“Infisical provided all of the functionality and security settings we would have liked to spice up our security posture and save engineering time. Whether you are working locally, running kubernetes clusters in production, or operating secrets inside CI/CD pipelines, Infisical has a seamless prebuilt workflow.”
Conclusion
Hugging Face’s migration to Infisical demonstrates how a technically driven, engineering-centric approach to managing secrets across multiple cloud platforms delivers significant advantages. For tackling similar challenges, using Infisical is a practical method to work more efficiently while keeping security strong.
When the secure path is made the simplest path, teams can give attention to constructing modern products as an alternative of worrying about managing secrets.
Resources
For teams thinking about adopting an identical approach:
This technical case study was adapted from the unique case study published at infisical.com/customers/hugging-face
