Generative AI has taken the world by storm, and cloud providers’ machine learning services have gotten increasingly popular. Amazon SageMaker is one in every of the leading ML platforms, and I recently decided to explore it out of curiosity. On this post, I’ll share my experience and highlight the 4 things that I loved about SageMaker (and few I’m missing).
Amazon SageMaker is a managed machine learning service provided by AWS that provides tools for constructing, training, and deploying machine learning models at scale.
Whether you’re a machine learning engineer or similar to hacking AWS, I hope you’ll find this post informative and interesting.
SageMaker’s managed Jupyter Notebooks are well designed for research, development and deployment. Having worked with other managed Jupyter solutions that were removed from optimal, I used to be pleasantly surprised by the flawless experience that SageMaker provided.
I experimented a bit with image-to-image diffusion models, and enjoyed deploying instruct-pix2pix on an ml.g5.2xlarge.
I used to be in a position to spin up a ml.g5.2xlarge instance in lower than two minutes and have my model fully up and running in five. I could literally transform images of Obama into an anime character or cyborg while waiting for my next construct to complete. This feature is awesome and showcases the speed and efficiency of SageMaker’s managed notebooks.
One other feature of SageMaker that I used to be impressed by is the recent launch of shared spaces. This permits multiple users to collaborate on notebooks in real-time and concurrently, which is actually convenient. That is great for data science teams, although I haven’t had a likelihood to try it myself.
While I’m not a machine learning engineer, I understand how difficult it could actually be to deploy and maintain models in production. That’s why I used to be excited to find SageMaker JumpStart, which provides pre-trained, open-source models for a big selection of problem types. With JumpStart, even beginners can start with machine learning in only just a few clicks. The models may be easily deployed to an endpoint and are able to handle traffic immediately, which is a big advantage for anyone seeking to experiment with machine learning without spending an excessive amount of time on infrastructure setup.
I used to be experimenting with Alexa TM 20B model, just look how easy the deployment is.
First, go to SageMaker Studio, hit and select a model. (I’ll go along with Alexa TM 20B)
After choosing the model, you click on the button (change configurations in case you need) and wait just a few minutes for the model to be ready.
I like how seamless was the experience. SageMaker provides a “plug and play” notebook to check your endpoint, which makes it incredibly easy to experiment together with your model.
I used to be excited to see that SageMaker permits you to deploy models with IaaC. I’m actually an AWS Cloud Development Kit (CDK) sucker, so I had plenty of fun deploying an HuggingFace model straight from code using CDK. Let’s take a more in-depth take a look at how that went.
Deploying a model with IaaC using CDK requires some setup work, resembling making a Docker container that might be utilized by SageMaker and adding the model.tar.gz. Nevertheless, when you’ve accomplished those steps, the deployment process is simple.
➜ cdk bootstrap
⏳ Bootstrapping environment aws://0123456789/us-west-2...
...
✅ Environment aws://0123456789/us-west-2 bootstrapped.➜ cdk deploy
✨ Synthesis time: 9.38s
SagemakerBlogPythonStack: constructing assets...
...
✅ SagemakerBlogPythonStack
✨ Deployment time: 587.42s
I used the HuggingFace GPT-J, and used the S3 model_data from their public bucket s3://huggingface-sagemaker-models/transformers/4.12.3/pytorch/1.9.1/gpt-j/model.tar.gz. For a picture, I used a regular HuggingFace PyTorch inference image from their public ECR arn:aws:ecr:us-west-2:763104351884:repository/huggingface-pytorch-inference. Note that the model required sufficient ECR and S3 permissions as a way to pull the image and skim the model data accurately.
As GPT-J is “only” 6B parameters, an ml.g4dn.xlarge was sufficient. I also added an auto-scaler feature to my endpoint, which was really cool to see in motion.
In lower than 10 minutes my model was served from an endpoint, able to generate text.
Check my GitHub code for more context.
It is a very nice feature.
Everyone knows that on-demand instances can get quite expensive, particularly when coping with large workloads like training your gigantic models. Managed Spot Training in SageMaker is designed to tackle this challenge by allowing customers to scale back their costs by as much as 90% through the usage of EC2 spot instances.
In case you’re not conversant in them, Spot Instances are EC2 instances that provide access to unused EC2 capability at a significantly discounted rate. Amazon offers these instances to assist EC2 users optimize their usage by benefiting from idle capability, as an alternative of relying solely on the costlier On-Demand Instances.
With Managed Spot Training in SageMaker, you possibly can specify which training jobs use spot instances and set a stopping condition that determines how long SageMaker waits for a job to run using Amazon EC2 Spot instances. Moreover, you possibly can arrange checkpoints on your training jobs, and SageMaker will mechanically restart the job from where it left off if it gets interrupted.
It’s very easy to set-up, take a look at this example.
I think that SageMaker may benefit from providing access to advanced models like StableDiffusion or Bloom through Amazon API endpoints, charged based on usage. This is able to be particularly useful for purchasers with lighter workloads, who could save costs while still leveraging cutting-edge models through the convenience of Amazon APIs.
Fortunately, I recently learned concerning the SageMaker JumpStart Foundation Models announcement, which is currently in closed-preview and may address this need. Considering the newest announcement of the expansion of the long-term strategic partnership between Amazon Web Services AWS and Hugging Face, along with the strong collaborations with Cohere, AI21 Labs, and Stability AI, we are able to only imagine the probabilities and anticipate recent breakthroughs in the sphere of AI and ML.
For my part, it could be helpful to have more low-code/no-code AI/ML tools in SageMaker. While SageMaker Canvas does have useful automatic data evaluation tools, I think that SageMaker’s no-code capabilities might be enhanced and expanded to incorporate features for computer vision and generative AI.
Can you be more specific about the content of your article? After reading it, I still have some doubts. Hope you can help me.
cozy living room ambience
instrumental jazz
Your article helped me a lot, is there any more related content? Thanks!