At Edge Analytics, we try to develop machine learning applications which might be transparent and reproducible. Machine learning projects are sometimes composed of many parts, and the rapidly available software solutions for managing these parts are collectively called Machine Learning Operations (MLOps).
We’ve found investing in MLOps infrastructure accelerates our ability to extract meaningful insights for the issues we solve. A strong MLOps infrastructure optimizes our iteration time and increases our trust within the models we develop for clients. On this blog series, we’ll discuss an example MLOps pipeline using a few of our favourite tools.
Our team develops and deploys ML models to each edge devices and the cloud. Tracking model development and deployment is critical to make sure algorithms behave as expected within the wild. We’ll display one example of a full pipeline using tools which have worked well for our team. The instance is a modular architecture with functionality broken into five major steps. These steps include:
- Data Storage and I/O
- Data Processing
- Model Development
- Model Tracking
- Model Deployment
The particular structures of MLOps pipelines will vary from project to project. At Edge Analytics, we use one of the best tools for the job at hand; nevertheless, we consistently use the above five points as guiding pillars for MLOps pipeline development.
Constructing an MLOps pipeline comes with countless trade-offs for balancing structure and adaptability. In choosing one of the best tools for our clients, we’re guided by two major principles:
- Code abstractions for third-party tools needs to be easy, consistent, and well documented.
- No single platform has one of the best of all solutions, and latest features are regularly available. We should always maintain a versatile pipeline able to interchanging third-party platforms.
There are several services that provide end-to-end MLOps support, akin to AWS SageMaker and Ray. Nonetheless, we aim to avoid tight coupling with any given service by enabling integration with quite a lot of Python packages and services.
Lastly, it needs to be acknowledged that there is no such thing as a one strategy to construct an MLOps pipeline. There are a lot of incredible tools that may assist in developing and tracking ML models. We hope this instance pipeline gives you a superb place to begin and encourages you to search out the methods that best serve your project!
Over this series of blog posts, we’ll examine each of the five central MLOps pipeline pillars more closely within the context of an example project classifying white blood cell images by cell type. And with that, let’s take a take a look at data storage…
Edge Analytics helps corporations construct MLOps solutions for his or her specific use cases. More broadly, we specialise in data science, machine learning, and algorithm development each on the sting and within the cloud. We offer end-to-end support throughout a product’s lifecycle, from quick exploratory prototypes to production-level AI/ML algorithms. We partner with our clients, who range from Fortune 500 corporations to revolutionary startups, to show their ideas into reality. Have a tough problem in mind? Get in contact at info@edgeanalytics.io.
