data scientist or ML engineer, learning machine learning system design is one of the crucial essential skills it is advisable know. It’s the bridge between constructing models and deploying solutions that drive actual business outcomes.
The power to show ML ideas into production systems that get monetary savings, boost revenue, and create measurable value determines your long-term profession growth and your salary.
I’ve built machine learning systems which have saved firms over $1.5 million per yr, and these same skills have helped me land job offers exceeding $100,000.
On this guide, I’ll break down how I take into consideration ML system design so you possibly can do the identical.
General Framework
Below is my framework on find out how to approach designing a machine learning system:
.
In case you need a PDF copy of this template, you possibly can get access using this link:
https://framework.egorhowell.com
Let’s break down these steps in a bit more detail.
Business Problem
The goal of this step is to:
- Make clear objectives — What’s the business or user problem you’re trying to unravel, and find out how to translate that to a machine learning solution?
- Define metrics — What metrics are we targeting: Accuracy, F1-score, ROC-AUC, precision/recall, RMSE, etc and the way that translates to business performance.
- Constraints and scope — How much compute resource is offered, do we would like live-time predictions or batch inference, will we even need machine learning?
- High-level design — What is going to the rough architecture appear like from data to inference?
Data
That is all about gathering and acquiring data:
- Discover data sources — Databases, APIs, logs, or user-generated data.
- Discover goal variable — What’s the goal variable and the way will we get it?
- Quality control — What state is the information in? Are there any legal issues with using the information?
Feature Engineering
Create novel features from the information to tackle the precise problem:
- Feature importance — Understanding what features are more likely to drive the goal variable.
- Data cleansing — Handle missing values, outliers, and inconsistent entries.
- Feature representation — One-hot encoding, goal encoding, embeddings, and scaling the information.
- Sampling and splits — Account for unbalanced datasets, data leakage, and appropriately split to training and testing datasets.
Model Design & Selection
That is where you showcase your theoretical knowledge of machine learning models:
- Benchmark — Start with a straightforward “silly” model or heuristic after which slowly construct complexity.
- Training — Cross-validation, hyperparameter tuning, early stopping.
- Tradeoffs — Consider tradeoffs like training speed, inference speed, latency, and interpretability.
Service & Deployment
Understanding the perfect option to serve and deploy the model in production.
- Infrastructure — Select cloud/on-prem, arrange CI/CD pipelines, and ensure scalability.
- Service — API endpoint, edge model, batch predictions vs online predictions.
Evaluation & Monitoring
The last part is establishing systems and frameworks to trace your model within the production environment.
- Metrics — What metrics to trace with the “online” model vs “offline” model.
- Monitoring — Setup a dashboard, monitoring notebook, Slack alerts.
- Experiment — Design an A/B experiment.
What To Learn?
Let me let you know a secret: machine learning system design just isn’t an entry-level interview or skill set.
It’s because machine learning system design is tested on the mid and above levels.
By that point, you should have solid knowledge across machine learning and software engineering, and can likely be developing a specialism.
Nevertheless, in case you need a comprehensive, but on no account exhaustive list, that is what it is advisable learn.
Machine Learning Theory
- Supervised learning — Classification (logistic regression, support vector machines, decision trees), regression (linear regression, decision trees. gradient boosted trees).
- Unsupervised learning — Clustering (k-means, DBSCAN), dimensionality reduction, latent semantic evaluation.
- Deep learning — Neural networks, convolutional neural networks and recurrent neural networks.
- Loss functions — Accuracy, F1-score, NDCG, precision/recall, RMSE etc.
- Feature selection — Methods to discover essential features, like correlation evaluation, recursive feature elimination, regularisation, cross-validation and hyperparameter tuning.
- Statistics — Bayesian statistics, hypothesis testing and A/B tests.
- Specialisms — Time series, computer vision, operations research, advice systems. natural language processing etc. Only need 1–2.
System Design & Engineering
- Cloud — The Important one is AWS, and it’s best to know S3, EC2, Lambda functions, and ECS. Most things are simply wrappers of storage and compute anyway.
- Containerization — Docker and Kubernetes.
- System design — Caching, networking, quantisation, APIs and storage.
- Version control — CircleCI, Jenkins, git, MLflow, Datadog, Weights and Biases.
- Deployment and orchestration frameworks — Argo, Metaflow, Databricks, Airflow and Kubeflow.
Resources
ML System Design Interviews
I plan to release a more detailed video on the machine learning system design interview process later, but for now, I’d wish to offer you a high-level overview together with some tricks to make it easier to prepare.
Machine learning system design interviews are typically aimed toward mid-level and senior machine learning engineers. In these interviews, you’ll often be presented with a broad, open-ended problem like designing a recommender system or a spam filter.
In case your role involves a selected specialisation, similar to computer vision, the interview query will often concentrate on that specific domain.
Certainly one of the most important challenges with machine learning system design interviews is their lack of standardisation. Unlike software engineering interviews, which follow a comparatively consistent format, ML design interviews vary widely in structure. There’s also rather a lot to cover: countless concepts, trade-offs, and potential solution paths.
That said, most hiring managers are likely to evaluate candidates on a couple of key dimensions:
- Problem translation — Can you are taking a business problem and frame it as a machine learning solution?
- Decision-making — Do you recognise trade-offs and justify your design selections logically?
- Breadth and depth — Do you show a solid understanding of ML theory, quite a lot of models, and find out how to apply them effectively in real-world scenarios?
How To Prepare For Interviews
When it comes to preparations, there’s one key thing I like to recommend.
Listed below are some resources to seek out such problems:
I also recommend testing large tech firms’ blog posts to learn more about how machine learning algorithms are deployed at scale:
Earlier, I discussed how system design interviews test greater than just your modelling skills.
But what are the underlying fundamentals they’re really testing for?
That’s precisely what I cover in certainly one of my previous articles, which can walk you thru all the pieces it is advisable know, together with the perfect resources.
The Ultimate AI/ML Roadmap For Beginners
One other Thing!
I offer 1:1 coaching calls where we will chat about whatever you would like — whether it’s projects, profession advice, or simply determining the next step. I’m here to make it easier to move forward!
1:1 Mentoring Call with Egor Howell
topmate.io
