Hey friends! Welcome to Machine Learning Experts. I’m your host, Britney Muller and today’s guest is Lewis Tunstall. Lewis is a Machine Learning Engineer at Hugging Face where he works on applying Transformers to automate business processes and solve MLOps challenges.
Lewis has built ML applications for startups and enterprises within the domains of NLP, topological data evaluation, and time series.
You’ll hear Lewis discuss his latest book, transformers, large scale model evaluation, how he’s helping ML engineers optimize for faster latency and better throughput, and more.
In a previous life, Lewis was a theoretical physicist and outdoors of labor likes to play guitar, go trail running, and contribute to open-source projects.
Very excited to introduce this fun and good episode to you! Here’s my conversation with Lewis Tunstall:
Note: Transcription has been barely modified/reformatted to deliver the highest-quality reading experience.
Welcome, Lewis! Thanks a lot for taking outing of your busy schedule to speak with me today about your awesome work!
Lewis: Thanks, Britney. It’s a pleasure to be here.
Curious for those who can do a temporary self-introduction and highlight what brought you to Hugging Face?
Lewis: What brought me to Hugging Face was transformers. In 2018, I used to be working with transformers at a startup in Switzerland. My first project was an issue answering task where you input some text and train a model to attempt to find the reply to an issue inside that text.
In those days the library was called: pytorch-pretrained-bert, it was a really focused code base with a few scripts and it was the primary time I worked with transformers. I had no idea what was occurring so I read the unique ‘Attention Is All You Need’ paper but I couldn’t understand it. So I began looking around for other resources to learn from.
In the method, Hugging Face exploded with their library growing into many architectures and I got really enthusiastic about contributing to open-source software. So around 2019, I had this kinda crazy idea to put in writing a book about transformers because I felt there was an information gap that was missing. So I partnered up with my friend, Leandro (von Werra) and we sent Thom (Wolf) a chilly email out of nowhere saying, “Hey we’re going to put in writing a book about transformers, are you interested?” and I used to be expecting no response. But to our great surprise, he responded “Yea, sure let’s have a chat.” and around 1.5 years later that is our book: NLP with Transformers.
This collaboration set the seeds for Leandro and I to eventually join Hugging Face. And I have been here now for around nine months.
That’s incredible. How does it feel to have a duplicate of your book in your hands?
Lewis: I actually have to say, I just became a parent a few 12 months and a half ago and it feels form of just like my son being born. You are holding this thing that you simply created.
It’s quite an exciting feeling and so different to truly hold it (in comparison with reading a PDF). Confirms that it’s actually real and I didn’t just dream about it.
Exactly. Congratulations!
Need to briefly read one endorsement that I like about this book;
“_Complexity made easy. This can be a rare and precious book about NLP, transformers, and the growing ecosystem around them, Hugging Face. Whether these are still buzzwords to you or you have already got a solid grasp of all of it, the authors will navigate you with humor, scientific rigor, and many code examples into the deepest secrets of the good technology around. From “off-the-shelf pre-trained” to “from-scratch custom” models, and from performance to missing labels issues, the authors address practically every real-life struggle of an ML engineer and supply state-of-the-art solutions, making this book destined to dictate the standards in the sphere for years to come back._”
—Luca Perrozi Ph.D., Data Science and Machine Learning Associate Manager at Accenture.
Checkout Natural Language Processing with Transformers.
Are you able to talk in regards to the work you’ve got done with the transformers library?
Lewis: One among the things that I experienced in my previous jobs before Hugging Face was there’s this challenge within the industry when deploying these models into production; these models are really large when it comes to the variety of parameters and this adds numerous complexity to the necessities you would possibly have.
So for instance, for those who’re attempting to construct a chatbot you wish this model to be very fast and responsive. And more often than not these models are a bit too slow for those who just take an off-the-shelf model, train it, after which attempt to integrate it into your application.
So what I have been working on for the previous few months on the transformers library is providing the functionality to export these models right into a format that enables you to run them far more efficiently using tools that we’ve got at Hugging Face, but in addition just general tools within the open-source ecosystem.
In a way, the philosophy of the transformers library is like writing numerous code in order that the users haven’t got to put in writing that code.
On this particular example, what we’re talking about is something called the ONNX format. It is a special format that’s utilized in industry where you may principally have a model that is written in PyTorch but you may then convert it to TensorFlow or you may run it on some very dedicated hardware.
And for those who actually have a look at what’s needed to make this conversion occur within the transformers library, it’s fairly gnarly. But we make it so that you simply only really must run one line of code and the library will deal with you.
So the concept is that this particular feature lets machine learning engineers and even data scientists take their model, convert it to this format, after which optimize it to get faster latency and better throughput.
That is very cool. Have there been, any standout applications of transformers?
Lewis: I feel there are a couple of. One is possibly emotional or personal, for instance lots of us when OpenAI released GPT-2, this very famous language model which might generate text.
OpenAI actually provided of their blog posts some examples of the essays that this model had created. And one in all them was really funny. One was an essay about why we shouldn’t recycle or why recycling is bad.
And the model wrote a compelling essay on why recycling was bad. Leandro and I were working at a startup on the time and I printed it out and stuck it right above the recycling bin within the office as a joke. And folks were like, “Woah, who wrote this?” and I said, “An algorithm.”
I feel there’s something kind of strangely human, right? Where if we see generated text we get more surprised when it looks like something I (or one other human) might need written versus other applications which have been happening like classifying text or more conventional tasks.
That is incredible. I remember once they released those examples for GPT-2, and one in all my favorites (that nearly gave me this sense of, whew, we’re not quite there yet) were among the more inaccurate mentions like “underwater fires”.
Lewis: Exactly!
Britney: But, then something had happened with an oil spill that next 12 months, where there have been actually fires underwater! And I immediately considered that text and thought, possibly AI is onto something already that we’re not quite aware of?
You and other experts at Hugging Face have been working hard on the Hugging Face Course. How did that come about & where is it headed?
Lewis: Once I joined Hugging Face, Sylvian and Lysandre, two of the core maintainers of the transformers library, were developing a course to principally bridge the gap between people who find themselves more like software engineers who’re inquisitive about natural language processing but specifically inquisitive about the transformers revolution that is been happening. So I worked with them and others within the open-source team to create a free course called the Hugging Face Course. And this course is designed to essentially help people go from knowing form of not a lot about ML throughout to having the power to coach models on many alternative tasks.
And, we have released two parts of this course and planning to release the third part this 12 months. I’m really excited in regards to the next part that we’re developing at once where we’ll explore different modalities where transformers are really powerful. More often than not we predict of transformers for NLP, but likely there’s been this explosion where transformers are getting used in things like audio or in computer vision and we’ll be taking a look at these intimately.
What are some transformers applications that you simply’re enthusiastic about?
Lewis: So one which’s form of fun is within the course we had an event last 12 months where we got people locally to make use of the course material to construct applications.
And one in all the participants on this event created a canopy letter generator for jobs. So the concept is that once you apply for a job there’s at all times this annoying thing you might have to put in writing a canopy letter and it is often like a bit like you might have to be witty. So this guy created a canopy letter generator where you provide some details about yourself after which it generates it from that.
And he actually used that to use to Hugging Face.
No way?!
Lewis: He’s joining the Big Science team as an intern. So. I mean that is a brilliant cool thing, right? Once you learn something after which use that thing to use which I believed was pretty awesome.
Where do you ought to see more ML applications?
Lewis: So I feel personally, the world that I’m most enthusiastic about is the appliance of machine learning into natural sciences. And that is partly due to my background. I was once a Physicist in a previous lifetime but I feel what’s also very exciting here is that in numerous fields. For instance, in physics or chemistry you already know what the say underlying laws are when it comes to equations that you may write down nevertheless it seems that most of the problems that you simply’re concerned with studying often require a simulation. Or they often require very hardcore supercomputers to know and solve these equations. And one of the exciting things to me is the mixture of deep learning with the prior knowledge that scientists have gathered to make breakthroughs that weren’t previously possible.
And I feel an incredible example is DeepMind’s Alpha Fold model for protein structure prediction where they were principally using a mixture of transformers with some extra information to generate predictions of proteins that I feel previously were taking over the order of months and now they will do them in days.
So this accelerates the entire field in a very powerful way. And I can imagine these applications ultimately result in hopefully a greater future for humanity.
The way you see the world of model evaluation evolving?
Lewis: That is an incredible query. So at Hugging Face, one in all the things I have been working on has been attempting to construct the infrastructure and the tooling that allows what we call ‘large-scale evaluation’. So you could know that the Hugging Face Hub has 1000’s of models and datasets. But for those who’re attempting to navigate this space you would possibly ask yourself, ‘I’m concerned with query answering and wish to know what the highest 10 models on this particular task are’.
And in the mean time, it’s hard to seek out the reply to that, not only on the Hub, but typically within the space of machine learning this is kind of hard. You frequently must read papers after which you might have to take those models and test them yourself manually and that is very slow and inefficient.
So one thing that we have been working on is to develop a way that you may evaluate models and datasets directly through the Hub. We’re still attempting to experiment there with the direction. But I’m hoping that we’ve got something cool to indicate later this 12 months.
And there is one other side to this which is that a big a part of the measuring progress in machine learning is thru using benchmarks. These benchmarks are traditionally a set of datasets with some tasks but what’s been possibly missing is that numerous researchers speak to us and say, “Hey, I’ve got this cool idea for a benchmark, but I do not actually need to implement all the nitty-gritty infrastructure for the submissions, and the upkeep, and all those things.”
And so we have been working with some really cool partners on hosting benchmarks on the Hub directly. In order that then people within the research community can use the tooling that we’ve got after which simplify the evaluation of those models.
That’s super interesting and powerful.
Lewis: Possibly one thing to say is that the entire evaluation query is a really subtle one. We all know from previous benchmarks, corresponding to SQuAD, a famous benchmark to measure how good models are at query answering, that lots of these transformer models are good at taking shortcuts.
Well, that is the aim nevertheless it seems that lots of these transformer models are really good at taking shortcuts. So, what they’re actually doing is that they’re getting a really high rating on a benchmark which does not necessarily translate into the actual thing you were concerned with which was answering questions.
And you might have all these subtle failure modes where the models will possibly provide completely flawed answers or they shouldn’t even answer in any respect. And so in the mean time within the research community there is a very energetic and vigorous discussion about what role benchmarks play in the way in which we measure progress.
But additionally, how do these benchmarks encode our values as a community? And one thing that I feel Hugging Face can really offer the community here is the means to diversify the space of values because traditionally most of those research papers come from the U.S. which is an incredible country nevertheless it’s a small slice of the human experience, right?
What are some common mistakes machine learning engineers or teams make?
Lewis: I can possibly let you know those that I’ve done.
Probably a superb representative of the remainder of the things. So I feel the most important lesson I learned once I was starting out in the sphere is using baseline models when starting out. It’s a typical problem that I did after which later saw other junior engineers doing is reaching for the fanciest state-of-the-art model.
Although that may match, numerous the time what happens is you introduce numerous complexity into the issue and your state-of-the-art model could have a bug and you will not really know tips on how to fix it since the model is so complex. It’s a quite common pattern in industry and particularly inside NLP is that you may actually get quite far with regular expressions and linear models like logistic regression and these sorts of things provides you with a superb start. Then for those who can construct a greater model then great, it’s best to do this, nevertheless it’s great to have a reference point.
After which I feel the second big lesson I’ve learned from constructing numerous projects is that you may get a bit obsessive about the modeling a part of the issue because that is the exciting bit once you’re doing machine learning but there’s this whole ecosystem. Especially for those who work in a big company there will be this whole ecosystem of services and things which can be around your application.
So the lesson there may be it’s best to really try to construct something end to finish that perhaps doesn’t even have any machine learning in any respect. However it’s the scaffolding upon which you’ll be able to construct the remainder of the system because you can spend all this time training an awesome mode, and you then go, oh, oops.
It doesn’t integrate with the necessities we’ve got in our application. And you then’ve wasted all this time.
That is a superb one! Don’t over-engineer. Something I at all times attempt to have in mind.
Lewis: Exactly. And it is a natural thing I feel as humans especially for those who’re nerdy you actually need to seek out essentially the most interesting solution to do something and more often than not easy is healthier.
Should you could return and do one thing in a different way in the beginning of your profession in machine learning, what wouldn’t it be?
Lewis: Oh, wow. That is a tricky one. Hmm. So, the rationale it is a really hard query to reply is that now that I’m working at Hugging Face, it’s essentially the most fulfilling sort of work that I’ve really done in my whole life. And the query is that if I modified something once I started off possibly I would not be here, right?
It’s one in all those things where it’s a tough one in that sense. I suppose one thing that perhaps I might’ve done barely in a different way is once I started off working as an information scientist you are likely to develop the talents that are about mapping business problems to software problems or ultimately machine learning problems.
And it is a really great skill to have. But what I later discovered is that my true driving passion is doing open source software development. So probably the thing I might have done in a different way would have been to begin that much earlier. Because at the tip of the day most open source is basically driven by community members.
So that might have been possibly a solution to shortcut my path to doing this full-time.
I like the concept of had you done something in a different way possibly you would not be at Hugging Face.
Lewis: It’s just like the butterfly effect movie, right? You return in time and you then have no legs or something.
Totally. Don’t need to mess with a superb thing!
Lewis: Exactly.
Rapid Fire Questions:
Best piece of recommendation for somebody trying to get into AI/Machine Learning?
Lewis: Just start. Just start coding. Just start contributing if you ought to do open-source. You possibly can at all times find reasons to not do it but you simply must get your hands dirty.
What are among the industries you are most excited to see machine learning applied?
Lewis: As I discussed before, I feel the natural sciences is the world I’m most enthusiastic about
That is where I feel that is most fun. If we have a look at something, say at the commercial side, I assume among the development of recent drugs through machine learning could be very exciting. Personally, I’d be really completely happy if there have been advancements in robotics where I could finally have a robot to love fold my laundry because I actually hate doing this and it could be nice if like there was an automatic way of handling that.
Should people be afraid of AI taking on the world?
Lewis: Possibly. It’s a tricky one because I feel we’ve got reasons to think that we may create systems which can be quite dangerous within the sense that they might be used to cause numerous harm. An analogy is probably with weapons you should use inside the sports like archery and shooting, but you can too use them for war. One big risk might be if we take into consideration combining these techniques with the military perhaps this results in some tricky situations.
But, I’m not super fearful in regards to the Terminator. I’m more fearful about, I do not know, a rogue agent on the financial stock market bankrupting the entire world.
That is a superb point.
Lewis: Sorry, that is a bit dark.
No, that was great. The following query is a follow-up in your folding laundry robot. When will AI-assisted robots be in homes in every single place?
Lewis: Honest answer. I do not know. Everyone, I do know who’s working on robotics says this continues to be an especially difficult task within the sense that robotics hasn’t quite experienced the identical form of revolutions that NLP and deep learning have had. But however, you may see some pretty exciting developments within the last 12 months, especially around the concept of having the ability to transfer knowledge from a simulation into the actual world.
I feel there’s hope that in my lifetime I may have a laundry-folding robot.
What have you ever been concerned with recently? It might be a movie, a recipe, a podcast, literally anything. And I’m just curious what that’s and the way someone concerned with which may find it or start.
Lewis: It’s an incredible query. So for me, I like podcasts typically. It’s my latest way of reading books because I actually have a young baby so I’m just doing chores and listening at the identical time.
One podcast that basically stands out recently is definitely the DeepMind podcast produced by Hannah Fry who’s a mathematician within the UK and he or she gives this beautiful journey through not only what Deep Mind does, but more generally, what deep learning and particularly reinforcement learning does and the way they’re impacting the world. Listening to this podcast seems like you are listening to love a BBC documentary because the English has such great accents and you are feeling really inspired because numerous the work that she discusses on this podcast has a powerful overlap with what we do at Hugging Face. You see this much larger picture of attempting to pave the way in which for a greater future.
It resonated strongly. And I just adore it because the reasons are super clear and you may share it together with your family and your mates and say, “Hey, if you ought to know what I’m doing? This will offer you a rough idea.”
It gives you a really interesting insight into the Deep Mind researchers and their backstory as well.
I’m definitely going to present that a listen. [Update: It’s one of my new favorite podcasts. 🙂 Thank you, Lewis!]
What are a few of your favorite Machine Learning papers?
Lewis: Is determined by how we measure this, but there’s one paper that stands out to me, which is kind of an old paper. It’s by the creator of random forests, Leo Breiman. Random forests is a really famous classic machine learning technique that is useful for tabular data that you simply see in industry and I needed to teach random forests at university a 12 months ago.
And I used to be like, okay, I’ll read this paper from the 2000s and see if I understand it. And it is a model of clarity. It is very short, and really clearly explains how the algorithm is implemented. You possibly can principally just take this paper and implement the code very very easily. And that to me was a very nice example of how papers were written in medieval times.
Whereas nowadays, most papers, have this formulaic approach of, okay, here’s an introduction, here’s a table with some numbers that improve, and here’s like some random related work section. So, I feel that is one which like stands out to me so much.
But one other one which’s a little bit bit newer is a paper by DeepMind again on using machine learning techniques to prove fundamental theorems like algebraic topology, which is a special branch of abstract mathematics. And at one point in my life, I used to work on these related topics.
So, to me, it’s a really exciting, perspective of augmenting the knowledge that a mathematician would have in attempting to narrow down the space of theorems that they could have to look for. I feel this to me was surprising because numerous the time I have been quite skeptical that machine learning will result in this fundamental scientific insight beyond the plain ones like making predictions.
But this instance showed that you may actually be quite creative and help mathematicians find latest ideas.
What’s the meaning of life?
Lewis: I feel that the honest answer is, I do not know. And possibly anyone who does let you know a solution probably is lying. That is a bit sarcastic. I dunno, I assume being a site scientist by training and particularly a physicist, you develop this worldview that could be very much that there is not really some kind of deeper intending to this.
It is very very like the universe is kind of random and I suppose the one thing you may take from that beyond being very sad is that you simply derive your individual meaning, right? And more often than not this comes either from the work that you simply do or from the family or from your mates that you might have.
But I feel once you discover a solution to derive your individual meaning and discover what you do is definitely interesting and meaningful that that is the most effective part. Life could be very up and down, right? At the least for me personally, the things which have at all times been very meaningful are generally in creating things. So, I was once a musician, in order that was a way of making music for other people and there was great pleasure in doing that. And now I form of, I assume, create code which is a type of creativity.
Absolutely. I feel that is beautiful, Lewis! Is there the rest you want to to share or mention before we log out?
Lewis: Possibly buy my book.
It’s so good!
Lewis: [shows book featuring a parrot on the cover] Do the story in regards to the parrot?
I do not think so.
Lewis: So when O’Reilly is telling you “We will get our illustrator now to design the duvet,” it is a secret, right?
They do not let you know what the logic is or you might have no say within the matter. So, principally, the illustrator comes up with an idea and in one in all the last chapters of the book we’ve got a bit where we principally train a GPT-2 like model on Python code, this was Thom’s idea, and he decided to call it code parrot.
I feel the concept or the joke he had was that there is numerous discussion locally about this paper that Meg Mitchell and others worked on called, ‘Stochastic Parrots’. And the concept was that you might have these very powerful language models which appear to exhibit human-like traits of their writing as we discussed earlier but deep down possibly they’re just performing some kind of like parrot parenting thing.
, for those who confer with like a cockatoo it is going to swear at you or make jokes. That might not be a real measure of intelligence, right? So I feel that the illustrator by some means possibly saw that and decided to place a parrot which I feel is an ideal metaphor for the book.
And the proven fact that there are transformers in it.
Had no concept that that was the way in which O’Reilly’s covers got here about. They do not let you know and just pull context from the book and create something?
Lewis: It looks as if it. I mean, we do not really know the method. I’m just kind of guessing that perhaps the illustrator was attempting to get an idea and saw a couple of animals within the book. In one in all the chapters we’ve got a discussion about giraffes and zebras and stuff. But yeah I’m completely happy with the parrot cover.
I adore it. Well, it looks absolutely amazing. A whole lot of most of these books are likely to be quite dry and technical and this one reads almost like a novel mixed with great applicable technical information, which is gorgeous.
Lewis: Thanks. Yeah, that’s one thing we realized afterward since it was the primary time we were writing a book we thought we ought to be sort of significant, right? But for those who kind of know me I’m like never really serious about anything. And in hindsight, we should always have been much more silly within the book.
I had to regulate my humor in various places but possibly there will be a second edition someday after which we are able to just inject it with memes.
Please do, I stay up for that!
Lewis: In actual fact, there may be one meme within the book. We tried to sneak this in past the Editor and have the DOGE dog contained in the book and we use a special vision transformer to attempt to classify what this meme is.
So glad you bought that one in there. Well done! Stay up for many more in the following edition. Thanks a lot for joining me today. I actually appreciate it. Where can our listeners find you online?
Lewis: I’m fairly energetic on Twitter. You possibly can just find me my handle @_lewtun. LinkedIn is an odd place and I’m probably not on there very much. And naturally, there’s Hugging Face, the Hugging Face Forums, and Discord.
Perfect. Thanks a lot, Lewis. And I’ll chat with you soon!
Lewis: See ya, Britney. Bye.
Thanks for listening to Machine Learning Experts!
