a decade old now.
Back then, OpenAI felt like one (well-baked) startup amongst others. DeepMind was already around, but not yet fully integrated into Google. And, back then, the “triad of deep learning” — LeCun, Hinton, and Bengio — published in *.
Today, AI is like a standard good. Back then, it was mostly scholars and tech nerds that knew and cared about it. Today, even kids know what AI is and interact with it (for worse or ).
It’s a fast-paced field, and I’m fortunate to have joined it only barely afterwards “back then” — eight years ago, when momentum was constructing but classic ML was still taught at the schools: clustering, k-means, SVMs. It also coincided with the yr that the community began to know that focus (and linear layers) is all we would want. It was, in other words, an amazing time to begin learning about machine learning.
Because the yr now closes, it seems like the proper time to zoom out. On a monthly basis I reflect on small, practical lessons and publish them. Roughly every half a yr, I then search for the larger themes underneath: the patterns that keep recurring, even when projects change.
This time, 4 threads show up all over the place in my notes:
- Deep Work (my all-time favorite)
- Over-identification with one’s work
- Sports (and movement on the whole)
- Blogging
Deep Work
Deep Work appears to be my favorite theme — and in machine learning it shows up all over the place.
Machine learning works can have several focus points, but most days revolve around some combination of:
- theory (math, proofs, careful reasoning),
- coding (pipelines, training loops, debugging),
- writing (project reports, papers, documentation).
All of them require sustained focus for prolonged time.
Theorem proofs don’t emerge from five-minute fragments. Coding, useless to say, punishes interruptions: if you happen to’re deep in a bug and someone pulls you out, you don’t just “resume” — it’s essential reconstruct, which just burns time**.
Writing, too, is fragile. Crafting good sentences needs attention, and a focus is the very first thing that disappears when your day becomes a sequence of small message pings.
I’m fortunate enough to work in an environment that enables multiple hours of deep work, several times every week. This isn’t the norm — truthfully, it is likely to be the exception. But it surely’s incredibly fulfilling. I .
Exhausted, but satisfied.
For me, deep work has all the time meant two things, and I already highlighted this half a yr ago:
- The skill: having the ability to concentrate deeply for long stretches.
- The environment: having conditions that allow and protect that concentration.
Normally, the skill is less complicated to amass (or re-acquire) if you happen to don’t have it. It’s the environment that’s harder to vary. You may train , but you’ll be able to’t single-handedly delete meetings out of your calendar, or change your organization’s culture overnight.
Still, it helps to call the 2 parts. In the event you’re fighting deep work, it won’t be a scarcity of discipline. Sometimes, as my experiences tell me, it’s simply that your environment doesn’t permit the thing you’re attempting to do.
Over-identification with one’s work
Do you want your job?
Let’s hope so, because an enormous fraction of your waking hours is spent doing it. But even if you happen to generally like your job, there will probably be times whenever you prefer it more — and times whenever you prefer it less.
Like all people, I’ve had each.
There have been periods where I felt jolted with energy just from the indisputable fact that I used to be “doing something with ML.”
Wow!
After which there have been periods where lack of progress — or a setback because an idea simply didn’t work — dragged me down hard.
Not-wow.
Through the years, I’ve come to consider that deriving an excessive amount of identity from the job is usually not a sensible strategy. Work on and with ML is stuffed with variance: experiments fail, baselines beat your fancy ideas, reviewers misunderstand, deadlines compress, data breaks, priorities shift. In case your sense of self rises and falls with the most recent training run, you can equally well be visiting Disneyland for a roller coaster ride.
An easy analogy: imagine you’re a gymnast. You train for years. You’re flexible, strong, accountable for your movements. Then you definately break your ankle. Suddenly, you’ll be able to’t even do the only jumps. You may’t train in the identical way you’ve done it the years before. In the event you’re an athlete — — it’s going to feel like losing yourself.
Thankfully most individuals are greater than their occupation. Even in the event that they forget it sometimes.
The identical applies to ML. You may be an ML engineer, or a researcher, or a “theory person” — and in addition be a friend, a partner, a sibling, a teammate, a reader, a runner, a author. When one part goes through a low, the others hold you regular.
This isn’t “I don’t care about my job”. It’s about .
Sports, or movement on the whole
Granted, it is a no-brainer.
Jobs in ML usually are not known for holding a number of movement. The miles you make are finger-miles on the keyboard. Meanwhile, the remainder of the body sits still.
I would like not go into what happens if you happen to just let occur.
The excellent news is: it’s easier than ever to counteract. There are a lot of boring but effective options now:
- height-adjustable desks
- meetings spent walking (especially when cameras are off anyway)
- walking pads under the desk
- short mobility routines (ideally, between deep work blocks)
Through the years, movement has develop into an integral part for my workday. It helps me start the day in a smoother state — not stiff, not slouched, not already “compressed.” And it helps me de-exhaust after deep work. Deep concentration is mentally tiring, but additionally has physical effects: shoulders stand up, neck falls forward, respiratory becomes shallow.
Moving resets that.
I don’t treat it as “fitness.” I treat it as an insurance that enables me to do my job for years to return.
Blogging
In the event you’ve been reading ML content on Towards Data Science for a very long time (a minimum of five, six years), that name might sound familiar. He published a number of ML articles (when TDS was still hosted on Medium), and his unique form of writing brought ML to a wider audience.
His example inspired me to begin blogging as well — also for TDS. I started at the top of 2019, starting of 2020.
At first, writing these articles was easy: write an article, publish it, move on. But over time, it became something else: a practice. Writing forces precision in putting your thoughts to paper. In the event you can’t explain something in a way that holds together, you almost certainly don’t understand it in addition to you’re thinking that you do.
Through the years, I covered machine learning roadmaps, wrote tutorials (like tips on how to handle TFRecords), and, yes, kept circling back to deep work — since it keeps proving itself essential for ML practitioners.
And blogging has been rewarding in two ways.
It’s been rewarding in monetary terms (to the purpose where, over time, it helped finance the pc I’m using to put in writing this). But more importantly, it has been rewarding as a practice in writing. I see blogging as a way of coaching my ability to translate: taking something technical and putting it into words that one other audience can actually carry.
In a field that moves quickly and loves novelty, such translation skill is oddly stable. Models change. Frameworks change (Theano, anybody?). But the power to think clearly and write clearly compounds.
Closing thoughts
Looking back after eight years of “doing ML”, none of those themes transform about a selected machine learning model or a selected trick to training faster.
Relatively, the teachings are about:
- Deep work, which makes progress possible
- Not over-identifying, which makes setbacks survivable
- Movement, which keeps your body from silently degrading
- Blogging, which turns trains your clarity by sharing experience
None of those lessons are strongly tied to machine learning.
But they’re those that keep showing up – and stayed with me during the last years of machine learning.
References
* The deep learning Nature article from LeCun, Bengio, and Hinton: https://www.nature.com/articles/nature14539; the annotated reference section is itself price a read.
** See a quite accessible digest by the American Psychological Association at https://www.apa.org/topics/research/multitasking.
*** Daniel Bourke’s homepage together with his posts on machine learning: https://www.mrdbourke.com/tag/machine-learning.
