Dr. Patrick M. Pilarski is a Canada CIFAR Artificial Intelligence Chair, past Canada Research Chair in Machine Intelligence for Rehabilitation, and an Associate Professor within the Division of Physical Medicine and Rehabilitation, Department of Medicine, University of Alberta.

In 2017, Dr. Pilarski co-founded DeepMind’s first international research office, positioned in Edmonton, Alberta, where he served as office co-lead and a Senior Staff Research Scientist until 2023. He’s a Fellow and Board of Directors member with the Alberta Machine Intelligence Institute (Amii), co-leads the Bionic Limbs for Improved Natural Control (BLINC) Laboratory, and is a principal investigator with the Reinforcement Learning and Artificial Intelligence Laboratory (RLAI) and the Sensory Motor Adaptive Rehabilitation Technology (SMART) Network on the University of Alberta.

Dr. Pilarski is the award-winning writer or co-author of greater than 120 peer-reviewed articles, a Senior Member of the IEEE, and has been supported by provincial, national, and international research grants.

We sat down for an interview on the annual 2023 Upper Certain conference on AI that’s held in Edmonton, AB and hosted by Amii (Alberta Machine Intelligence Institute).

How did you end up in AI? What attracted you to the industry?

Those are two separate questions.  By way of what attracts me to AI, there’s something beautiful about how complexity can emerge and the way structure can emerge out of complexity. Intelligence is just one among these amazing examples of that, so whether it’s coming from biology or whether it’s coming from how we see elaborate behavior emerge in machines, I believe there’s something beautiful about that. That is at all times fascinated me for a really very long time, and my very long winding trajectory to work in the world of AI I work in now, which is machines that learn through trial and error, reinforcement systems that interact with humans while they’re each immersed in it, the stream of experience, flow of time, got here through all kinds of different kind of plateaus. I studied how machines and humans could interact by way of biomechatronic devices and biotechnology, things like artificial limbs and prosthesis.

I checked out how AI will be used to support medical diagnostics, how we will use machine intelligence to begin to know patterns that result in disease or how different disease might present by way of recordings on a machine. But that is all a part of this long-winded drive to essentially start to understand how you would possibly have the opportunity to get very complex behaviors out of quite simple foundations. And that is what I actually love, especially about reinforcement learning, is the concept the machine can embed itself throughout the flow of time and learn from its own experience to exhibit very complex behaviors and capture each the complex phenomenon’s, really, on the earth around it. That is been a driving force.

The mechanics of it, I actually did a variety of sports medicine training and things like that back in highschool. I studied sports medicine and now here I’m working in a environment where I take a look at how machine intelligence and rehabilitation technologies come together to support people of their day by day life. It’s a really interesting journey, just like the side fascination with complex systems and complexity, after which very practical pragmatics of how can we begin to take into consideration how humans will be higher supported, live lives they need to live.

How did sports initially lead you to prosthetics?

What’s really interesting about fields like sports medicine is the human body and the way someone’s unique needs, whether it’s sporting or otherwise, can in truth be supported by other people, by procedures and processes. The bionic limbs and prosthetic technologies are about constructing devices, constructing systems, constructing technology that helps people live the lives that they need to live. These two things are really tightly connected. It’s actually really exciting to have the opportunity to come back full circle and have a few of those much earlier interests come to fruition in, again, co-leading a lab where we take a look at… And particularly machine learning systems that work with in a tightly coupled way, the individual that they’re designed to support.

You’ve previous discussed how a prosthetic adapts to the person as a substitute of the person adapting to the prosthetics. Could you talk concerning the machine learning behind this?

Absolutely. As a foundation within the history of tool use, humans have adapted ourselves to our tools after which we have adapted our tools to the needs that we now have. And so there’s this iterative means of us adapting to our tools. We’re, at once, at an inflection point where for the primary time, you have possibly heard me say this before in talks in the event you’ve checked out among the talks that I’ve given. But really, we’re at this essential point in history where we will now imagine constructing tools that herald a few of those hallmarks of human intelligence. Tools that can actually adapt and improve while they’re getting used by an individual. The underlying technologies support continual learning. Systems that may continually learn from an ongoing stream experience. On this case, reinforcement learning and the mechanisms that underpin it, things like temporal difference learning, are really critical to constructing systems that may continually adapt while they’re interacting with an individual and while they’re in use by an individual supporting them of their day by day life.

Could you define temporal difference learning?

Absolutely, what I actually like about that is that we will think concerning the core technologies, temporal difference learning and the elemental prediction learning algorithms that underpin much of what we work on the lab. You’ve a system that, very similar to we do, is making a prediction about what the long run goes to appear like with respect to some signal, with respect to something like the long run reward is what we often see. But some other signal you would possibly imagine like, how much force am I exerting at once? How hot is it going to be? What number of donuts am I going to have tomorrow? These are the possible things that you just may think predicting. And so the core algorithm is basically the difference between my guess about what is going on to occur at once and my guess about what is going on to occur in the long run together with any sort of signal that I’m currently receiving.

How much force am I exerting as a robot arm is lifting up a cup of coffee or a cup of water? This is perhaps the difference between the prediction concerning the amount of force it’s going to be exerting at once or the quantity it’s going to over some period of the long run. After which comparing that to its expectations concerning the future and the force it’s actually exerting. Put those all together, and also you get this error, the temporal difference error. It is that this nice accumulation of the temporally prolonged forecast in the long run and the differences between them, which you may then use to update the structure of the training machine itself.

And so this, again, for conventional reinforcement learning based on reward, this could possibly be updating the way in which the machine acts based on the long run expected reward you would possibly perceive. For a variety of what we do, it’s different kinds of signals, using generalized value functions, which is the variation of the reinforcement learning process, temporal difference learning of reward signals to any sort of signal of interest that is perhaps applicable to the operation of the machine.

You regularly discuss a prosthetic called the Cairo Toe in your presentations. What does it need to teach us?

I like using the instance of the Cairo Toe, a 3000-year-old prosthesis. I work in the world of neuro prosthetics, we now see very advanced robotic systems that may in some cases have the identical level of control or the degrees of control as biological body parts. And yet, I am going back to a really stylized wood toe from 3000 years ago. I believe what’s neat is it’s an example of humans extending themselves with technology. That’s what we’re seeing at once by way of neuro prosthetics and human machine interaction will not be something that’s weird, recent or wacky. We have now at all times been tool users, animals, non-human animals also use tools. There’s many great books on this, especially by Frans de Waal, “Are We Smart Enough to Know How Smart Animals Are?”.

This extension of ourselves, the augmentation and enhancement of ourselves through using tools will not be something recent, it’s something ancient. It’s something that has been happening since time and memorial within the very land that we’re on at once by the individuals who lived here. The opposite interesting thing concerning the Cairo Toe is that the evidence, a minimum of from the scholarly reports on it, show that it was adapted multiple times over the course of its interactions with its users. They really went in and customised it and altered it, modified it during its use.

My understanding, it was not only a set tool that was attached to an individual during their lifetime, it was a set tool that was attached but in addition modified. It’s an example of how, again, the concept tools are adapted during their span of use and a sustained span of use is definitely something that can be quite ancient. It isn’t something recent, and there is numerous lessons we will learn from the co-adaptation of individuals and tools over many, a few years.

You’ve previously mentioned the feedback pathway between prosthetics and the human, could you elaborate on feedback pathway?

We’re also in a special time by way of how we’re viewing the connection between an individual and the machine that goals to support them of their day by day life. When someone is using a synthetic limb, as an example someone with limb difference, someone with an amputation is using a synthetic limb. Traditionally, they might be using it very very similar to a tool, like an extension of their body, but we’ll see them largely counting on what we consider the control pathway. That some sense of their wheel or their intent is being passed right down to that device, which is then tasked with determining what it’s, after which executing upon that, whether it’s opening and shutting a hand or bending an elbow or making a pinch grip to grab a key. We regularly don’t see people studying or considering the feedback pathway.

So numerous artificial limbs that you just might see deployed commercially, the pathway of data flowing from the device back to the person is perhaps the mechanical coupling, the way in which that they really feel the forces of the limb and act upon them. It is perhaps them hearing the worrying of the motors or them watching as they pick up a cuff and move it across a desk or they grab it from one other a part of their workspace. And so, those pathways are the normal way of doing it. There are amazing things which might be happening across the globe to take a look at how information is perhaps higher fed back from a artificial limb to the person using it. Especially even here in Edmonton, there’s a variety of really cool work using the rewiring of the nervous system, targeted nerve renovation and other things to support that pathway. However it remains to be a extremely popular emerging area of study to take into consideration how machine learning supports the interactions with respect to that feedback pathway.

How machine learning can support a system that is perhaps perceiving and predicting loads about its world actually transmit, having that information transmitted clearly and effectively back to the person using it. How can machine learning support that? I believe that is an excellent topic, because if you will have each that feedback pathway and that control pathway, each pathways are adapting and each the device getting used by the person and the person themself are constructing models of one another. You’ll be able to do something almost miraculous. You’ll be able to almost transmit information totally free. If you will have each these systems which might be actually well attuned to one another, they’ve built a really powerful model of one another and so they have an adaptation each to regulate the feedback pathways, you may form very tight partnerships between humans and machines that may pass an enormous amount of data with little or no effort and little or no bandwidth.

And that opens up whole recent realms of human machine machine coordination, especially in the world of neuroprosthetics. I’m really think this can be a pretty miraculous time for us to begin studying this area.

Do you think that these are going to be 3D printed in the long run or how do you think that the manufacturing will proceed?

I do not feel like I’m the perfect place to take a position on how which may occur. I can say though, that we’re seeing a big uptick in industrial providers of neuroprosthetic devices using additive manufacturing, 3D printing, and other types of additive on the spot manufacturing to create their devices. This can be really neat to see, that it isn’t only a prototype using additive manufacturing or 3D printing, it’s 3D printing becoming an integral a part of how we offer devices to individuals and the way we optimize those devices to the precise those that are using them.

Additive manufacturing or bespoke manufacturing, customized prosthesis fitting happens in hospitals on a regular basis. This can be a natural a part of care provision to individuals with limb difference who need assisted technologies or other sort of rehabilitation technologies. I believe we’re beginning to see that a variety of that customization is beginning to mix into the manufacturers of the devices, and never just left to the purpose of care providers. And that is also really exciting. I believe there’s an excellent opportunity for devices that do not just appear like hands or are used hands, but devices that very precisely meet the needs of the person using them, that enables them to precise themselves in the way in which that they need to precise themselves, and lets them live lives that they need to live the way in which they need to live it, not only the way in which we expect a hand needs to be utilized in day by day life.

You’ve written over 120 papers. Is there one which stands out to you that we must always learn about?

There is a recently published paper in neural computing applications, but it surely represents the tip of an iceberg of considering that we have recommend for well over a decade now, on frameworks for the way humans and machines interact, especially how humans and prosthetic device interact. It’s the concept of communicative capital. And so that is the paper that we recently published.

And this paper lays forward our view on how predictions which might be learned and maintained in real time by a, say, prosthetic device interacting with the person, the person themself can form essentially capital, can form a resource that each of those parties can depend on. Remember, previously I said we will do something really spectacular when we now have a human and a machine which might be each constructing models of one another, adapting the real-time based on experience, and beginning to pass information in a bidirectional channel. As a sidebar, because we live in a magical world where there’s recordings and you may cut things out of it.

It’s essentially like magic.

Exactly. It’s feels like magic. If we return to thinkers like as Ashby, W. Ross Ashby, back within the Nineteen Sixties and his book “Introduction of Cybernetics” talked about how we would amplify the human intellect. And he really said it comes right down to amplifying the flexibility of an individual to make a choice from one among many options. And that is made possible by systems where an individual is interacting with, say, a machine, where there is a channel of communication open between them. So if we now have that channeled communication open, whether it is bidirectional, and if each systems are constructing capital in the shape of predictions and other things, you then can begin to see them really align themselves and to change into greater than the sum of their parts. You’ll be able to get more out than they’re putting in.

And I believe this is the reason I consider this to be one among our most fun papers, since it does represent a thought shift. It represents a thought shift towards considering of neuro prosthetic devices as systems with agency, systems that we won’t just describe agency to, but depend on to have the opportunity to co adapt with us to accumulate these resources. The communicative capital that lets us multiply our ability to interact with the world, lets us get more out than we’re putting in and permit people to, I will say, from a prosthetic lens, stop fascinated with the prosthesis of their day by day life and begin fascinated with living their day by day life. Not the device that is helping them live their day by day life.

What are among the applications you’d see for brain machine interfaces with what you only discussed?

One in all my favorites is something we recommend, again, over the past almost 10 years, is a technology called adaptive switching. Adaptive switching relies on the knowledge that many systems we interact with on a day by day basis depend on us switching between many modes or functions. Whether I’m switching between apps on my phone or I’m attempting to determine the appropriate setting on my drill or whether I’m adapting other tools in my life, we switch between many modes or functions on a regular basis, considering back to Ashby, our ability to make a choice from many options. So in adaptive switching, we use temporal difference learning to permit a artificial limb to learn what motor function an individual might need to use and after they need to use it. So really quite a straightforward premise is that, just the act of me reaching over to a cup and shutting my hand.

Well, a system should have the opportunity to accumulate predictions through experience that in this case, I’m likely going to be using the hand open close function. I will be opening and shutting my hand. After which in the long run, in similar situations, to have the opportunity to predict that. And after I’m navigating the swirling cloud of modes and functions, give me kind of those that I would like without having to sort through all of those many options. And this can be a quite simple example of increase that communicative capital. You’ve a system that’s in truth increase predictions through interaction, they’re predictions about that person, that machine, their relationship in that situation at the moment. And that shared resource then allows the system to reconfigure its control interface on the fly, such that the person get what they need and after they want. And really, in a situation where the system could be very, very sure about what motor function an individual might want, it may well in truth just select that for them as they’re getting in.

And the cool thing is, is that the person at all times has the flexibility to say, “Ah, that is what I actually wanted,” And switch to a different motor function. In a robotic arm, that is perhaps different sorts of hand grasps, whether it’s shaping the grip to grab a doorknob or pick up a key or to shake someone’s hand. Those are different modes of functions, different grabs patterns. It is extremely interesting that the system can start to accumulate an appreciation of what is appropriate in what situation. Units of capital that each of those parties can depend on to maneuver more swiftly through the world, and with less cognitive burden, especially within the a part of the unit.