How Python inserts 'self' into methods
Zeeff, Sebastiaan
When someone starts learning about classes in Python, one of the first things they'll come across is "self" in the parameter list of a method. To keep it simple, it's usually explained that Python will automatically pass the current instance as the first argument to the method: "self" will refer to the instance the method was called on. This high-level explanation really helps with keeping the focus on learning the basics of classes, but it also side-steps what is really going on: It makes it sound like process of inserting "self" is something automagical that the language just does for you. In reality, the mechanism behind inserting self isn't magical at all and it's something you can very much play with yourself. In this intermediate level talk, Sebastiaan Zeeff will take you down into the heart of the Python data model to explain how the mechanism behind inserting "self" works. He will talk about the descriptor protocol and how it allows you to modify how attributes are accessed, assigned, or deleted in Python. He hopes that understanding how descriptors work will demystify "self" in Python.
 Hello, my name is Sebastian. I hope you're having fun today. I know I sure am. Today, I'm going to, well, first I'm going to introduce myself, but after I'm done talking about myself, I will be talking about self in Python. And more specifically, I will talk about how Python inserts self into methods. To do that, we need to talk about something called descriptors, and we basically have to dive right into the middle of the Python data model. Don't worry, I'll start slowly and we'll build it up from there. Now, as said, my name is Sebastian. I live with my girlfriend in Reichweig, that's near the Hague in the Netherlands. And in my daily job, I'm a software engineer. I work for Ordina and more specifically for the Python years unit. And that means that I get to play with Python every day. And that's, that's pretty nice. Another big part of my life is Python Discord. I'm one of the three owners of Python Discord. And it's a large online community for Python enthusiasts. We have about 140,000 members. So I'm also really happy that we have about 100 volunteers to help us run the community. Now, if you're not familiar with Python Discord, we mainly focus on Python education, both for beginners and advanced users, but we also organize Python related events and maintain several open source projects. One thing we started doing recently is supporting other organizations in the Python ecosystem. And one specific example is that we help the Python software foundation with organizing the C Python core developers print a few months ago. Well, anyway, I think I've been talking about myself enough. So let's talk about Python and more specifically, let's talk about self in Python. When talking about self in Python, I think it's really important to go back to the basics, because there's a little bit of magic about self in Python, but you don't really see that magic anymore. If you've already been programming in Python for quite a while now, you get used to self, you get used to how it works and what it does for you. That means that you don't really notice anymore that self in Python doesn't really work like similar concepts in other languages. So let's go back to the basics and see what the magic of self is all about. To do that, I wrote a very simple class. It's a guitar class. And that's because I really like guitars. This one is really simple. Basically, if you have a guitar, you can give it a name and then the Dunder init method will assign that name as an attribute of the guitar instance. And if you're unfamiliar with classes, that means that you can do something like this. You can create a guitar, give it a name. In this case, it's Warwick streamer, which is just the name of my favorite bass. And then I assign the name Warwick to that guitar instance. And because we've assigned the name as an attribute of the instance, we can now do something like this, Warwick dot name. So we access the name attribute on this specific guitar. And we see that we get the name right back out. Now, this isn't really interesting. So I'm going to add a little method in there. It's a really simple method. It's a play note method. That's because the actual implementation isn't really important here. But anyway, what you can do with this method is you can play a note with one of the guitar instances you've created. So if I wanted to play a note with my Warwick, I could do Warwick dot play notes. So I'm accessing the I'm calling the method on this specific Warwick guitar. And then I pass in the note. In this case, it's C sharp, which to me is a musical note. It's not a programming language. And then if I hit enter, the message gets printed by Python, my Warwick streamer plays the note C sharp. And it doesn't really look all that exciting. But there is actually a little bit of magic going on here. And in order for us to see that magic, we have to zoom in on the function definition of plain notes. So if you look at the function definition here, and I'm calling it a function definition on purpose, not a math definition, because it's still a function. You can see that this function has two parameters in the parameter list. It has a self parameter, and it has a note parameter. Now, if a function has two parameters, it really needs to get two arguments when you call it. Otherwise, it won't work. You'll get an exception. Now, so this method, this function really needs to get two arguments in order to work. But if you now look at how we've called our method, you can see that we've only provided it with one argument, the musical note. Now, that musical note will end up here with the note parameter. But where's the value for self coming from? There's no value for self here. At least there's no obvious value for self here. So what's going to happen? Well, if you've been programming Python for a while, you probably know that Python is going to give you a value for self. More specifically, what it's going to do is it's going to insert the instance you called the method on. So in this case, we called the play note method on the Warwick base. So it's going to insert the Warwick base as the first argument into the function. So self will be equal to Warwick, to my Warwick base. That's why we could use the name of the Warwick base. But if you think about it, this is not really obvious. This is just something that you have to know that you have to get used to. And then you can start working with it. But there's no indication here of how it works, why it should happen in the first place. And in fact, a lot of beginners are a bit confused by this. But at the end of the day, this is what Python does for you. So it really looks like some kind of magical language feature that you just have to learn. And you can't really do anything like it yourself. It's just something that Python does for you. Well, in fact, that isn't true. Otherwise, this talk would end now. There's actually a well defined way in which this happens. You can actually use that way yourself as well. So that brings me to the central question in this talk. And that is, how does Python insert itself into methods? Now, in order for us to answer this question, we have to look at what happens if you define a function within a class. In part, that's because there's a common myth. And that says that as soon as you define a function within a class, it will turn into a method at definition time. That's not true. But there is something important happening there that that in part will explain what's going what what's actually going on. So let's get back to our guitar class. I've got it here. And now we have to think about what will happen if Python is going to define this guitar class. Now, first of all, Python needs to create this class. It needs to keep track of all the attributes and all the properties of this specific guitar class. And since everything in Python is an object, this class will also be an object. This is a little bit confusing for some people, but classes are in fact objects themselves. And in this case, I've drawn that with this little rectangle here just to keep track of stuff. So this is where it starts now. And now we're going to focus on what happens when Python gets to this section. This is a defined function statement. And we have to think about what happens when Python executes such a defined function statement. Well, from a very high level perspective, the perspective that we need to understand what's going on, Python basically does two things. The first thing is that it has to create that function. Now, since everything in Python is an object, so is a function. So if you create a function, you will actually get a function object floating around somewhere in memory. So this function object is an instance of the class function. And it keeps track of what of everything there is to know about this individual function, like the code that needs to be executed when you call this function, it keeps track of the parameter lists. If there are default values, this function object will track them. So everything there is to know about this function will be contained in this function instance. Now, in Python, we don't really have use for for objects that are just floating around somewhere in memory, we need a way to reach and use them. And that's why Python, we have to use them. And that's why Python will assign a name to the function in the current namespace. Now, that's really important here, because we're defining a class, so that namespace is the current class. And that means that as soon as we try to define a name, it will end up as a class attribute. So I've tried to indicate that here with this little label inside of the class object. And then Python will assign the class attribute to the function. What this means is that we can now use this class attribute guitar dot play note to access the function that's somewhere in memory. So let's demonstrate that if I do guitar dot play note, and I print the result, you'll see that this is just the function object. So function, guitar, the play note, and then some memory address. So importantly, this is still a function. So one of the important conclusions here is that this function object is still a regular function object. It's not it has not magically turned into method. It's not a method yet. Self hasn't been inserted. This is still a regular function. And in fact, if you were to call the function like this, so guitar dot play note, it really needs two arguments. Otherwise, you'll get an exception. Now, another thing to realize here is that it's really important that we've assigned a class attribute to that function object. And that's that that turns out to be quite essential to how methods work in Python. And to understand why that is, we're going to zoom in on that a little bit. So let's zoom in on the fact that we've assigned a class attribute to that function. And what we're specifically going to do, we're going to look at what happens if we now try to use this function as a as a method on an instance. So let's recreate my Warwick guitar. So this is the same thing as we did before, Warwick is equal to guitar with the name Warwick Streamer. And then think about what will happen if we try to use that play note attribute, that class attribute directly on this specific Warwick guitar. Now, why is it, why is this a little bit odd? Well, if you think about it, play note is a class attribute, it's an attribute of the guitar class. It's not an attribute of this specific guitar of this specific Warwick Streamer guitar. So why would this work? Warwick doesn't even have a play note attribute. It has a name attribute, it has some default attributes, but it doesn't have a play note attribute, because that belongs to the guitar class. Well, in fact, as it turns out, if you access such an attribute on an instance, Python doesn't only look at that instance, at that specific object, but it will look at the class of that object as well. So in this case, if we try to access play note, that's not an attribute on Warwick, but it is an attribute on the guitar class. So Python will still return a value for you. And we can show that, we can hit enter. What we see is that we indeed get a value back. But hey, that's odd. It's not a function anymore. We get a bound, something called a bound method back. As you can see, it's still based on the function. It's the bound method, guitar, guitar dot play note. But it's not the same function object anymore. So what's going on here? Well, as it turns out, attribute access in in Python isn't as straightforward as you may think. And in fact, you can objects can modify what happens when attributes are accessed. And that is precisely what function object objects do. What they do is whenever they notice that they've been accessed as an instance attribute, instead of a class attribute, they say, hey, we're being called as a method on an instance, we need to bind this specific instance to to ourselves to the function. And so that function object will then create a bound method by inserting that specific guitar already into the function for the self parameter. And then it will return that. And that is what a bound method literally means. It means that we've taken the function, we've bound it to the specific Warwick guitar, we've inserted Warwick already as the first argument. And then we have a bound method and everything we call it with after that will be additional arguments to that instance. So if we now call Warwick dot play note, we only have to provide one argument, because the instance itself has already been bound to the function simply by accessing the class attribute on the Warwick instance. So that's pretty neat. And basically, this is a really abstract explanation of how methods work. But it's a bit too abstract. So let's look at a more concrete example. So and to do that, we need to talk about something called descriptors. Now, what are descriptors? And what do they have to do with attribute access? Basically, descriptor objects can customize attribute lookup. So you're simply looking an attribute up attribute assignments. So trying to assign something to an attribute and attribute deletion when you do something like del object dot dot attribute. And the way they do that is that they can implement special methods from the Python data model, so called Dunder methods. And specifically, if they implement a get method, they can customize attribute lookup. So if we do Warwick dot play note, if the object we're accessing here has a get method that get method can influence or customize what happens when we try to do this. If the object has a set method, we can customize attribute assignments. So whenever we try to do Warwick dot play note is equal to a new value. If the object implements a set method, that set methods can now determine what will happen here. And finally, there's a delete method. And that and it customizes attribute deletion. So whenever you do del Warwick dot play note, instead of just executing this, if the object implements a delete method, that delete method will be called and will do the work instead. And what is the most important thing here? Well, functions are descriptors that implement such a get method. And so that's why we're going to focus on that get method right now by simply implementing one. So let's look at this guitar class again. To make a little bit of room here, I'm going to collapse the methods. So they're still there. You can assume they're still there. I've just made them smaller. And what I'm going to do is I'm going to add a new class attribute. And it's is my favorite. And then I'm going to add a new class attribute. And that is my favorite class attribute will be assigned to a favorite descriptor. Now, what is the idea here? If I create a guitar instance like my Warwick streamer, what is supposed to happen is that is that I can do Warwick is my favorite. And then it should return true or false based on whether or not this is my favorite guitar. And this needs to be taken the descriptor, because the descriptor has to check which name the guitar has. And in this case, because Warwick streamer is my favorite guitar, this should print true. Let's look at how we can do that. In order to do that, we need to write a new class, a favorite descriptor class. Since we're only dealing with attribute access here, and that is determined by a Dunder get method, this class only needs to implement a Dunder get method. It gets two arguments besides self. It's the instance. And that is the instance you're accessing the attribute on. So in the example above, that would be the Warwick guitar. And then there's the owner. And the owner is the owner class. And that has the attribute. So in this case, it would be guitar, because the descriptor is assigned to the guitar dot is my favorite attribute. Now, what do we need to do next? Well, this method, this get method will be called whenever we try to access that dot is my favorite attribute. And now we can check if the instance we're calling it on has Warwick streamer as the name. So I'm going to add an if statement. I'm using get attribute here to get the name attribute out of the instance. The reason I'm not doing instance dot name is really simple. I don't want this to break. In principle, there should not be guitar instances without a name, because the init methods always assigns that name attribute. But it could be that Python will pass none as the argument as the argument for instance. And that happens specifically when we try to access the is my favorite attributes directly on the class. So if you do guitar dot is my favorite, there is no instance yet. So the instance parameter in the get method will get none. And obviously none doesn't have a name attribute. So we need to be a bit careful here. So that's why you get attribute with a non default value. Then I can simply check if the name is equal to the Warwick streamer. If so, we return true. And if not, we return false. Now let's see that in action. So I create a Warwick guitar instance. I then access warwick dot is my favorite. This means that the Dunder get method will get called with Warwick, for instance, and guitar for owner. It then checks the name attribute. In this case, it's equal to Warwick streamer. So it will return true. So the result of accessing this attribute is determined by the descriptors get method. Now let's create another one of my guitars, which is a vendor, a vendor jazz base in this case. Now I'm going to access the vendor dot is my favorite attribute. This will call the Dunder get method again. But now with the vendor base as the instance, it will then get the name of the vendor base, which is not equal to Warwick streamer, which means it will return false. So as you can see, instead of just being a simple attribute is my favorite, it will actually call a method, the Dunder get method and whatever that method computes has as the return value, that will be the return value of the attributes. So now that we've seen a descriptor in action, I want to get back to functions. And in fact, functions also have such a get method. That's a little bit difficult to show you the actual implementation. But we can actually call that get method manually just to see what it does. And that's what I did here. And in order to do that, I need to access the original function objects. And if you remember, we can do that by accessing the function as a class attribute. So that's what I did here. So what I did is a guitar, a plain note that gives me the original function object, then we can call the Dunder get method on the original function objects. Well, a get method expects the instance. In this case, I pass in Warwick, and it expects the owner class. And I pass in guitar. Now, if I hit enter, if everything goes well, we should get a bound method. And indeed, that's what we get. So by calling the get method manually, we can actually bind the instance we pass in to the function to get a bound method. And this is also what happens if you access a method on an instance. Well, that brings us to the end of the talk. I have a short summary for you. And the most important thing of today is that the descriptor protocol allows you to customize how attributes work in Python. And the second thing is that functions implement the descriptor protocol to create bound methods. I don't really have any other important take home messages, except that you can do a lot of cool things with descriptors. There are also a few things that I haven't covered. But there are nonetheless important, I haven't given you any examples with done their set and done their delete, that's some purpose, because they only had 25 minutes. And I also did not explain the difference between data and non data descriptors. There's actually something interesting that happens as soon as you add a set or delete method. But it's kind of complicated. So I did not have time for that today as well. But I do want to recommend you to read the how to descriptor guide in the official documentation. It's written by Raymond Hettinger. It really explains these concepts really well. And another good recommendation is this book. It's fluent Python. It's my favorite Python book. And it also really, it really does a good job at explaining descriptors. It calls it overriding and non overriding descriptors. But it's all the same. Well, finally, I want you to be aware of a special done their method. It's called set name. And it really works well in combination with descriptors. Raymond Hettinger also talks about it in his how to descriptor guide. It's not yet in this version of fluent Python. But I've read that a new version, a new addition will come out in a few months. Well, that's it for me. Thanks for listening. I'll be here for the Q&A, so you can ask me questions and hopefully until until next year.
 Hello, my name is Sebastian. I hope you're having fun today. I know I sure am. Today,  I'm going to, well, first I'm going to introduce myself, but after I'm done talking about myself,  I will be talking about self in Python. And more specifically, I will talk about how Python  inserts self into methods. To do that, we need to talk about something called descriptors,  and we basically have to dive right into the middle of the Python data model.  Don't worry, I'll start slowly and we'll build it up from there. Now, as said, my name is Sebastian.  I live with my girlfriend in Reichweig, that's near the Hague in the Netherlands.  And in my daily job, I'm a software engineer. I work for Ordina and more specifically for the  Python years unit. And that means that I get to play with Python every day. And that's,  that's pretty nice. Another big part of my life is Python Discord. I'm one of the three owners  of Python Discord. And it's a large online community for Python enthusiasts. We have about 140,000  members. So I'm also really happy that we have about 100 volunteers to help us run the community.  Now, if you're not familiar with Python Discord, we mainly focus on Python education,  both for beginners and advanced users, but we also organize Python related events and maintain  several open source projects. One thing we started doing recently is supporting other  organizations in the Python ecosystem. And one specific example is that we help the Python  software foundation with organizing the C Python core developers print a few months ago. Well,  anyway, I think I've been talking about myself enough. So let's talk about Python and more  specifically, let's talk about self in Python. When talking about self in Python, I think it's  really important to go back to the basics, because there's a little bit of magic about  self in Python, but you don't really see that magic anymore. If you've already been programming  in Python for quite a while now, you get used to self, you get used to how it works and what it does  for you. That means that you don't really notice anymore that self in Python doesn't really work  like similar concepts in other languages. So let's go back to the basics and see what the  magic of self is all about. To do that, I wrote a very simple class. It's a guitar class. And  that's because I really like guitars. This one is really simple. Basically, if you have a guitar,  you can give it a name and then the Dunder init method will assign that name as an attribute  of the guitar instance. And if you're unfamiliar with classes, that means that you can do something  like this. You can create a guitar, give it a name. In this case, it's Warwick streamer,  which is just the name of my favorite bass. And then I assign the name Warwick to that guitar  instance. And because we've assigned the name as an attribute of the instance, we can now do  something like this, Warwick dot name. So we access the name attribute on this specific guitar.  And we see that we get the name right back out. Now, this isn't really interesting. So I'm going  to add a little method in there. It's a really simple method. It's a play note method. That's  because the actual implementation isn't really important here. But anyway, what you can do  with this method is you can play a note with one of the guitar instances you've created.  So if I wanted to play a note with my Warwick, I could do Warwick dot play notes. So I'm accessing  the I'm calling the method on this specific Warwick guitar. And then I pass in the note. In this case,  it's C sharp, which to me is a musical note. It's not a programming language. And then if I hit enter,  the message gets printed by Python, my Warwick streamer plays the note C sharp.  And it doesn't really look all that exciting. But there is actually a little bit of magic going on  here. And in order for us to see that magic, we have to zoom in on the function definition of  plain notes. So if you look at the function definition here, and I'm calling it a function  definition on purpose, not a math definition, because it's still a function. You can see that  this function has two parameters in the parameter list. It has a self parameter,  and it has a note parameter. Now, if a function has two parameters, it really needs to get two  arguments when you call it. Otherwise, it won't work. You'll get an exception.  Now, so this method, this function really needs to get two arguments in order to work.  But if you now look at how we've called our method, you can see that we've only provided it with  one argument, the musical note. Now, that musical note will end up here with the note parameter.  But where's the value for self coming from? There's no value for self here. At least there's no  obvious value for self here. So what's going to happen? Well, if you've been programming Python  for a while, you probably know that Python is going to give you a value for self. More specifically,  what it's going to do is it's going to insert the instance you called the method on. So in this case,  we called the play note method on the Warwick base. So it's going to insert the Warwick base  as the first argument into the function. So self will be equal to Warwick, to my Warwick base.  That's why we could use the name of the Warwick base. But if you think about it,  this is not really obvious. This is just something that you have to know that you have to get used  to. And then you can start working with it. But there's no indication here of how it works,  why it should happen in the first place. And in fact, a lot of beginners are a bit confused by this.  But at the end of the day, this is what Python does for you. So it really looks like some kind of  magical language feature that you just have to learn. And you can't really do anything like it  yourself. It's just something that Python does for you. Well, in fact, that isn't true. Otherwise,  this talk would end now. There's actually a well defined way in which this happens. You can actually  use that way yourself as well. So that brings me to the central question in this talk. And that is,  how does Python insert itself into methods? Now, in order for us to answer this question,  we have to look at what happens if you define a function within a class. In part, that's because  there's a common myth. And that says that as soon as you define a function within a class,  it will turn into a method at definition time. That's not true. But there is something important  happening there that that in part will explain what's going what what's actually going on.  So let's get back to our guitar class. I've got it here. And now we have to think about what will  happen if Python is going to define this guitar class. Now, first of all, Python needs to create  this class. It needs to keep track of all the attributes and all the properties of this specific  guitar class. And since everything in Python is an object, this class will also be an object.  This is a little bit confusing for some people, but classes are in fact objects themselves.  And in this case, I've drawn that with this little rectangle here just to keep track of stuff.  So this is where it starts now. And now we're going to focus on what happens when Python gets to this  section. This is a defined function statement. And we have to think about what happens when Python  executes such a defined function statement. Well, from a very high level perspective,  the perspective that we need to understand what's going on, Python basically does two things.  The first thing is that it has to create that function. Now, since everything in Python is an  object, so is a function. So if you create a function, you will actually get a function object  floating around somewhere in memory. So this function object is an instance of the class function.  And it keeps track of what of everything there is to know about this individual function,  like the code that needs to be executed when you call this function, it keeps track of the  parameter lists. If there are default values, this function object will track them.  So everything there is to know about this function will be contained in this function instance.  Now, in Python, we don't really have use for for objects that are just floating around somewhere  in memory, we need a way to reach and use them. And that's why Python, we have to use  them. And that's why Python will assign a name to the function in the current namespace. Now,  that's really important here, because we're defining a class, so that namespace is the current class.  And that means that as soon as we try to define a name, it will end up as a class attribute.  So I've tried to indicate that here with this little label inside of the class object.  And then Python will assign the class attribute to the function. What this means is that we can  now use this class attribute guitar dot play note to access the function that's somewhere in memory.  So let's demonstrate that if I do guitar dot play note, and I print the result, you'll see that  this is just the function object. So function, guitar, the play note, and then some memory address.  So importantly, this is still a function. So one of the important conclusions here is that this  function object is still a regular function object. It's not it has not magically turned into method.  It's not a method yet. Self hasn't been inserted. This is still a regular function. And in fact,  if you were to call the function like this, so guitar dot play note, it really needs two arguments.  Otherwise, you'll get an exception. Now, another thing to realize here is that it's really important  that we've assigned a class attribute to that function object. And that's that that turns out to be  quite essential to how methods work in Python. And to understand why that is, we're going to zoom  in on that a little bit. So let's zoom in on the fact that we've assigned a class attribute to  that function. And what we're specifically going to do, we're going to look at what happens if we  now try to use this function as a as a method on an instance. So let's recreate my Warwick guitar.  So this is the same thing as we did before, Warwick is equal to guitar with the name Warwick  Streamer. And then think about what will happen if we try to use that play note attribute,  that class attribute directly on this specific Warwick guitar. Now, why is it, why is this a  little bit odd? Well, if you think about it, play note is a class attribute, it's an attribute  of the guitar class. It's not an attribute of this specific guitar of this specific Warwick  Streamer guitar. So why would this work? Warwick doesn't even have a play note attribute. It has  a name attribute, it has some default attributes, but it doesn't have a play note attribute, because  that belongs to the guitar class. Well, in fact, as it turns out, if you access such an attribute  on an instance, Python doesn't only look at that instance, at that specific object, but it will  look at the class of that object as well. So in this case, if we try to access play note, that's  not an attribute on Warwick, but it is an attribute on the guitar class. So Python will still return  a value for you. And we can show that, we can hit enter. What we see is that we indeed get a value  back. But hey, that's odd. It's not a function anymore. We get a bound, something called a bound  method back. As you can see, it's still based on the function. It's the bound method, guitar,  guitar dot play note. But it's not the same function object anymore. So what's going on here?  Well, as it turns out, attribute access in in Python isn't as straightforward as you may think.  And in fact, you can objects can modify what happens when attributes are accessed. And that is  precisely what function object objects do. What they do is whenever they notice that they've been  accessed as an instance attribute, instead of a class attribute, they say, hey, we're being called  as a method on an instance, we need to bind this specific instance to to ourselves to the function.  And so that function object will then create a bound method by inserting that specific guitar  already into the function for the self parameter. And then it will return that. And that is what a  bound method literally means. It means that we've taken the function, we've bound it to the specific  Warwick guitar, we've inserted Warwick already as the first argument. And then we have a bound  method and everything we call it with after that will be additional arguments to that instance.  So if we now call Warwick dot play note, we only have to provide one argument, because the instance  itself has already been bound to the function simply by accessing the class attribute on the  Warwick instance. So that's pretty neat. And basically, this is a really abstract explanation  of how methods work. But it's a bit too abstract. So let's look at a more concrete example.  So and to do that, we need to talk about something called descriptors.  Now, what are descriptors? And what do they have to do with attribute access?  Basically, descriptor objects can customize attribute lookup. So you're simply looking an  attribute up attribute assignments. So trying to assign something to an attribute and attribute  deletion when you do something like del object dot dot attribute. And the way they do that is that  they can implement special methods from the Python data model, so called Dunder methods.  And specifically, if they implement a get method, they can customize attribute lookup. So if we do  Warwick dot play note, if the object we're accessing here has a get method that get method can  influence or customize what happens when we try to do this. If the object has a set method,  we can customize attribute assignments. So whenever we try to do Warwick dot play note  is equal to a new value. If the object implements a set method, that set methods can now  determine what will happen here. And finally, there's a delete method. And that and it  customizes attribute deletion. So whenever you do del Warwick dot play note, instead of just  executing this, if the object implements a delete method, that delete method will be called and  will do the work instead. And what is the most important thing here? Well, functions are descriptors  that implement such a get method. And so that's why we're going to focus on  that get method right now by simply implementing one. So let's look at this guitar class again.  To make a little bit of room here, I'm going to collapse the methods. So they're still there.  You can assume they're still there. I've just made them smaller. And what I'm going to do  is I'm going to add a new class attribute. And it's is my favorite. And then I'm going to add  a new class attribute. And that is my favorite class attribute will be assigned to a favorite  descriptor. Now, what is the idea here? If I create a guitar instance like my Warwick streamer,  what is supposed to happen is that is that I can do Warwick is my favorite. And then it should  return true or false based on whether or not this is my favorite guitar. And this needs to be taken  the descriptor, because the descriptor has to check which name the guitar has. And in this case,  because Warwick streamer is my favorite guitar, this should print true.  Let's look at how we can do that. In order to do that, we need to write a new class,  a favorite descriptor class. Since we're only dealing with attribute access here,  and that is determined by a Dunder get method, this class only needs to implement a Dunder  get method. It gets two arguments besides self. It's the instance. And that is the instance you're  accessing the attribute on. So in the example above, that would be the Warwick guitar. And then  there's the owner. And the owner is the owner class. And that has the attribute. So in this case,  it would be guitar, because the descriptor is assigned to the guitar dot is my favorite attribute.  Now, what do we need to do next? Well, this method, this get method will be called whenever we try  to access that dot is my favorite attribute. And now we can check if the instance we're calling it on  has Warwick streamer as the name. So I'm going to add an if statement. I'm using get attribute here  to get the name attribute out of the instance. The reason I'm not doing instance dot name is  really simple. I don't want this to break. In principle, there should not be guitar instances  without a name, because the init methods always assigns that name attribute. But it could be  that Python will pass none as the argument as the argument for instance. And that happens specifically  when we try to access the is my favorite attributes directly on the class. So if you do guitar dot is  my favorite, there is no instance yet. So the instance parameter in the get method will get none.  And obviously none doesn't have a name attribute. So we need to be a bit careful here.  So that's why you get attribute with a non default value. Then I can simply check if the name is  equal to the Warwick streamer. If so, we return true. And if not, we return false. Now let's see  that in action. So I create a Warwick guitar instance. I then access warwick dot is my favorite.  This means that the Dunder get method will get called with Warwick, for instance, and guitar for  owner. It then checks the name attribute. In this case, it's equal to Warwick streamer. So it will  return true. So the result of accessing this attribute is determined by the descriptors get  method. Now let's create another one of my guitars, which is a vendor, a vendor jazz base in this case.  Now I'm going to access the vendor dot is my favorite attribute. This will call the Dunder get  method again. But now with the vendor base as the instance, it will then get the name of the  vendor base, which is not equal to Warwick streamer, which means it will return false. So as you can  see, instead of just being a simple attribute is my favorite, it will actually call a method,  the Dunder get method and whatever that method computes has as the return value, that will be  the return value of the attributes. So now that we've seen a descriptor in action, I want to get back  to functions. And in fact, functions also have such a get method. That's a little bit difficult to  show you the actual implementation. But we can actually call that get method manually just to  see what it does. And that's what I did here. And in order to do that, I need to access the original  function objects. And if you remember, we can do that by accessing the function as a class attribute.  So that's what I did here. So what I did is a guitar, a plain note that gives me the original  function object, then we can call the Dunder get method on the original function objects. Well,  a get method expects the instance. In this case, I pass in Warwick, and it expects the owner class.  And I pass in guitar. Now, if I hit enter, if everything goes well, we should get a bound method.  And indeed, that's what we get. So by calling the get method manually, we can actually bind the  instance we pass in to the function to get a bound method. And this is also what happens if you access  a method on an instance. Well, that brings us to the end of the talk. I have a short summary for you.  And the most important thing of today is that the descriptor protocol allows you to customize how  attributes work in Python. And the second thing is that functions implement the descriptor protocol  to create bound methods. I don't really have any other important take home messages, except that  you can do a lot of cool things with descriptors. There are also a few things that I haven't covered.  But there are nonetheless important, I haven't given you any examples with done their set and done  their delete, that's some purpose, because they only had 25 minutes. And I also did not explain  the difference between data and non data descriptors. There's actually something interesting that happens  as soon as you add a set or delete method. But it's kind of complicated. So I did not have time  for that today as well. But I do want to recommend you to read the how to descriptor guide in the  official documentation. It's written by Raymond Hettinger. It really explains these concepts  really well. And another good recommendation is this book. It's fluent Python. It's my favorite  Python book. And it also really, it really does a good job at explaining descriptors. It calls it  overriding and non overriding descriptors. But it's all the same. Well, finally, I want you to be  aware of a special done their method. It's called set name. And it really works well in combination  with descriptors. Raymond Hettinger also talks about it in his how to descriptor guide. It's not  yet in this version of fluent Python. But I've read that a new version, a new addition will come  out in a few months. Well, that's it for me. Thanks for listening. I'll be here for the Q&A,  so you can ask me questions and hopefully until until next year.
python_self_0_179.png python_self_180_239.png python_self_240_563.png python_self_564_1067.png python_self_1068_1199.png python_self_1200_1343.png python_self_1344_1403.png python_self_1404_1703.png python_self_1704_3131.png python_self_3132_10595.png python_self_10596_11051.png python_self_11052_11483.png python_self_11484_11579.png python_self_11580_11915.png python_self_11916_12095.png python_self_12096_12155.png python_self_12156_17135.png python_self_17136_17315.png python_self_17316_22955.png python_self_22956_23099.png python_self_23100_23195.png python_self_23196_23579.png python_self_23580_23627.png python_self_23628_23831.png python_self_23832_23915.png python_self_23916_24443.png python_self_24444_25463.png python_self_25464_25883.png python_self_25884_26423.png python_self_26424_27503.png python_self_27504_32807.png python_self_32808_34739.png python_self_34740_34823.png python_self_34824_35231.png python_self_35232_35363.png python_self_35364_37403.png python_self_37404_37631.png python_self_37632_37907.png python_self_37908_38025.png
Hello, my name is Sebastian. I hope you're having fun today. I know I sure am. Today, I'm going to, well, first I'm going to introduce myself, but after I'm done talking about myself, I will be talking about self in Python. And more specifically, I will talk about how Python inserts self into methods. To do that, we need to talk about something called descriptors, and we basically have to dive right into the middle of the Python data model. Don't worry, I'll start slowly and we'll build it up from there. Now, as said, my name is Sebastian. I live with my girlfriend in Reichweig, that's near the Hague in the Netherlands. And in my daily job, I'm a software engineer. I work for Ordina and more specifically for the Python years unit. And that means that I get to play with Python every day. And that's, that's pretty nice. Another big part of my life is Python Discord. I'm one of the three owners of Python Discord. And it's a large online community for Python enthusiasts. We have about 140,000 members. So I'm also really happy that we have about 100 volunteers to help us run the community. Now, if you're not familiar with Python Discord, we mainly focus on Python education, both for beginners and advanced users, but we also organize Python related events and maintain several open source projects. One thing we started doing recently is supporting other organizations in the Python ecosystem. And one specific example is that we help the Python software foundation with organizing the C Python core developers print a few months ago. Well, anyway, I think I've been talking about myself enough. So let's talk about Python and more specifically, let's talk about self in Python. When talking about self in Python, I think it's really important to go back to the basics, because there's a little bit of magic about self in Python, but you don't really see that magic anymore. If you've already been programming in Python for quite a while now, you get used to self, you get used to how it works and what it does for you. That means that you don't really notice anymore that self in Python doesn't really work like similar concepts in other languages. So let's go back to the basics and see what the magic of self is all about. To do that, I wrote a very simple class. It's a guitar class. And that's because I really like guitars. This one is really simple. Basically, if you have a guitar, you can give it a name and then the Dunder init method will assign that name as an attribute of the guitar instance. And if you're unfamiliar with classes, that means that you can do something like this. You can create a guitar, give it a name. In this case, it's Warwick streamer, which is just the name of my favorite bass. And then I assign the name Warwick to that guitar instance. And because we've assigned the name as an attribute of the instance, we can now do something like this, Warwick dot name. So we access the name attribute on this specific guitar. And we see that we get the name right back out. Now, this isn't really interesting. So I'm going to add a little method in there. It's a really simple method. It's a play note method. That's because the actual implementation isn't really important here. But anyway, what you can do with this method is you can play a note with one of the guitar instances you've created. So if I wanted to play a note with my Warwick, I could do Warwick dot play notes. So I'm accessing the I'm calling the method on this specific Warwick guitar. And then I pass in the note. In this case, it's C sharp, which to me is a musical note. It's not a programming language. And then if I hit enter, the message gets printed by Python, my Warwick streamer plays the note C sharp. And it doesn't really look all that exciting. But there is actually a little bit of magic going on here. And in order for us to see that magic, we have to zoom in on the function definition of plain notes. So if you look at the function definition here, and I'm calling it a function definition on purpose, not a math definition, because it's still a function. You can see that this function has two parameters in the parameter list. It has a self parameter, and it has a note parameter. Now, if a function has two parameters, it really needs to get two arguments when you call it. Otherwise, it won't work. You'll get an exception. Now, so this method, this function really needs to get two arguments in order to work. But if you now look at how we've called our method, you can see that we've only provided it with one argument, the musical note. Now, that musical note will end up here with the note parameter. But where's the value for self coming from? There's no value for self here. At least there's no obvious value for self here. So what's going to happen? Well, if you've been programming Python for a while, you probably know that Python is going to give you a value for self. More specifically, what it's going to do is it's going to insert the instance you called the method on. So in this case, we called the play note method on the Warwick base. So it's going to insert the Warwick base as the first argument into the function. So self will be equal to Warwick, to my Warwick base. That's why we could use the name of the Warwick base. But if you think about it, this is not really obvious. This is just something that you have to know that you have to get used to. And then you can start working with it. But there's no indication here of how it works, why it should happen in the first place. And in fact, a lot of beginners are a bit confused by this. But at the end of the day, this is what Python does for you. So it really looks like some kind of magical language feature that you just have to learn. And you can't really do anything like it yourself. It's just something that Python does for you. Well, in fact, that isn't true. Otherwise, this talk would end now. There's actually a well defined way in which this happens. You can actually use that way yourself as well. So that brings me to the central question in this talk. And that is, how does Python insert itself into methods? Now, in order for us to answer this question, we have to look at what happens if you define a function within a class. In part, that's because there's a common myth. And that says that as soon as you define a function within a class, it will turn into a method at definition time. That's not true. But there is something important happening there that that in part will explain what's going what what's actually going on. So let's get back to our guitar class. I've got it here. And now we have to think about what will happen if Python is going to define this guitar class. Now, first of all, Python needs to create this class. It needs to keep track of all the attributes and all the properties of this specific guitar class. And since everything in Python is an object, this class will also be an object. This is a little bit confusing for some people, but classes are in fact objects themselves. And in this case, I've drawn that with this little rectangle here just to keep track of stuff. So this is where it starts now. And now we're going to focus on what happens when Python gets to this section. This is a defined function statement. And we have to think about what happens when Python executes such a defined function statement. Well, from a very high level perspective, the perspective that we need to understand what's going on, Python basically does two things. The first thing is that it has to create that function. Now, since everything in Python is an object, so is a function. So if you create a function, you will actually get a function object floating around somewhere in memory. So this function object is an instance of the class function. And it keeps track of what of everything there is to know about this individual function, like the code that needs to be executed when you call this function, it keeps track of the parameter lists. If there are default values, this function object will track them. So everything there is to know about this function will be contained in this function instance. Now, in Python, we don't really have use for for objects that are just floating around somewhere in memory, we need a way to reach and use them. And that's why Python, we have to use them. And that's why Python will assign a name to the function in the current namespace. Now, that's really important here, because we're defining a class, so that namespace is the current class. And that means that as soon as we try to define a name, it will end up as a class attribute. So I've tried to indicate that here with this little label inside of the class object. And then Python will assign the class attribute to the function. What this means is that we can now use this class attribute guitar dot play note to access the function that's somewhere in memory. So let's demonstrate that if I do guitar dot play note, and I print the result, you'll see that this is just the function object. So function, guitar, the play note, and then some memory address. So importantly, this is still a function. So one of the important conclusions here is that this function object is still a regular function object. It's not it has not magically turned into method. It's not a method yet. Self hasn't been inserted. This is still a regular function. And in fact, if you were to call the function like this, so guitar dot play note, it really needs two arguments. Otherwise, you'll get an exception. Now, another thing to realize here is that it's really important that we've assigned a class attribute to that function object. And that's that that turns out to be quite essential to how methods work in Python. And to understand why that is, we're going to zoom in on that a little bit. So let's zoom in on the fact that we've assigned a class attribute to that function. And what we're specifically going to do, we're going to look at what happens if we now try to use this function as a as a method on an instance. So let's recreate my Warwick guitar. So this is the same thing as we did before, Warwick is equal to guitar with the name Warwick Streamer. And then think about what will happen if we try to use that play note attribute, that class attribute directly on this specific Warwick guitar. Now, why is it, why is this a little bit odd? Well, if you think about it, play note is a class attribute, it's an attribute of the guitar class. It's not an attribute of this specific guitar of this specific Warwick Streamer guitar. So why would this work? Warwick doesn't even have a play note attribute. It has a name attribute, it has some default attributes, but it doesn't have a play note attribute, because that belongs to the guitar class. Well, in fact, as it turns out, if you access such an attribute on an instance, Python doesn't only look at that instance, at that specific object, but it will look at the class of that object as well. So in this case, if we try to access play note, that's not an attribute on Warwick, but it is an attribute on the guitar class. So Python will still return a value for you. And we can show that, we can hit enter. What we see is that we indeed get a value back. But hey, that's odd. It's not a function anymore. We get a bound, something called a bound method back. As you can see, it's still based on the function. It's the bound method, guitar, guitar dot play note. But it's not the same function object anymore. So what's going on here? Well, as it turns out, attribute access in in Python isn't as straightforward as you may think. And in fact, you can objects can modify what happens when attributes are accessed. And that is precisely what function object objects do. What they do is whenever they notice that they've been accessed as an instance attribute, instead of a class attribute, they say, hey, we're being called as a method on an instance, we need to bind this specific instance to to ourselves to the function. And so that function object will then create a bound method by inserting that specific guitar already into the function for the self parameter. And then it will return that. And that is what a bound method literally means. It means that we've taken the function, we've bound it to the specific Warwick guitar, we've inserted Warwick already as the first argument. And then we have a bound method and everything we call it with after that will be additional arguments to that instance. So if we now call Warwick dot play note, we only have to provide one argument, because the instance itself has already been bound to the function simply by accessing the class attribute on the Warwick instance. So that's pretty neat. And basically, this is a really abstract explanation of how methods work. But it's a bit too abstract. So let's look at a more concrete example. So and to do that, we need to talk about something called descriptors. Now, what are descriptors? And what do they have to do with attribute access? Basically, descriptor objects can customize attribute lookup. So you're simply looking an attribute up attribute assignments. So trying to assign something to an attribute and attribute deletion when you do something like del object dot dot attribute. And the way they do that is that they can implement special methods from the Python data model, so called Dunder methods. And specifically, if they implement a get method, they can customize attribute lookup. So if we do Warwick dot play note, if the object we're accessing here has a get method that get method can influence or customize what happens when we try to do this. If the object has a set method, we can customize attribute assignments. So whenever we try to do Warwick dot play note is equal to a new value. If the object implements a set method, that set methods can now determine what will happen here. And finally, there's a delete method. And that and it customizes attribute deletion. So whenever you do del Warwick dot play note, instead of just executing this, if the object implements a delete method, that delete method will be called and will do the work instead. And what is the most important thing here? Well, functions are descriptors that implement such a get method. And so that's why we're going to focus on that get method right now by simply implementing one. So let's look at this guitar class again. To make a little bit of room here, I'm going to collapse the methods. So they're still there. You can assume they're still there. I've just made them smaller. And what I'm going to do is I'm going to add a new class attribute. And it's is my favorite. And then I'm going to add a new class attribute. And that is my favorite class attribute will be assigned to a favorite descriptor. Now, what is the idea here? If I create a guitar instance like my Warwick streamer, what is supposed to happen is that is that I can do Warwick is my favorite. And then it should return true or false based on whether or not this is my favorite guitar. And this needs to be taken the descriptor, because the descriptor has to check which name the guitar has. And in this case, because Warwick streamer is my favorite guitar, this should print true. Let's look at how we can do that. In order to do that, we need to write a new class, a favorite descriptor class. Since we're only dealing with attribute access here, and that is determined by a Dunder get method, this class only needs to implement a Dunder get method. It gets two arguments besides self. It's the instance. And that is the instance you're accessing the attribute on. So in the example above, that would be the Warwick guitar. And then there's the owner. And the owner is the owner class. And that has the attribute. So in this case, it would be guitar, because the descriptor is assigned to the guitar dot is my favorite attribute. Now, what do we need to do next? Well, this method, this get method will be called whenever we try to access that dot is my favorite attribute. And now we can check if the instance we're calling it on has Warwick streamer as the name. So I'm going to add an if statement. I'm using get attribute here to get the name attribute out of the instance. The reason I'm not doing instance dot name is really simple. I don't want this to break. In principle, there should not be guitar instances without a name, because the init methods always assigns that name attribute. But it could be that Python will pass none as the argument as the argument for instance. And that happens specifically when we try to access the is my favorite attributes directly on the class. So if you do guitar dot is my favorite, there is no instance yet. So the instance parameter in the get method will get none. And obviously none doesn't have a name attribute. So we need to be a bit careful here. So that's why you get attribute with a non default value. Then I can simply check if the name is equal to the Warwick streamer. If so, we return true. And if not, we return false. Now let's see that in action. So I create a Warwick guitar instance. I then access warwick dot is my favorite. This means that the Dunder get method will get called with Warwick, for instance, and guitar for owner. It then checks the name attribute. In this case, it's equal to Warwick streamer. So it will return true. So the result of accessing this attribute is determined by the descriptors get method. Now let's create another one of my guitars, which is a vendor, a vendor jazz base in this case. Now I'm going to access the vendor dot is my favorite attribute. This will call the Dunder get method again. But now with the vendor base as the instance, it will then get the name of the vendor base, which is not equal to Warwick streamer, which means it will return false. So as you can see, instead of just being a simple attribute is my favorite, it will actually call a method, the Dunder get method and whatever that method computes has as the return value, that will be the return value of the attributes. So now that we've seen a descriptor in action, I want to get back to functions. And in fact, functions also have such a get method. That's a little bit difficult to show you the actual implementation. But we can actually call that get method manually just to see what it does. And that's what I did here. And in order to do that, I need to access the original function objects. And if you remember, we can do that by accessing the function as a class attribute. So that's what I did here. So what I did is a guitar, a plain note that gives me the original function object, then we can call the Dunder get method on the original function objects. Well, a get method expects the instance. In this case, I pass in Warwick, and it expects the owner class. And I pass in guitar. Now, if I hit enter, if everything goes well, we should get a bound method. And indeed, that's what we get. So by calling the get method manually, we can actually bind the instance we pass in to the function to get a bound method. And this is also what happens if you access a method on an instance. Well, that brings us to the end of the talk. I have a short summary for you. And the most important thing of today is that the descriptor protocol allows you to customize how attributes work in Python. And the second thing is that functions implement the descriptor protocol to create bound methods. I don't really have any other important take home messages, except that you can do a lot of cool things with descriptors. There are also a few things that I haven't covered. But there are nonetheless important, I haven't given you any examples with done their set and done their delete, that's some purpose, because they only had 25 minutes. And I also did not explain the difference between data and non data descriptors. There's actually something interesting that happens as soon as you add a set or delete method. But it's kind of complicated. So I did not have time for that today as well. But I do want to recommend you to read the how to descriptor guide in the official documentation. It's written by Raymond Hettinger. It really explains these concepts really well. And another good recommendation is this book. It's fluent Python. It's my favorite Python book. And it also really, it really does a good job at explaining descriptors. It calls it overriding and non overriding descriptors. But it's all the same. Well, finally, I want you to be aware of a special done their method. It's called set name. And it really works well in combination with descriptors. Raymond Hettinger also talks about it in his how to descriptor guide. It's not yet in this version of fluent Python. But I've read that a new version, a new addition will come out in a few months. Well, that's it for me. Thanks for listening. I'll be here for the Q&A, so you can ask me questions and hopefully until until next year.
talk about something called descriptors, and we When talking about self in Python, I think it's really important to go back to the basics, self, you get used to how it works and what it magic of self is all about. To do that, I wrote a very simple class. It's a guitar class. And with classes, that means that you can do do with this method is you can play a note with guitar. And then I pass in the note. In this the parameter list. It has a self parameter, and first argument into the function. So self will be really looks like some kind of magical language just something that Python does for you. Well, in starts now. And now we're going to focus on what is the current class. And that means that as soon still a function. So one of the important happens when attributes are accessed. And that is really abstract explanation of how methods work. special methods from the Python data model, so guitar, because the descriptor is assigned to the favorite. This means that the Dunder get method with the vendor base as the instance, it will descriptor protocol allows you to customize how And it also really, it really does a good job at