How you can benefit from type hints
Vlasovskikh, Andrey
Andrey Vlasovskikh - How you can benefit from type hints PEP 484 introduces type hints for Python 3. Type hints can increase readability of our code for both humans and tools and lead to better and safer outcomes. And we'll prove it in this talk! We're going to take a closer look at type hints, see practical examples of where they can be used and the value they provide. We'll see that simple class types and built-in collection types are often enough for our public API's. We'll also discuss how you can benefit from type hinting stubs for third-party libraries and briefly cover more advanced scenarios like generic types.
 So, welcome everybody. I'm Andrey Blasovsky and I'm going to talk about how you can benefit from type hints. So, I come from the city called St. Petersburg. It's a big city in Russia and I work for a company called JetBrains and I'm one of the developers of the ID called PyCharm. Also, I've been contributing to this new PEP for it for called type hints and I'm very glad that it will be the part of Python 3.5. I would really like to say thank you to the authors of this PEP, Guido, Yuka and Lukash and also to the BDFL for this PEP, Mark, who actually accepted this PEP. So, what are type hints? Type hints are like any information provided by the user about the types in their programs, like types of function parameters, types of class attributes and so on. And the PEP 4.4 defines the formal notation for type hints. So, what I'm going to be talking about here is basically the motivation behind type hints in Python and how you can get started with them. And also, I will talk about the best practices for annotating your code with type hints. What is out of the scope here is the overview of the type system, what kind of types can you write in your code and also I want to walk through the whole type in module and this is out of the scope here. Probably some of you have visited the talk by Guido and Rossum earlier and I have a couple of questions for you. Who of you has actually listened to Guido about type hints? Please raise your hands. Great. Almost everyone. And how many of you have actually looked into this document, PEP 4.4? All right. About 2,000, I guess. And have you ever had a chance to actually play with type annotations in your code? Just a couple of you. Great. So, you came to the right talk for you and, yeah, I'm very glad. All right. So, type hints in Python, they are used to be informal. So, basically, Python programmers would define type hints inside their doc strings and those definitions were usually given in an informal language or natural language such as English, for example. Here's a piece of documentation from the standard library about the filter function. As you can see, there are lots of words highlighted in blue that actually refer to type hints. And the PEP 4.4 basically defines the formal notation. And as you can see, this is the type hints for the same filter function and they are a lot dense. And if you have a bit of understanding of what type hints are, this notation, I think, is much more clear. And just one thing I want to remind you from Guido's talk is that Python will always be a dynamic language and the type hints are completely optional. All right. So, from my perspective, the main benefits of type annotations is readability. Type hints are all about readability. For both humans and tools. What do I mean by readability by humans? Actually, I mean documentation. Documentation with type hints is a lot more easier to grasp. If you already have some familiarity with type hints, then lengthy natural language descriptions. And by readability by tools, I mean the possibility of creating type checkers and linters for Python. And also the tools that help you write code, for example, for code completion. Type information is crucial for those kinds of tools. And that's what we are going to talk about in this talk. So, let's start with type hints for tools. What tools do support type hints right now? Actually, I know only two of them. Is that MyPy, the original tool that was the main inspiration for this, for it for written by Yucca LectoSalo. And PyCharm. And I should know that we have a free community edition of PyCharm that is completely open sourced under the PyCharm license. And all our type inference and type analysis code and stuff like that is also open sourced. PyCharm 2.5 that was released earlier this year has already some support, some basic support for type hints. But the PEP wasn't ready yet at the time we were releasing PyCharm. So, only some parts of the specifications are supported. And PyCharm 5 is coming this fall and it will include full support for type hints, PEP for it for. Several other developers of code analysis tools like PyLint have already announced that they are going to support type hints in their tools. And for PyLint, I guess it will support it in 2016. All right. So, let's pick an example for understanding type hints. And as an example, I've chosen the library called elementary. It's a part of the standard Python library. And I would like to ask you how many of you are familiar with elementary. Please raise your hands. Almost everyone. Right. For those of you who are not familiar with it, it's a library for handling XML data, for passing XML and for dealing with it with an API that is nicer than the standard DOM API. So, let's start with this library and let's write some type hints in order to see the benefits of them. And just for the sake of gravity, let's call it e3. It's our own modified version of elementary. Later on, I will show you how to deal with the library without modifying it. So, here's a little example of using the elementary library. And if you are quite familiar with the API of this library, probably you would notice some errors here. Does anybody see the problem here? What's wrong here? Sorry? Can you hear? Yeah, yeah, bite literal. Right. That's it. So, basically, when you run this code, the result will be a type error. Because elementary cannot serialize binary data. By the way, there is another error here. You cannot specify integers as values of XML attributes. So, yes. You have to basically run this program in order to find this error if you are not that familiar with the elementary API. And as we will see right now, the type checker tools will find this kind of errors for you automatically with a little bit of help from type hints. So, let's start annotating just the constructor of the element class. Here, we add the stir annotation for the tag parameter. We expect it to be a string, not a by-string. And we expect the attribute parameter to be a dictionary of string keys and string values. So it's a generic type. All right. And with the help of this type hints, what we can get from tools. Here is the output of those two tools. PyCharm will underline the problematic elements with this yellow squiggly line. And it basically says that it expects the type string, but it goes by instead. And the MyPy, the command line tool, outputs basically the same information, but only in the texture form. All right. So, type hints are already somewhat useful. Let's continue annotating the API of the elementary library. Most type hints you will write in practice are quite simple. For example, the register namespace function accepts just two strings at return of nothing. Basically, there is no return statement in this function. So we annotated with the return type none. And the element, and the is element function over there, it accepts basically any value and checks if it is actually an XML element. So we specify the return, the parameter type of the element parameter as any. Some type hints are a little bit more complex. For example, below we are using some genetic types. The iter function returns an iterator over elements. And you will notice this string quotation Widow has mentioned in his talk because we are referring to the class itself while we are still defining it. And the iter text function returns just an iterator over strings. All right. So with the help, suppose we have annotated the whole API of the elementary. What could we get from that using the tools? Here's a little piece of code in PyCharm. Let's find out what this program does. It's a program that creates an XML element inside the h1 tag. So it's under HTML. That contains the string uppercase. So heading, the heading variable is an element with the uppercase hello in h1. And then we print basically the text of the first node, textual node of this element by iterating over text nodes, sub elements inside this XML element h1, and print the string world here. So the result would be uppercase hello, world, exclamation mark. And actually there is a problem in this code. That PyCharm would notice with the help of those type hints. And in order to find out what this problem is and why PyCharm will find it, we should think of an algorithm called type inference here. So let's think as the type inference error would think about this program. We have to find out if the indexing operator, the square bracket operator is available on the result of the iter text function. So let's find what iter text returns. But we still don't know which class contains this iter text method because we don't know the class of the heading variable. So we need to go and look through and look up the definition of the heading variable. It's right there on the previous line. And it is assigned a return value of the uppercase heading function. And it knows that uppercase heading has no type annotations whatsoever. But a type checker would try to infer the return type of the uppercase heading function. In this case, it returns a local element variable. And it is assigned a value of basically calling a constructor on this element tree.element class. So we know that the local element variable is actually a variable of type element. It's an information inference from this very basic example. And we know that the return value of the uppercase heading function would be element. So heading variable, the global variable heading is also of type element. And let's go to the previous slide to find out what the iter text on element is returning according to our written type annotations. Here it is. It's the last line on the slide. We have annotated the iter text to return the iterator over strings. As many of you would probably know, the iterator is an abstract base class. And it defines only two methods. It's the dunder iter method and dunder next. So there is no dunder get item method that is available on the iterator. So we cannot access, have access to the elements of the iterator by using the indexing operator. So it is clearly an error. And PyCharm with the help of some type inference, quite basic type inference in this example, but we could come up with quite more advanced ones, can infer, can find this kind of error. And know that we don't have to annotate any function. Some annotations would be enough in most cases. Another good thing you can get from type hints is code completion. So in PyCharm, we offer so-called context-sensitive code completion, meaning that it is based on the actual type of the variable. It shows only methods and attributes of the type of the variable. So this is the same iter text function. We are iterating over an iterator of strings and we have invoked code completion for the variable we are iterating over. And we know that it's type of strings. So we show the methods for the string class and the methods of their, its superclasses. So those are the benefits you can get from tools. Now let's switch to benefits from the documentation. Often documentation could become quite worthy if you are carefully writing down all the types of your function definition in English. Here's in, is a real-world example from the standard library documentation for the function called common of the elementary module. And not only is it worthy, but it also contains an actual error here marked in red. It says that we can pass a byte string in here. But as you can see from the example below, if we are actually passed the byte string literal and then try to serialize it, it will print the byte string literal as its wrapper, b, quotation marks and so on. This is crazy. This is absolutely not what you are trying to do here. So either the documentation is written badly and not tested or the implementation has bug in it. Something is definitely wrong here. Or maybe I just don't get the English text here. I'm a foreign speaker, so maybe. All right. So what would be, would it be better if we annotated this function with type hints and used just the doc string from the library? I did exactly this here. I used the Sphinx tool, the famous documentation generator for Python with the auto doc extension. Auto doc extension can extract the doc strings from your code as well as type annotations. And from my point of view, this piece of documentation is much more clear and it explains what is going on here using better, more formally defined syntax. And I'm more, I'm happier with this kind of documentation than this lengthy cumbersome documentation that has errors. Know that you can actually, you check your type hints in some kind of test. You can write a test suite for your code and run a static type checker in order to check functional annotations, but you cannot check the English natural language documentation. This is the idea. All right. So now I'll go in, I'm going to give some tips about how to actually use type hints in order to be productive with them. And the most important thing, actually, I want to tell you in this presentation is that you should annotate your public APIs. If you are an author of some kind of library that is used inside your company, your colleagues are using it, or you are an author of some open source piece and you want people on the Internet to start using your library and you want to get more users, please, please do use type hints for your public APIs. This is the most important slide I would like you to remember from this whole talk. And it is so important that I put a reference here to this very slide so you would find it on the Internet later. So please, if you would like, take a photo on something like that. Yes, this is the main idea. Documentation for your public APIs will make your users happier because it would act like, first of all, the better documentation and it would act like a safety net around your library. Basically, you are defining a clear interface between your code and the user code. And if some kind of error is happening in your library while your users are using it, they could run a type checker or they can just look the errors in their ID and they will see that they are using it wrongly. They will not report wrong bugs to your tracker just because they won't get your documentation right. So this is the idea. Types are like better documentation, better for IDs and so on. And as I've already told you, you don't have to annotate everything. Just annotate your public API and see if it's good for you. I think that in most cases that would be enough. Another tip is start with simple types. In most cases, it is enough to use types like stir, built-in types int and so on. You can use built-in collection types like list or dict and if you want to specify the types of elements inside your collection, please use collection types from the typing module, the uppercase versions, capital case dict and so on. The path for it for, as you know, defines some advanced type constructs as well, but use them only when you really need them. And remember that you can always put any if some particular type is tricky as you don't know how to express it or it's too long and so on. So try to stick with simple things. They would be simpler for your users and you won't spend your time trying to invent some crazy static typing construct in your code. All right. So another tip is be liberal in what you accept in your functions and conservative in what you've done. Here's a little piece of code. It's basically a function that iterates over iterable integers and filters out the even elements and returns a list of them. Here you are using only the dunder iter operator on numbers because you are using the generated comprehension in the code. So you don't have to restrict your users to pass a list here, for example, or an iterator. It is enough to say that the numbers should be an iterable. The more generic type you could probably mention that would work for this function. And speaking about the return values, in this case, I think you could be quite specific about what you return because you might want to allow your users to use some methods on the concrete type you are returning. For example, here we are returning the list explicitly and you might want to allow your users to avoid, for example, the append method or something like that. Note that the typing module defines all kinds of these abstract collection types like mapping or sequence when you can access stuff by index operator, set, and so on. So they are good for your parameters, but they are somewhat less good for your return values. It depends actually on you. All right. So far we've seen some type hints written formally using Python 3 function annotations, right? And in order to use those annotations, it is enough for the user to just keep on stall your library. The type annotations are in the code. They are available. Everything's good. And this is compatible with Python 2 and some other Python versions if you install the typing module from PyPR. Right now you have to specify typing equals equals 3.5.0 beta 1. Otherwise you will get just an empty package. But this is because the library is very new and the developers haven't had the chance to fix it. All right. And what about Python 2? I guess many of you actually are using Python 2. So here's another quid poll about Python 2 versus Python 3. Who of you are using Python 2? Please raise your hands. 90%, I guess. And who of you are using Python 3? Wow. This is really good. Python 3 is finally coming. Great. All right. So those of you who still need to support Python 2, there is this thing called Python Stops. Just a quid reminder of what they are. They are Python files with the different PyI extension. And they contain no bodies for methods. Just the ellipses literal. And here's our little stop for the elementary library. The actual elementary library of the standard library of Python 3.5. And this is how we should define type hints for the library. Not by modifying it, obviously, but by writing this kind of stops. And you can use these Python 3 stops with your Python 2 code. And the type checkers will understand that. So stops in Python 3, code for Python 2. All right. So stops are very good for annotating not only the standard library, but for third party libraries as well. Type checkers would expect them to be somewhere on your assistive pass. So if you are creating some kind of project that uses lots of third party libraries and you want to annotate them in order to get better code completion, you can just create a folder inside your project. For example, a folder called type hints. And put your annotations for all the libraries in there. Just add it to your assistive pass. And then you can, for example, share this folder with the rest of your project using Git or Mirkorel or whatnot. And your colleagues will have your type annotations. All right. So if you do that kind of stuff, if you are interested in creating Python sub files, please share your stuff with others if you are annotating third party libraries. There are several ways of doing that. The main is to send a pull request to this new type shared repo. It's the wrapper mentioned in PEP 4.4. And it is supposed to be the universal collection of all the steps available. Basically, it's modeled after the definitely typed repo for the type script, the language that has type hints from its early days. It's still quite new. And this model of creating steps for libraries is very successful. It's very, very well established in the type script community and in the JavaScript community at the whole. For example, people who are still using JavaScript, they clone this definitely typed repo in type script for their project and check their code against these type script definitions of libraries. So in Python, we will have the same situation where you can actually check your code against this large collection of steps for all kinds of libraries. Another way to specify it in the PEP is to create a package with steps on PyPy and use it. All right. What if you don't want to invest in writing type hints, either in functional annotations or in steps? Then you still can benefit from that. For example, if any of you are PyCharm users, you might not know it, but you are already using Python annotations, functional annotations. Because in PyCharm, we have this kind of, behind the scenes repository of type hints for the standard library. Not all of it, but for some parts. So basically, you are already using type hints, but they are written using our own legacy syntax and we are going to switch to this new syntax. That's why we are interested in this proposal and we are contributing to it. We want all the tools to use the same shared collection of steps in order to be more productive as a community. Another way of using steps by other people would be cloning this repo and start using it. All right. So in conclusion, I urge you to try type hints. They are really good. I'm a fan of them, as you probably already know, but they are really good. And there are a few steps, a few very simple steps, how to, you can get started with it. Just grab the latest version of Python 3.5 beta or use some older version of Python 3 and get some of the tools, mypy or pycharm, and just start using type hints. Try it. I do want to know your thoughts about it. So please contact me. Here's me on Twitter and my website, PyRx.ru. I really want to hear what is your experience with type hints. Do you like them? Do you find them good for your users or maybe bad for your users? Are you going to switch to Python 3 with type hints? I'm very interested to hearing all those kinds of stuff from you. So please, please contact me. And the slides are already available on the Internet via this URL. So thank you for your time. Very interesting talk from Andrei. Any questions? Thanks. Thanks for the talk. I'm interested if you can do an easy introspection into the type hints to do like validation for RPC, for example. For RPC, well, you have to write some kind of tool that would understand the introspection API of your RPC library and generate those kind of stops. So it's a piece of software that is perfectly valid and it's a good idea to create these kind of things for your RPC mechanisms. And can you use type hints to do that, to inspect the type hints and do validation based on that? Yeah, yeah. That would be also a good idea, yes. Thank you for the talk. So you said we should all annotate our public APIs and you also said that PyCharm, for example, has already the tooling to infer type hints from the code itself, in some simple cases. Maybe there is also a way to create stops for us because we are lazy to annotate everything? We are thinking about this kind of feature in PyCharm 5.0, but it's still work in progress. So let's see, we will see what is possible here. We are quite good at inferring the return type for functions, but it is very tricky to infer the type of parameters to the function because the only source of information for parameters, actually there are two sources, the attributes to accessing of the parameter inside the body of the function and the usages of the function, but they are all kind of not so, they contain much less information than you can get for the return type based on calling to constructors or literals and so on. So yeah, we will see what's possible here. Is it possible to use if-punches in sub files in case, like, when type is different from Python 2 and Python 3? Or, sorry, could you repeat the question? Is it possible to have, like, an if-pyton2 in a sub file? Yeah, definitely, yes. The PEP 44 defines several ways of using conditionals like that, but we don't support them in PyCharm 4.5, but we are going to support them in PyCharm 5. Do you think that it would be a good way to provide source compatibility with Python 2 to make tools like 3 to 2 to generate the stub files from the source code instead of repeating yourself and defining the API basically two times? You mean, like, generating- No, I mean, targeting Python 3 in development and then generating the stub files from the Python 3 source code? Ah, yeah, yeah. It is actually a very simple task. I guess it's a good-it would be a good addition, for example, for the MyPy tool. Yes, it's definitely possible. Hi. Is there any effort or project that is putting the type hintings on all the standard library? Because it could be a nice party to put it all there, and when Python 3.5 is released, having all the standard library annotated. The project is called TypeShad. So TypeShad will eventually have the stubs for the whole Python 3.5 library and Python 2.7 library. There will be stubs for the standard library, yes. Is there any tool to generate the stub file from dog strings type information? I'm not aware of any kind of tool like that. It would be not so hard to create one, but I'm not aware of any of these tools. Thank you. So I played around a bit with MyPy, and the first thing I looked into was trying to detect if a function returns an optional value, so it can be none or it can be something else. And that's something that MyPy does not currently pick up. Does PyCharm do that? It's a question with some bit of history. We used to support those kinds of checks when we still didn't have any type annotations whatsoever, and we used only the inferred type information. And we used to have these checks when you have to actually write something like if value, and then you would be able to use the value, and we will be sure that it is not none. But then we received a lot of reports from our users that this is not what they were expecting from PyCharm, because the inference type could be not so accurate. And if you are explicitly specifying that you are expecting an optional type, or you are returning an optional type, and then using this type, then I guess we can trust your annotations, because this is what you are expected from the defined behavior of your functions, and we will support checking for optional types in PyCharm 5. Any more questions? No? Wait put back what I created, not necessarily an external variable in PyCharm cross just
 So, welcome everybody. I'm Andrey Blasovsky and I'm going to talk about how you can benefit  from type hints. So, I come from the city called St. Petersburg. It's a big city in  Russia and I work for a company called JetBrains and I'm one of the developers of the ID called  PyCharm. Also, I've been contributing to this new PEP for it for called type hints and I'm very  glad that it will be the part of Python 3.5. I would really like to say thank you to the authors  of this PEP, Guido, Yuka and Lukash and also to the BDFL for this PEP, Mark, who actually accepted  this PEP. So, what are type hints? Type hints are like any information provided by the user about  the types in their programs, like types of function parameters, types of class attributes and so on.  And the PEP 4.4 defines the formal notation for type hints. So, what I'm going to be talking about  here is basically the motivation behind type hints in Python and how you can get started with them.  And also, I will talk about the best practices for annotating your code with type hints. What is  out of the scope here is the overview of the type system, what kind of types can you write in your  code and also I want to walk through the whole type in module and this is out of the scope here.  Probably some of you have visited the talk by Guido and Rossum earlier and I have a couple of  questions for you. Who of you has actually listened to Guido about type hints? Please raise your  hands. Great. Almost everyone. And how many of you have actually looked into this document,  PEP 4.4? All right. About 2,000, I guess. And have you ever had a chance to actually play with  type annotations in your code? Just a couple of you. Great. So, you came to the right talk for you  and, yeah, I'm very glad. All right. So, type hints in Python, they are used to be informal. So,  basically, Python programmers would define type hints inside their doc strings and those  definitions were usually given in an informal language or natural language such as English,  for example. Here's a piece of documentation from the standard library about the filter function.  As you can see, there are lots of words highlighted in blue that actually refer to type  hints. And the PEP 4.4 basically defines the formal notation. And as you can see,  this is the type hints for the same filter function and they are a lot dense. And if you  have a bit of understanding of what type hints are, this notation, I think, is much more clear.  And just one thing I want to remind you from Guido's talk is that Python will always be a  dynamic language and the type hints are completely optional. All right. So, from my perspective,  the main benefits of type annotations is readability. Type hints are all about readability. For  both humans and tools. What do I mean by readability by humans? Actually, I mean documentation.  Documentation with type hints is a lot more easier to grasp. If you already have some  familiarity with type hints, then lengthy natural language descriptions. And by readability  by tools, I mean the possibility of creating type checkers and linters for Python. And also  the tools that help you write code, for example, for code completion. Type information is crucial  for those kinds of tools. And that's what we are going to talk about in this talk. So,  let's start with type hints for tools. What tools do support type hints right now? Actually,  I know only two of them. Is that MyPy, the original tool that was the main inspiration  for this, for it for written by Yucca LectoSalo. And PyCharm. And I should know that we have  a free community edition of PyCharm that is completely open sourced under the PyCharm  license. And all our type inference and type analysis code and stuff like that is also  open sourced. PyCharm 2.5 that was released earlier this year has already some support,  some basic support for type hints. But the PEP wasn't ready yet at the time we were  releasing PyCharm. So, only some parts of the specifications are supported. And PyCharm  5 is coming this fall and it will include full support for type hints, PEP for it for.  Several other developers of code analysis tools like PyLint have already announced that  they are going to support type hints in their tools. And for PyLint, I guess it will support  it in 2016. All right. So, let's pick an example for understanding type hints. And as an example,  I've chosen the library called elementary. It's a part of the standard Python library.  And I would like to ask you how many of you are familiar with elementary. Please raise  your hands. Almost everyone. Right. For those of you who are not familiar with it, it's  a library for handling XML data, for passing XML and for dealing with it with an API that  is nicer than the standard DOM API. So, let's start with this library and let's write some  type hints in order to see the benefits of them. And just for the sake of gravity, let's  call it e3. It's our own modified version of elementary. Later on, I will show you how  to deal with the library without modifying it. So, here's a little example of using the  elementary library. And if you are quite familiar with the API of this library, probably you  would notice some errors here. Does anybody see the problem here? What's wrong here? Sorry?  Can you hear? Yeah, yeah, bite literal. Right. That's it. So, basically, when you run this  code, the result will be a type error. Because elementary cannot serialize binary data. By  the way, there is another error here. You cannot specify integers as values of XML attributes.  So, yes. You have to basically run this program in order to find this error if you are not  that familiar with the elementary API. And as we will see right now, the type checker  tools will find this kind of errors for you automatically with a little bit of help from  type hints. So, let's start annotating just the constructor of the element class. Here,  we add the stir annotation for the tag parameter. We expect it to be a string, not a by-string.  And we expect the attribute parameter to be a dictionary of string keys and string values.  So it's a generic type. All right. And with the help of this type hints, what we can get  from tools. Here is the output of those two tools. PyCharm will underline the problematic  elements with this yellow squiggly line. And it basically says that it expects the type  string, but it goes by instead. And the MyPy, the command line tool, outputs basically the  same information, but only in the texture form. All right. So, type hints are already somewhat  useful. Let's continue annotating the API of the elementary library. Most type hints  you will write in practice are quite simple. For example, the register namespace function  accepts just two strings at return of nothing. Basically, there is no return statement in  this function. So we annotated with the return type none. And the element, and the is element  function over there, it accepts basically any value and checks if it is actually an XML  element. So we specify the return, the parameter type of the element parameter as any. Some  type hints are a little bit more complex. For example, below we are using some genetic  types. The iter function returns an iterator over elements. And you will notice this string  quotation Widow has mentioned in his talk because we are referring to the class itself  while we are still defining it. And the iter text function returns just an iterator over  strings. All right. So with the help, suppose we have annotated the whole API of the elementary.  What could we get from that using the tools? Here's a little piece of code in PyCharm. Let's  find out what this program does. It's a program that creates an XML element inside the h1  tag. So it's under HTML. That contains the string uppercase. So heading, the heading  variable is an element with the uppercase hello in h1. And then we print basically the text  of the first node, textual node of this element by iterating over text nodes, sub elements  inside this XML element h1, and print the string world here. So the result would be  uppercase hello, world, exclamation mark. And actually there is a problem in this code.  That PyCharm would notice with the help of those type hints. And in order to find out  what this problem is and why PyCharm will find it, we should think of an algorithm called  type inference here. So let's think as the type inference error would think about this  program. We have to find out if the indexing operator, the square bracket operator is available  on the result of the iter text function. So let's find what iter text returns. But we  still don't know which class contains this iter text method because we don't know the  class of the heading variable. So we need to go and look through and look up the definition  of the heading variable. It's right there on the previous line. And it is assigned a return  value of the uppercase heading function. And it knows that uppercase heading has no type  annotations whatsoever. But a type checker would try to infer the return type of the  uppercase heading function. In this case, it returns a local element variable. And it  is assigned a value of basically calling a constructor on this element tree.element class.  So we know that the local element variable is actually a variable of type element. It's  an information inference from this very basic example. And we know that the return value  of the uppercase heading function would be element. So heading variable, the global variable  heading is also of type element. And let's go to the previous slide to find out what  the iter text on element is returning according to our written type annotations. Here it is.  It's the last line on the slide. We have annotated the iter text to return the iterator over  strings. As many of you would probably know, the iterator is an abstract base class. And  it defines only two methods. It's the dunder iter method and dunder next. So there is no  dunder get item method that is available on the iterator. So we cannot access, have access  to the elements of the iterator by using the indexing operator. So it is clearly an error.  And PyCharm with the help of some type inference, quite basic type inference in this example,  but we could come up with quite more advanced ones, can infer, can find this kind of error.  And know that we don't have to annotate any function. Some annotations would be enough  in most cases. Another good thing you can get from type hints is code completion. So  in PyCharm, we offer so-called context-sensitive code completion, meaning that it is based  on the actual type of the variable. It shows only methods and attributes of the type of  the variable. So this is the same iter text function. We are iterating over an iterator  of strings and we have invoked code completion for the variable we are iterating over. And  we know that it's type of strings. So we show the methods for the string class and the methods  of their, its superclasses. So those are the benefits you can get from tools. Now let's  switch to benefits from the documentation. Often documentation could become quite worthy  if you are carefully writing down all the types of your function definition in English.  Here's in, is a real-world example from the standard library documentation for the function  called common of the elementary module. And not only is it worthy, but it also contains  an actual error here marked in red. It says that we can pass a byte string in here. But  as you can see from the example below, if we are actually passed the byte string literal  and then try to serialize it, it will print the byte string literal as its wrapper, b,  quotation marks and so on. This is crazy. This is absolutely not what you are trying  to do here. So either the documentation is written badly and not tested or the implementation  has bug in it. Something is definitely wrong here. Or maybe I just don't get the English  text here. I'm a foreign speaker, so maybe. All right. So what would be, would it be better  if we annotated this function with type hints and used just the doc string from the library?  I did exactly this here. I used the Sphinx tool, the famous documentation generator for  Python with the auto doc extension. Auto doc extension can extract the doc strings from  your code as well as type annotations. And from my point of view, this piece of documentation  is much more clear and it explains what is going on here using better, more formally  defined syntax. And I'm more, I'm happier with this kind of documentation than this  lengthy cumbersome documentation that has errors. Know that you can actually, you check  your type hints in some kind of test. You can write a test suite for your code and run  a static type checker in order to check functional annotations, but you cannot check the English  natural language documentation. This is the idea. All right. So now I'll go in, I'm going  to give some tips about how to actually use type hints in order to be productive with them.  And the most important thing, actually, I want to tell you in this presentation is that  you should annotate your public APIs. If you are an author of some kind of library that  is used inside your company, your colleagues are using it, or you are an author of some  open source piece and you want people on the Internet to start using your library and you  want to get more users, please, please do use type hints for your public APIs. This is  the most important slide I would like you to remember from this whole talk. And it is  so important that I put a reference here to this very slide so you would find it on the  Internet later. So please, if you would like, take a photo on something like that. Yes,  this is the main idea. Documentation for your public APIs will make your users happier because  it would act like, first of all, the better documentation and it would act like a safety  net around your library. Basically, you are defining a clear interface between your code  and the user code. And if some kind of error is happening in your library while your users  are using it, they could run a type checker or they can just look the errors in their  ID and they will see that they are using it wrongly. They will not report wrong bugs to  your tracker just because they won't get your documentation right. So this is the idea.  Types are like better documentation, better for IDs and so on. And as I've already told  you, you don't have to annotate everything. Just annotate your public API and see if it's  good for you. I think that in most cases that would be enough. Another tip is start with  simple types. In most cases, it is enough to use types like stir, built-in types int  and so on. You can use built-in collection types like list or dict and if you want to  specify the types of elements inside your collection, please use collection types from  the typing module, the uppercase versions, capital case dict and so on. The path for  it for, as you know, defines some advanced type constructs as well, but use them only  when you really need them. And remember that you can always put any if some particular  type is tricky as you don't know how to express it or it's too long and so on. So try to stick  with simple things. They would be simpler for your users and you won't spend your time  trying to invent some crazy static typing construct in your code. All right. So another  tip is be liberal in what you accept in your functions and conservative in what you've  done. Here's a little piece of code. It's basically a function that iterates over iterable integers  and filters out the even elements and returns a list of them. Here you are using only the  dunder iter operator on numbers because you are using the generated comprehension in the  code. So you don't have to restrict your users to pass a list here, for example, or an iterator.  It is enough to say that the numbers should be an iterable. The more generic type you  could probably mention that would work for this function. And speaking about the return  values, in this case, I think you could be quite specific about what you return because  you might want to allow your users to use some methods on the concrete type you are  returning. For example, here we are returning the list explicitly and you might want to  allow your users to avoid, for example, the append method or something like that. Note  that the typing module defines all kinds of these abstract collection types like mapping  or sequence when you can access stuff by index operator, set, and so on. So they are good  for your parameters, but they are somewhat less good for your return values. It depends  actually on you. All right. So far we've seen some type hints written formally using Python  3 function annotations, right? And in order to use those annotations, it is enough for  the user to just keep on stall your library. The type annotations are in the code. They  are available. Everything's good. And this is compatible with Python 2 and some other  Python versions if you install the typing module from PyPR. Right now you have to specify  typing equals equals 3.5.0 beta 1. Otherwise you will get just an empty package. But this  is because the library is very new and the developers haven't had the chance to fix it.  All right. And what about Python 2? I guess many of you actually are using Python 2. So  here's another quid poll about Python 2 versus Python 3. Who of you are using Python 2? Please  raise your hands. 90%, I guess. And who of you are using Python 3? Wow. This is really  good. Python 3 is finally coming. Great. All right. So those of you who still need to  support Python 2, there is this thing called Python Stops. Just a quid reminder of what  they are. They are Python files with the different PyI extension. And they contain no bodies  for methods. Just the ellipses literal. And here's our little stop for the elementary  library. The actual elementary library of the standard library of Python 3.5. And this  is how we should define type hints for the library. Not by modifying it, obviously, but  by writing this kind of stops. And you can use these Python 3 stops with your Python 2  code. And the type checkers will understand that. So stops in Python 3, code for Python  2. All right. So stops are very good for annotating not only the standard library, but for third  party libraries as well. Type checkers would expect them to be somewhere on your assistive  pass. So if you are creating some kind of project that uses lots of third party libraries  and you want to annotate them in order to get better code completion, you can just create  a folder inside your project. For example, a folder called type hints. And put your annotations  for all the libraries in there. Just add it to your assistive pass. And then you can,  for example, share this folder with the rest of your project using Git or Mirkorel or whatnot.  And your colleagues will have your type annotations. All right. So if you do that kind of stuff,  if you are interested in creating Python sub files, please share your stuff with others  if you are annotating third party libraries. There are several ways of doing that. The main  is to send a pull request to this new type shared repo. It's the wrapper mentioned in  PEP 4.4. And it is supposed to be the universal collection of all the steps available. Basically,  it's modeled after the definitely typed repo for the type script, the language that has  type hints from its early days. It's still quite new. And this model of creating steps  for libraries is very successful. It's very, very well established in the type script community  and in the JavaScript community at the whole. For example, people who are still using JavaScript,  they clone this definitely typed repo in type script for their project and check their code  against these type script definitions of libraries. So in Python, we will have the same situation  where you can actually check your code against this large collection of steps for all kinds  of libraries. Another way to specify it in the PEP is to create a package with steps on  PyPy and use it. All right. What if you don't want to invest in writing type hints, either  in functional annotations or in steps? Then you still can benefit from that. For example,  if any of you are PyCharm users, you might not know it, but you are already using Python  annotations, functional annotations. Because in PyCharm, we have this kind of, behind the  scenes repository of type hints for the standard library. Not all of it, but for some parts.  So basically, you are already using type hints, but they are written using our own legacy syntax  and we are going to switch to this new syntax. That's why we are interested in this proposal  and we are contributing to it. We want all the tools to use the same shared collection  of steps in order to be more productive as a community. Another way of using steps by  other people would be cloning this repo and start using it. All right. So in conclusion,  I urge you to try type hints. They are really good. I'm a fan of them, as you probably already  know, but they are really good. And there are a few steps, a few very simple steps, how  to, you can get started with it. Just grab the latest version of Python 3.5 beta or use  some older version of Python 3 and get some of the tools, mypy or pycharm, and just start  using type hints. Try it. I do want to know your thoughts about it. So please contact  me. Here's me on Twitter and my website, PyRx.ru. I really want to hear what is your  experience with type hints. Do you like them? Do you find them good for your users or maybe  bad for your users? Are you going to switch to Python 3 with type hints? I'm very interested  to hearing all those kinds of stuff from you. So please, please contact me. And the slides  are already available on the Internet via this URL. So thank you for your time.  Very interesting talk from Andrei. Any questions?  Thanks. Thanks for the talk. I'm interested if you can do an easy introspection into the  type hints to do like validation for RPC, for example.  For RPC, well, you have to write some kind of tool that would understand the introspection  API of your RPC library and generate those kind of stops. So it's a piece of software  that is perfectly valid and it's a good idea to create these kind of things for your RPC  mechanisms.  And can you use type hints to do that, to inspect the type hints and do validation based  on that?  Yeah, yeah. That would be also a good idea, yes.  Thank you for the talk. So you said we should all annotate our public APIs and you also  said that PyCharm, for example, has already the tooling to infer type hints from the code  itself, in some simple cases. Maybe there is also a way to create stops for us because  we are lazy to annotate everything?  We are thinking about this kind of feature in PyCharm 5.0, but it's still work in progress.  So let's see, we will see what is possible here. We are quite good at inferring the return  type for functions, but it is very tricky to infer the type of parameters to the function  because the only source of information for parameters, actually there are two sources,  the attributes to accessing of the parameter inside the body of the function and the usages  of the function, but they are all kind of not so, they contain much less information  than you can get for the return type based on calling to constructors or literals and  so on. So yeah, we will see what's possible here.  Is it possible to use if-punches in sub files in case, like, when type is different from  Python 2 and Python 3?  Or, sorry, could you repeat the question? Is it possible to have, like, an if-pyton2  in a sub file?  Yeah, definitely, yes. The PEP 44 defines several ways of using conditionals like that,  but we don't support them in PyCharm 4.5, but we are going to support them in PyCharm  5.  Do you think that it would be a good way to provide source compatibility with Python  2 to make tools like 3 to 2 to generate the stub files from the source code instead of  repeating yourself and defining the API basically two times?  You mean, like, generating-  No, I mean, targeting Python 3 in development and then generating the stub files from the  Python 3 source code?  Ah, yeah, yeah. It is actually a very simple task. I guess it's a good-it would be a good  addition, for example, for the MyPy tool. Yes, it's definitely possible.  Hi. Is there any effort or project that is putting the type hintings on all the standard  library?  Because it could be a nice party to put it all there, and when Python 3.5 is released,  having all the standard library annotated.  The project is called TypeShad. So TypeShad will eventually have the stubs for the whole  Python 3.5 library and Python 2.7 library. There will be stubs for the standard library,  yes.  Is there any tool to generate the stub file from dog strings type information?  I'm not aware of any kind of tool like that. It would be not so hard to create one, but  I'm not aware of any of these tools.  Thank you.  So I played around a bit with MyPy, and the first thing I looked into was trying to detect  if a function returns an optional value, so it can be none or it can be something else.  And that's something that MyPy does not currently pick up.  Does PyCharm do that?  It's a question with some bit of history. We used to support those kinds of checks when  we still didn't have any type annotations whatsoever, and we used only the inferred  type information. And we used to have these checks when you have to actually write something  like if value, and then you would be able to use the value, and we will be sure that  it is not none.  But then we received a lot of reports from our users that this is not what they were  expecting from PyCharm, because the inference type could be not so accurate.  And if you are explicitly specifying that you are expecting an optional type, or you  are returning an optional type, and then using this type, then I guess we can trust your  annotations, because this is what you are expected from the defined behavior of your  functions, and we will support checking for optional types in PyCharm 5.  Any more questions?  No?  Wait put back what I created, not necessarily an external variable in PyCharm cross just
Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_0_1703.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_1704_1931.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_1932_2927.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_2928_3887.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_3888_4043.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_4044_4655.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_4656_5495.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_5496_6131.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_6132_7031.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_7032_7811.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_7812_8963.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_8964_10115.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_10116_10919.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_10920_11639.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_11640_12503.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_12504_14435.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_14436_14999.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_15000_21143.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_21144_22043.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_22044_22151.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_22152_22211.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_22212_23927.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_23928_24779.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_24780_25055.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25056_25127.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25128_25223.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25224_25283.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25284_25571.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25572_25619.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25620_25727.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25728_25775.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_25776_34535.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_34536_34583.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_34584_35279.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_35280_35963.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_35964_36203.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_36204_37163.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_37164_39527.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_39528_40991.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_40992_43175.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_43176_43247.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_43248_44471.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_44472_44519.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_44520_44579.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_44580_45167.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_45168_46031.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_46032_46775.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_46776_48083.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_48084_51011.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_51012_52667.png Andrey_Vlasovskikh_-_How_you_can_benefit_from_type_hints_52668_52671.png
So, welcome everybody. I'm Andrey Blasovsky and I'm going to talk about how you can benefit from type hints. So, I come from the city called St. Petersburg. It's a big city in Russia and I work for a company called JetBrains and I'm one of the developers of the ID called PyCharm. Also, I've been contributing to this new PEP for it for called type hints and I'm very glad that it will be the part of Python 3.5. I would really like to say thank you to the authors of this PEP, Guido, Yuka and Lukash and also to the BDFL for this PEP, Mark, who actually accepted this PEP. So, what are type hints? Type hints are like any information provided by the user about the types in their programs, like types of function parameters, types of class attributes and so on. And the PEP 4.4 defines the formal notation for type hints. So, what I'm going to be talking about here is basically the motivation behind type hints in Python and how you can get started with them. And also, I will talk about the best practices for annotating your code with type hints. What is out of the scope here is the overview of the type system, what kind of types can you write in your code and also I want to walk through the whole type in module and this is out of the scope here. Probably some of you have visited the talk by Guido and Rossum earlier and I have a couple of questions for you. Who of you has actually listened to Guido about type hints? Please raise your hands. Great. Almost everyone. And how many of you have actually looked into this document, PEP 4.4? All right. About 2,000, I guess. And have you ever had a chance to actually play with type annotations in your code? Just a couple of you. Great. So, you came to the right talk for you and, yeah, I'm very glad. All right. So, type hints in Python, they are used to be informal. So, basically, Python programmers would define type hints inside their doc strings and those definitions were usually given in an informal language or natural language such as English, for example. Here's a piece of documentation from the standard library about the filter function. As you can see, there are lots of words highlighted in blue that actually refer to type hints. And the PEP 4.4 basically defines the formal notation. And as you can see, this is the type hints for the same filter function and they are a lot dense. And if you have a bit of understanding of what type hints are, this notation, I think, is much more clear. And just one thing I want to remind you from Guido's talk is that Python will always be a dynamic language and the type hints are completely optional. All right. So, from my perspective, the main benefits of type annotations is readability. Type hints are all about readability. For both humans and tools. What do I mean by readability by humans? Actually, I mean documentation. Documentation with type hints is a lot more easier to grasp. If you already have some familiarity with type hints, then lengthy natural language descriptions. And by readability by tools, I mean the possibility of creating type checkers and linters for Python. And also the tools that help you write code, for example, for code completion. Type information is crucial for those kinds of tools. And that's what we are going to talk about in this talk. So, let's start with type hints for tools. What tools do support type hints right now? Actually, I know only two of them. Is that MyPy, the original tool that was the main inspiration for this, for it for written by Yucca LectoSalo. And PyCharm. And I should know that we have a free community edition of PyCharm that is completely open sourced under the PyCharm license. And all our type inference and type analysis code and stuff like that is also open sourced. PyCharm 2.5 that was released earlier this year has already some support, some basic support for type hints. But the PEP wasn't ready yet at the time we were releasing PyCharm. So, only some parts of the specifications are supported. And PyCharm 5 is coming this fall and it will include full support for type hints, PEP for it for. Several other developers of code analysis tools like PyLint have already announced that they are going to support type hints in their tools. And for PyLint, I guess it will support it in 2016. All right. So, let's pick an example for understanding type hints. And as an example, I've chosen the library called elementary. It's a part of the standard Python library. And I would like to ask you how many of you are familiar with elementary. Please raise your hands. Almost everyone. Right. For those of you who are not familiar with it, it's a library for handling XML data, for passing XML and for dealing with it with an API that is nicer than the standard DOM API. So, let's start with this library and let's write some type hints in order to see the benefits of them. And just for the sake of gravity, let's call it e3. It's our own modified version of elementary. Later on, I will show you how to deal with the library without modifying it. So, here's a little example of using the elementary library. And if you are quite familiar with the API of this library, probably you would notice some errors here. Does anybody see the problem here? What's wrong here? Sorry? Can you hear? Yeah, yeah, bite literal. Right. That's it. So, basically, when you run this code, the result will be a type error. Because elementary cannot serialize binary data. By the way, there is another error here. You cannot specify integers as values of XML attributes. So, yes. You have to basically run this program in order to find this error if you are not that familiar with the elementary API. And as we will see right now, the type checker tools will find this kind of errors for you automatically with a little bit of help from type hints. So, let's start annotating just the constructor of the element class. Here, we add the stir annotation for the tag parameter. We expect it to be a string, not a by-string. And we expect the attribute parameter to be a dictionary of string keys and string values. So it's a generic type. All right. And with the help of this type hints, what we can get from tools. Here is the output of those two tools. PyCharm will underline the problematic elements with this yellow squiggly line. And it basically says that it expects the type string, but it goes by instead. And the MyPy, the command line tool, outputs basically the same information, but only in the texture form. All right. So, type hints are already somewhat useful. Let's continue annotating the API of the elementary library. Most type hints you will write in practice are quite simple. For example, the register namespace function accepts just two strings at return of nothing. Basically, there is no return statement in this function. So we annotated with the return type none. And the element, and the is element function over there, it accepts basically any value and checks if it is actually an XML element. So we specify the return, the parameter type of the element parameter as any. Some type hints are a little bit more complex. For example, below we are using some genetic types. The iter function returns an iterator over elements. And you will notice this string quotation Widow has mentioned in his talk because we are referring to the class itself while we are still defining it. And the iter text function returns just an iterator over strings. All right. So with the help, suppose we have annotated the whole API of the elementary. What could we get from that using the tools? Here's a little piece of code in PyCharm. Let's find out what this program does. It's a program that creates an XML element inside the h1 tag. So it's under HTML. That contains the string uppercase. So heading, the heading variable is an element with the uppercase hello in h1. And then we print basically the text of the first node, textual node of this element by iterating over text nodes, sub elements inside this XML element h1, and print the string world here. So the result would be uppercase hello, world, exclamation mark. And actually there is a problem in this code. That PyCharm would notice with the help of those type hints. And in order to find out what this problem is and why PyCharm will find it, we should think of an algorithm called type inference here. So let's think as the type inference error would think about this program. We have to find out if the indexing operator, the square bracket operator is available on the result of the iter text function. So let's find what iter text returns. But we still don't know which class contains this iter text method because we don't know the class of the heading variable. So we need to go and look through and look up the definition of the heading variable. It's right there on the previous line. And it is assigned a return value of the uppercase heading function. And it knows that uppercase heading has no type annotations whatsoever. But a type checker would try to infer the return type of the uppercase heading function. In this case, it returns a local element variable. And it is assigned a value of basically calling a constructor on this element tree.element class. So we know that the local element variable is actually a variable of type element. It's an information inference from this very basic example. And we know that the return value of the uppercase heading function would be element. So heading variable, the global variable heading is also of type element. And let's go to the previous slide to find out what the iter text on element is returning according to our written type annotations. Here it is. It's the last line on the slide. We have annotated the iter text to return the iterator over strings. As many of you would probably know, the iterator is an abstract base class. And it defines only two methods. It's the dunder iter method and dunder next. So there is no dunder get item method that is available on the iterator. So we cannot access, have access to the elements of the iterator by using the indexing operator. So it is clearly an error. And PyCharm with the help of some type inference, quite basic type inference in this example, but we could come up with quite more advanced ones, can infer, can find this kind of error. And know that we don't have to annotate any function. Some annotations would be enough in most cases. Another good thing you can get from type hints is code completion. So in PyCharm, we offer so-called context-sensitive code completion, meaning that it is based on the actual type of the variable. It shows only methods and attributes of the type of the variable. So this is the same iter text function. We are iterating over an iterator of strings and we have invoked code completion for the variable we are iterating over. And we know that it's type of strings. So we show the methods for the string class and the methods of their, its superclasses. So those are the benefits you can get from tools. Now let's switch to benefits from the documentation. Often documentation could become quite worthy if you are carefully writing down all the types of your function definition in English. Here's in, is a real-world example from the standard library documentation for the function called common of the elementary module. And not only is it worthy, but it also contains an actual error here marked in red. It says that we can pass a byte string in here. But as you can see from the example below, if we are actually passed the byte string literal and then try to serialize it, it will print the byte string literal as its wrapper, b, quotation marks and so on. This is crazy. This is absolutely not what you are trying to do here. So either the documentation is written badly and not tested or the implementation has bug in it. Something is definitely wrong here. Or maybe I just don't get the English text here. I'm a foreign speaker, so maybe. All right. So what would be, would it be better if we annotated this function with type hints and used just the doc string from the library? I did exactly this here. I used the Sphinx tool, the famous documentation generator for Python with the auto doc extension. Auto doc extension can extract the doc strings from your code as well as type annotations. And from my point of view, this piece of documentation is much more clear and it explains what is going on here using better, more formally defined syntax. And I'm more, I'm happier with this kind of documentation than this lengthy cumbersome documentation that has errors. Know that you can actually, you check your type hints in some kind of test. You can write a test suite for your code and run a static type checker in order to check functional annotations, but you cannot check the English natural language documentation. This is the idea. All right. So now I'll go in, I'm going to give some tips about how to actually use type hints in order to be productive with them. And the most important thing, actually, I want to tell you in this presentation is that you should annotate your public APIs. If you are an author of some kind of library that is used inside your company, your colleagues are using it, or you are an author of some open source piece and you want people on the Internet to start using your library and you want to get more users, please, please do use type hints for your public APIs. This is the most important slide I would like you to remember from this whole talk. And it is so important that I put a reference here to this very slide so you would find it on the Internet later. So please, if you would like, take a photo on something like that. Yes, this is the main idea. Documentation for your public APIs will make your users happier because it would act like, first of all, the better documentation and it would act like a safety net around your library. Basically, you are defining a clear interface between your code and the user code. And if some kind of error is happening in your library while your users are using it, they could run a type checker or they can just look the errors in their ID and they will see that they are using it wrongly. They will not report wrong bugs to your tracker just because they won't get your documentation right. So this is the idea. Types are like better documentation, better for IDs and so on. And as I've already told you, you don't have to annotate everything. Just annotate your public API and see if it's good for you. I think that in most cases that would be enough. Another tip is start with simple types. In most cases, it is enough to use types like stir, built-in types int and so on. You can use built-in collection types like list or dict and if you want to specify the types of elements inside your collection, please use collection types from the typing module, the uppercase versions, capital case dict and so on. The path for it for, as you know, defines some advanced type constructs as well, but use them only when you really need them. And remember that you can always put any if some particular type is tricky as you don't know how to express it or it's too long and so on. So try to stick with simple things. They would be simpler for your users and you won't spend your time trying to invent some crazy static typing construct in your code. All right. So another tip is be liberal in what you accept in your functions and conservative in what you've done. Here's a little piece of code. It's basically a function that iterates over iterable integers and filters out the even elements and returns a list of them. Here you are using only the dunder iter operator on numbers because you are using the generated comprehension in the code. So you don't have to restrict your users to pass a list here, for example, or an iterator. It is enough to say that the numbers should be an iterable. The more generic type you could probably mention that would work for this function. And speaking about the return values, in this case, I think you could be quite specific about what you return because you might want to allow your users to use some methods on the concrete type you are returning. For example, here we are returning the list explicitly and you might want to allow your users to avoid, for example, the append method or something like that. Note that the typing module defines all kinds of these abstract collection types like mapping or sequence when you can access stuff by index operator, set, and so on. So they are good for your parameters, but they are somewhat less good for your return values. It depends actually on you. All right. So far we've seen some type hints written formally using Python 3 function annotations, right? And in order to use those annotations, it is enough for the user to just keep on stall your library. The type annotations are in the code. They are available. Everything's good. And this is compatible with Python 2 and some other Python versions if you install the typing module from PyPR. Right now you have to specify typing equals equals 3.5.0 beta 1. Otherwise you will get just an empty package. But this is because the library is very new and the developers haven't had the chance to fix it. All right. And what about Python 2? I guess many of you actually are using Python 2. So here's another quid poll about Python 2 versus Python 3. Who of you are using Python 2? Please raise your hands. 90%, I guess. And who of you are using Python 3? Wow. This is really good. Python 3 is finally coming. Great. All right. So those of you who still need to support Python 2, there is this thing called Python Stops. Just a quid reminder of what they are. They are Python files with the different PyI extension. And they contain no bodies for methods. Just the ellipses literal. And here's our little stop for the elementary library. The actual elementary library of the standard library of Python 3.5. And this is how we should define type hints for the library. Not by modifying it, obviously, but by writing this kind of stops. And you can use these Python 3 stops with your Python 2 code. And the type checkers will understand that. So stops in Python 3, code for Python 2. All right. So stops are very good for annotating not only the standard library, but for third party libraries as well. Type checkers would expect them to be somewhere on your assistive pass. So if you are creating some kind of project that uses lots of third party libraries and you want to annotate them in order to get better code completion, you can just create a folder inside your project. For example, a folder called type hints. And put your annotations for all the libraries in there. Just add it to your assistive pass. And then you can, for example, share this folder with the rest of your project using Git or Mirkorel or whatnot. And your colleagues will have your type annotations. All right. So if you do that kind of stuff, if you are interested in creating Python sub files, please share your stuff with others if you are annotating third party libraries. There are several ways of doing that. The main is to send a pull request to this new type shared repo. It's the wrapper mentioned in PEP 4.4. And it is supposed to be the universal collection of all the steps available. Basically, it's modeled after the definitely typed repo for the type script, the language that has type hints from its early days. It's still quite new. And this model of creating steps for libraries is very successful. It's very, very well established in the type script community and in the JavaScript community at the whole. For example, people who are still using JavaScript, they clone this definitely typed repo in type script for their project and check their code against these type script definitions of libraries. So in Python, we will have the same situation where you can actually check your code against this large collection of steps for all kinds of libraries. Another way to specify it in the PEP is to create a package with steps on PyPy and use it. All right. What if you don't want to invest in writing type hints, either in functional annotations or in steps? Then you still can benefit from that. For example, if any of you are PyCharm users, you might not know it, but you are already using Python annotations, functional annotations. Because in PyCharm, we have this kind of, behind the scenes repository of type hints for the standard library. Not all of it, but for some parts. So basically, you are already using type hints, but they are written using our own legacy syntax and we are going to switch to this new syntax. That's why we are interested in this proposal and we are contributing to it. We want all the tools to use the same shared collection of steps in order to be more productive as a community. Another way of using steps by other people would be cloning this repo and start using it. All right. So in conclusion, I urge you to try type hints. They are really good. I'm a fan of them, as you probably already know, but they are really good. And there are a few steps, a few very simple steps, how to, you can get started with it. Just grab the latest version of Python 3.5 beta or use some older version of Python 3 and get some of the tools, mypy or pycharm, and just start using type hints. Try it. I do want to know your thoughts about it. So please contact me. Here's me on Twitter and my website, PyRx.ru. I really want to hear what is your experience with type hints. Do you like them? Do you find them good for your users or maybe bad for your users? Are you going to switch to Python 3 with type hints? I'm very interested to hearing all those kinds of stuff from you. So please, please contact me. And the slides are already available on the Internet via this URL. So thank you for your time. Very interesting talk from Andrei. Any questions? Thanks. Thanks for the talk. I'm interested if you can do an easy introspection into the type hints to do like validation for RPC, for example. For RPC, well, you have to write some kind of tool that would understand the introspection API of your RPC library and generate those kind of stops. So it's a piece of software that is perfectly valid and it's a good idea to create these kind of things for your RPC mechanisms. And can you use type hints to do that, to inspect the type hints and do validation based on that? Yeah, yeah. That would be also a good idea, yes. Thank you for the talk. So you said we should all annotate our public APIs and you also said that PyCharm, for example, has already the tooling to infer type hints from the code itself, in some simple cases. Maybe there is also a way to create stops for us because we are lazy to annotate everything? We are thinking about this kind of feature in PyCharm 5.0, but it's still work in progress. So let's see, we will see what is possible here. We are quite good at inferring the return type for functions, but it is very tricky to infer the type of parameters to the function because the only source of information for parameters, actually there are two sources, the attributes to accessing of the parameter inside the body of the function and the usages of the function, but they are all kind of not so, they contain much less information than you can get for the return type based on calling to constructors or literals and so on. So yeah, we will see what's possible here. Is it possible to use if-punches in sub files in case, like, when type is different from Python 2 and Python 3? Or, sorry, could you repeat the question? Is it possible to have, like, an if-pyton2 in a sub file? Yeah, definitely, yes. The PEP 44 defines several ways of using conditionals like that, but we don't support them in PyCharm 4.5, but we are going to support them in PyCharm 5. Do you think that it would be a good way to provide source compatibility with Python 2 to make tools like 3 to 2 to generate the stub files from the source code instead of repeating yourself and defining the API basically two times? You mean, like, generating- No, I mean, targeting Python 3 in development and then generating the stub files from the Python 3 source code? Ah, yeah, yeah. It is actually a very simple task. I guess it's a good-it would be a good addition, for example, for the MyPy tool. Yes, it's definitely possible. Hi. Is there any effort or project that is putting the type hintings on all the standard library? Because it could be a nice party to put it all there, and when Python 3.5 is released, having all the standard library annotated. The project is called TypeShad. So TypeShad will eventually have the stubs for the whole Python 3.5 library and Python 2.7 library. There will be stubs for the standard library, yes. Is there any tool to generate the stub file from dog strings type information? I'm not aware of any kind of tool like that. It would be not so hard to create one, but I'm not aware of any of these tools. Thank you. So I played around a bit with MyPy, and the first thing I looked into was trying to detect if a function returns an optional value, so it can be none or it can be something else. And that's something that MyPy does not currently pick up. Does PyCharm do that? It's a question with some bit of history. We used to support those kinds of checks when we still didn't have any type annotations whatsoever, and we used only the inferred type information. And we used to have these checks when you have to actually write something like if value, and then you would be able to use the value, and we will be sure that it is not none. But then we received a lot of reports from our users that this is not what they were expecting from PyCharm, because the inference type could be not so accurate. And if you are explicitly specifying that you are expecting an optional type, or you are returning an optional type, and then using this type, then I guess we can trust your annotations, because this is what you are expected from the defined behavior of your functions, and we will support checking for optional types in PyCharm 5. Any more questions? No? Wait put back what I created, not necessarily an external variable in PyCharm cross just
I'm going to talk about how you can benefit from So, type hints in Python, they are used to be of type annotations is readability. Type hints are all about readability. For both humans and in this talk. So, let's start with type hints for element variable. And it is assigned a value of You can use built-in collection types like list order to use those annotations, it is enough for that. So stops in Python 3, code for Python 2. annotate our public APIs and you also said that eventually have the stubs for the whole Python