Python 3: ten years later
Stinner, Victor
Draft of the talk: Growing populary of the Python programming language Defacto language in the scientific world, replacing other closed source and more specialized but limited language Favorite programming language used as the first language to learn programming Port Python 2 code to Python 3 Port Python 2 code to Python 3 No: "Add support for Python 3", don't loose Python 2 support Port Python 3 code to Python 2 Python changes to make the transition smoother: PEP 414: u"syntax" reintroduced in Python 3.3 PEP 461: bytes % args, Python 3.5 More "Py3k" warnings added to Python 2.7.x Linters like flake8 detect some issues six, futures, modernize, 2to6, etc. Analysis of the transition Good: At Pycon US 2014, Guido van Rossum announced that the Python 2.7 support was extended from 2015 to 2020 to give more time to companies to port their applications to Python 3. Bad: 2to3: "drop Python2 support at once", don't work when you have dependencies. If done again, would it be different? Yes, obviously. Python 4 will be different than Python 3: no more "break the world" release, but a "regular deprecation period" release, as any other release. Break things, one by one :-) Bugs that won't be fixed in Python 2 anymore Some bugs cannot be fixed without breaking the backward compatibility Unicode Support Python 2 I/O stack bugs: rely on libc stdio.h Security: hash DoS, enabled by default in Python 3.3, Python 3.4 now uses SipHash subprocess is not thread-safe in Python 2. Python 2 subprocess has many race conditions: fixed in Python 3 with a C implementation which has less or no race condition. Handling signals while forking in complex. threading.RLock is not "signal safe" in Python 2 Python 2 requires polling to wait for a lock or for I/O. Python 3 uses native threading API with timeout and has asyncio. Python 3 uses a monotonic clock to not crash on system clock update (ex: DST change). Python 3 has a better GIL. Python 2 inherits file descriptors on fork+exec by default. Python 3 don't: PEP 446. Functions can fail with OSError(EINTR) when interrupted by a signal, need to be very careful everywhere. SIGCHLD when a child process completes, SIGWINCH when using ncurses. Python 3.5 restarts the interrupted system call for you. Performance Python 3.6 is now faster than Python 2.7 Evolutions of the Python language Python 3.5 PEP 492: async/await "keywords" for asyncio. (Really keywords in Python 3.7.) PEP 461: bytes % args and bytearray % args PEP 465, a new matrix multiplication operator: a @ b. PEP 448: Generalized unpacking: head, *tail = list mylist = [1, 2, **other_list] mydict = {"key": "value", **other_dict} Python 3.6 PEP 515: million = 1_000_000 name = "World"; print(f"Hello {name}!") PEP 526, syntax for variable annotations. PEP 525, asynchronous generators. PEP 530: asynchronous comprehensions. Bury Python 2? "Python 3 only" Fedora 23, Ubuntu 17.10: no python2 (/usr/bin/python) in the base system 2017, April: IPython 6.0 2017, December: Django 2
 At the very beginning of Python 3, there was a project called Python 3000. It was in 2006. It started with the PEP 3000 called Python 3000. The idea was to focus on what is called Python warts because Python 2 got more and more users and slowly people started to complain about some design issues. This design issues started to be collected as a list and they were called the Python warts. For example, in Python 2, you have two different types for integers. You have the long integer and the small integer. Depending on the number, depending on the operation, you may get a small or large integer. In that case, to test the type of a variable, you have to test for both types at the same time. This can be surprising for newcomers. They may not understand why we have two different things for the same kind of value, a number. There is also something called New Class. It's a new addition during the Python 2 development. The New Class should be used to get new features like, for example, Python properties. If you don't use the New Class, the properties don't work as expected. Again, it's a little bit surprising that you have to generate explicitly from the object type to get the New Class. It's surprising for newcomers. For the division of two integers in Python 2, you get an integer. This was a choice made from the beginning of Python. But when Python becomes more and more popular, some people were surprised that they didn't get a float because in other languages, using a simple calculator, you get a float. It has been decided to change that to always return a float because there is already an operator in Python 2. Instead of a single slash, you put a double slash and you always get an integer. For the Unicode, in short, I would say that if you only use bytes, you're fine. If all your inputs, all your outputs, all data inside your application and bytes, there is no issue. But if you use Unicode as inputs, outputs, Unicode everywhere in your application, you are also fine. You start to get some nasty issues when you have bytes and Unicode in your same application. Especially you don't get errors all the time, but only if a single character has an accent. It's a little bit difficult to debug such issue. It may happen in production. And more and more people were angry against this design. The comparison, Python 2 again made a choice from the beginning is that you are always able to compare two variables in Python 2. This means that if some object doesn't make any sense for comparison, Python takes the name of the type and compares the name. For example, if you compare the string hello and the number 3, you get the string s tier for the type string and you get the string int for the number 3 and you compare these two values to decide how to order values. And this behavior can be surprising and maybe not so one that you may expect. So we also decided to change that and not try to guess, but raise type error in Python 3. It means that in Python 3, you have to make it very explicit. And finally, for the imports, another trap for newcomers is that if you have a module which has the same name as the module of the standard library like the CIS module, you get a conflict between the module from the standard library and in your project. Because Python decided to use something called relative imports, which means that you can import the module in the same directory without having to specify the full path to the module. So again, we wanted to change that to always use absolute imports, which means that you always have to write the full path to the module to avoid such name conflicts. So to make sure that Python 3 is not going to be a big mistake from the very beginning of the project, we tried to have something called risk management to not break everything. And the idea is to only change acknowledged awards, so only the very short list of design issues. And we also decided to have an open community process to decide what should be changed and the process where the PEP, PEP with a number starting with 3000. And we also decided to not reimplement Python from scratch, the C Python, the implementation of the language. It's not done from scratch, but it's just a fork from Python 2. Again it's to limit the number of subtle changes that you may get if you do it from scratch. And again, from the very beginning, we decided to put a deadline for Python 2. At the beginning, it was 2015, I think, to say that today you are fine with Python 2, but be prepared that at this time you will have to switch. And in 2008, the Holy Grail, Python 2, the first release. And it was 10 years ago. You take your whole application, all dependencies, everything, you run an application called Python 2 to 3 to convert your Python 2 code base to Python 3 and done. Okay. Maybe it was not that easy in practice. Maybe there are some practical issues. Because the thing is that when you have third party dependency, the author of the module maybe don't want to drop the Python 2 user base because when you run 2 to 3, you lose Python 2 support. And when Python 3 has been released, Python 3 was a new thing. It was not supported by Linux distribution, for example. It was not installed by most users. It was really something very new. So many users didn't want to hear anything about Python 3. And another practical issue is that even if you succeed to port a dependency to Python 3, it's not enough. You really have to port all your dependencies and dependency of dependency of dependency of dependency. And so you cannot start to port your own application before you port dependencies. So it takes a lot of time just to start on your own code. And another bad or good surprise is that we didn't know that Python 2 was so much popular and it was heavily used in production. To explain you another problem called technical depth, imagine that you are talking with a new manager. And your manager asks, why should I let you work on Python 3 support? And the developer says, for all this new cool Python 3 features, and manager, okay, but can I use all these features? Well, we still have to support Python 2? No. Because when you support Python 2 and Python 3 in the same code base, technically you cannot use Python 3 features because you have to support Python 2. So you can spend a lot of time to port everything, but at the end you don't get any kind of direct benefit from the port. So you have to spend a lot of time for it. So again, it's difficult to justify if you are a manager of you. If you would like to port an application to Python 3, you have different options for the code base. For example, you can decide to have two different branches in Git. And one example is that the DNS Python project has been forked. There were two different projects, DNS Python for Python 2, and DNS Python 3 for Python 3. Sometimes the maintainer of the project doesn't care about Python 3 because, for example, of the Harry manager. And so the community decided to fork the projects. For example, the PIL module has been forked as PILO. It's a module to manipulate pictures, to resize and do whatever you want with pictures. And not only they added Python 3 support, but they also added new features, which is really cool. And in the worst case, you are stuck by a dependency like MySQL Python. I was working on OpenStack and all OpenStack rely on this module. But the maintainer decided to not reply to any break reports. He didn't reply to pull requests. So we were stuck and we didn't know what to do. So someone decided to maybe just fork the project and put it as a new name. But then you get a new issue is that for Linux vendors like Red Hat, we didn't know if we should rely on the old version or the new version. Because we don't know if the old version is going to die or maybe it's going to be maintained for the maintainer decided to wake up. To give you an idea of what was Python 2 at this time, you have to know that when Python 3 has been released, the stable version of Python 2 was 2.6. And the issue with Python 2.6 is that you have to make a lot of changes to make your code compatible with Python 3. For example, for all your nicole string, we add the U prefix to annotate in the nicole string. But if you do that, you get a syntax error in Python 3. So to be able to have a single code base for your nicole string, you have to remove the prefix and call a function from the 6th module. So you have to write 6.U parenthesis your string. It was a little bit annoying to do that on all your string. Because of that, some project just decided to not do anything for Python 3. Another issue is that if you really want to use new Python 3 features like the new method of the unit test module, you can use something called unit backports. Like unit test 2, which backports new features from Python 3. But it can be an issue to have dependency because 10 years ago, it was a little bit difficult to install all dependency and take care of that. It was more difficult to install it. So we had to wait up to Python 3.2, you had the U issue and you have to wait for the next release to get a fix for that. So after autumn comes the cold winter time. And we will start with our website called the Python 3 wall of shame. So in 2011, someone decided to create a website called Python 3 wall of shame. But the idea was not to blame the author of the modules. The intent was to motivate the package manager to port the code to Python 3. But when the website has been created, only 9% of the top 200 modules were compatible with Python 3. And to recall what was Python 10 years ago, we had higher notified 3 big players in the Python community. There is a tree steal, which is a very good networking solution for clients on server to do asynchronous programming, which is a very efficient way to do networking in Python. There is a mercurial, which is similar to Git, it's a source control management. And there is also Django, a very famous framework to do websites. And this free project had different issues with Python 3. Tristed and mercurial are very close to the hardware, to the low level. For example, Tristed works on the socket level. And at the socket level, the network level, there is nothing called Unicode. There are only bytes. So all the project is designed to use bytes. And in Python 3, using bytes is not the first citizen class type. We prefer to use Unicode. So at the beginning, it was a little bit difficult to Tristed to port the code base to Python 3. And it's the same issue for mercurial because they used bytes not only for the file name for very good reason, but also for the file content. For this two reasons, it was also difficult and it's still difficult to mercurial to be ported to Python 3. And for Django, when Python 3 has been released, the support of Unicode was only at the beginning. So, because it was very difficult to port code, because nobody wanted to be the first one to port code, we started to see this Frenchman with Python Troll. And the Python Troll, considered that Python 3 doesn't bring anything. Because as I explained with the manager, when you port code Python 3, you still have the same features. You don't bring anything. And when you move to Python 3, you are forced to use properly Unicode. And nobody wants to do that because it's very difficult for good reason. And it's much more simpler to use Python 2 with bytes because you don't have to worry about Unicode, people speaking a different language, just pass text and it works. So Python 2 is a failure. So maybe what we can do is just continue the development of Python 2.7. So there is an idea from the trolls is that maybe we can just continue the development and create a new version called 2.8 to add new features from Python 3, but don't break the backward compatibility. And four years ago, there was still an article saying that we were still debating transitional Python 2.8 and they were concerned that Python 3 would never take off. And also that Python 3 only represents 2% of packages. So to make things more explicit, we wrote a pep called the pep not found, the 404. It's a Python 2.8 unreleased schedule. So it's a very explicit document to say that we know that we are breaking the backward compatibility but we are not going to spend more time on Python 2 because we are a small team of core developers. We don't have enough time to maintain two versions. And if you use the numbers differently in 2013, someone else counted that there are 80% of the top 50 projects which were compatible with Python 3. So it wasn't that bad. And Python 2.7 as his end of life extended at the Python in 2014. And for me, it's one of the best moves that we did for Python 3 is that we say that, okay, we know that it's difficult to port your code. You have very good reason to not port your code today because not all dependencies are compatible because it costs a lot of money and we don't want to force people to move to Python 3. We didn't want to force people to do anything. So we just say that you have five more years to do your work and we extended the deadline to 2020. So after the cold winter comes fresh air, flowers, it's a new beginning. And the first great move for Python is that our first problem has been solved. So what is our first problem is that when you want it, we install a dependency which was a very common question because using dependency is a good practice. The question is how can I install a dependency? And the answer was, oh, you have to install Saptor tools. Okay, but how can I install Saptor tools? And it was a little bit difficult because you had to find a script somewhere on the Internet to download the script. You need the system administrator permission to be able to run the script. It does install something on your system. It's not from the system, it's from the Internet, so you have to trust it. It was not the perfect solution. But in 2011, PIP 1.0 has been released. And in 2014, we decided to include not PIP itself, but Ensure PIP, which is a new module to install PIP directly into Python. So in Python 2.7.9 and Python 3.4, there is no Ensure PIP to be installed to VAT. So very easily install PIP. In practice, even if you don't have the Internet connection to the Internet, you are still able to install PIP using that. And thanks to this change, slowly, PIP became the de facto installer. And you don't have to answer how to install dependency. You just know that PIP is the correct way to do that. And we also see a Linux distribution to include PIP directly in the system. So today, it's very easy to have dependencies. Support and application to Python 3, we saw that 2.2.3 was not the best approach. So what we saw is that maybe we should not remove Python 2.0 support, but say differently like add Python 3.0 support. And this is a big, big change because you don't lose your users. You only move to Python 3 step by step. And you don't have to force all your dependencies, all your code base at once. You can start with a simple change or a single file and do it incrementally. And we started to see new tools like Modernize. It's a tool to convert your code base without losing the Python 2.0 support. And let me tell you the story of Sixer, which is my own tool. So I'm working on OpenStack. And on OpenStack, if you send a pull request generated by Modernize, you get a giant pull request. And you are sure that in a few hours, you will get a conflict because OpenStack is moving very quickly. And all files are modified. So it's really difficult to get a pull request with no conflict for long hours or one day, especially a very large pull request. So what I did is to write a new tool which only apply selected operation, like, for example, just add parentages to the print statement. And by doing that, you can generate much smaller pull requests. You get less conflicts. And it's much more easier to review also for the maintainer because you see that the change only add parentages. So it's a simple change and it's easy to approve. And thanks to this tool, I was able to pass all unit tests of OpenStack to Python 3. And another good advantage of this method is that if you are a CI to validate all your work, you can make sure that each change doesn't break the backward compatibility. It doesn't introduce regression. So you can apply changes one by one and make sure that everything is fine. If you have a larger code base, maybe you need to do it differently because some companies have very, very large code base, like, for example, Instagram or GrubBox have at least two million lines of code. So what they need to do is that when you have a very, very large code base, what you can do is first to add new tests because sometimes in a big company, people like to add new features but they just forget to add new tests. So adding a new test is a good practice for your existing code base, but it also helps to make sure that the migration doesn't break anything. And GrubBox is also trying to a different approach. As I decided to work on the type annotation, the idea is that you annotate the type of the arguments of your function but also the return type of your function. And by doing that, they expect that it will be much easier to pass the application to Python 3 without introducing bugs because if you have a type annotation, you know which type are expected and it's much more easier to make sure to validate everything. To make the migration simpler, we made the different changes in Python itself but also in tooling around Python. And the first big change is that in 2012, we decided to reintroduce the U prefix. Even if it doesn't make sense, if you only look at the Python 3 language to have test U prefix because all strings are unique by default, adding the U prefix was very useful to write a single code base for Python 2 and Python 3. So it's fixed a lot of issues to help people to port that code. In 2015, Python 3.7 has been released with a new operator for bytes. It's a way to format a byte string. This is very useful for projects like Twisted or Mercure, which manipulates a lot of bytes. And we also added more Python 3000 warnings. So Python 3 warnings. We added these warnings in Python 2. So the idea is that even if you don't port anything, just enable the warning and you will see which code may have some troubles on Python 3 and you have to first focus on this code before starting the big migration plan. And to be able to use Python 3 features, we saw more and more backports of Python 3 features to Python 2. For example, unit test 2, Enum 3.4, which is a new Enum module. And thanks to all these backports, moving to Python 3 really had something. And after the spring comes the summertime. So to come back to the website, the wall of shame, the author of the website decided to change the title from wall of shame to wall of superpower. And today we got almost 100% of the module which are compatible with Python 3. In practice, it's a little bit less. It's only 90% 5. Because in this long list, you have some modules which have been replaced by new modules because the old module doesn't make sense on Python 3 or the maintainer decided to restart from scratch. So in this list, you have some modules which you can just ignore. And you can consider that the top 200 modules on PAPPI are compatible with Python 3. And another good news for Python 3 is that finally after a long development process, we had enough time to optimize Python 3. And Python 3 is now faster than Python 2. On this graphic, you can see the result of a few benchmarks and the orange line in the normalized result of Python 2. So it's always one. And if the other bar is lower, it means that Python 3 is faster. And you can see that depending on the benchmark, it's more or less faster. And in some cases, it's way faster. To give you these numbers differently, there is a company Instagram which has 700 million users. So it's a very, very big Django application running every day on mobile platform on everything. So it's a big, big product. It's fully written in Python. And they switched to Python 2 to Python 3. And thanks to this move to Python 3, they saved 12% of CPU just by doing the change. And they also saved 30% of memory, which is very something very important for them because when you have so many users, you need a lot of servers and at this scale, as a performance and a really, really matter. It's really expensive to have so much hardware. And if you didn't notice, Python 2 is a little bit old today. And even if we are doing our best to fix all bugs, sometimes we just cannot fix some issues. Because the thing with Python 2 is that it's called Super Stable because we have so many people which rely on Python 2. This means that if you do any kind of change in Python 2, even to fix a bug, there is a risk that someone rely on the bug and also the application rely on this bug. So we think twice when we have to back port a fix to Python 2. And sometimes it's more a design issue that has to require a big change in the language. For example, the first bullet is Unicode. Even if technically we could try different options in Python 2 to have new warnings or new mode for Unicode, we decided to not touch the current implementation of Unicode. Just because, as I said, a lot of people rely on it. And we cannot change anything because it would break too much code in the wild. Another example is that for security reasons, we decided to randomize the order of items in a dictionary on a set object in Python 3. Because if you inject a specific case in a dictionary, you may get the worst complexity of the dictionary type. And in that case, you may crash the server just by sending a few bytes to an HTTP server. So what we did is to randomize the order. But we cannot do that in Python 2 because if you change the order of the dictionary, the representation of a dictionary change and it can break too many unit tests. So for your information, you have to explicitly enable randomization using dash uppercase error to enable the workaround for the security issue. Another very annoying issue of Python 2 is that the subprocess module is not thread safe. So maybe you did not notice because you don't use threads or maybe you did not notice because you're lucky. But the idea of not being thread safe means that everything is fine. Everything is fine. Everything is fine until one day in production, you get a crash and nobody is able to reproduce a crash because it really depends on the workload. It really depends on the order of operation and on your developer laptop. You have different timings. So it's very common that race conditions only occur on production. So nobody wants to rely on something which is not thread safe. So be careful with the subprocess. And for your information, there is a back port called subprocess 2.3.2 which backpots the thread safe implementation. Another example is the recursive lock which again is not as a race condition. As the race conditions are not threads but signals. It means that if you use recursive lock and you get a signal, signal is for example that you spawn a process and when the child process completes, you get a notification to say that hey, your child completed, it's time to read the exit status of the process. So when you get the signal, you may get a race condition and your lock may become inconsistent. So you may get a dead lock just because of a signal. And this is really annoying because a signal can come anytime. You don't, it really depends on the timing of the child process and you don't control the timing of the child process. So it's really dangerous to use a recursive lock on Python 2. Another example is that clocks in Python 2 use a system time. So during winter time or summer time when you use a change of clock for DST, your application may crash just because of that. Or for example, if the system administrator decided to change the time manually, again you may get a crash. And the solution for that is to use something called monotonic lock. The monotonic lock only increases, there is no jump in the future or in the past. So the good news is that you just have to switch the Python 3 and you're fine. So for example, for the time in Python 3.3, we added the time.monotonic function and slowly we modified Python to use monotonic lock inside Python. And to explain you something else about Python 3 is that as I said, Python 2 is super stable, we cannot touch anything. And the good thing with Python 3 is that it's still kind of not as popular as Python 2. So we can afford to make more backward and uncomfortable changes. For example, I change how file descriptors are inherited by the child process. And since Python 3.3.4, when you spawn a child process, you don't inherit the file descriptor. It means that you didn't inherit Open Socket, you don't inherit the sensitive file which contains passwords or all your sequets. But this was our backward compatible changes and we cannot do such a change in Python 2. And another example is that in Python 3.5, I made another change which is to when you get a signal, the CIS call is interrupted, you get an error. And the behavior in Python 2 is that you get an exception and you have to do something. And in Python 3.4.5 is that if the signal handler doesn't raise an exception, Python will just retry the blocking CIS call for you and you don't notice anything. You're just fine. And we have an awesome when he approved my change for the file descriptor and he returns, he wrote that we are aware that the code breakage, this is likely to cause and doing it anyway for the good of mankind. And again, Python 3.0 is much, much better because you have a lot of new modules in the standard library. I counted 21 modules for 3.7. Just to give you some examples, there is a asynchronous programming, there is a new which is very useful, there is pathlib which is very popular to manipulate paths, there is unit test.mock which is a very, very nice way to reduce the dependency of a unit test on really test something very specific. So to mock another function. And not only we fixed bugs and we added new features, we also made changes in the language. And for example, in Python 3.6, we added something called the f-string. It's a new way to format string. And in my opinion, it's just the best way to format string because it's very short, it's very obvious when you read the string. And yeah, it's just the best because not only you can format a variable, but you can even call a method. You can, in practice, you can add any Python expression. It just works. It's amazing. And therefore, asynchronous programming, we made different changes. First we added the yield from. After that, we added async on our weight keywords. And we also added support for asynchronous generators. So yield inside an asynchronous method. There are asynchronous list comprehension. But you have also more changes. Like for example, we have the keywords only arguments which are very useful to write a better API. The prints is no longer a statement, but it's a function. And I really prefer to have this function. You can use the start to unpack. But also to inside a list, now you can use the start. This is a new feature. It's a new way to build a list. And in my opinion, it's straightforward and very useful. You can also do it for dictionary. You can use underscore inside numbers to make numbers easier to read. You can annotate the type of variable. For the with context manager, with statements, you can add multiple expressions. There is, again, bytes formatting. And there is something very useful for non-py users. There is a new operator for matrix multiplication, which is the hats. It avoids to, this is an in fix way for the matrix operation. Because previously you had to call a function and it was harder to read. So I spent a lot of time to explain that Python 3 is way better. And we are reaching slowly the deadline, which is 2020, the deadline of Python 2. So the question is now, is it time to maybe bury Python 2? So the good news is that in Fedora, we already started to do that three years ago. In Fedora, in 2023, there is no Python 2 by default. There is only Python 3 by default. And in the release of human 2 last year, again, there is no Python 2 by default, only Python 3. And there is a Python 3 statement, which is a common timeline for all applications for the end of support for Python 2 in the application. And there are big projects like IPyton 6 and Django 2, which dropped Python 2 support. And I'm working for Red Hat. So for your information, in the latest release, Red Hat 7.5, we announced that Python 2 is deprecated. So if your manager is still concerned about Python 2, you can say that even Red Hat, which has a very, very long support, we are aware that something is wrong with Python 2 and we are working on that. And for your information, the next release of Red Hat Enterprise Linux will be Python 3 only. And if you... This is thanks to my very cool colleagues that were working on that for two years. And if you use Red Hat Enterprise Linux or CentOS, there is also something called Software Collections, which is a way to install the latest version of Python on your old system. So technically, it's already possible to use Python 3 on Red Hat. And one open question for me is that we saw that changing the syntax and changing Python was really difficult. So maybe we can try to handle this issue differently and maybe try the JavaScript issue. The solution, which is to use transpiling using Babel and Polyfill. And by doing that, you are able to use the latest language on your old browser. And thanks to that, they are able to move way faster because you are no longer stuck by the version of JavaScript on your browser. You can use new features. Okay. Question? Thank you so much. Thank you. We have time for a few questions maybe. If there's... I've seen one over there. Just to come back to the latest slide, to be honest, I think that we understood that Python 3 was a huge pain for everybody. And if we do that one more time, we will not afford to break everything. And my expectation is that Python 4 will just be the release after the previous Python 3 release. And we will follow the same deprecation process to have a slow process to give at least two release to remove something. So we are not going to break Python one more time. That's good to hear. I was wondering whether the chart about performance was generic case because it puzzles me a bit. And the point is that from what I saw around the Internet, usually Python 2.7 performed faster on the general computing. But is there a usage of a sync or something like that which drastically changes the chart or it's the general case? I spent a lot of time to work on benchmark because I wanted to optimize Python. So this chart comes from a project called Perfomance. It's a benchmark suite. So I'm not sure that they are real representative of real applications, but we are trying to be... to use more complex benchmark to have a better idea of what is a Python performance. And what I just can say that on this benchmark suite, Python 3 is always faster. So there is only one case where Python 3 is slower is a start-up time. But we are working on that and it's getting better. So I'm sure that in some specific cases, Python 2 is still slower. But the good news is that this chart is for Python 2.6. It's an old chart. But Python 3.7 is even faster. So maybe it should try the latest version of Python. Because we optimized the method call and many other small changes. One last question. Is there any? So knowing how the transition went, what would you do differently if you could do Python 3 again? I try to explain that ten years ago, the choice that we made was a good way to do that. Because we didn't have enough information that we didn't know that we had so much users. We are not aware of the big dependency chain. So yes, if I do that again, what I would do is just to have a very smooth transition to make sure that even if we break something, we have a lot of time to port the code and not to push all backward and compatible changes at once. But do them one by one and make sure that we have the right tooling to do the conversion, right tooling to detect issues in your code. Because one of the very difficult issues with Python 3 is that when you divide two integers, you get a float. This specific issue is really difficult to catch in your code because just by reading the code, you don't know the type. So we need more tooling to detect that and to help users. Okay. Thank you so much. Thank you.
 At the very beginning of Python 3, there was a project called Python 3000.  It was in 2006.  It started with the PEP 3000 called Python 3000.  The idea was to focus on what is called Python warts because Python 2 got more and more users  and slowly people started to complain about some design issues.  This design issues started to be collected as a list and they were called the Python warts.  For example, in Python 2, you have two different types for integers.  You have the long integer and the small integer.  Depending on the number, depending on the operation, you may get a small or large integer.  In that case, to test the type of a variable, you have to test for both types at the same time.  This can be surprising for newcomers.  They may not understand why we have two different things for the same kind of value, a number.  There is also something called New Class.  It's a new addition during the Python 2 development.  The New Class should be used to get new features like, for example, Python properties.  If you don't use the New Class, the properties don't work as expected.  Again, it's a little bit surprising that you have to generate explicitly from the object  type to get the New Class.  It's surprising for newcomers.  For the division of two integers in Python 2, you get an integer.  This was a choice made from the beginning of Python.  But when Python becomes more and more popular, some people were surprised that they didn't  get a float because in other languages, using a simple calculator, you get a float.  It has been decided to change that to always return a float because there is already an  operator in Python 2.  Instead of a single slash, you put a double slash and you always get an integer.  For the Unicode, in short, I would say that if you only use bytes, you're fine.  If all your inputs, all your outputs, all data inside your application and bytes, there  is no issue.  But if you use Unicode as inputs, outputs, Unicode everywhere in your application, you  are also fine.  You start to get some nasty issues when you have bytes and Unicode in your same application.  Especially you don't get errors all the time, but only if a single character has an accent.  It's a little bit difficult to debug such issue.  It may happen in production.  And more and more people were angry against this design.  The comparison, Python 2 again made a choice from the beginning is that you are always  able to compare two variables in Python 2.  This means that if some object doesn't make any sense for comparison, Python takes the  name of the type and compares the name.  For example, if you compare the string hello and the number 3, you get the string s tier  for the type string and you get the string int for the number 3 and you compare these  two values to decide how to order values.  And this behavior can be surprising and maybe not so one that you may expect.  So we also decided to change that and not try to guess, but raise type error in Python  3.  It means that in Python 3, you have to make it very explicit.  And finally, for the imports, another trap for newcomers is that if you have a module  which has the same name as the module of the standard library like the CIS module, you get  a conflict between the module from the standard library and in your project.  Because Python decided to use something called relative imports, which means that you can  import the module in the same directory without having to specify the full path to the module.  So again, we wanted to change that to always use absolute imports, which means that you  always have to write the full path to the module to avoid such name conflicts.  So to make sure that Python 3 is not going to be a big mistake from the very beginning  of the project, we tried to have something called risk management to not break everything.  And the idea is to only change acknowledged awards, so only the very short list of design  issues.  And we also decided to have an open community process to decide what should be changed and  the process where the PEP, PEP with a number starting with 3000.  And we also decided to not reimplement Python from scratch, the C Python, the implementation  of the language.  It's not done from scratch, but it's just a fork from Python 2.  Again it's to limit the number of subtle changes that you may get if you do it from scratch.  And again, from the very beginning, we decided to put a deadline for Python 2.  At the beginning, it was 2015, I think, to say that today you are fine with Python 2,  but be prepared that at this time you will have to switch.  And in 2008, the Holy Grail, Python 2, the first release.  And it was 10 years ago.  You take your whole application, all dependencies, everything, you run an application called  Python 2 to 3 to convert your Python 2 code base to Python 3 and done.  Okay.  Maybe it was not that easy in practice.  Maybe there are some practical issues.  Because the thing is that when you have third party dependency, the author of the module  maybe don't want to drop the Python 2 user base because when you run 2 to 3, you lose  Python 2 support.  And when Python 3 has been released, Python 3 was a new thing.  It was not supported by Linux distribution, for example.  It was not installed by most users.  It was really something very new.  So many users didn't want to hear anything about Python 3.  And another practical issue is that even if you succeed to port a dependency to Python  3, it's not enough.  You really have to port all your dependencies and dependency of dependency of dependency  of dependency.  And so you cannot start to port your own application before you port dependencies.  So it takes a lot of time just to start on your own code.  And another bad or good surprise is that we didn't know that Python 2 was so much popular  and it was heavily used in production.  To explain you another problem called technical depth, imagine that you are talking with a  new manager.  And your manager asks, why should I let you work on Python 3 support?  And the developer says, for all this new cool Python 3 features, and manager, okay, but  can I use all these features?  Well, we still have to support Python 2?  No.  Because when you support Python 2 and Python 3 in the same code base, technically you  cannot use Python 3 features because you have to support Python 2.  So you can spend a lot of time to port everything, but at the end you don't get any kind of direct  benefit from the port.  So you have to spend a lot of time for it.  So again, it's difficult to justify if you are a manager of you.  If you would like to port an application to Python 3, you have different options for  the code base.  For example, you can decide to have two different branches in Git.  And one example is that the DNS Python project has been forked.  There were two different projects, DNS Python for Python 2, and DNS Python 3 for Python  3.  Sometimes the maintainer of the project doesn't care about Python 3 because, for example, of  the Harry manager.  And so the community decided to fork the projects.  For example, the PIL module has been forked as PILO.  It's a module to manipulate pictures, to resize and do whatever you want with pictures.  And not only they added Python 3 support, but they also added new features, which is  really cool.  And in the worst case, you are stuck by a dependency like MySQL Python.  I was working on OpenStack and all OpenStack rely on this module.  But the maintainer decided to not reply to any break reports.  He didn't reply to pull requests.  So we were stuck and we didn't know what to do.  So someone decided to maybe just fork the project and put it as a new name.  But then you get a new issue is that for Linux vendors like Red Hat, we didn't know if we  should rely on the old version or the new version.  Because we don't know if the old version is going to die or maybe it's going to be maintained  for the maintainer decided to wake up.  To give you an idea of what was Python 2 at this time, you have to know that when Python  3 has been released, the stable version of Python 2 was 2.6.  And the issue with Python 2.6 is that you have to make a lot of changes to make your  code compatible with Python 3.  For example, for all your nicole string, we add the U prefix to annotate in the nicole  string.  But if you do that, you get a syntax error in Python 3.  So to be able to have a single code base for your nicole string, you have to remove the  prefix and call a function from the 6th module.  So you have to write 6.U parenthesis your string.  It was a little bit annoying to do that on all your string.  Because of that, some project just decided to not do anything for Python 3.  Another issue is that if you really want to use new Python 3 features like the new method  of the unit test module, you can use something called unit backports.  Like unit test 2, which backports new features from Python 3.  But it can be an issue to have dependency because 10 years ago, it was a little bit difficult  to install all dependency and take care of that.  It was more difficult to install it.  So we had to wait up to Python 3.2, you had the U issue and you have to wait for the next  release to get a fix for that.  So after autumn comes the cold winter time.  And we will start with our website called the Python 3 wall of shame.  So in 2011, someone decided to create a website called Python 3 wall of shame.  But the idea was not to blame the author of the modules.  The intent was to motivate the package manager to port the code to Python 3.  But when the website has been created, only 9% of the top 200 modules were compatible  with Python 3.  And to recall what was Python 10 years ago, we had higher notified 3 big players in the  Python community.  There is a tree steal, which is a very good networking solution for clients on server  to do asynchronous programming, which is a very efficient way to do networking in Python.  There is a mercurial, which is similar to Git, it's a source control management.  And there is also Django, a very famous framework to do websites.  And this free project had different issues with Python 3.  Tristed and mercurial are very close to the hardware, to the low level.  For example, Tristed works on the socket level.  And at the socket level, the network level, there is nothing called Unicode.  There are only bytes.  So all the project is designed to use bytes.  And in Python 3, using bytes is not the first citizen class type.  We prefer to use Unicode.  So at the beginning, it was a little bit difficult to Tristed to port the code base to Python  3.  And it's the same issue for mercurial because they used bytes not only for the file name  for very good reason, but also for the file content.  For this two reasons, it was also difficult and it's still difficult to mercurial to be  ported to Python 3.  And for Django, when Python 3 has been released, the support of Unicode was only at the beginning.  So, because it was very difficult to port code, because nobody wanted to be the first  one to port code, we started to see this Frenchman with Python Troll.  And the Python Troll, considered that Python 3 doesn't bring anything.  Because as I explained with the manager, when you port code Python 3, you still have the  same features.  You don't bring anything.  And when you move to Python 3, you are forced to use properly Unicode.  And nobody wants to do that because it's very difficult for good reason.  And it's much more simpler to use Python 2 with bytes because you don't have to worry  about Unicode, people speaking a different language, just pass text and it works.  So Python 2 is a failure.  So maybe what we can do is just continue the development of Python 2.7.  So there is an idea from the trolls is that maybe we can just continue the development  and create a new version called 2.8 to add new features from Python 3, but don't break  the backward compatibility.  And four years ago, there was still an article saying that we were still debating transitional  Python 2.8 and they were concerned that Python 3 would never take off.  And also that Python 3 only represents 2% of packages.  So to make things more explicit, we wrote a pep called the pep not found, the 404.  It's a Python 2.8 unreleased schedule.  So it's a very explicit document to say that we know that we are breaking the backward  compatibility but we are not going to spend more time on Python 2 because we are a small  team of core developers.  We don't have enough time to maintain two versions.  And if you use the numbers differently in 2013, someone else counted that there are 80%  of the top 50 projects which were compatible with Python 3.  So it wasn't that bad.  And Python 2.7 as his end of life extended at the Python in 2014.  And for me, it's one of the best moves that we did for Python 3 is that we say that, okay,  we know that it's difficult to port your code.  You have very good reason to not port your code today because not all dependencies are  compatible because it costs a lot of money and we don't want to force people to move  to Python 3.  We didn't want to force people to do anything.  So we just say that you have five more years to do your work and we extended the deadline  to 2020.  So after the cold winter comes fresh air, flowers, it's a new beginning.  And the first great move for Python is that our first problem has been solved.  So what is our first problem is that when you want it, we install a dependency which  was a very common question because using dependency is a good practice.  The question is how can I install a dependency?  And the answer was, oh, you have to install Saptor tools.  Okay, but how can I install Saptor tools?  And it was a little bit difficult because you had to find a script somewhere on the  Internet to download the script.  You need the system administrator permission to be able to run the script.  It does install something on your system.  It's not from the system, it's from the Internet, so you have to trust it.  It was not the perfect solution.  But in 2011, PIP 1.0 has been released.  And in 2014, we decided to include not PIP itself, but Ensure PIP, which is a new module  to install PIP directly into Python.  So in Python 2.7.9 and Python 3.4, there is no Ensure PIP to be installed to VAT.  So very easily install PIP.  In practice, even if you don't have the Internet connection to the Internet, you are still able  to install PIP using that.  And thanks to this change, slowly, PIP became the de facto installer.  And you don't have to answer how to install dependency.  You just know that PIP is the correct way to do that.  And we also see a Linux distribution to include PIP directly in the system.  So today, it's very easy to have dependencies.  Support and application to Python 3, we saw that 2.2.3 was not the best approach.  So what we saw is that maybe we should not remove Python 2.0 support, but say differently  like add Python 3.0 support.  And this is a big, big change because you don't lose your users.  You only move to Python 3 step by step.  And you don't have to force all your dependencies, all your code base at once.  You can start with a simple change or a single file and do it incrementally.  And we started to see new tools like Modernize.  It's a tool to convert your code base without losing the Python 2.0 support.  And let me tell you the story of Sixer, which is my own tool.  So I'm working on OpenStack.  And on OpenStack, if you send a pull request generated by Modernize, you get a giant pull  request.  And you are sure that in a few hours, you will get a conflict because OpenStack is moving  very quickly.  And all files are modified.  So it's really difficult to get a pull request with no conflict for long hours or one day,  especially a very large pull request.  So what I did is to write a new tool which only apply selected operation, like, for example,  just add parentages to the print statement.  And by doing that, you can generate much smaller pull requests.  You get less conflicts.  And it's much more easier to review also for the maintainer because you see that the change  only add parentages.  So it's a simple change and it's easy to approve.  And thanks to this tool, I was able to pass all unit tests of OpenStack to Python 3.  And another good advantage of this method is that if you are a CI to validate all your  work, you can make sure that each change doesn't break the backward compatibility.  It doesn't introduce regression.  So you can apply changes one by one and make sure that everything is fine.  If you have a larger code base, maybe you need to do it differently because some companies  have very, very large code base, like, for example, Instagram or GrubBox have at least  two million lines of code.  So what they need to do is that when you have a very, very large code base, what you can  do is first to add new tests because sometimes in a big company, people like to add new features  but they just forget to add new tests.  So adding a new test is a good practice for your existing code base, but it also helps  to make sure that the migration doesn't break anything.  And GrubBox is also trying to a different approach.  As I decided to work on the type annotation, the idea is that you annotate the type of  the arguments of your function but also the return type of your function.  And by doing that, they expect that it will be much easier to pass the application to  Python 3 without introducing bugs because if you have a type annotation, you know which  type are expected and it's much more easier to make sure to validate everything.  To make the migration simpler, we made the different changes in Python itself but also  in tooling around Python.  And the first big change is that in 2012, we decided to reintroduce the U prefix.  Even if it doesn't make sense, if you only look at the Python 3 language to have test  U prefix because all strings are unique by default, adding the U prefix was very useful  to write a single code base for Python 2 and Python 3.  So it's fixed a lot of issues to help people to port that code.  In 2015, Python 3.7 has been released with a new operator for bytes.  It's a way to format a byte string.  This is very useful for projects like Twisted or Mercure, which manipulates a lot of bytes.  And we also added more Python 3000 warnings.  So Python 3 warnings.  We added these warnings in Python 2.  So the idea is that even if you don't port anything, just enable the warning and you  will see which code may have some troubles on Python 3 and you have to first focus on  this code before starting the big migration plan.  And to be able to use Python 3 features, we saw more and more backports of Python 3 features  to Python 2.  For example, unit test 2, Enum 3.4, which is a new Enum module.  And thanks to all these backports, moving to Python 3 really had something.  And after the spring comes the summertime.  So to come back to the website, the wall of shame, the author of the website decided to  change the title from wall of shame to wall of superpower.  And today we got almost 100% of the module which are compatible with Python 3.  In practice, it's a little bit less.  It's only 90% 5.  Because in this long list, you have some modules which have been replaced by new modules because  the old module doesn't make sense on Python 3 or the maintainer decided to restart from  scratch.  So in this list, you have some modules which you can just ignore.  And you can consider that the top 200 modules on PAPPI are compatible with Python 3.  And another good news for Python 3 is that finally after a long development process,  we had enough time to optimize Python 3.  And Python 3 is now faster than Python 2.  On this graphic, you can see the result of a few benchmarks and the orange line in the  normalized result of Python 2.  So it's always one.  And if the other bar is lower, it means that Python 3 is faster.  And you can see that depending on the benchmark, it's more or less faster.  And in some cases, it's way faster.  To give you these numbers differently, there is a company Instagram which has 700 million  users.  So it's a very, very big Django application running every day on mobile platform on everything.  So it's a big, big product.  It's fully written in Python.  And they switched to Python 2 to Python 3.  And thanks to this move to Python 3, they saved 12% of CPU just by doing the change.  And they also saved 30% of memory, which is very something very important for them because  when you have so many users, you need a lot of servers and at this scale, as a performance  and a really, really matter.  It's really expensive to have so much hardware.  And if you didn't notice, Python 2 is a little bit old today.  And even if we are doing our best to fix all bugs, sometimes we just cannot fix some issues.  Because the thing with Python 2 is that it's called Super Stable because we have so many  people which rely on Python 2.  This means that if you do any kind of change in Python 2, even to fix a bug, there is a  risk that someone rely on the bug and also the application rely on this bug.  So we think twice when we have to back port a fix to Python 2.  And sometimes it's more a design issue that has to require a big change in the language.  For example, the first bullet is Unicode.  Even if technically we could try different options in Python 2 to have new warnings or  new mode for Unicode, we decided to not touch the current implementation of Unicode.  Just because, as I said, a lot of people rely on it.  And we cannot change anything because it would break too much code in the wild.  Another example is that for security reasons, we decided to randomize the order of items  in a dictionary on a set object in Python 3.  Because if you inject a specific case in a dictionary, you may get the worst complexity  of the dictionary type.  And in that case, you may crash the server just by sending a few bytes to an HTTP server.  So what we did is to randomize the order.  But we cannot do that in Python 2 because if you change the order of the dictionary,  the representation of a dictionary change and it can break too many unit tests.  So for your information, you have to explicitly enable randomization using dash uppercase  error to enable the workaround for the security issue.  Another very annoying issue of Python 2 is that the subprocess module is not thread safe.  So maybe you did not notice because you don't use threads or maybe you did not notice because  you're lucky.  But the idea of not being thread safe means that everything is fine.  Everything is fine.  Everything is fine until one day in production, you get a crash and nobody is able to reproduce  a crash because it really depends on the workload.  It really depends on the order of operation and on your developer laptop.  You have different timings.  So it's very common that race conditions only occur on production.  So nobody wants to rely on something which is not thread safe.  So be careful with the subprocess.  And for your information, there is a back port called subprocess 2.3.2 which backpots  the thread safe implementation.  Another example is the recursive lock which again is not as a race condition.  As the race conditions are not threads but signals.  It means that if you use recursive lock and you get a signal, signal is for example that  you spawn a process and when the child process completes, you get a notification to say that  hey, your child completed, it's time to read the exit status of the process.  So when you get the signal, you may get a race condition and your lock may become inconsistent.  So you may get a dead lock just because of a signal.  And this is really annoying because a signal can come anytime.  You don't, it really depends on the timing of the child process and you don't control  the timing of the child process.  So it's really dangerous to use a recursive lock on Python 2.  Another example is that clocks in Python 2 use a system time.  So during winter time or summer time when you use a change of clock for DST, your application  may crash just because of that.  Or for example, if the system administrator decided to change the time manually, again  you may get a crash.  And the solution for that is to use something called monotonic lock.  The monotonic lock only increases, there is no jump in the future or in the past.  So the good news is that you just have to switch the Python 3 and you're fine.  So for example, for the time in Python 3.3, we added the time.monotonic function and slowly  we modified Python to use monotonic lock inside Python.  And to explain you something else about Python 3 is that as I said, Python 2 is super stable,  we cannot touch anything.  And the good thing with Python 3 is that it's still kind of not as popular as Python 2.  So we can afford to make more backward and uncomfortable changes.  For example, I change how file descriptors are inherited by the child process.  And since Python 3.3.4, when you spawn a child process, you don't inherit the file descriptor.  It means that you didn't inherit Open Socket, you don't inherit the sensitive file which  contains passwords or all your sequets.  But this was our backward compatible changes and we cannot do such a change in Python 2.  And another example is that in Python 3.5, I made another change which is to when you  get a signal, the CIS call is interrupted, you get an error.  And the behavior in Python 2 is that you get an exception and you have to do something.  And in Python 3.4.5 is that if the signal handler doesn't raise an exception, Python  will just retry the blocking CIS call for you and you don't notice anything.  You're just fine.  And we have an awesome when he approved my change for the file descriptor and he returns,  he wrote that we are aware that the code breakage, this is likely to cause and doing it anyway  for the good of mankind.  And again, Python 3.0 is much, much better because you have a lot of new modules in the  standard library.  I counted 21 modules for 3.7.  Just to give you some examples, there is a asynchronous programming, there is a new  which is very useful, there is pathlib which is very popular to manipulate paths, there  is unit test.mock which is a very, very nice way to reduce the dependency of a unit test  on really test something very specific.  So to mock another function.  And not only we fixed bugs and we added new features, we also made changes in the language.  And for example, in Python 3.6, we added something called the f-string.  It's a new way to format string.  And in my opinion, it's just the best way to format string because it's very short, it's  very obvious when you read the string.  And yeah, it's just the best because not only you can format a variable, but you can  even call a method.  You can, in practice, you can add any Python expression.  It just works.  It's amazing.  And therefore, asynchronous programming, we made different changes.  First we added the yield from.  After that, we added async on our weight keywords.  And we also added support for asynchronous generators.  So yield inside an asynchronous method.  There are asynchronous list comprehension.  But you have also more changes.  Like for example, we have the keywords only arguments which are very useful to write a  better API.  The prints is no longer a statement, but it's a function.  And I really prefer to have this function.  You can use the start to unpack.  But also to inside a list, now you can use the start.  This is a new feature.  It's a new way to build a list.  And in my opinion, it's straightforward and very useful.  You can also do it for dictionary.  You can use underscore inside numbers to make numbers easier to read.  You can annotate the type of variable.  For the with context manager, with statements, you can add multiple expressions.  There is, again, bytes formatting.  And there is something very useful for non-py users.  There is a new operator for matrix multiplication, which is the hats.  It avoids to, this is an in fix way for the matrix operation.  Because previously you had to call a function and it was harder to read.  So I spent a lot of time to explain that Python 3 is way better.  And we are reaching slowly the deadline, which is 2020, the deadline of Python 2.  So the question is now, is it time to maybe bury Python 2?  So the good news is that in Fedora, we already started to do that three years ago.  In Fedora, in 2023, there is no Python 2 by default.  There is only Python 3 by default.  And in the release of human 2 last year, again, there is no Python 2 by default, only Python  3.  And there is a Python 3 statement, which is a common timeline for all applications for  the end of support for Python 2 in the application.  And there are big projects like IPyton 6 and Django 2, which dropped Python 2 support.  And I'm working for Red Hat.  So for your information, in the latest release, Red Hat 7.5, we announced that Python 2 is  deprecated.  So if your manager is still concerned about Python 2, you can say that even Red Hat, which  has a very, very long support, we are aware that something is wrong with Python 2 and  we are working on that.  And for your information, the next release of Red Hat Enterprise Linux will be Python  3 only.  And if you...  This is thanks to my very cool colleagues that were working on that for two years.  And if you use Red Hat Enterprise Linux or CentOS, there is also something called Software  Collections, which is a way to install the latest version of Python on your old system.  So technically, it's already possible to use Python 3 on Red Hat.  And one open question for me is that we saw that changing the syntax and changing Python  was really difficult.  So maybe we can try to handle this issue differently and maybe try the JavaScript issue.  The solution, which is to use transpiling using Babel and Polyfill.  And by doing that, you are able to use the latest language on your old browser.  And thanks to that, they are able to move way faster because you are no longer stuck  by the version of JavaScript on your browser.  You can use new features.  Okay.  Question?  Thank you so much.  Thank you.  We have time for a few questions maybe.  If there's...  I've seen one over there.  Just to come back to the latest slide, to be honest, I think that we understood that Python  3 was a huge pain for everybody.  And if we do that one more time, we will not afford to break everything.  And my expectation is that Python 4 will just be the release after the previous Python 3  release.  And we will follow the same deprecation process to have a slow process to give at least two  release to remove something.  So we are not going to break Python one more time.  That's good to hear.  I was wondering whether the chart about performance was generic case because it puzzles me a bit.  And the point is that from what I saw around the Internet, usually Python 2.7 performed  faster on the general computing.  But is there a usage of a sync or something like that which drastically changes the chart  or it's the general case?  I spent a lot of time to work on benchmark because I wanted to optimize Python.  So this chart comes from a project called Perfomance.  It's a benchmark suite.  So I'm not sure that they are real representative of real applications, but we are trying to  be... to use more complex benchmark to have a better idea of what is a Python performance.  And what I just can say that on this benchmark suite, Python 3 is always faster.  So there is only one case where Python 3 is slower is a start-up time.  But we are working on that and it's getting better.  So I'm sure that in some specific cases, Python 2 is still slower.  But the good news is that this chart is for Python 2.6.  It's an old chart.  But Python 3.7 is even faster.  So maybe it should try the latest version of Python.  Because we optimized the method call and many other small changes.  One last question.  Is there any?  So knowing how the transition went, what would you do differently if you could do Python  3 again?  I try to explain that ten years ago, the choice that we made was a good way to do that.  Because we didn't have enough information that we didn't know that we had so much users.  We are not aware of the big dependency chain.  So yes, if I do that again, what I would do is just to have a very smooth transition  to make sure that even if we break something, we have a lot of time to port the code and  not to push all backward and compatible changes at once.  But do them one by one and make sure that we have the right tooling to do the conversion,  right tooling to detect issues in your code.  Because one of the very difficult issues with Python 3 is that when you divide two integers,  you get a float.  This specific issue is really difficult to catch in your code because just by reading  the code, you don't know the type.  So we need more tooling to detect that and to help users.  Okay.  Thank you so much.  Thank you.
Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_0_59.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_60_131.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_132_7127.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_7128_9047.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_9048_9263.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_9264_9347.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_9348_9623.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_9624_11135.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_11136_12155.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_12156_13583.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_13584_16151.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_16152_18659.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_18660_18803.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_18804_19643.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_19644_22007.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_22008_22787.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_22788_23243.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_23244_24419.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_24420_26723.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_26724_26963.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_26964_27155.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_27156_27779.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_27780_29723.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_29724_33071.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_33072_34979.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_34980_37859.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_37860_38015.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_38016_38255.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_38256_39383.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_39384_40427.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_40428_41735.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_41736_49223.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_49224_52319.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_52320_53279.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_53280_54323.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_54324_54791.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_54792_54899.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_54900_55835.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_55836_56855.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_56856_58511.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_58512_58631.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_58632_60143.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_60144_61127.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_61128_61295.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_61296_61607.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_61608_62555.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62556_62675.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62676_62723.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62724_62771.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62772_62819.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62820_62867.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62868_62915.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62916_62963.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_62964_65663.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_65664_67307.png Victor_Stinner_-_Python_3__ten_years_later_OHaxQZPKURg_67308_67410.png
At the very beginning of Python 3, there was a project called Python 3000. It was in 2006. It started with the PEP 3000 called Python 3000. The idea was to focus on what is called Python warts because Python 2 got more and more users and slowly people started to complain about some design issues. This design issues started to be collected as a list and they were called the Python warts. For example, in Python 2, you have two different types for integers. You have the long integer and the small integer. Depending on the number, depending on the operation, you may get a small or large integer. In that case, to test the type of a variable, you have to test for both types at the same time. This can be surprising for newcomers. They may not understand why we have two different things for the same kind of value, a number. There is also something called New Class. It's a new addition during the Python 2 development. The New Class should be used to get new features like, for example, Python properties. If you don't use the New Class, the properties don't work as expected. Again, it's a little bit surprising that you have to generate explicitly from the object type to get the New Class. It's surprising for newcomers. For the division of two integers in Python 2, you get an integer. This was a choice made from the beginning of Python. But when Python becomes more and more popular, some people were surprised that they didn't get a float because in other languages, using a simple calculator, you get a float. It has been decided to change that to always return a float because there is already an operator in Python 2. Instead of a single slash, you put a double slash and you always get an integer. For the Unicode, in short, I would say that if you only use bytes, you're fine. If all your inputs, all your outputs, all data inside your application and bytes, there is no issue. But if you use Unicode as inputs, outputs, Unicode everywhere in your application, you are also fine. You start to get some nasty issues when you have bytes and Unicode in your same application. Especially you don't get errors all the time, but only if a single character has an accent. It's a little bit difficult to debug such issue. It may happen in production. And more and more people were angry against this design. The comparison, Python 2 again made a choice from the beginning is that you are always able to compare two variables in Python 2. This means that if some object doesn't make any sense for comparison, Python takes the name of the type and compares the name. For example, if you compare the string hello and the number 3, you get the string s tier for the type string and you get the string int for the number 3 and you compare these two values to decide how to order values. And this behavior can be surprising and maybe not so one that you may expect. So we also decided to change that and not try to guess, but raise type error in Python 3. It means that in Python 3, you have to make it very explicit. And finally, for the imports, another trap for newcomers is that if you have a module which has the same name as the module of the standard library like the CIS module, you get a conflict between the module from the standard library and in your project. Because Python decided to use something called relative imports, which means that you can import the module in the same directory without having to specify the full path to the module. So again, we wanted to change that to always use absolute imports, which means that you always have to write the full path to the module to avoid such name conflicts. So to make sure that Python 3 is not going to be a big mistake from the very beginning of the project, we tried to have something called risk management to not break everything. And the idea is to only change acknowledged awards, so only the very short list of design issues. And we also decided to have an open community process to decide what should be changed and the process where the PEP, PEP with a number starting with 3000. And we also decided to not reimplement Python from scratch, the C Python, the implementation of the language. It's not done from scratch, but it's just a fork from Python 2. Again it's to limit the number of subtle changes that you may get if you do it from scratch. And again, from the very beginning, we decided to put a deadline for Python 2. At the beginning, it was 2015, I think, to say that today you are fine with Python 2, but be prepared that at this time you will have to switch. And in 2008, the Holy Grail, Python 2, the first release. And it was 10 years ago. You take your whole application, all dependencies, everything, you run an application called Python 2 to 3 to convert your Python 2 code base to Python 3 and done. Okay. Maybe it was not that easy in practice. Maybe there are some practical issues. Because the thing is that when you have third party dependency, the author of the module maybe don't want to drop the Python 2 user base because when you run 2 to 3, you lose Python 2 support. And when Python 3 has been released, Python 3 was a new thing. It was not supported by Linux distribution, for example. It was not installed by most users. It was really something very new. So many users didn't want to hear anything about Python 3. And another practical issue is that even if you succeed to port a dependency to Python 3, it's not enough. You really have to port all your dependencies and dependency of dependency of dependency of dependency. And so you cannot start to port your own application before you port dependencies. So it takes a lot of time just to start on your own code. And another bad or good surprise is that we didn't know that Python 2 was so much popular and it was heavily used in production. To explain you another problem called technical depth, imagine that you are talking with a new manager. And your manager asks, why should I let you work on Python 3 support? And the developer says, for all this new cool Python 3 features, and manager, okay, but can I use all these features? Well, we still have to support Python 2? No. Because when you support Python 2 and Python 3 in the same code base, technically you cannot use Python 3 features because you have to support Python 2. So you can spend a lot of time to port everything, but at the end you don't get any kind of direct benefit from the port. So you have to spend a lot of time for it. So again, it's difficult to justify if you are a manager of you. If you would like to port an application to Python 3, you have different options for the code base. For example, you can decide to have two different branches in Git. And one example is that the DNS Python project has been forked. There were two different projects, DNS Python for Python 2, and DNS Python 3 for Python 3. Sometimes the maintainer of the project doesn't care about Python 3 because, for example, of the Harry manager. And so the community decided to fork the projects. For example, the PIL module has been forked as PILO. It's a module to manipulate pictures, to resize and do whatever you want with pictures. And not only they added Python 3 support, but they also added new features, which is really cool. And in the worst case, you are stuck by a dependency like MySQL Python. I was working on OpenStack and all OpenStack rely on this module. But the maintainer decided to not reply to any break reports. He didn't reply to pull requests. So we were stuck and we didn't know what to do. So someone decided to maybe just fork the project and put it as a new name. But then you get a new issue is that for Linux vendors like Red Hat, we didn't know if we should rely on the old version or the new version. Because we don't know if the old version is going to die or maybe it's going to be maintained for the maintainer decided to wake up. To give you an idea of what was Python 2 at this time, you have to know that when Python 3 has been released, the stable version of Python 2 was 2.6. And the issue with Python 2.6 is that you have to make a lot of changes to make your code compatible with Python 3. For example, for all your nicole string, we add the U prefix to annotate in the nicole string. But if you do that, you get a syntax error in Python 3. So to be able to have a single code base for your nicole string, you have to remove the prefix and call a function from the 6th module. So you have to write 6.U parenthesis your string. It was a little bit annoying to do that on all your string. Because of that, some project just decided to not do anything for Python 3. Another issue is that if you really want to use new Python 3 features like the new method of the unit test module, you can use something called unit backports. Like unit test 2, which backports new features from Python 3. But it can be an issue to have dependency because 10 years ago, it was a little bit difficult to install all dependency and take care of that. It was more difficult to install it. So we had to wait up to Python 3.2, you had the U issue and you have to wait for the next release to get a fix for that. So after autumn comes the cold winter time. And we will start with our website called the Python 3 wall of shame. So in 2011, someone decided to create a website called Python 3 wall of shame. But the idea was not to blame the author of the modules. The intent was to motivate the package manager to port the code to Python 3. But when the website has been created, only 9% of the top 200 modules were compatible with Python 3. And to recall what was Python 10 years ago, we had higher notified 3 big players in the Python community. There is a tree steal, which is a very good networking solution for clients on server to do asynchronous programming, which is a very efficient way to do networking in Python. There is a mercurial, which is similar to Git, it's a source control management. And there is also Django, a very famous framework to do websites. And this free project had different issues with Python 3. Tristed and mercurial are very close to the hardware, to the low level. For example, Tristed works on the socket level. And at the socket level, the network level, there is nothing called Unicode. There are only bytes. So all the project is designed to use bytes. And in Python 3, using bytes is not the first citizen class type. We prefer to use Unicode. So at the beginning, it was a little bit difficult to Tristed to port the code base to Python 3. And it's the same issue for mercurial because they used bytes not only for the file name for very good reason, but also for the file content. For this two reasons, it was also difficult and it's still difficult to mercurial to be ported to Python 3. And for Django, when Python 3 has been released, the support of Unicode was only at the beginning. So, because it was very difficult to port code, because nobody wanted to be the first one to port code, we started to see this Frenchman with Python Troll. And the Python Troll, considered that Python 3 doesn't bring anything. Because as I explained with the manager, when you port code Python 3, you still have the same features. You don't bring anything. And when you move to Python 3, you are forced to use properly Unicode. And nobody wants to do that because it's very difficult for good reason. And it's much more simpler to use Python 2 with bytes because you don't have to worry about Unicode, people speaking a different language, just pass text and it works. So Python 2 is a failure. So maybe what we can do is just continue the development of Python 2.7. So there is an idea from the trolls is that maybe we can just continue the development and create a new version called 2.8 to add new features from Python 3, but don't break the backward compatibility. And four years ago, there was still an article saying that we were still debating transitional Python 2.8 and they were concerned that Python 3 would never take off. And also that Python 3 only represents 2% of packages. So to make things more explicit, we wrote a pep called the pep not found, the 404. It's a Python 2.8 unreleased schedule. So it's a very explicit document to say that we know that we are breaking the backward compatibility but we are not going to spend more time on Python 2 because we are a small team of core developers. We don't have enough time to maintain two versions. And if you use the numbers differently in 2013, someone else counted that there are 80% of the top 50 projects which were compatible with Python 3. So it wasn't that bad. And Python 2.7 as his end of life extended at the Python in 2014. And for me, it's one of the best moves that we did for Python 3 is that we say that, okay, we know that it's difficult to port your code. You have very good reason to not port your code today because not all dependencies are compatible because it costs a lot of money and we don't want to force people to move to Python 3. We didn't want to force people to do anything. So we just say that you have five more years to do your work and we extended the deadline to 2020. So after the cold winter comes fresh air, flowers, it's a new beginning. And the first great move for Python is that our first problem has been solved. So what is our first problem is that when you want it, we install a dependency which was a very common question because using dependency is a good practice. The question is how can I install a dependency? And the answer was, oh, you have to install Saptor tools. Okay, but how can I install Saptor tools? And it was a little bit difficult because you had to find a script somewhere on the Internet to download the script. You need the system administrator permission to be able to run the script. It does install something on your system. It's not from the system, it's from the Internet, so you have to trust it. It was not the perfect solution. But in 2011, PIP 1.0 has been released. And in 2014, we decided to include not PIP itself, but Ensure PIP, which is a new module to install PIP directly into Python. So in Python 2.7.9 and Python 3.4, there is no Ensure PIP to be installed to VAT. So very easily install PIP. In practice, even if you don't have the Internet connection to the Internet, you are still able to install PIP using that. And thanks to this change, slowly, PIP became the de facto installer. And you don't have to answer how to install dependency. You just know that PIP is the correct way to do that. And we also see a Linux distribution to include PIP directly in the system. So today, it's very easy to have dependencies. Support and application to Python 3, we saw that 2.2.3 was not the best approach. So what we saw is that maybe we should not remove Python 2.0 support, but say differently like add Python 3.0 support. And this is a big, big change because you don't lose your users. You only move to Python 3 step by step. And you don't have to force all your dependencies, all your code base at once. You can start with a simple change or a single file and do it incrementally. And we started to see new tools like Modernize. It's a tool to convert your code base without losing the Python 2.0 support. And let me tell you the story of Sixer, which is my own tool. So I'm working on OpenStack. And on OpenStack, if you send a pull request generated by Modernize, you get a giant pull request. And you are sure that in a few hours, you will get a conflict because OpenStack is moving very quickly. And all files are modified. So it's really difficult to get a pull request with no conflict for long hours or one day, especially a very large pull request. So what I did is to write a new tool which only apply selected operation, like, for example, just add parentages to the print statement. And by doing that, you can generate much smaller pull requests. You get less conflicts. And it's much more easier to review also for the maintainer because you see that the change only add parentages. So it's a simple change and it's easy to approve. And thanks to this tool, I was able to pass all unit tests of OpenStack to Python 3. And another good advantage of this method is that if you are a CI to validate all your work, you can make sure that each change doesn't break the backward compatibility. It doesn't introduce regression. So you can apply changes one by one and make sure that everything is fine. If you have a larger code base, maybe you need to do it differently because some companies have very, very large code base, like, for example, Instagram or GrubBox have at least two million lines of code. So what they need to do is that when you have a very, very large code base, what you can do is first to add new tests because sometimes in a big company, people like to add new features but they just forget to add new tests. So adding a new test is a good practice for your existing code base, but it also helps to make sure that the migration doesn't break anything. And GrubBox is also trying to a different approach. As I decided to work on the type annotation, the idea is that you annotate the type of the arguments of your function but also the return type of your function. And by doing that, they expect that it will be much easier to pass the application to Python 3 without introducing bugs because if you have a type annotation, you know which type are expected and it's much more easier to make sure to validate everything. To make the migration simpler, we made the different changes in Python itself but also in tooling around Python. And the first big change is that in 2012, we decided to reintroduce the U prefix. Even if it doesn't make sense, if you only look at the Python 3 language to have test U prefix because all strings are unique by default, adding the U prefix was very useful to write a single code base for Python 2 and Python 3. So it's fixed a lot of issues to help people to port that code. In 2015, Python 3.7 has been released with a new operator for bytes. It's a way to format a byte string. This is very useful for projects like Twisted or Mercure, which manipulates a lot of bytes. And we also added more Python 3000 warnings. So Python 3 warnings. We added these warnings in Python 2. So the idea is that even if you don't port anything, just enable the warning and you will see which code may have some troubles on Python 3 and you have to first focus on this code before starting the big migration plan. And to be able to use Python 3 features, we saw more and more backports of Python 3 features to Python 2. For example, unit test 2, Enum 3.4, which is a new Enum module. And thanks to all these backports, moving to Python 3 really had something. And after the spring comes the summertime. So to come back to the website, the wall of shame, the author of the website decided to change the title from wall of shame to wall of superpower. And today we got almost 100% of the module which are compatible with Python 3. In practice, it's a little bit less. It's only 90% 5. Because in this long list, you have some modules which have been replaced by new modules because the old module doesn't make sense on Python 3 or the maintainer decided to restart from scratch. So in this list, you have some modules which you can just ignore. And you can consider that the top 200 modules on PAPPI are compatible with Python 3. And another good news for Python 3 is that finally after a long development process, we had enough time to optimize Python 3. And Python 3 is now faster than Python 2. On this graphic, you can see the result of a few benchmarks and the orange line in the normalized result of Python 2. So it's always one. And if the other bar is lower, it means that Python 3 is faster. And you can see that depending on the benchmark, it's more or less faster. And in some cases, it's way faster. To give you these numbers differently, there is a company Instagram which has 700 million users. So it's a very, very big Django application running every day on mobile platform on everything. So it's a big, big product. It's fully written in Python. And they switched to Python 2 to Python 3. And thanks to this move to Python 3, they saved 12% of CPU just by doing the change. And they also saved 30% of memory, which is very something very important for them because when you have so many users, you need a lot of servers and at this scale, as a performance and a really, really matter. It's really expensive to have so much hardware. And if you didn't notice, Python 2 is a little bit old today. And even if we are doing our best to fix all bugs, sometimes we just cannot fix some issues. Because the thing with Python 2 is that it's called Super Stable because we have so many people which rely on Python 2. This means that if you do any kind of change in Python 2, even to fix a bug, there is a risk that someone rely on the bug and also the application rely on this bug. So we think twice when we have to back port a fix to Python 2. And sometimes it's more a design issue that has to require a big change in the language. For example, the first bullet is Unicode. Even if technically we could try different options in Python 2 to have new warnings or new mode for Unicode, we decided to not touch the current implementation of Unicode. Just because, as I said, a lot of people rely on it. And we cannot change anything because it would break too much code in the wild. Another example is that for security reasons, we decided to randomize the order of items in a dictionary on a set object in Python 3. Because if you inject a specific case in a dictionary, you may get the worst complexity of the dictionary type. And in that case, you may crash the server just by sending a few bytes to an HTTP server. So what we did is to randomize the order. But we cannot do that in Python 2 because if you change the order of the dictionary, the representation of a dictionary change and it can break too many unit tests. So for your information, you have to explicitly enable randomization using dash uppercase error to enable the workaround for the security issue. Another very annoying issue of Python 2 is that the subprocess module is not thread safe. So maybe you did not notice because you don't use threads or maybe you did not notice because you're lucky. But the idea of not being thread safe means that everything is fine. Everything is fine. Everything is fine until one day in production, you get a crash and nobody is able to reproduce a crash because it really depends on the workload. It really depends on the order of operation and on your developer laptop. You have different timings. So it's very common that race conditions only occur on production. So nobody wants to rely on something which is not thread safe. So be careful with the subprocess. And for your information, there is a back port called subprocess 2.3.2 which backpots the thread safe implementation. Another example is the recursive lock which again is not as a race condition. As the race conditions are not threads but signals. It means that if you use recursive lock and you get a signal, signal is for example that you spawn a process and when the child process completes, you get a notification to say that hey, your child completed, it's time to read the exit status of the process. So when you get the signal, you may get a race condition and your lock may become inconsistent. So you may get a dead lock just because of a signal. And this is really annoying because a signal can come anytime. You don't, it really depends on the timing of the child process and you don't control the timing of the child process. So it's really dangerous to use a recursive lock on Python 2. Another example is that clocks in Python 2 use a system time. So during winter time or summer time when you use a change of clock for DST, your application may crash just because of that. Or for example, if the system administrator decided to change the time manually, again you may get a crash. And the solution for that is to use something called monotonic lock. The monotonic lock only increases, there is no jump in the future or in the past. So the good news is that you just have to switch the Python 3 and you're fine. So for example, for the time in Python 3.3, we added the time.monotonic function and slowly we modified Python to use monotonic lock inside Python. And to explain you something else about Python 3 is that as I said, Python 2 is super stable, we cannot touch anything. And the good thing with Python 3 is that it's still kind of not as popular as Python 2. So we can afford to make more backward and uncomfortable changes. For example, I change how file descriptors are inherited by the child process. And since Python 3.3.4, when you spawn a child process, you don't inherit the file descriptor. It means that you didn't inherit Open Socket, you don't inherit the sensitive file which contains passwords or all your sequets. But this was our backward compatible changes and we cannot do such a change in Python 2. And another example is that in Python 3.5, I made another change which is to when you get a signal, the CIS call is interrupted, you get an error. And the behavior in Python 2 is that you get an exception and you have to do something. And in Python 3.4.5 is that if the signal handler doesn't raise an exception, Python will just retry the blocking CIS call for you and you don't notice anything. You're just fine. And we have an awesome when he approved my change for the file descriptor and he returns, he wrote that we are aware that the code breakage, this is likely to cause and doing it anyway for the good of mankind. And again, Python 3.0 is much, much better because you have a lot of new modules in the standard library. I counted 21 modules for 3.7. Just to give you some examples, there is a asynchronous programming, there is a new which is very useful, there is pathlib which is very popular to manipulate paths, there is unit test.mock which is a very, very nice way to reduce the dependency of a unit test on really test something very specific. So to mock another function. And not only we fixed bugs and we added new features, we also made changes in the language. And for example, in Python 3.6, we added something called the f-string. It's a new way to format string. And in my opinion, it's just the best way to format string because it's very short, it's very obvious when you read the string. And yeah, it's just the best because not only you can format a variable, but you can even call a method. You can, in practice, you can add any Python expression. It just works. It's amazing. And therefore, asynchronous programming, we made different changes. First we added the yield from. After that, we added async on our weight keywords. And we also added support for asynchronous generators. So yield inside an asynchronous method. There are asynchronous list comprehension. But you have also more changes. Like for example, we have the keywords only arguments which are very useful to write a better API. The prints is no longer a statement, but it's a function. And I really prefer to have this function. You can use the start to unpack. But also to inside a list, now you can use the start. This is a new feature. It's a new way to build a list. And in my opinion, it's straightforward and very useful. You can also do it for dictionary. You can use underscore inside numbers to make numbers easier to read. You can annotate the type of variable. For the with context manager, with statements, you can add multiple expressions. There is, again, bytes formatting. And there is something very useful for non-py users. There is a new operator for matrix multiplication, which is the hats. It avoids to, this is an in fix way for the matrix operation. Because previously you had to call a function and it was harder to read. So I spent a lot of time to explain that Python 3 is way better. And we are reaching slowly the deadline, which is 2020, the deadline of Python 2. So the question is now, is it time to maybe bury Python 2? So the good news is that in Fedora, we already started to do that three years ago. In Fedora, in 2023, there is no Python 2 by default. There is only Python 3 by default. And in the release of human 2 last year, again, there is no Python 2 by default, only Python 3. And there is a Python 3 statement, which is a common timeline for all applications for the end of support for Python 2 in the application. And there are big projects like IPyton 6 and Django 2, which dropped Python 2 support. And I'm working for Red Hat. So for your information, in the latest release, Red Hat 7.5, we announced that Python 2 is deprecated. So if your manager is still concerned about Python 2, you can say that even Red Hat, which has a very, very long support, we are aware that something is wrong with Python 2 and we are working on that. And for your information, the next release of Red Hat Enterprise Linux will be Python 3 only. And if you... This is thanks to my very cool colleagues that were working on that for two years. And if you use Red Hat Enterprise Linux or CentOS, there is also something called Software Collections, which is a way to install the latest version of Python on your old system. So technically, it's already possible to use Python 3 on Red Hat. And one open question for me is that we saw that changing the syntax and changing Python was really difficult. So maybe we can try to handle this issue differently and maybe try the JavaScript issue. The solution, which is to use transpiling using Babel and Polyfill. And by doing that, you are able to use the latest language on your old browser. And thanks to that, they are able to move way faster because you are no longer stuck by the version of JavaScript on your browser. You can use new features. Okay. Question? Thank you so much. Thank you. We have time for a few questions maybe. If there's... I've seen one over there. Just to come back to the latest slide, to be honest, I think that we understood that Python 3 was a huge pain for everybody. And if we do that one more time, we will not afford to break everything. And my expectation is that Python 4 will just be the release after the previous Python 3 release. And we will follow the same deprecation process to have a slow process to give at least two release to remove something. So we are not going to break Python one more time. That's good to hear. I was wondering whether the chart about performance was generic case because it puzzles me a bit. And the point is that from what I saw around the Internet, usually Python 2.7 performed faster on the general computing. But is there a usage of a sync or something like that which drastically changes the chart or it's the general case? I spent a lot of time to work on benchmark because I wanted to optimize Python. So this chart comes from a project called Perfomance. It's a benchmark suite. So I'm not sure that they are real representative of real applications, but we are trying to be... to use more complex benchmark to have a better idea of what is a Python performance. And what I just can say that on this benchmark suite, Python 3 is always faster. So there is only one case where Python 3 is slower is a start-up time. But we are working on that and it's getting better. So I'm sure that in some specific cases, Python 2 is still slower. But the good news is that this chart is for Python 2.6. It's an old chart. But Python 3.7 is even faster. So maybe it should try the latest version of Python. Because we optimized the method call and many other small changes. One last question. Is there any? So knowing how the transition went, what would you do differently if you could do Python 3 again? I try to explain that ten years ago, the choice that we made was a good way to do that. Because we didn't have enough information that we didn't know that we had so much users. We are not aware of the big dependency chain. So yes, if I do that again, what I would do is just to have a very smooth transition to make sure that even if we break something, we have a lot of time to port the code and not to push all backward and compatible changes at once. But do them one by one and make sure that we have the right tooling to do the conversion, right tooling to detect issues in your code. Because one of the very difficult issues with Python 3 is that when you divide two integers, you get a float. This specific issue is really difficult to catch in your code because just by reading the code, you don't know the type. So we need more tooling to detect that and to help users. Okay. Thank you so much. Thank you.
At the very beginning of Python 3, there was a why we have two different things for the same used to get new features like, for example, Unicode everywhere in your application, you are if you have a module which has the same name as beginning, it was 2015, I think, to say that don't want to drop the Python 2 user base because have to support Python 2? No. Because when you justify if you are a manager of you. If you would you have to make a lot of changes to make your that, you get a syntax error in Python 3. So to you had the U issue and you have to wait for the is designed to use bytes. And in Python 3, using Django, when Python 3 has been released, the because nobody wanted to be the first one to port Python 2.7. So there is an idea from the trolls break the backward compatibility. And four years we are breaking the backward compatibility but we and we extended the deadline to 2020. So after 2.7.9 and Python 3.4, there is no Ensure PIP to dependencies. Support and application to Python at once. You can start with a simple change or a regression. So you can apply changes one by one differently because some companies have very, very large code base, like, for example, without introducing bugs because if you have a also added more Python 3000 warnings. So Python 3 optimize Python 3. And Python 3 is now faster you have so many users, you need a lot of servers to not touch the current implementation of rely on it. And we cannot change anything because Python 2 is that the subprocess module is not thread safe. So be careful with the subprocess. recursive lock which again is not as a race condition. As the race conditions are not threads process completes, you get a notification to say when you use a change of clock for DST, your crash. And the solution for that is to use fine. So for example, for the time in Python 3.3, we modified Python to use monotonic lock inside example, I change how file descriptors are 3.3.4, when you spawn a child process, you don't Python 3.5, I made another change which is to when you get a signal, the CIS call is interrupted, you get an error. And the behavior modules for 3.7. Just to give you some examples, there is a asynchronous programming, there is a function. And not only we fixed bugs and we added language. And for example, in Python 3.6, we asynchronous generators. So yield inside an Like for example, we have the keywords only also to inside a list, now you can use the start. a new operator for matrix multiplication, which is no Python 2 by default, only Python 3. And support for Python 2 in the application. And Enterprise Linux will be Python 3 only. And if expectation is that Python 4 will just be the release after the previous Python 3 release. And we will follow the same deprecation process to Python one more time. That's good to hear. I was spent a lot of time to work on benchmark because are trying to be... to use more complex benchmark to have a better idea of what is a Python there any? So knowing how the transition went, to detect issues in your code. Because one of the