Writing faster Python
Witowski, Sebastian
Sebastian Witowski - Writing faster Python Presentation on how you can write faster Python in your daily work. I will briefly explain ways of profiling the code, discuss different code structures and show how they can be improved. You will see what is the fastest way to remove duplicates from a list, what is faster than a _for_ loop or how “asking for permission” is slower than “begging for forgiveness”. - Did you know that Python preallocates integers from -5 to 257 ? Reusing them 1000 times, instead of allocating memory for a bigger integer, can save you a couple of milliseconds of code’s execution time. If you want to learn more about this kind of optimizations then, … well, probably this presentation is not for you :) Instead of going into such small details, I will talk about more _"sane"_ ideas for writing faster code. After a very brief overview of how to optimize Python code (rule 1: don’t do this, rule 2: don’t do this yet, rule 3: ok, but what if I really want to do this ?), I will show simple and fast ways of measuring the execution time and finally, discuss examples of how some code structures could be improved. You will see: - What is the fastest way of removing duplicates from a list - How much faster your code is when you reuse the built-in functions instead of trying to reinvent the wheel - What is faster than the good ol’ _for_ loop - If the lookup is faster in a list or a set (and when it makes sense to use each) - How the “It's better to beg for forgiveness than to ask for permission” rule works in practice
 Good. Without further ado, I'd like to present our next speaker, Sebastian. Sebastian will be talking about writing faster code. Give him a big hand. Okay. Hi, everyone. Can you hear me? Okay. So, I would like to talk with you about writing faster code. And last time I was giving a short talk, short lightning talk about writing faster code, I remember someone pointed out that, well, basically, you can take your Python code, rewrite it in C or C++, and it will be faster. And, I mean, the guy was right. So, take any piece of Python code, rewrite it in C or C++, and probably it will be automatically faster. So, I was thinking, hmm, if I say just writing faster code, it might not be clear if I mean only Python or not. So, I had to fix the title of my presentation. And I was very happy with the new title. I mean, it makes it clear we are not going to talk about C or Java today. But then I realized the title is too long. I mean, even though it's very clear, it barely fits on the slide. So, I have to change it again. So, I got the version that means exactly the same thing, but it's shorter. So, this is how I ended up with writing faster Python as a title for my today's talk. So, let's put aside the question. I'm going to explain more about which programming language is better. We all know the answer. That's why you're here. Python was not created to be a fast language that you would use for some computations where every nanosecond counts. And that's fine with me. I mean, Python is a great programming language that is very easy and fun to use. Python is very easy to learn. The fact that it's so easy to read and write code in Python, it's very encouraging for people new to software development. I see that it's getting more and more popular in schools or at the universities as the first programming language. And I'm not surprised. I mean, imagine you're completely new to programming and someone tells you, hey, let me show you how much fun programming is and let's start with something super simple. Let's just bring some text to the screen. And then he shows you those two examples. I mean, one of them is clearly not something that you want to show to a beginner to encourage him or her to start programming. But also, Python is not only useful for learning. There are many big companies that are using Python. Companies with millions of users and billions of requests per month. So your website is going to be fun using Python. So Python is usually fast enough. But what if you decide that it's not fast enough anymore? For example, your website starts giving you timeouts or maybe a faster code will bring more money if you're a company. So it's time for optimization. But how do we optimize the code? Well, probably you need to follow some rules. So let's try to Google for that. And if we open the first link, we see that there are only three rules. Wow. It's going to be easier than we expected. So let's take a closer look at those rules. First rule of optimization, don't. Okay, that was easy. Any questions? Well, actually, no, there is more to that. So what does it mean don't? Well, nine out of ten times when you think that you need optimization, you don't. Especially in the early stage of your product's life. You might think it would be nice to optimize your code a bit, but, well, first of all, you will waste time doing something that is probably not needed. If you want your code to run faster, you can start with getting a faster hardware in the first place. And second of all, optimization comes with a cost. Most often, the only cost is the time that you spend optimizing your code. Well, sometimes you will, it's also the time that you need to fix what you just broke with your optimization. But also optimize it, the optimized code might not be as readable as it was in the first place. And maybe your code is running faster, but it's now using more memory. So unless you have really good reasons to optimize, don't do this. And if you know that you have reasons to optimize, then we can move to the second rule of optimization. Don't do this yet. And this is how I understand this rule. So first, make sure your code works. Then make sure you have a good test suit. And only then you're ready for optimization. I love this rule. It always reminds me how many times I broke it. I mean, so many times I was working on a piece of software and I started thinking, hmm, maybe I can change this piece of code and it will be faster. And maybe I will save like two lines of code. Was it a good idea? No. And not only because I ended up breaking things, but most often I did end up breaking things. But also I started jumping around the code and at some point I got confused what I was writing in the first place. Did it make my code faster? I have no idea because I had nothing to compare it to in the first place. If I would finish writing the code and then try to improve it, I could measure both solutions and compare them. But in that scenario I could only guess. And that brings me to the last rule of optimization. Don't guess. Always profile your code. Humano are terrible in predicting the bottlenecks of your code. So if you think that your code is slow, first profile it and then see what part of it takes most of the time. Otherwise you may end up spending time rewriting one piece of your code to just get like 1% of speed improvement. While there are other parts of your software where you can gain way more improvement with less effort. So there are plenty of profiling tools. There were already quite a few talks during Eurofighting about profiling. So I will not go into the details. But if you don't know where to start, you can take a look at the C profile module. It will show you a clear overview of how many times each function is called and where the code is spent. And if you want to have some more advanced formatting you can add the PSTADS module. Also if you prefer the graphic interface you can take a look at the run, snake run or snake vis libraries. So once we are ready for optimization we have to decide on which area we want to focus. So there are different levels of optimization. Starting from the highest level you have the design level optimization. So depending on the constraints and priorities of your system you can optimize it by redesigning it. It might require rewriting your software in a different programming language or changing the database type or changing the architecture to perform less database queries. So this kind of optimization will usually give you the best improvement but it also takes most time to do this. So I don't encourage you to rewrite your software from the scratch but if you have some critical parts of your code that are run often you can optimize them by rewriting them in C or C++. Because C is faster you will have some good speed improvement for free. Well not really for free now you will have Python and C code in the same project. So one level lower we have algorithms and data structures. So knowing good algorithms together with their complexity definitely helps you creating a good and efficient software. For example if you want to get the sum of numbers from 1 to N the first idea might be to get a loop that goes through all the elements and ask them. It will work but it won't be fast. Instead you can use the algorithm for the arithmetic sum which will give you the same results and it will be more efficient. So the next level is the source code optimization. And this is something that I will talk about in the second part of the presentation. Now we are moving to the build level which involves setting up some specific build flags. So in your daily work it's not something that you will do often. You can optimize Python for a specific architecture but if you're a web developer like me this is either something that you will do once per machine or you won't bother at all. Next we have the compile level. So we can make some optimizations if your programming language has an ahead of time compiler. Since I'm talking about Cpyton today which doesn't really have a head of time compiler we're going to skip that part as well. And last but not least we have the runtime level. So it's related with a specific compiler that you're using. Some compilers are faster than the others. So for example if you replace Cpyton with PyPy you can get some improvements depending on the use case of your software. But it really depends on what kind of piece of code you're writing. So most of the time once you set up on a specific language implementation there's nothing you have to do to benefit from this kind of optimization. It's usually up to the creators of the compilers to optimize them. So simply updating to the new version of the programming language you're using can make your code run a bit faster. So when you optimize you probably want your code to run faster. And also use less memory. And basically less of everything. The bad news is you can't have all of it. Optimization in one area will usually cause deterioration in other areas. So you always have to decide which resources are crucial and you have to optimize in that direction. So it's possible that optimization will have nothing to do with the speed because there are other resources more important than the raw speed. For example who cares that your program is now ten times faster when it's crashing half of the time because it's running out of memory. Also another important resource that people are often forgetting is the sanity. A sanity of a person that will be maintaining your code. So please be nice to that person. You never know who that might be. Yeah. So unless you're really writing a throwaway code, if you're making your code harder to read and maintain then you're probably doing it wrong. So having those things clear let's jump straight to how you can write faster Python. Also known as source code optimization. My examples I'm using the version 3.5.1 of Python together with Python. Although the example should work in both Python 2 and Python 3. So for measuring the execution time of my code I will be using the magic time it function. It has some overhead comparing to the standard time in library but it doesn't really matter because as long as we use the same method to measure execution time of different functions we only need to know which method is faster than by how much. So for each of my examples I will write different versions of code, measure the execution time and compare them. So let's start with something simple. Let's say you want to count the number of elements in a list. You can easily write a simple loop that will increment the counter and this will work. It will be very so. You can achieve the same results using the build in land function. As you can see for only one million of results the difference is insanely huge. So my first advice is not to reinvent the wheel but first check if there is a function that you can use. Python 3.5 has 68 building functions. So it's nice to take a look at them and keep them in the back of your head because they might be handy at some point. Also, before you start writing your own version of order dictionary or dictionary with default values, take a look at the collection module from the standard library. Even though it contains only like 10 different data types, those are probably the data types you are looking for if the standard ones are not enough. So let's say you have a list of one million elements and you want to select only the odd numbers. So the naive version would be to use the for loop. So for each element of the list you check if it's odd and if it is, you add it to another list. But I already show you in the previous example that in most cases for loops can be replaced with something better. In this case, you could use the build in filter function instead. And in Python 2, filter was returning a list directly. Python 3 is returning an iterator. So I have to call the list function to get the same results as in case of the for loop. And even though the list function has some impact on the performance, it's negligible comparing to the time spent in the filter function. Yet you can see that filter performs even slower than the for loop. Why does this happen? Well, the fact that filter is returning now an iterator is a clear sign that it's a wrong use case for this kind of function. So if you want to get the whole list as the result, it's better to use the list comprehension. It's around 75% faster than the for loop and at least for me it looks more clear. When you want to execute a piece of code, but you are not sure if it will be successful. Maybe some variables are not set or like in this case, the class might be missing some attributes. So you want to protect yourself somehow. The first way you can do this is look before you leave or ask for permissions. What it means is that you first check if the class has a specific attribute and then you perform the operations. Usually this checking is done with the if statement. However, there's different approach that you could use and it's called back for forgiveness. So in this scenario, you perform the operation without checking the conditions first. But in case you expect that something might break, you wrap your code in a tri-except block and you catch the exceptions that were raised. And as you can see in the simple example, begging for forgiveness is like three times faster. But it gets even better if you're checking for more conditions. So here we are checking if three attributes are present. And asking for permission is still slower and now it's also getting more difficult to read. So following the back for forgiveness approach will result in a faster and more readable code. So we could say that asking for forgiveness instead of checking the permissions is always a better way. But we won't say that because it's not true. Exceptional handling is still quite expensive. So if the attribute is actually missing, then begging for forgiveness will be slower than asking for permissions. So as a rule of thumb, you can use the ask for permissions way if you know that it's very likely that the attribute will be missing or there will be some other problems that you can predict. Otherwise, if you're expecting that your code will work in most of the times, try using try accept will result in a faster and quite often more readable code. So for example, if you're fetching some files from the Internet and you expect that everything will be fine unless there is no Internet connection. So instead of checking if there is Internet connection, if it's fast enough, if there are no timeouts, just go for the try accept. But then again, I strongly advise you to measure both solutions and see maybe in your case it will be different. So let's tackle another problem, the membership testing. So if you have a list and you want to check if it contains a specific element, you can use a for loop. But the problem is you are iterating over the whole list even though you're not really doing anything with all those elements. So you can replace the for loop with the in statement. It will check if a specific element belongs to a given set of data and it will do this twice as fast. But there is still one big problem with this approach. The lookup time depends on where your element is located in that list. If it's at the beginning of the list, you're lucky and you will get it fast. If it's at the end of the list, you have to wait. So what would be really nice here if we had the data structure that would have a constant lookup time. And actually in Python we have. We have both sets and dictionary that have constant lookup time. So if we replace the list with a set, then the lookup time becomes faster. From just few times faster to hundreds of thousand times faster. So where's the catch? Well, you pay some time to convert this to a set. And in this scenario, converting this list to a set takes more time than any of the lookups in that list. So it doesn't really make sense. However, if you're checking membership of different elements, quite often it makes sense to first convert it to a set. So speaking of sets, they have another interesting feature. They don't contain duplicates. So basically if you have a list of elements and you want to remove the duplicates, the fastest way to do this is to convert this list to a set. But be aware that sets are not ordered. So if you need to preserve the order, take a look at the order dictionary from the collection module. So if you want to sort your list, you can either do this in place using the list.sort function, or you can call this sorted function that will create a new list. And unless you really need to have a new list, sorting in place will be like six times faster in this scenario. This is for one million of random numbers. If you want to perform the same operation on a large set of data, then you have two options. You can write a function that performs the operation and call this function 1,000 times. Or you can call a function that takes this set of data and performs the operation inside. And the second approach will be faster. So if you can in an easy way replace multiple calls to one function with just one function, then quite often it's a good idea. So what's the best way to check if a variable expression is true? Well, you can explicitly compare this variable to true, but in most cases you're adding additional redundancy. So you can simplify your condition to just if variable. And it will return true unless the variable is false, non-zero, empty string, empty list, or other false expression. And by doing that, your comparison gets faster by like 70%. And the same rule applies when checking for false. So the fastest way to do this is to use if not variable unless you really need to distinguish false from, let's say, non or zero or other false values. It also applies to empty data structures. So simply doing if not a list is, will be almost three times faster than explicitly checking the length of a list. So let's take a look at different ways of defining functions in Python. The most common one is to create a function with def keyword. The other way is to declare an anonymous function with lambda. If you assign this lambda to a variable, it will act in the same way as a function created with a def keyword. And as you can see, they are both equally fast. Why? Because both versions do exactly the same thing. We can disassemble the code of both versions with the disk library, and we'll see that inside is the same code. So is there any difference? Well, if your function has more than one line, you can't use lambda. And you can't really put documentation inside of lambda. Although you could have Pepe it enabled in your code editor, it will complain each time you try to assign lambda to a variable. And in his right, lambdas work really nice when you need a simple one-liner callback for your functions, especially for functions like filter, mabel, reduce. And there are also some quite few narrow use cases where it might be necessary to use lambda as a callback. So if you want to read more, you can check the link at the bottom. But however, in any other case, I would definitely recommend to use that. It's much cleaner. You can document it properly. And the performance is exactly the same. So there are two ways how you can create an empty list. So you can either call a list function or you can just use the literal syntax. And as you can see, the literal syntax is faster. Why is it faster? Because if you call a function, Python first needs to resolve this function. And with the literal syntax, there is no overhead for that. And exact same thing happens for creating a dictionary. Okay, I have two more examples that should be treated with quotient. Even though the code can run faster, I would not advise you to do this kind of optimization. So sometimes, even though you can squeeze some additional performance from your code, it doesn't mean that you should do this. So one thing is a variable assignment. If you have a bunch of variables that you need to assign, you can do this the normal sequential way or you can go for this crazy parallel assignment. And I mean, you can gain some speed, but with this speed comes the hate of your colleagues that will be reading this code later. So in my opinion, it's not a problem. Okay, and another interesting property of Python is that the lookup for local variables is faster than the lookup for globals or buildings. So you can save some time if you store the reference to a building function or a global function in a local variable. So in this example, the only difference is on the line three where I'm storing the reference to the global app in a local append variable. And thanks to that, this function is like 35% faster. But then again, if you see this code for the first time, it's not very clear what it is supposed to do. It might be confusing to see this kind of append function because we are most used to see the list.append version. To sum up, there are different kind of optimizations. It's quite often about the speed, but not always. And there are also different levels of optimization. So sometimes if you cannot rewrite your whole application, maybe you can use a different approach. Even though the source code optimization is not the fastest way to optimize your code, those small improvements will add up. And the main advantage of it is cheap. So you can optimize the code at the moment of writing. You don't really need to rewrite something. And as long as you're writing idiomatic code and you don't reinvent the wheel but already use the existing functions and data structures in Python, then you're already doing it correctly. So be curious when you're coding. If you think that the different code structure can be faster, you can quickly check it with the time it and then you can improve it. All right. My name is Sebastian Witowski and I work at CERN. So if you guys want to talk about physics, then you're probably on the wrong conference. But if you want to talk about Python, you can catch me somewhere on the corridor. Thank you. All right. Brilliant. We have about two minutes for questions. Sebastian, if you're happy to take one or two questions, shall we have them? Fantastic. Who's got a question? You, sir. Awesome talk. I have a quick question. You showed us some profilers, code profilers. Did you have any preference? Any favorites? It really depends what you want to profile. Because if you care about the speed, then the basic ones are fine. But if you want to profile the memory users, then you might need to use different profilers. So it really depends what you want to profile. Any other questions? Yep. Any good recommendation on books or source where we can find best practices regarding this idiomatic Python? Not from the top of my head, but definitely there is some guides on the official Python documentation. But also for me, it's a lot of Googling for best practices, also reading a lot of stag overflow. There are some books, but I can't give you the names right now. Any more questions? Yes? Was that you sticking your hand up, sir, or just explaining something excitedly? Pretty really not a question. Any more questions? No? In that case, let's thank our speaker for a fantastic talking. We're very sorry.
 Good. Without further ado, I'd like to present our next speaker, Sebastian.  Sebastian will be talking about writing faster code. Give him a big hand.  Okay. Hi, everyone. Can you hear me? Okay. So, I would like to talk with you about writing  faster code. And last time I was giving a short talk, short lightning talk about writing  faster code, I remember someone pointed out that, well, basically, you can take your  Python code, rewrite it in C or C++, and it will be faster. And, I mean, the guy was  right. So, take any piece of Python code, rewrite it in C or C++, and probably it will  be automatically faster. So, I was thinking, hmm, if I say just writing faster code, it  might not be clear if I mean only Python or not. So, I had to fix the title of my presentation.  And I was very happy with the new title. I mean, it makes it clear we are not going  to talk about C or Java today. But then I realized the title is too long. I mean, even  though it's very clear, it barely fits on the slide. So, I have to change it again. So,  I got the version that means exactly the same thing, but it's shorter. So, this is how I  ended up with writing faster Python as a title for my today's talk. So, let's put aside the  question. I'm going to explain more about which programming language is better. We all  know the answer. That's why you're here. Python was not created to be a fast language that  you would use for some computations where every nanosecond counts. And that's fine with  me. I mean, Python is a great programming language that is very easy and fun to use.  Python is very easy to learn. The fact that it's so easy to read and write code in Python,  it's very encouraging for people new to software development. I see that it's getting more  and more popular in schools or at the universities as the first programming language. And I'm  not surprised. I mean, imagine you're completely new to programming and someone tells you,  hey, let me show you how much fun programming is and let's start with something super simple.  Let's just bring some text to the screen. And then he shows you those two examples.  I mean, one of them is clearly not something that you want to show to a beginner to encourage  him or her to start programming. But also, Python is not only useful for learning. There  are many big companies that are using Python. Companies with millions of users and billions  of requests per month. So your website is going to be fun using Python. So Python is  usually fast enough. But what if you decide that it's not fast enough anymore? For example,  your website starts giving you timeouts or maybe a faster code will bring more money  if you're a company. So it's time for optimization. But how do we optimize the code? Well, probably  you need to follow some rules. So let's try to Google for that. And if we open the first  link, we see that there are only three rules. Wow. It's going to be easier than we expected.  So let's take a closer look at those rules. First rule of optimization, don't. Okay, that  was easy. Any questions? Well, actually, no, there is more to that. So what does it mean  don't? Well, nine out of ten times when you think that you need optimization, you don't.  Especially in the early stage of your product's life. You might think it would be nice to  optimize your code a bit, but, well, first of all, you will waste time doing something  that is probably not needed. If you want your code to run faster, you can start with getting  a faster hardware in the first place. And second of all, optimization comes with a cost.  Most often, the only cost is the time that you spend optimizing your code. Well, sometimes  you will, it's also the time that you need to fix what you just broke with your optimization.  But also optimize it, the optimized code might not be as readable as it was in the first  place. And maybe your code is running faster, but it's now using more memory. So unless  you have really good reasons to optimize, don't do this. And if you know that you have  reasons to optimize, then we can move to the second rule of optimization. Don't do this  yet. And this is how I understand this rule. So first, make sure your code works. Then  make sure you have a good test suit. And only then you're ready for optimization. I love  this rule. It always reminds me how many times I broke it. I mean, so many times I was working  on a piece of software and I started thinking, hmm, maybe I can change this piece of code  and it will be faster. And maybe I will save like two lines of code. Was it a good idea?  No. And not only because I ended up breaking things, but most often I did end up breaking  things. But also I started jumping around the code and at some point I got confused  what I was writing in the first place. Did it make my code faster? I have no idea because  I had nothing to compare it to in the first place. If I would finish writing the code  and then try to improve it, I could measure both solutions and compare them. But in that  scenario I could only guess. And that brings me to the last rule of optimization. Don't  guess. Always profile your code. Humano are terrible in predicting the bottlenecks of your  code. So if you think that your code is slow, first profile it and then see what part of  it takes most of the time. Otherwise you may end up spending time rewriting one piece of  your code to just get like 1% of speed improvement. While there are other parts of your software  where you can gain way more improvement with less effort. So there are plenty of profiling  tools. There were already quite a few talks during Eurofighting about profiling. So I  will not go into the details. But if you don't know where to start, you can take a look at  the C profile module. It will show you a clear overview of how many times each function is  called and where the code is spent. And if you want to have some more advanced formatting  you can add the PSTADS module. Also if you prefer the graphic interface you can take a  look at the run, snake run or snake vis libraries. So once we are ready for optimization we  have to decide on which area we want to focus. So there are different levels of optimization.  Starting from the highest level you have the design level optimization. So depending on  the constraints and priorities of your system you can optimize it by redesigning it. It  might require rewriting your software in a different programming language or changing  the database type or changing the architecture to perform less database queries. So this  kind of optimization will usually give you the best improvement but it also takes most  time to do this. So I don't encourage you to rewrite your software from the scratch  but if you have some critical parts of your code that are run often you can optimize them  by rewriting them in C or C++. Because C is faster you will have some good speed improvement  for free. Well not really for free now you will have Python and C code in the same project.  So one level lower we have algorithms and data structures. So knowing good algorithms together  with their complexity definitely helps you creating a good and efficient software. For  example if you want to get the sum of numbers from 1 to N the first idea might be to get  a loop that goes through all the elements and ask them. It will work but it won't be  fast. Instead you can use the algorithm for the arithmetic sum which will give you the  same results and it will be more efficient. So the next level is the source code optimization.  And this is something that I will talk about in the second part of the presentation. Now  we are moving to the build level which involves setting up some specific build flags. So in  your daily work it's not something that you will do often. You can optimize Python for  a specific architecture but if you're a web developer like me this is either something  that you will do once per machine or you won't bother at all. Next we have the compile level.  So we can make some optimizations if your programming language has an ahead of time  compiler. Since I'm talking about Cpyton today which doesn't really have a head of time  compiler we're going to skip that part as well. And last but not least we have the runtime  level. So it's related with a specific compiler that you're using. Some compilers are faster  than the others. So for example if you replace Cpyton with PyPy you can get some improvements  depending on the use case of your software. But it really depends on what kind of piece  of code you're writing. So most of the time once you set up on a specific language implementation  there's nothing you have to do to benefit from this kind of optimization. It's usually  up to the creators of the compilers to optimize them. So simply updating to the new version  of the programming language you're using can make your code run a bit faster. So when you  optimize you probably want your code to run faster. And also use less memory. And basically  less of everything. The bad news is you can't have all of it. Optimization in one area  will usually cause deterioration in other areas. So you always have to decide which  resources are crucial and you have to optimize in that direction. So it's possible that optimization  will have nothing to do with the speed because there are other resources more important than  the raw speed. For example who cares that your program is now ten times faster when  it's crashing half of the time because it's running out of memory. Also another important  resource that people are often forgetting is the sanity. A sanity of a person that will  be maintaining your code. So please be nice to that person. You never know who that might  be. Yeah. So unless you're really writing a throwaway code, if you're making your code  harder to read and maintain then you're probably doing it wrong. So having those things clear  let's jump straight to how you can write faster Python. Also known as source code optimization.  My examples I'm using the version 3.5.1 of Python together with Python. Although the  example should work in both Python 2 and Python 3. So for measuring the execution time of  my code I will be using the magic time it function. It has some overhead comparing to  the standard time in library but it doesn't really matter because as long as we use the  same method to measure execution time of different functions we only need to know which method  is faster than by how much. So for each of my examples I will write different versions  of code, measure the execution time and compare them. So let's start with something simple.  Let's say you want to count the number of elements in a list. You can easily write a  simple loop that will increment the counter and this will work. It will be very so. You  can achieve the same results using the build in land function. As you can see for only  one million of results the difference is insanely huge. So my first advice is not to reinvent  the wheel but first check if there is a function that you can use. Python 3.5 has 68 building  functions. So it's nice to take a look at them and keep them in the back of your head  because they might be handy at some point. Also, before you start writing your own version  of order dictionary or dictionary with default values, take a look at the collection module  from the standard library. Even though it contains only like 10 different data types,  those are probably the data types you are looking for if the standard ones are not enough.  So let's say you have a list of one million elements and you want to select only the odd  numbers. So the naive version would be to use the for loop. So for each element of the  list you check if it's odd and if it is, you add it to another list. But I already show  you in the previous example that in most cases for loops can be replaced with something better.  In this case, you could use the build in filter function instead. And in Python 2, filter was  returning a list directly. Python 3 is returning an iterator. So I have to call the list function  to get the same results as in case of the for loop. And even though the list function  has some impact on the performance, it's negligible comparing to the time spent in the filter  function. Yet you can see that filter performs even slower than the for loop. Why does this  happen? Well, the fact that filter is returning now an iterator is a clear sign that it's  a wrong use case for this kind of function. So if you want to get the whole list as the  result, it's better to use the list comprehension. It's around 75% faster than the for loop and  at least for me it looks more clear. When you want to execute a piece of code, but you are  not sure if it will be successful. Maybe some variables are not set or like in this case,  the class might be missing some attributes. So you want to protect yourself somehow. The  first way you can do this is look before you leave or ask for permissions. What it means  is that you first check if the class has a specific attribute and then you perform the  operations. Usually this checking is done with the if statement. However, there's different  approach that you could use and it's called back for forgiveness. So in this scenario,  you perform the operation without checking the conditions first. But in case you expect  that something might break, you wrap your code in a tri-except block and you catch the exceptions  that were raised. And as you can see in the simple example, begging for forgiveness is  like three times faster. But it gets even better if you're checking for more conditions.  So here we are checking if three attributes are present. And asking for permission is  still slower and now it's also getting more difficult to read. So following the back for  forgiveness approach will result in a faster and more readable code. So we could say that  asking for forgiveness instead of checking the permissions is always a better way. But  we won't say that because it's not true. Exceptional handling is still quite expensive. So if the  attribute is actually missing, then begging for forgiveness will be slower than asking  for permissions. So as a rule of thumb, you can use the ask for permissions way if you  know that it's very likely that the attribute will be missing or there will be some other  problems that you can predict. Otherwise, if you're expecting that your code will work  in most of the times, try using try accept will result in a faster and quite often more  readable code. So for example, if you're fetching some files from the Internet and you expect  that everything will be fine unless there is no Internet connection. So instead of checking  if there is Internet connection, if it's fast enough, if there are no timeouts, just go for  the try accept. But then again, I strongly advise you to measure both solutions and see  maybe in your case it will be different. So let's tackle another problem, the membership  testing. So if you have a list and you want to check if it contains a specific element,  you can use a for loop. But the problem is you are iterating over the whole list even  though you're not really doing anything with all those elements. So you can replace the  for loop with the in statement. It will check if a specific element belongs to a given  set of data and it will do this twice as fast. But there is still one big problem with this  approach. The lookup time depends on where your element is located in that list. If it's  at the beginning of the list, you're lucky and you will get it fast. If it's at the  end of the list, you have to wait. So what would be really nice here if we had the data  structure that would have a constant lookup time. And actually in Python we have. We have  both sets and dictionary that have constant lookup time. So if we replace the list with  a set, then the lookup time becomes faster. From just few times faster to hundreds of  thousand times faster. So where's the catch? Well, you pay some time to convert this to  a set. And in this scenario, converting this list to a set takes more time than any of the  lookups in that list. So it doesn't really make sense. However, if you're checking membership  of different elements, quite often it makes sense to first convert it to a set. So speaking  of sets, they have another interesting feature. They don't contain duplicates. So basically  if you have a list of elements and you want to remove the duplicates, the fastest way to  do this is to convert this list to a set. But be aware that sets are not ordered. So if you  need to preserve the order, take a look at the order dictionary from the collection module.  So if you want to sort your list, you can either do this in place using the list.sort  function, or you can call this sorted function that will create a new list. And unless you  really need to have a new list, sorting in place will be like six times faster in this  scenario. This is for one million of random numbers. If you want to perform the same operation  on a large set of data, then you have two options. You can write a function that performs  the operation and call this function 1,000 times. Or you can call a function that takes  this set of data and performs the operation inside. And the second approach will be faster.  So if you can in an easy way replace multiple calls to one function with just one function,  then quite often it's a good idea. So what's the best way to check if a variable expression  is true? Well, you can explicitly compare this variable to true, but in most cases you're  adding additional redundancy. So you can simplify your condition to just if variable. And it  will return true unless the variable is false, non-zero, empty string, empty list, or other  false expression. And by doing that, your comparison gets faster by like 70%. And the  same rule applies when checking for false. So the fastest way to do this is to use if  not variable unless you really need to distinguish false from, let's say, non or zero or other  false values. It also applies to empty data structures. So simply doing if not a list  is, will be almost three times faster than explicitly checking the length of a list.  So let's take a look at different ways of defining functions in Python. The most common  one is to create a function with def keyword. The other way is to declare an anonymous function  with lambda. If you assign this lambda to a variable, it will act in the same way as  a function created with a def keyword. And as you can see, they are both equally fast.  Why? Because both versions do exactly the same thing. We can disassemble the code of  both versions with the disk library, and we'll see that inside is the same code. So is there  any difference? Well, if your function has more than one line, you can't use lambda.  And you can't really put documentation inside of lambda. Although you could have Pepe it  enabled in your code editor, it will complain each time you try to assign lambda to a variable.  And in his right, lambdas work really nice when you need a simple one-liner callback  for your functions, especially for functions like filter, mabel, reduce. And there are  also some quite few narrow use cases where it might be necessary to use lambda as a callback.  So if you want to read more, you can check the link at the bottom. But however, in any  other case, I would definitely recommend to use that. It's much cleaner. You can document  it properly. And the performance is exactly the same. So there are two ways how you can  create an empty list. So you can either call a list function or you can just use the literal  syntax. And as you can see, the literal syntax is faster. Why is it faster? Because if you  call a function, Python first needs to resolve this function. And with the literal syntax,  there is no overhead for that. And exact same thing happens for creating a dictionary. Okay,  I have two more examples that should be treated with quotient. Even though the code can run  faster, I would not advise you to do this kind of optimization. So sometimes, even though  you can squeeze some additional performance from your code, it doesn't mean that you should  do this. So one thing is a variable assignment. If you have a bunch of variables that you  need to assign, you can do this the normal sequential way or you can go for this crazy  parallel assignment. And I mean, you can gain some speed, but with this speed comes the  hate of your colleagues that will be reading this code later. So in my opinion, it's not  a problem. Okay, and another interesting property of Python is that the lookup for local variables  is faster than the lookup for globals or buildings. So you can save some time if you store the  reference to a building function or a global function in a local variable. So in this example,  the only difference is on the line three where I'm storing the reference to the global app  in a local append variable. And thanks to that, this function is like 35% faster. But  then again, if you see this code for the first time, it's not very clear what it is supposed  to do. It might be confusing to see this kind of append function because we are most used  to see the list.append version. To sum up, there are different kind of optimizations.  It's quite often about the speed, but not always. And there are also different levels  of optimization. So sometimes if you cannot rewrite your whole application, maybe you  can use a different approach. Even though the source code optimization is not the fastest  way to optimize your code, those small improvements will add up. And the main advantage of it  is cheap. So you can optimize the code at the moment of writing. You don't really need  to rewrite something. And as long as you're writing idiomatic code and you don't reinvent  the wheel but already use the existing functions and data structures in Python, then you're  already doing it correctly. So be curious when you're coding. If you think that the  different code structure can be faster, you can quickly check it with the time it and  then you can improve it. All right. My name is Sebastian Witowski and I work at CERN.  So if you guys want to talk about physics, then you're probably on the wrong conference.  But if you want to talk about Python, you can catch me somewhere on the corridor. Thank  you. All right. Brilliant. We have about two minutes for questions. Sebastian, if you're  happy to take one or two questions, shall we have them? Fantastic. Who's got a question?  You, sir. Awesome talk. I have a quick question. You showed us some profilers, code profilers.  Did you have any preference? Any favorites? It really depends what you want to profile.  Because if you care about the speed, then the basic ones are fine. But if you want to profile  the memory users, then you might need to use different profilers. So it really depends  what you want to profile. Any other questions? Yep. Any good recommendation on books or  source where we can find best practices regarding this idiomatic Python?  Not from the top of my head, but definitely there is some guides on the official Python  documentation. But also for me, it's a lot of Googling for best practices, also reading  a lot of stag overflow. There are some books, but I can't give you the names right now.  Any more questions? Yes? Was that you sticking your hand up, sir, or just explaining something  excitedly? Pretty really not a question. Any more questions? No? In that case, let's  thank our speaker for a fantastic talking. We're very sorry.
Sebastian_Witowski_-_Writing_faster_Python_0_2195.png Sebastian_Witowski_-_Writing_faster_Python_2196_3491.png Sebastian_Witowski_-_Writing_faster_Python_3492_3803.png Sebastian_Witowski_-_Writing_faster_Python_3804_5771.png Sebastian_Witowski_-_Writing_faster_Python_5772_6107.png Sebastian_Witowski_-_Writing_faster_Python_6108_6359.png Sebastian_Witowski_-_Writing_faster_Python_6360_6455.png Sebastian_Witowski_-_Writing_faster_Python_6456_7199.png Sebastian_Witowski_-_Writing_faster_Python_7200_7739.png Sebastian_Witowski_-_Writing_faster_Python_7740_7799.png Sebastian_Witowski_-_Writing_faster_Python_7800_8291.png Sebastian_Witowski_-_Writing_faster_Python_8292_9671.png Sebastian_Witowski_-_Writing_faster_Python_9672_11435.png Sebastian_Witowski_-_Writing_faster_Python_11436_13823.png Sebastian_Witowski_-_Writing_faster_Python_13824_19067.png Sebastian_Witowski_-_Writing_faster_Python_19068_28247.png Sebastian_Witowski_-_Writing_faster_Python_28248_30395.png Sebastian_Witowski_-_Writing_faster_Python_30396_32039.png Sebastian_Witowski_-_Writing_faster_Python_32040_34211.png Sebastian_Witowski_-_Writing_faster_Python_34212_34295.png Sebastian_Witowski_-_Writing_faster_Python_34296_36226.png
Good. Without further ado, I'd like to present our next speaker, Sebastian. Sebastian will be talking about writing faster code. Give him a big hand. Okay. Hi, everyone. Can you hear me? Okay. So, I would like to talk with you about writing faster code. And last time I was giving a short talk, short lightning talk about writing faster code, I remember someone pointed out that, well, basically, you can take your Python code, rewrite it in C or C++, and it will be faster. And, I mean, the guy was right. So, take any piece of Python code, rewrite it in C or C++, and probably it will be automatically faster. So, I was thinking, hmm, if I say just writing faster code, it might not be clear if I mean only Python or not. So, I had to fix the title of my presentation. And I was very happy with the new title. I mean, it makes it clear we are not going to talk about C or Java today. But then I realized the title is too long. I mean, even though it's very clear, it barely fits on the slide. So, I have to change it again. So, I got the version that means exactly the same thing, but it's shorter. So, this is how I ended up with writing faster Python as a title for my today's talk. So, let's put aside the question. I'm going to explain more about which programming language is better. We all know the answer. That's why you're here. Python was not created to be a fast language that you would use for some computations where every nanosecond counts. And that's fine with me. I mean, Python is a great programming language that is very easy and fun to use. Python is very easy to learn. The fact that it's so easy to read and write code in Python, it's very encouraging for people new to software development. I see that it's getting more and more popular in schools or at the universities as the first programming language. And I'm not surprised. I mean, imagine you're completely new to programming and someone tells you, hey, let me show you how much fun programming is and let's start with something super simple. Let's just bring some text to the screen. And then he shows you those two examples. I mean, one of them is clearly not something that you want to show to a beginner to encourage him or her to start programming. But also, Python is not only useful for learning. There are many big companies that are using Python. Companies with millions of users and billions of requests per month. So your website is going to be fun using Python. So Python is usually fast enough. But what if you decide that it's not fast enough anymore? For example, your website starts giving you timeouts or maybe a faster code will bring more money if you're a company. So it's time for optimization. But how do we optimize the code? Well, probably you need to follow some rules. So let's try to Google for that. And if we open the first link, we see that there are only three rules. Wow. It's going to be easier than we expected. So let's take a closer look at those rules. First rule of optimization, don't. Okay, that was easy. Any questions? Well, actually, no, there is more to that. So what does it mean don't? Well, nine out of ten times when you think that you need optimization, you don't. Especially in the early stage of your product's life. You might think it would be nice to optimize your code a bit, but, well, first of all, you will waste time doing something that is probably not needed. If you want your code to run faster, you can start with getting a faster hardware in the first place. And second of all, optimization comes with a cost. Most often, the only cost is the time that you spend optimizing your code. Well, sometimes you will, it's also the time that you need to fix what you just broke with your optimization. But also optimize it, the optimized code might not be as readable as it was in the first place. And maybe your code is running faster, but it's now using more memory. So unless you have really good reasons to optimize, don't do this. And if you know that you have reasons to optimize, then we can move to the second rule of optimization. Don't do this yet. And this is how I understand this rule. So first, make sure your code works. Then make sure you have a good test suit. And only then you're ready for optimization. I love this rule. It always reminds me how many times I broke it. I mean, so many times I was working on a piece of software and I started thinking, hmm, maybe I can change this piece of code and it will be faster. And maybe I will save like two lines of code. Was it a good idea? No. And not only because I ended up breaking things, but most often I did end up breaking things. But also I started jumping around the code and at some point I got confused what I was writing in the first place. Did it make my code faster? I have no idea because I had nothing to compare it to in the first place. If I would finish writing the code and then try to improve it, I could measure both solutions and compare them. But in that scenario I could only guess. And that brings me to the last rule of optimization. Don't guess. Always profile your code. Humano are terrible in predicting the bottlenecks of your code. So if you think that your code is slow, first profile it and then see what part of it takes most of the time. Otherwise you may end up spending time rewriting one piece of your code to just get like 1% of speed improvement. While there are other parts of your software where you can gain way more improvement with less effort. So there are plenty of profiling tools. There were already quite a few talks during Eurofighting about profiling. So I will not go into the details. But if you don't know where to start, you can take a look at the C profile module. It will show you a clear overview of how many times each function is called and where the code is spent. And if you want to have some more advanced formatting you can add the PSTADS module. Also if you prefer the graphic interface you can take a look at the run, snake run or snake vis libraries. So once we are ready for optimization we have to decide on which area we want to focus. So there are different levels of optimization. Starting from the highest level you have the design level optimization. So depending on the constraints and priorities of your system you can optimize it by redesigning it. It might require rewriting your software in a different programming language or changing the database type or changing the architecture to perform less database queries. So this kind of optimization will usually give you the best improvement but it also takes most time to do this. So I don't encourage you to rewrite your software from the scratch but if you have some critical parts of your code that are run often you can optimize them by rewriting them in C or C++. Because C is faster you will have some good speed improvement for free. Well not really for free now you will have Python and C code in the same project. So one level lower we have algorithms and data structures. So knowing good algorithms together with their complexity definitely helps you creating a good and efficient software. For example if you want to get the sum of numbers from 1 to N the first idea might be to get a loop that goes through all the elements and ask them. It will work but it won't be fast. Instead you can use the algorithm for the arithmetic sum which will give you the same results and it will be more efficient. So the next level is the source code optimization. And this is something that I will talk about in the second part of the presentation. Now we are moving to the build level which involves setting up some specific build flags. So in your daily work it's not something that you will do often. You can optimize Python for a specific architecture but if you're a web developer like me this is either something that you will do once per machine or you won't bother at all. Next we have the compile level. So we can make some optimizations if your programming language has an ahead of time compiler. Since I'm talking about Cpyton today which doesn't really have a head of time compiler we're going to skip that part as well. And last but not least we have the runtime level. So it's related with a specific compiler that you're using. Some compilers are faster than the others. So for example if you replace Cpyton with PyPy you can get some improvements depending on the use case of your software. But it really depends on what kind of piece of code you're writing. So most of the time once you set up on a specific language implementation there's nothing you have to do to benefit from this kind of optimization. It's usually up to the creators of the compilers to optimize them. So simply updating to the new version of the programming language you're using can make your code run a bit faster. So when you optimize you probably want your code to run faster. And also use less memory. And basically less of everything. The bad news is you can't have all of it. Optimization in one area will usually cause deterioration in other areas. So you always have to decide which resources are crucial and you have to optimize in that direction. So it's possible that optimization will have nothing to do with the speed because there are other resources more important than the raw speed. For example who cares that your program is now ten times faster when it's crashing half of the time because it's running out of memory. Also another important resource that people are often forgetting is the sanity. A sanity of a person that will be maintaining your code. So please be nice to that person. You never know who that might be. Yeah. So unless you're really writing a throwaway code, if you're making your code harder to read and maintain then you're probably doing it wrong. So having those things clear let's jump straight to how you can write faster Python. Also known as source code optimization. My examples I'm using the version 3.5.1 of Python together with Python. Although the example should work in both Python 2 and Python 3. So for measuring the execution time of my code I will be using the magic time it function. It has some overhead comparing to the standard time in library but it doesn't really matter because as long as we use the same method to measure execution time of different functions we only need to know which method is faster than by how much. So for each of my examples I will write different versions of code, measure the execution time and compare them. So let's start with something simple. Let's say you want to count the number of elements in a list. You can easily write a simple loop that will increment the counter and this will work. It will be very so. You can achieve the same results using the build in land function. As you can see for only one million of results the difference is insanely huge. So my first advice is not to reinvent the wheel but first check if there is a function that you can use. Python 3.5 has 68 building functions. So it's nice to take a look at them and keep them in the back of your head because they might be handy at some point. Also, before you start writing your own version of order dictionary or dictionary with default values, take a look at the collection module from the standard library. Even though it contains only like 10 different data types, those are probably the data types you are looking for if the standard ones are not enough. So let's say you have a list of one million elements and you want to select only the odd numbers. So the naive version would be to use the for loop. So for each element of the list you check if it's odd and if it is, you add it to another list. But I already show you in the previous example that in most cases for loops can be replaced with something better. In this case, you could use the build in filter function instead. And in Python 2, filter was returning a list directly. Python 3 is returning an iterator. So I have to call the list function to get the same results as in case of the for loop. And even though the list function has some impact on the performance, it's negligible comparing to the time spent in the filter function. Yet you can see that filter performs even slower than the for loop. Why does this happen? Well, the fact that filter is returning now an iterator is a clear sign that it's a wrong use case for this kind of function. So if you want to get the whole list as the result, it's better to use the list comprehension. It's around 75% faster than the for loop and at least for me it looks more clear. When you want to execute a piece of code, but you are not sure if it will be successful. Maybe some variables are not set or like in this case, the class might be missing some attributes. So you want to protect yourself somehow. The first way you can do this is look before you leave or ask for permissions. What it means is that you first check if the class has a specific attribute and then you perform the operations. Usually this checking is done with the if statement. However, there's different approach that you could use and it's called back for forgiveness. So in this scenario, you perform the operation without checking the conditions first. But in case you expect that something might break, you wrap your code in a tri-except block and you catch the exceptions that were raised. And as you can see in the simple example, begging for forgiveness is like three times faster. But it gets even better if you're checking for more conditions. So here we are checking if three attributes are present. And asking for permission is still slower and now it's also getting more difficult to read. So following the back for forgiveness approach will result in a faster and more readable code. So we could say that asking for forgiveness instead of checking the permissions is always a better way. But we won't say that because it's not true. Exceptional handling is still quite expensive. So if the attribute is actually missing, then begging for forgiveness will be slower than asking for permissions. So as a rule of thumb, you can use the ask for permissions way if you know that it's very likely that the attribute will be missing or there will be some other problems that you can predict. Otherwise, if you're expecting that your code will work in most of the times, try using try accept will result in a faster and quite often more readable code. So for example, if you're fetching some files from the Internet and you expect that everything will be fine unless there is no Internet connection. So instead of checking if there is Internet connection, if it's fast enough, if there are no timeouts, just go for the try accept. But then again, I strongly advise you to measure both solutions and see maybe in your case it will be different. So let's tackle another problem, the membership testing. So if you have a list and you want to check if it contains a specific element, you can use a for loop. But the problem is you are iterating over the whole list even though you're not really doing anything with all those elements. So you can replace the for loop with the in statement. It will check if a specific element belongs to a given set of data and it will do this twice as fast. But there is still one big problem with this approach. The lookup time depends on where your element is located in that list. If it's at the beginning of the list, you're lucky and you will get it fast. If it's at the end of the list, you have to wait. So what would be really nice here if we had the data structure that would have a constant lookup time. And actually in Python we have. We have both sets and dictionary that have constant lookup time. So if we replace the list with a set, then the lookup time becomes faster. From just few times faster to hundreds of thousand times faster. So where's the catch? Well, you pay some time to convert this to a set. And in this scenario, converting this list to a set takes more time than any of the lookups in that list. So it doesn't really make sense. However, if you're checking membership of different elements, quite often it makes sense to first convert it to a set. So speaking of sets, they have another interesting feature. They don't contain duplicates. So basically if you have a list of elements and you want to remove the duplicates, the fastest way to do this is to convert this list to a set. But be aware that sets are not ordered. So if you need to preserve the order, take a look at the order dictionary from the collection module. So if you want to sort your list, you can either do this in place using the list.sort function, or you can call this sorted function that will create a new list. And unless you really need to have a new list, sorting in place will be like six times faster in this scenario. This is for one million of random numbers. If you want to perform the same operation on a large set of data, then you have two options. You can write a function that performs the operation and call this function 1,000 times. Or you can call a function that takes this set of data and performs the operation inside. And the second approach will be faster. So if you can in an easy way replace multiple calls to one function with just one function, then quite often it's a good idea. So what's the best way to check if a variable expression is true? Well, you can explicitly compare this variable to true, but in most cases you're adding additional redundancy. So you can simplify your condition to just if variable. And it will return true unless the variable is false, non-zero, empty string, empty list, or other false expression. And by doing that, your comparison gets faster by like 70%. And the same rule applies when checking for false. So the fastest way to do this is to use if not variable unless you really need to distinguish false from, let's say, non or zero or other false values. It also applies to empty data structures. So simply doing if not a list is, will be almost three times faster than explicitly checking the length of a list. So let's take a look at different ways of defining functions in Python. The most common one is to create a function with def keyword. The other way is to declare an anonymous function with lambda. If you assign this lambda to a variable, it will act in the same way as a function created with a def keyword. And as you can see, they are both equally fast. Why? Because both versions do exactly the same thing. We can disassemble the code of both versions with the disk library, and we'll see that inside is the same code. So is there any difference? Well, if your function has more than one line, you can't use lambda. And you can't really put documentation inside of lambda. Although you could have Pepe it enabled in your code editor, it will complain each time you try to assign lambda to a variable. And in his right, lambdas work really nice when you need a simple one-liner callback for your functions, especially for functions like filter, mabel, reduce. And there are also some quite few narrow use cases where it might be necessary to use lambda as a callback. So if you want to read more, you can check the link at the bottom. But however, in any other case, I would definitely recommend to use that. It's much cleaner. You can document it properly. And the performance is exactly the same. So there are two ways how you can create an empty list. So you can either call a list function or you can just use the literal syntax. And as you can see, the literal syntax is faster. Why is it faster? Because if you call a function, Python first needs to resolve this function. And with the literal syntax, there is no overhead for that. And exact same thing happens for creating a dictionary. Okay, I have two more examples that should be treated with quotient. Even though the code can run faster, I would not advise you to do this kind of optimization. So sometimes, even though you can squeeze some additional performance from your code, it doesn't mean that you should do this. So one thing is a variable assignment. If you have a bunch of variables that you need to assign, you can do this the normal sequential way or you can go for this crazy parallel assignment. And I mean, you can gain some speed, but with this speed comes the hate of your colleagues that will be reading this code later. So in my opinion, it's not a problem. Okay, and another interesting property of Python is that the lookup for local variables is faster than the lookup for globals or buildings. So you can save some time if you store the reference to a building function or a global function in a local variable. So in this example, the only difference is on the line three where I'm storing the reference to the global app in a local append variable. And thanks to that, this function is like 35% faster. But then again, if you see this code for the first time, it's not very clear what it is supposed to do. It might be confusing to see this kind of append function because we are most used to see the list.append version. To sum up, there are different kind of optimizations. It's quite often about the speed, but not always. And there are also different levels of optimization. So sometimes if you cannot rewrite your whole application, maybe you can use a different approach. Even though the source code optimization is not the fastest way to optimize your code, those small improvements will add up. And the main advantage of it is cheap. So you can optimize the code at the moment of writing. You don't really need to rewrite something. And as long as you're writing idiomatic code and you don't reinvent the wheel but already use the existing functions and data structures in Python, then you're already doing it correctly. So be curious when you're coding. If you think that the different code structure can be faster, you can quickly check it with the time it and then you can improve it. All right. My name is Sebastian Witowski and I work at CERN. So if you guys want to talk about physics, then you're probably on the wrong conference. But if you want to talk about Python, you can catch me somewhere on the corridor. Thank you. All right. Brilliant. We have about two minutes for questions. Sebastian, if you're happy to take one or two questions, shall we have them? Fantastic. Who's got a question? You, sir. Awesome talk. I have a quick question. You showed us some profilers, code profilers. Did you have any preference? Any favorites? It really depends what you want to profile. Because if you care about the speed, then the basic ones are fine. But if you want to profile the memory users, then you might need to use different profilers. So it really depends what you want to profile. Any other questions? Yep. Any good recommendation on books or source where we can find best practices regarding this idiomatic Python? Not from the top of my head, but definitely there is some guides on the official Python documentation. But also for me, it's a lot of Googling for best practices, also reading a lot of stag overflow. There are some books, but I can't give you the names right now. Any more questions? Yes? Was that you sticking your hand up, sir, or just explaining something excitedly? Pretty really not a question. Any more questions? No? In that case, let's thank our speaker for a fantastic talking. We're very sorry.
thinking, hmm, if I say just writing faster code, something that is probably not needed. If you using more memory. So unless you have really good Don't do this yet. And this is how I understand this rule. So first, make sure your code works. it and then see what part of it takes most of the profiling. So I will not go into the details. But clear overview of how many times each function is be fast. Instead you can use the algorithm for this is something that I will talk about in the up some specific build flags. So in your daily You can optimize Python for a specific you have to do to benefit from this kind of how you can write faster Python. Also known as and Python 3. So for measuring the execution time result, it's better to use the list And asking for permission is still slower and now could say that asking for forgiveness instead of begging for forgiveness will be slower than you can use the ask for permissions way if you know that it's very likely that the attribute you can use a for loop. But the problem is you converting this list to a set takes more time often it makes sense to first convert it to a elements and you want to remove the duplicates, 1,000 times. Or you can call a function that work really nice when you need a simple one-liner than the lookup for globals or buildings. So you up, there are different kind of optimizations. optimization is not the fastest way to optimize don't reinvent the wheel but already use the existing functions and data structures in Python, different code structure can be faster, you can