Your Name Is Invalid!
Šedivý, Miroslav
People have names. Most people do. People have first names and last names. Many people do. People have any sorts of names that often don't fit fixed fields in the forms. These names may contain letters, accented letters, and other characters, that may cause problems to your code depending on the encoding you use. They may look differently in uppercase and lowercase, or may not be case foldable at all. Searching and sorting these names may be tricky too. And if you design an application, web form, and/or database dealing with personal names, you'll have to take that into account. This talk is not about GDPR, but will help you to use the best tools to handle encoding and locales in Python and prevent your application from appearing in my talk and in uxfails memes.
 Welcome to the next afternoon session. Our next speaker is Miroslav Shadivi. He will talk about your name is Invalid. Names of people cannot be Invalid. Your app shouldn't. He will mention many things I guess about these wrong names. So, okay. Miro, please start. Thank you. Hi, let me introduce myself. My name is Miroslav. We have still some problems with the transmission. Wait, wait, wait. Hey, we couldn't prepare it beforehand. I will have to switch something here. No, no, that's not still. Ah, no. Okay. My name is Miroslav Shadivi. And sometimes I still see somewhere that my name is Invalid. If you want to pronounce it correctly, this is the IPA, how to pronounce it, and this is how to type it on a standard keyboard layout with Komposki. I very get a salute to GmbH and Karlsruhe. And if you would like to see the region with the best German wine and the best French beer, come to see us to Karlsruhe. We do a lot of great stuff with Python and other open source technology. Originally, I was born in Czechoslovakia, studied in France, and now I'm a German. What we are going to speak about today, first about strings and names in Python, how the strings bytes work, encoding, normalizing, case folding, sorting, and regular expressions, and then about names on the web, about the apps, about databases, how to work with the names correctly, and how to allow or test whether a name is valid or not. So in Python 3, there are two types of objects that work with strings. The first is standard string that, for each character, knows more than 1 million possible characters. That's Unicode, and that is something that is in the working memory. So you work with it, but you don't send it over network or store it into a file. And there are bytes with 256 possible combinations that are used to store data on file and or network. And you have to combine or you have to convert between these somehow. So for example, if your string is or name is Chagnaris and you just encode it, you get a series of the bytes that looks almost the same. And the other way around from bytes, back to string, you convert it with using the decode method. The bytes, you can see them with, you can recognize them, but they have a small b before this quote. And the strings, they have nothing. If you encode them standardly in Python, it uses UTF-8 as a standard encoding. But in this case, it will be 12 characters as a string or 12 bytes as bytes. So there is no difference. But for example, if you have the German name Mühler and you encode it, then from six characters, you will get seven bytes because the letter U, umlaut, is will be written or stored as two bytes. Yeah, if you have Chinese name, this is not a name. This is Mihao. So a simple hello in Chinese from two Chinese characters. In UTF-8, it will be stored as six bytes. There are other encodings. UTF-8 is relatively the newest one, although it is over two decades old. There is ASCII, but it's around since many, many dozens of years. If your name is Tregnorys, of course, you can be encoded in any encoding. And actually, you are the one who decides about the encoding. But in this case, Tregnorys with 12 characters will be encoded into 12 bytes using ASCII. But if you have the name Mühler, you cannot do it with ASCII. You can do it, for example, with Latin 1 that knows more characters, that knows even this UTF-8. And in this case, the six characters will be encoded as six bytes using Latin 1. My last name, Shaivi, comes from Czechoslovakia, which doesn't use Latin 1. It uses Latin 2 alphabet or encoding. And our six characters will be encoded as six bytes. But always a byte is something that has only 256 possibilities. So you don't have the information which encoding it is in. In this case, we knew that we are going to use Latin 1 or Latin 2. But normally, you don't know which encoding it is in. And sometimes, it can happen something like this, that I get a mail here in Germany that types my last name wrongly. How is it possible that instead of the s with Karen, I get a question mark? Would they use Python? So if I take my last name and encode it with Latin 2, I will get the correct six bytes that mean the rising. But there is no question mark. If I do it with Latin 1, I will get a unique error because the hash cannot be encoded in Latin 1. There is in those 256 possible bytes of Latin 1, the letter h is not there. But in Python, you can define an error. What to do in the case of error? Default is raised an exception. But it is possible to say replace it with something. And replace default is a question mark. So if you encode some string that contains characters that are outside of the encoding, so for example, sure, in the case of Latin 1. And you say, in the case of error, replace it, it will automatically replace it with a question mark. And this is probably how this happened. You can even do something more. You can define your own function if you don't want question mark the whole time. And you can register an error and encode it. And then tell, OK, if I use the errors method, replace randomly, for example, call this lambda function that gives a random digit. And this x start plus plus 1 means that then go on with the next character. You can define more movements around such cases. So in this case, I can tell if I want to encode my last name with Latin 1. And if a character cannot be encoded, I then put there some random digit. In this case, number 5, for example. And this is probably how they are doing it, random characters. Sometimes there is a space. Sometimes there is no space. Sometimes there is something like this. Sometimes there is an inverted question mark. Actually, all three pictures come from the same company that has different means of communication, like letter, like card, customer card, or online form. Although I have entered it correctly in the web form when I signed up for their service. There are some other cases. Here, some non-name German flying company that has huge airplanes but have problem with their form. They tell me, hey, you can only enter letters in the adults last name field. What are letters? My last name consists only of letters. What is a letter? In Python 3, you can do something like this. You can give variable the name that is not pure ASCII, that consists of other characters. But you cannot give variable a name consisting of starting with a number. But sure, it's not a number. It's a character. It's a letter. How does Python know that sure is a letter but a smiley is not a letter? Question mark is not a letter. There is a unique of data that has the standard module, unique of data, that has the method category and name. So we give it any character and it will tell you which category and name it is. So in this case, I gave a few numbers and smiley and dot and so on. There you see all these characters. And then you see two letters. There is LL, LU, ZS, POSO, and so on. And then there is the official name of the character. So everything that these two characters, when they start with an L, it means a letter. LL is a lowercase letter. LU is uppercase letter. And so it is defined. Every character in the mini code is defined. It belongs to some category and it is a letter or not. So you can use the capital letter sharp as a variable name in Python. These are all the categories that we are going to see a little bit later how we can use them. Also, if you open the character map application, you will see all the information about the character. And you can use actually the name to write the character with the capital N and with the character. Case folding. Case folding means the conversion between uppercase and lowercase. So you see we have some letters in the lowercase. We got uppercase and vice versa. It has a few exceptions. So for example, the German sharp as lowercase converts to S as uppercase, although there is official sharp as uppercase character. The other way around, it converts correctly. So it works for most other characters between uppercase and lowercase. Of course, you have, for example, Chinese characters that support something like that. But there is also with this uppercase and lowercase, there is one problem even in the basic ASCII. There is one ASCII pair, over case uppercase, that we, like the Western English, European language group, we've wrong. Which one is it? It's I. You see, if I take lowercase I, I do it to uppercase. I get this. The other way around as well, from the uppercase I, I get the lowercase I. You see the difference? The lowercase I has a dot. The uppercase I doesn't have a dot. How do you explain that to someone from Turkey? Because they have the I with the lowercase uppercase that is E and I without dot, a lowercase uppercase that is I. And when they convert between these two pairs of characters, they always get the other one. So if you want to work with Turkish text, you have to import international components for a unique code library and then work a little bit more complicated codes that converts between the Turkish pairs of I and I correctly. Normalization is something different. If you have already your unique code text and you think, OK, that's great. I can now work with that. I can access the characters, the length, and everything. You will get probably two strings that look the same and they are not the same. Because when they are normalized, they will be divided or some characters will be split in two characters. So for example, the first word has three characters. The second one is U with dots like the resist mode. And the second version has this the resist behind the U. So the first U is standard U with value of 117. And the second one then comes this combining the resist that is of the category MN, which means a combining character. And also, this is not an error, but you see that the whole line is a little bit offset to the left. This means that this combining the resist doesn't have a width. And if you can convert between those two versions, like the normalized and the denormalized, and between the two normal forms, and they will not be the same. So it means that if you have a lot of text, try to normalize them or denormalize them to one form and only then compare them. With this combining the resist and there are plenty of combining characters, you can do some fun stuff. You can put more characters after each other. And then you can get something like this. This is actually a mistake overflow answer to the question how to use, how to parse HTML with the Red War expression. So you see at the end, there are some characters with plenty, plenty, plenty of combining characters. Alphabetic sorting, that's not so easy if you just sort the standard Python sort using the standard Python sort, you will not get the alphabetical order because you see that there are some uppercase, lowercase, then you have some other letters. What's the new, like, letters, then there is some letters, and then the shuffles comes at the end because it has a really high unique value. You can use locale. So you start locale with the German, for example, and then you sort the characters and you will get it out of the out between A and B. But if your locale is, for example, Swedish, then it will be different because in Swedish all the umlauts are A, B, N. And there are some other exceptions, for example, in Hungarian. The CS pronounced as ch is extra letter between C and D. In the Czech and Slovak, we have plenty of characters with carons that we put just behind the respective letter. There is CH pronounced as ch, which is between H and I. And for example, in French, they have even more rules. It means that they sort everything according to ASCII. But if there are exceptions, they sort them from the end of the word. So first they sort the last syllable and then the penultimate and then and so on. So there are plenty of rules. The problem with locale is if you do that in your library, then the locale of the whole process will be changed. This means that if you have some multilingual program that has to have the input of several users at the same time, such as WebSaver, and you switch the locale the whole time, it will, there will be a chaos. So that's why there is this ICU that we have used already to convert the lowercase uppercase to the other case that you can use to sort something without changing the system. I have read that there are some recommendations to use by use EA, although some unique code collocation algorithm, collator algorithm. The difference between these two is that by use EA doesn't define which language it uses. It has a great database included. And it's announced that it works very well for characters outside of English. Yes, you can use it if you only need some English list sorted list, and you want to have those other characters included at not so bad position. But don't use this if you have pure German, Swedish, or Czech text, because it doesn't use the specific sorting of the specific language. For that case, I would prefer to use ICU. At the end, unique code regular expressions. So let's have a look at this text. There is a new space 1, 2, 3. And I want to extract the word minkin. If I do this, so all characters between A, Z, uppercase, lowercase, it will ignore this. So umlaut and return to two words like that. It's wrong. If I say I want a backslash w, so all alphanumeric characters, there are also numeric characters. So I will return to return also the 1, 2, 3 that I want to have. And in that case, we will have to use the reggis. It is a third party library that works 100% like the standard. RE works and only has a few functions extra. And in this case, with backslash p, you can define unique code characters that belong to some category. And you have seen at the beginning, this capital L means that it is a letter. And you can also define something like L U for uppercase letters, L L for lowercase letters, and so on. You have seen that at the beginning in the list of categories. And in that case, we get the word. I came here for Python, but I say for the names. So this was the Python part. Now let's have a look at the names. If I were with you in the normal offline conference, I would just ask, raise your hand, who can write their first and last name like this? I don't know how many of you can write it later in the discussion. But probably, OK, I can tell my first name, it'll slow my last name. But maybe some of you have something like the middle name or something Petronomic or Metronomic surname in Portuguese or Swedish, sorry, in Spanish. There is last name, first name in many Asian languages or in Hungarian. There is name number if we have some popes or queens or things here. Or there is some simple name. But there is also something like the name of the father and the daughter of the father. So for example, in Icelandic, if your name is Sigur Johansson, your father was Johan, it means that you have to use or work with these names a little bit differently because Sigur Johansson doesn't mean that it's a mystery Johansson. There is no mystery Johansson. Sigur Johansson is Sigur or Mr. Sigur Johansson. And you will find them in the phone list, the phone book, you will find them as S, like Sigur Johansson and not as J, because they don't sort their names according to the Petronomic surname or the name of their family. Of course, there are plenty of phononsu and funder, dedede, and so on in different languages. And many times you just don't know whether to put it together with the last name or first name or somewhere in the beginning and how to sort them. Yeah, of course, some doctors in some languages, the doctor, became the part of your last name. So how to do it correctly? Actually, if you ask for a name, just ask for the full name. Let someone write the full name, and then you can ask how they should call you. So for example, Mr. T, he says, oh, my first name is Mr. My middle name is dot or period, and my last name is T. And then even if you can write it correctly, you will sometimes get an error message, like please enter characters from the European character set only. This happened in Germany. My last name, Shaggy, is not far away from Germany. Sorry, what's European character set? And the same service a few pages later, please enter a full valid name. They didn't want to accept my name. And my name is valid. So the names of people cannot be valid. So if you work with names in some forms, don't assume anything. Don't put random limit on their length. Minimum, maximum, there is no minimum. There is no maximum. So for example, Pipi, lunchroom, logstocking, her name is like this. Or a character or got that a dolphin, which Gutenberg, this was some German minister a few years ago, his name even appeared on Wikipedia with a typo, and even he didn't see that Twitter as a typo. Don't use stop words. So if it's a stop word or like root word in your language, it can be a perfect word in another language. The all family members don't have necessarily the same name. So for example, all the male members of my family are Shaggy, or the female members of my family are Shaggy. Don't assume anything, different transcription from non-letton alphabet. So for example, if you have the Russian name Chekhov, in every European languages, types it a little bit differently. And other way around, so for example, I was in Russia twice with the same passport, and I got to visa. And most times, my last name was typed differently in the serial alphabet. Or in Chinese, the same. If you use something like made the name or name, know that then they change their family names too. One letter name is probably not an initial, so binwa bi lander, mother brother bi is not an abbreviation. So actually, you should accept almost all printable characters. Hello, my name is Mr. Nol. My name makes me invisible to computers and articles. I don't understand how someone can program something that doesn't accept the last name. Or the same with this little Bobby Teigos. Please don't do that. Yeah, the names of the streets and cities and so on, the most widespread name of street in Germany is, of course, Einbahnstraße. So one-way street, you see it almost everywhere now. How in German, this Straße, like street, is with the sharp S, you will find a lot of American lists of German companies with this B capital and instead of the sharp S. So the names of the places, all somewhere in Scandinavia, the Zipstil Union, so the Y in somewhere in France. And then the hint question is, what is your mother's name? The main, at least six characters, or they ask you for your place of birth, at least six characters. No, it doesn't work like that. Or on the other hand, you have places like Hanweil, and the other places that are a little bit longer. Or, sometimes you even don't take a mattress, because you just described where you want to take your mother. There is a long list of falsehoods, programmers believe about names, I'm not going to read everything. So if you are working with the names of people, I now ignore GDPR, everything. I programmatically respect their names. And don't break the locale, so use the ICU. Convert hamburger principle, convert from bytes to string as soon as possible, and convert back to bytes as late as possible. UTFAB school, Python 3D school, because 2 use Python 3D and UTF8. If you tell your user because of your spitefulness, ignorance, laziness, or something, their name is invalid, this is not cool. There is a Twitter account, your name is valid. It's not invalid, it is, your name is valid, because there is a limit on the characters at Twitter that will maybe, your case can appear if you do something like that. Be nice. That's all. Thank you very much. Are there any questions? I see there is a first question. Unfortunately, I can't read it. It's probably the wrong encoding. So while you can type in questions, I do actually have a question. At the moment, I'm always using UTF8 for anything, everything actually. And is there a reason not to use UTF8 and when and why? If you control both ends of the communication or you have some power to do that, use UTF8. Of course, there are cases where you can use it if you are programming some special printer or something that doesn't support it. But if you control both ends of communication, use UTF8. OK. Any more questions? You can click this Q&A button down on your screen if you are in Zoom and ask your questions there. Am I in Zoom? Yes, there is a question coming in. No, don't worry. I'll read the questions for you. OK. OK. Johnny Zhang is asking. Johnny Zhang is asking. Johnny Zhang is asking. Johnny Zhang is asking. Johnny Zhang is asking. Some auto-Chinese software is still using Big Five. So what are your thoughts about that? Revrite it if you can. Revrite it, OK. Why that? What's the major drawback of Big Five? I've never used that. But sometimes it would be nice to have one encoding that works everywhere. Because you have also combined documents. In my case, I have also used some Chinese characters within European and Sierra League and so on. So it's nice to have, if you want to have bootlabel documents, anything, anywhere you need something that works for everything. OK. So are there more questions? You can always ask Miro later in Discord. Just press Command and K in Discord. And then you see Window and you can enter Invalid there. And then you will see the channel talk. Your name is Invalid. And there, oh, you get already got big applause there. Really great talk, poignant and entertaining. While other people are maybe asking more questions. Last chance. If not, please go. Oh, even more applause in the channel. So if not, please, Miro, go to the channel. Your name is Invalid. And maybe there are more questions there. And in the meantime, we maybe have time for another advertisement. OK. Bye. Thank you. Thank you again, Miro. Oh, that's the first second one.
 Welcome to the next afternoon session.  Our next speaker is Miroslav Shadivi.  He will talk about your name is Invalid.  Names of people cannot be Invalid.  Your app shouldn't.  He will mention many things I guess about these wrong names.  So, okay.  Miro, please start.  Thank you.  Hi, let me introduce myself.  My name is Miroslav.  We have still some problems with the transmission.  Wait, wait, wait.  Hey, we couldn't prepare it beforehand.  I will have to switch something here.  No, no, that's not still.  Ah, no.  Okay.  My name is Miroslav Shadivi.  And sometimes I still see somewhere that my name is Invalid.  If you want to pronounce it correctly, this is the IPA,  how to pronounce it, and this is how to type it  on a standard keyboard layout with Komposki.  I very get a salute to GmbH and Karlsruhe.  And if you would like to see the region with the best German wine  and the best French beer, come to see us to Karlsruhe.  We do a lot of great stuff with Python  and other open source technology.  Originally, I was born in Czechoslovakia,  studied in France, and now I'm a German.  What we are going to speak about today,  first about strings and names in Python,  how the strings bytes work, encoding, normalizing,  case folding, sorting, and regular expressions,  and then about names on the web, about the apps,  about databases, how to work with the names  correctly, and how to allow or test  whether a name is valid or not.  So in Python 3, there are two types of objects  that work with strings.  The first is standard string that, for each character,  knows more than 1 million possible characters.  That's Unicode, and that is something  that is in the working memory.  So you work with it, but you don't send it over network  or store it into a file.  And there are bytes with 256 possible combinations  that are used to store data on file and or network.  And you have to combine or you have  to convert between these somehow.  So for example, if your string is or name is  Chagnaris and you just encode it,  you get a series of the bytes that looks almost the same.  And the other way around from bytes, back to string,  you convert it with using the decode method.  The bytes, you can see them with, you can recognize them,  but they have a small b before this quote.  And the strings, they have nothing.  If you encode them standardly in Python,  it uses UTF-8 as a standard encoding.  But in this case, it will be 12 characters as a string  or 12 bytes as bytes.  So there is no difference.  But for example, if you have the German name Mühler  and you encode it, then from six characters,  you will get seven bytes because the letter U, umlaut,  is will be written or stored as two bytes.  Yeah, if you have Chinese name, this is not a name.  This is Mihao.  So a simple hello in Chinese from two Chinese characters.  In UTF-8, it will be stored as six bytes.  There are other encodings.  UTF-8 is relatively the newest one,  although it is over two decades old.  There is ASCII, but it's around since many,  many dozens of years.  If your name is Tregnorys, of course,  you can be encoded in any encoding.  And actually, you are the one who decides about the encoding.  But in this case, Tregnorys with 12 characters  will be encoded into 12 bytes using ASCII.  But if you have the name Mühler, you cannot do it with ASCII.  You can do it, for example, with Latin 1  that knows more characters, that knows even this UTF-8.  And in this case, the six characters  will be encoded as six bytes using Latin 1.  My last name, Shaivi, comes from Czechoslovakia,  which doesn't use Latin 1.  It uses Latin 2 alphabet or encoding.  And our six characters will be encoded as six bytes.  But always a byte is something that has only 256 possibilities.  So you don't have the information which encoding it is in.  In this case, we knew that we are going to use Latin 1  or Latin 2.  But normally, you don't know which encoding it is in.  And sometimes, it can happen something like this,  that I get a mail here in Germany  that types my last name wrongly.  How is it possible that instead of the s with Karen,  I get a question mark?  Would they use Python?  So if I take my last name and encode it with Latin 2,  I will get the correct six bytes that mean the rising.  But there is no question mark.  If I do it with Latin 1, I will get a unique error  because the hash cannot be encoded in Latin 1.  There is in those 256 possible bytes of Latin 1,  the letter h is not there.  But in Python, you can define an error.  What to do in the case of error?  Default is raised an exception.  But it is possible to say replace it with something.  And replace default is a question mark.  So if you encode some string that contains characters  that are outside of the encoding, so for example,  sure, in the case of Latin 1.  And you say, in the case of error, replace it,  it will automatically replace it with a question mark.  And this is probably how this happened.  You can even do something more.  You can define your own function if you don't want  question mark the whole time.  And you can register an error and encode it.  And then tell, OK, if I use the errors method,  replace randomly, for example, call this lambda function that  gives a random digit.  And this x start plus plus 1 means that then go on  with the next character.  You can define more movements around such cases.  So in this case, I can tell if I want to encode my last name  with Latin 1.  And if a character cannot be encoded,  I then put there some random digit.  In this case, number 5, for example.  And this is probably how they are doing it,  random characters.  Sometimes there is a space.  Sometimes there is no space.  Sometimes there is something like this.  Sometimes there is an inverted question mark.  Actually, all three pictures come from the same company  that has different means of communication,  like letter, like card, customer card, or online form.  Although I have entered it correctly in the web form  when I signed up for their service.  There are some other cases.  Here, some non-name German flying company  that has huge airplanes but have problem with their form.  They tell me, hey, you can only enter letters  in the adults last name field.  What are letters?  My last name consists only of letters.  What is a letter?  In Python 3, you can do something like this.  You can give variable the name that is not pure ASCII,  that consists of other characters.  But you cannot give variable a name consisting  of starting with a number.  But sure, it's not a number.  It's a character.  It's a letter.  How does Python know that sure is a letter  but a smiley is not a letter?  Question mark is not a letter.  There is a unique of data that has the standard  module, unique of data, that has the method category and name.  So we give it any character and it  will tell you which category and name it is.  So in this case, I gave a few numbers and smiley and dot  and so on.  There you see all these characters.  And then you see two letters.  There is LL, LU, ZS, POSO, and so on.  And then there is the official name of the character.  So everything that these two characters,  when they start with an L, it means a letter.  LL is a lowercase letter.  LU is uppercase letter.  And so it is defined.  Every character in the mini code is defined.  It belongs to some category and it is a letter or not.  So you can use the capital letter sharp as a variable name  in Python.  These are all the categories that we  are going to see a little bit later how we can use them.  Also, if you open the character map application,  you will see all the information about the character.  And you can use actually the name to write the character  with the capital N and with the character.  Case folding.  Case folding means the conversion  between uppercase and lowercase.  So you see we have some letters in the lowercase.  We got uppercase and vice versa.  It has a few exceptions.  So for example, the German sharp as lowercase  converts to S as uppercase, although there  is official sharp as uppercase character.  The other way around, it converts correctly.  So it works for most other characters  between uppercase and lowercase.  Of course, you have, for example, Chinese characters  that support something like that.  But there is also with this uppercase and lowercase,  there is one problem even in the basic ASCII.  There is one ASCII pair, over case uppercase,  that we, like the Western English, European language  group, we've wrong.  Which one is it?  It's I. You see, if I take lowercase I,  I do it to uppercase.  I get this.  The other way around as well, from the uppercase I,  I get the lowercase I. You see the difference?  The lowercase I has a dot.  The uppercase I doesn't have a dot.  How do you explain that to someone from Turkey?  Because they have the I with the lowercase uppercase that  is E and I without dot, a lowercase uppercase that is  I. And when they convert between these two pairs  of characters, they always get the other one.  So if you want to work with Turkish text,  you have to import international components  for a unique code library and then work a little bit  more complicated codes that converts  between the Turkish pairs of I and I correctly.  Normalization is something different.  If you have already your unique code text  and you think, OK, that's great.  I can now work with that.  I can access the characters, the length,  and everything.  You will get probably two strings that look the same  and they are not the same.  Because when they are normalized, they  will be divided or some characters will be split  in two characters.  So for example, the first word has three characters.  The second one is U with dots like the resist mode.  And the second version has this the resist behind the U.  So the first U is standard U with value of 117.  And the second one then comes this combining  the resist that is of the category  MN, which means a combining character.  And also, this is not an error, but you  see that the whole line is a little bit offset to the left.  This means that this combining the resist doesn't have a width.  And if you can convert between those two versions,  like the normalized and the denormalized,  and between the two normal forms,  and they will not be the same.  So it means that if you have a lot of text,  try to normalize them or denormalize them to one form  and only then compare them.  With this combining the resist and there  are plenty of combining characters,  you can do some fun stuff.  You can put more characters after each other.  And then you can get something like this.  This is actually a mistake overflow answer  to the question how to use, how to parse HTML  with the Red War expression.  So you see at the end, there are some characters  with plenty, plenty, plenty of combining characters.  Alphabetic sorting, that's not so easy  if you just sort the standard Python sort  using the standard Python sort, you will not  get the alphabetical order because you see that there  are some uppercase, lowercase, then you  have some other letters.  What's the new, like, letters, then there  is some letters, and then the shuffles  comes at the end because it has a really high unique value.  You can use locale.  So you start locale with the German, for example,  and then you sort the characters and you will get it out  of the out between A and B. But if your locale is,  for example, Swedish, then it will be different  because in Swedish all the umlauts are A, B, N.  And there are some other exceptions, for example,  in Hungarian.  The CS pronounced as ch is extra letter between C and D.  In the Czech and Slovak, we have plenty of characters  with carons that we put just behind the respective letter.  There is CH pronounced as ch, which is between H and I.  And for example, in French, they have even more rules.  It means that they sort everything according to ASCII.  But if there are exceptions, they sort them  from the end of the word.  So first they sort the last syllable  and then the penultimate and then and so on.  So there are plenty of rules.  The problem with locale is if you do that in your library,  then the locale of the whole process will be changed.  This means that if you have some multilingual program that  has to have the input of several users at the same time,  such as WebSaver, and you switch the locale the whole time,  it will, there will be a chaos.  So that's why there is this ICU that we have used already  to convert the lowercase uppercase  to the other case that you can use to sort something  without changing the system.  I have read that there are some recommendations  to use by use EA, although some unique code  collocation algorithm, collator algorithm.  The difference between these two is that by use EA  doesn't define which language it uses.  It has a great database included.  And it's announced that it works very well for characters  outside of English.  Yes, you can use it if you only need some English list  sorted list, and you want to have those other characters  included at not so bad position.  But don't use this if you have pure German, Swedish,  or Czech text, because it doesn't use  the specific sorting of the specific language.  For that case, I would prefer to use ICU.  At the end, unique code regular expressions.  So let's have a look at this text.  There is a new space 1, 2, 3.  And I want to extract the word minkin.  If I do this, so all characters between A, Z, uppercase,  lowercase, it will ignore this.  So umlaut and return to two words like that.  It's wrong.  If I say I want a backslash w, so all alphanumeric characters,  there are also numeric characters.  So I will return to return also the 1, 2, 3 that I want to have.  And in that case, we will have to use the reggis.  It is a third party library that works 100% like the standard.  RE works and only has a few functions extra.  And in this case, with backslash p,  you can define unique code characters that  belong to some category.  And you have seen at the beginning, this capital L  means that it is a letter.  And you can also define something like L U for uppercase letters,  L L for lowercase letters, and so on.  You have seen that at the beginning in the list of categories.  And in that case, we get the word.  I came here for Python, but I say for the names.  So this was the Python part.  Now let's have a look at the names.  If I were with you in the normal offline conference,  I would just ask, raise your hand,  who can write their first and last name like this?  I don't know how many of you can write it  later in the discussion.  But probably, OK, I can tell my first name,  it'll slow my last name.  But maybe some of you have something like the middle name  or something Petronomic or Metronomic  surname in Portuguese or Swedish, sorry, in Spanish.  There is last name, first name in many Asian languages  or in Hungarian.  There is name number if we have some popes or queens or things  here.  Or there is some simple name.  But there is also something like the name of the father  and the daughter of the father.  So for example, in Icelandic, if your name is Sigur Johansson,  your father was Johan, it means that you have to use or work  with these names a little bit differently  because Sigur Johansson doesn't mean  that it's a mystery Johansson.  There is no mystery Johansson.  Sigur Johansson is Sigur or Mr. Sigur Johansson.  And you will find them in the phone list, the phone book,  you will find them as S, like Sigur Johansson and not as J,  because they don't sort their names according to the Petronomic  surname or the name of their family.  Of course, there are plenty of phononsu and funder,  dedede, and so on in different languages.  And many times you just don't know  whether to put it together with the last name or first name  or somewhere in the beginning and how to sort them.  Yeah, of course, some doctors in some languages,  the doctor, became the part of your last name.  So how to do it correctly?  Actually, if you ask for a name, just ask for the full name.  Let someone write the full name, and then you  can ask how they should call you.  So for example, Mr. T, he says, oh, my first name is Mr.  My middle name is dot or period, and my last name is T.  And then even if you can write it correctly,  you will sometimes get an error message,  like please enter characters from the European character set  only.  This happened in Germany.  My last name, Shaggy, is not far away from Germany.  Sorry, what's European character set?  And the same service a few pages later,  please enter a full valid name.  They didn't want to accept my name.  And my name is valid.  So the names of people cannot be valid.  So if you work with names in some forms,  don't assume anything.  Don't put random limit on their length.  Minimum, maximum, there is no minimum.  There is no maximum.  So for example, Pipi, lunchroom, logstocking,  her name is like this.  Or a character or got that a dolphin,  which Gutenberg, this was some German minister a few years  ago, his name even appeared on Wikipedia with a typo,  and even he didn't see that Twitter as a typo.  Don't use stop words.  So if it's a stop word or like root word in your language,  it can be a perfect word in another language.  The all family members don't have necessarily  the same name.  So for example, all the male members of my family  are Shaggy, or the female members of my family are Shaggy.  Don't assume anything, different transcription  from non-letton alphabet.  So for example, if you have the Russian name Chekhov,  in every European languages, types it a little bit differently.  And other way around, so for example,  I was in Russia twice with the same passport,  and I got to visa.  And most times, my last name was typed differently  in the serial alphabet.  Or in Chinese, the same.  If you use something like made the name or name,  know that then they change their family names too.  One letter name is probably not an initial,  so binwa bi lander, mother brother bi is not an abbreviation.  So actually, you should accept almost all printable characters.  Hello, my name is Mr. Nol.  My name makes me invisible to computers and articles.  I don't understand how someone can program something  that doesn't accept the last name.  Or the same with this little Bobby Teigos.  Please don't do that.  Yeah, the names of the streets and cities and so on,  the most widespread name of street in Germany is, of course,  Einbahnstraße.  So one-way street, you see it almost everywhere now.  How in German, this Straße, like street, is with the sharp S,  you will find a lot of American lists of German companies  with this B capital and instead of the sharp S.  So the names of the places, all somewhere in Scandinavia,  the Zipstil Union, so the Y in somewhere in France.  And then the hint question is, what is your mother's name?  The main, at least six characters,  or they ask you for your place of birth, at least six characters.  No, it doesn't work like that.  Or on the other hand, you have places like Hanweil,  and the other places that are a little bit longer.  Or, sometimes you even don't take a mattress,  because you just described where you want to take your mother.  There is a long list of falsehoods,  programmers believe about names, I'm not going to read everything.  So if you are working with the names of people,  I now ignore GDPR, everything.  I programmatically respect their names.  And don't break the locale, so use the ICU.  Convert hamburger principle, convert from bytes  to string as soon as possible, and convert back  to bytes as late as possible.  UTFAB school, Python 3D school, because 2 use Python 3D  and UTF8.  If you tell your user because of your spitefulness,  ignorance, laziness, or something,  their name is invalid, this is not cool.  There is a Twitter account, your name is valid.  It's not invalid, it is, your name is valid,  because there is a limit on the characters at Twitter  that will maybe, your case can appear if you do something like that.  Be nice.  That's all.  Thank you very much.  Are there any questions?  I see there is a first question.  Unfortunately, I can't read it.  It's probably the wrong encoding.  So while you can type in questions,  I do actually have a question.  At the moment, I'm always using UTF8 for anything,  everything actually.  And is there a reason not to use UTF8 and when and why?  If you control both ends of the communication  or you have some power to do that, use UTF8.  Of course, there are cases where you can use it  if you are programming some special printer or something  that doesn't support it.  But if you control both ends of communication, use UTF8.  OK.  Any more questions?  You can click this Q&A button down on your screen  if you are in Zoom and ask your questions there.  Am I in Zoom?  Yes, there is a question coming in.  No, don't worry.  I'll read the questions for you.  OK.  OK.  Johnny Zhang is asking.  Johnny Zhang is asking.  Johnny Zhang is asking.  Johnny Zhang is asking.  Johnny Zhang is asking.  Some auto-Chinese software is still using Big Five.  So what are your thoughts about that?  Revrite it if you can.  Revrite it, OK.  Why that?  What's the major drawback of Big Five?  I've never used that.  But sometimes it would be nice to have one encoding that works  everywhere.  Because you have also combined documents.  In my case, I have also used some Chinese characters within  European and Sierra League and so on.  So it's nice to have, if you want to have bootlabel documents,  anything, anywhere you need something that works for everything.  OK.  So are there more questions?  You can always ask Miro later in Discord.  Just press Command and K in Discord.  And then you see Window and you can enter Invalid there.  And then you will see the channel talk.  Your name is Invalid.  And there, oh, you get already got big applause there.  Really great talk, poignant and entertaining.  While other people are maybe asking more questions.  Last chance.  If not, please go.  Oh, even more applause in the channel.  So if not, please, Miro, go to the channel.  Your name is Invalid.  And maybe there are more questions there.  And in the meantime, we maybe have time for another advertisement.  OK.  Bye.  Thank you.  Thank you again, Miro.  Oh, that's the first second one.
Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_0_155.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_156_263.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_264_383.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_384_995.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_996_9287.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_9288_10487.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_10488_11351.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_11352_30347.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_30348_33011.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_33012_33263.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_33264_33779.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_33780_34283.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_34284_34451.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_34452_36323.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_36324_36443.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_36444_36671.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_36672_36851.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_36852_36899.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_36900_37163.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_37164_37307.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_37308_37391.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_37392_37511.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_37512_37619.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_37620_37991.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_37992_38039.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_38040_38087.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_38088_38807.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_38808_38891.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_38892_39431.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_39432_39839.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_39840_40319.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_40320_41231.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_41232_41279.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_41280_41423.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_41424_41471.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_41472_41519.png Miroslav_Sedivy_-_Your_Name_Is_Invalid__IuZBTjaphsY_41520_41622.png
Welcome to the next afternoon session. Our next speaker is Miroslav Shadivi. He will talk about your name is Invalid. Names of people cannot be Invalid. Your app shouldn't. He will mention many things I guess about these wrong names. So, okay. Miro, please start. Thank you. Hi, let me introduce myself. My name is Miroslav. We have still some problems with the transmission. Wait, wait, wait. Hey, we couldn't prepare it beforehand. I will have to switch something here. No, no, that's not still. Ah, no. Okay. My name is Miroslav Shadivi. And sometimes I still see somewhere that my name is Invalid. If you want to pronounce it correctly, this is the IPA, how to pronounce it, and this is how to type it on a standard keyboard layout with Komposki. I very get a salute to GmbH and Karlsruhe. And if you would like to see the region with the best German wine and the best French beer, come to see us to Karlsruhe. We do a lot of great stuff with Python and other open source technology. Originally, I was born in Czechoslovakia, studied in France, and now I'm a German. What we are going to speak about today, first about strings and names in Python, how the strings bytes work, encoding, normalizing, case folding, sorting, and regular expressions, and then about names on the web, about the apps, about databases, how to work with the names correctly, and how to allow or test whether a name is valid or not. So in Python 3, there are two types of objects that work with strings. The first is standard string that, for each character, knows more than 1 million possible characters. That's Unicode, and that is something that is in the working memory. So you work with it, but you don't send it over network or store it into a file. And there are bytes with 256 possible combinations that are used to store data on file and or network. And you have to combine or you have to convert between these somehow. So for example, if your string is or name is Chagnaris and you just encode it, you get a series of the bytes that looks almost the same. And the other way around from bytes, back to string, you convert it with using the decode method. The bytes, you can see them with, you can recognize them, but they have a small b before this quote. And the strings, they have nothing. If you encode them standardly in Python, it uses UTF-8 as a standard encoding. But in this case, it will be 12 characters as a string or 12 bytes as bytes. So there is no difference. But for example, if you have the German name Mühler and you encode it, then from six characters, you will get seven bytes because the letter U, umlaut, is will be written or stored as two bytes. Yeah, if you have Chinese name, this is not a name. This is Mihao. So a simple hello in Chinese from two Chinese characters. In UTF-8, it will be stored as six bytes. There are other encodings. UTF-8 is relatively the newest one, although it is over two decades old. There is ASCII, but it's around since many, many dozens of years. If your name is Tregnorys, of course, you can be encoded in any encoding. And actually, you are the one who decides about the encoding. But in this case, Tregnorys with 12 characters will be encoded into 12 bytes using ASCII. But if you have the name Mühler, you cannot do it with ASCII. You can do it, for example, with Latin 1 that knows more characters, that knows even this UTF-8. And in this case, the six characters will be encoded as six bytes using Latin 1. My last name, Shaivi, comes from Czechoslovakia, which doesn't use Latin 1. It uses Latin 2 alphabet or encoding. And our six characters will be encoded as six bytes. But always a byte is something that has only 256 possibilities. So you don't have the information which encoding it is in. In this case, we knew that we are going to use Latin 1 or Latin 2. But normally, you don't know which encoding it is in. And sometimes, it can happen something like this, that I get a mail here in Germany that types my last name wrongly. How is it possible that instead of the s with Karen, I get a question mark? Would they use Python? So if I take my last name and encode it with Latin 2, I will get the correct six bytes that mean the rising. But there is no question mark. If I do it with Latin 1, I will get a unique error because the hash cannot be encoded in Latin 1. There is in those 256 possible bytes of Latin 1, the letter h is not there. But in Python, you can define an error. What to do in the case of error? Default is raised an exception. But it is possible to say replace it with something. And replace default is a question mark. So if you encode some string that contains characters that are outside of the encoding, so for example, sure, in the case of Latin 1. And you say, in the case of error, replace it, it will automatically replace it with a question mark. And this is probably how this happened. You can even do something more. You can define your own function if you don't want question mark the whole time. And you can register an error and encode it. And then tell, OK, if I use the errors method, replace randomly, for example, call this lambda function that gives a random digit. And this x start plus plus 1 means that then go on with the next character. You can define more movements around such cases. So in this case, I can tell if I want to encode my last name with Latin 1. And if a character cannot be encoded, I then put there some random digit. In this case, number 5, for example. And this is probably how they are doing it, random characters. Sometimes there is a space. Sometimes there is no space. Sometimes there is something like this. Sometimes there is an inverted question mark. Actually, all three pictures come from the same company that has different means of communication, like letter, like card, customer card, or online form. Although I have entered it correctly in the web form when I signed up for their service. There are some other cases. Here, some non-name German flying company that has huge airplanes but have problem with their form. They tell me, hey, you can only enter letters in the adults last name field. What are letters? My last name consists only of letters. What is a letter? In Python 3, you can do something like this. You can give variable the name that is not pure ASCII, that consists of other characters. But you cannot give variable a name consisting of starting with a number. But sure, it's not a number. It's a character. It's a letter. How does Python know that sure is a letter but a smiley is not a letter? Question mark is not a letter. There is a unique of data that has the standard module, unique of data, that has the method category and name. So we give it any character and it will tell you which category and name it is. So in this case, I gave a few numbers and smiley and dot and so on. There you see all these characters. And then you see two letters. There is LL, LU, ZS, POSO, and so on. And then there is the official name of the character. So everything that these two characters, when they start with an L, it means a letter. LL is a lowercase letter. LU is uppercase letter. And so it is defined. Every character in the mini code is defined. It belongs to some category and it is a letter or not. So you can use the capital letter sharp as a variable name in Python. These are all the categories that we are going to see a little bit later how we can use them. Also, if you open the character map application, you will see all the information about the character. And you can use actually the name to write the character with the capital N and with the character. Case folding. Case folding means the conversion between uppercase and lowercase. So you see we have some letters in the lowercase. We got uppercase and vice versa. It has a few exceptions. So for example, the German sharp as lowercase converts to S as uppercase, although there is official sharp as uppercase character. The other way around, it converts correctly. So it works for most other characters between uppercase and lowercase. Of course, you have, for example, Chinese characters that support something like that. But there is also with this uppercase and lowercase, there is one problem even in the basic ASCII. There is one ASCII pair, over case uppercase, that we, like the Western English, European language group, we've wrong. Which one is it? It's I. You see, if I take lowercase I, I do it to uppercase. I get this. The other way around as well, from the uppercase I, I get the lowercase I. You see the difference? The lowercase I has a dot. The uppercase I doesn't have a dot. How do you explain that to someone from Turkey? Because they have the I with the lowercase uppercase that is E and I without dot, a lowercase uppercase that is I. And when they convert between these two pairs of characters, they always get the other one. So if you want to work with Turkish text, you have to import international components for a unique code library and then work a little bit more complicated codes that converts between the Turkish pairs of I and I correctly. Normalization is something different. If you have already your unique code text and you think, OK, that's great. I can now work with that. I can access the characters, the length, and everything. You will get probably two strings that look the same and they are not the same. Because when they are normalized, they will be divided or some characters will be split in two characters. So for example, the first word has three characters. The second one is U with dots like the resist mode. And the second version has this the resist behind the U. So the first U is standard U with value of 117. And the second one then comes this combining the resist that is of the category MN, which means a combining character. And also, this is not an error, but you see that the whole line is a little bit offset to the left. This means that this combining the resist doesn't have a width. And if you can convert between those two versions, like the normalized and the denormalized, and between the two normal forms, and they will not be the same. So it means that if you have a lot of text, try to normalize them or denormalize them to one form and only then compare them. With this combining the resist and there are plenty of combining characters, you can do some fun stuff. You can put more characters after each other. And then you can get something like this. This is actually a mistake overflow answer to the question how to use, how to parse HTML with the Red War expression. So you see at the end, there are some characters with plenty, plenty, plenty of combining characters. Alphabetic sorting, that's not so easy if you just sort the standard Python sort using the standard Python sort, you will not get the alphabetical order because you see that there are some uppercase, lowercase, then you have some other letters. What's the new, like, letters, then there is some letters, and then the shuffles comes at the end because it has a really high unique value. You can use locale. So you start locale with the German, for example, and then you sort the characters and you will get it out of the out between A and B. But if your locale is, for example, Swedish, then it will be different because in Swedish all the umlauts are A, B, N. And there are some other exceptions, for example, in Hungarian. The CS pronounced as ch is extra letter between C and D. In the Czech and Slovak, we have plenty of characters with carons that we put just behind the respective letter. There is CH pronounced as ch, which is between H and I. And for example, in French, they have even more rules. It means that they sort everything according to ASCII. But if there are exceptions, they sort them from the end of the word. So first they sort the last syllable and then the penultimate and then and so on. So there are plenty of rules. The problem with locale is if you do that in your library, then the locale of the whole process will be changed. This means that if you have some multilingual program that has to have the input of several users at the same time, such as WebSaver, and you switch the locale the whole time, it will, there will be a chaos. So that's why there is this ICU that we have used already to convert the lowercase uppercase to the other case that you can use to sort something without changing the system. I have read that there are some recommendations to use by use EA, although some unique code collocation algorithm, collator algorithm. The difference between these two is that by use EA doesn't define which language it uses. It has a great database included. And it's announced that it works very well for characters outside of English. Yes, you can use it if you only need some English list sorted list, and you want to have those other characters included at not so bad position. But don't use this if you have pure German, Swedish, or Czech text, because it doesn't use the specific sorting of the specific language. For that case, I would prefer to use ICU. At the end, unique code regular expressions. So let's have a look at this text. There is a new space 1, 2, 3. And I want to extract the word minkin. If I do this, so all characters between A, Z, uppercase, lowercase, it will ignore this. So umlaut and return to two words like that. It's wrong. If I say I want a backslash w, so all alphanumeric characters, there are also numeric characters. So I will return to return also the 1, 2, 3 that I want to have. And in that case, we will have to use the reggis. It is a third party library that works 100% like the standard. RE works and only has a few functions extra. And in this case, with backslash p, you can define unique code characters that belong to some category. And you have seen at the beginning, this capital L means that it is a letter. And you can also define something like L U for uppercase letters, L L for lowercase letters, and so on. You have seen that at the beginning in the list of categories. And in that case, we get the word. I came here for Python, but I say for the names. So this was the Python part. Now let's have a look at the names. If I were with you in the normal offline conference, I would just ask, raise your hand, who can write their first and last name like this? I don't know how many of you can write it later in the discussion. But probably, OK, I can tell my first name, it'll slow my last name. But maybe some of you have something like the middle name or something Petronomic or Metronomic surname in Portuguese or Swedish, sorry, in Spanish. There is last name, first name in many Asian languages or in Hungarian. There is name number if we have some popes or queens or things here. Or there is some simple name. But there is also something like the name of the father and the daughter of the father. So for example, in Icelandic, if your name is Sigur Johansson, your father was Johan, it means that you have to use or work with these names a little bit differently because Sigur Johansson doesn't mean that it's a mystery Johansson. There is no mystery Johansson. Sigur Johansson is Sigur or Mr. Sigur Johansson. And you will find them in the phone list, the phone book, you will find them as S, like Sigur Johansson and not as J, because they don't sort their names according to the Petronomic surname or the name of their family. Of course, there are plenty of phononsu and funder, dedede, and so on in different languages. And many times you just don't know whether to put it together with the last name or first name or somewhere in the beginning and how to sort them. Yeah, of course, some doctors in some languages, the doctor, became the part of your last name. So how to do it correctly? Actually, if you ask for a name, just ask for the full name. Let someone write the full name, and then you can ask how they should call you. So for example, Mr. T, he says, oh, my first name is Mr. My middle name is dot or period, and my last name is T. And then even if you can write it correctly, you will sometimes get an error message, like please enter characters from the European character set only. This happened in Germany. My last name, Shaggy, is not far away from Germany. Sorry, what's European character set? And the same service a few pages later, please enter a full valid name. They didn't want to accept my name. And my name is valid. So the names of people cannot be valid. So if you work with names in some forms, don't assume anything. Don't put random limit on their length. Minimum, maximum, there is no minimum. There is no maximum. So for example, Pipi, lunchroom, logstocking, her name is like this. Or a character or got that a dolphin, which Gutenberg, this was some German minister a few years ago, his name even appeared on Wikipedia with a typo, and even he didn't see that Twitter as a typo. Don't use stop words. So if it's a stop word or like root word in your language, it can be a perfect word in another language. The all family members don't have necessarily the same name. So for example, all the male members of my family are Shaggy, or the female members of my family are Shaggy. Don't assume anything, different transcription from non-letton alphabet. So for example, if you have the Russian name Chekhov, in every European languages, types it a little bit differently. And other way around, so for example, I was in Russia twice with the same passport, and I got to visa. And most times, my last name was typed differently in the serial alphabet. Or in Chinese, the same. If you use something like made the name or name, know that then they change their family names too. One letter name is probably not an initial, so binwa bi lander, mother brother bi is not an abbreviation. So actually, you should accept almost all printable characters. Hello, my name is Mr. Nol. My name makes me invisible to computers and articles. I don't understand how someone can program something that doesn't accept the last name. Or the same with this little Bobby Teigos. Please don't do that. Yeah, the names of the streets and cities and so on, the most widespread name of street in Germany is, of course, Einbahnstraße. So one-way street, you see it almost everywhere now. How in German, this Straße, like street, is with the sharp S, you will find a lot of American lists of German companies with this B capital and instead of the sharp S. So the names of the places, all somewhere in Scandinavia, the Zipstil Union, so the Y in somewhere in France. And then the hint question is, what is your mother's name? The main, at least six characters, or they ask you for your place of birth, at least six characters. No, it doesn't work like that. Or on the other hand, you have places like Hanweil, and the other places that are a little bit longer. Or, sometimes you even don't take a mattress, because you just described where you want to take your mother. There is a long list of falsehoods, programmers believe about names, I'm not going to read everything. So if you are working with the names of people, I now ignore GDPR, everything. I programmatically respect their names. And don't break the locale, so use the ICU. Convert hamburger principle, convert from bytes to string as soon as possible, and convert back to bytes as late as possible. UTFAB school, Python 3D school, because 2 use Python 3D and UTF8. If you tell your user because of your spitefulness, ignorance, laziness, or something, their name is invalid, this is not cool. There is a Twitter account, your name is valid. It's not invalid, it is, your name is valid, because there is a limit on the characters at Twitter that will maybe, your case can appear if you do something like that. Be nice. That's all. Thank you very much. Are there any questions? I see there is a first question. Unfortunately, I can't read it. It's probably the wrong encoding. So while you can type in questions, I do actually have a question. At the moment, I'm always using UTF8 for anything, everything actually. And is there a reason not to use UTF8 and when and why? If you control both ends of the communication or you have some power to do that, use UTF8. Of course, there are cases where you can use it if you are programming some special printer or something that doesn't support it. But if you control both ends of communication, use UTF8. OK. Any more questions? You can click this Q&A button down on your screen if you are in Zoom and ask your questions there. Am I in Zoom? Yes, there is a question coming in. No, don't worry. I'll read the questions for you. OK. OK. Johnny Zhang is asking. Johnny Zhang is asking. Johnny Zhang is asking. Johnny Zhang is asking. Johnny Zhang is asking. Some auto-Chinese software is still using Big Five. So what are your thoughts about that? Revrite it if you can. Revrite it, OK. Why that? What's the major drawback of Big Five? I've never used that. But sometimes it would be nice to have one encoding that works everywhere. Because you have also combined documents. In my case, I have also used some Chinese characters within European and Sierra League and so on. So it's nice to have, if you want to have bootlabel documents, anything, anywhere you need something that works for everything. OK. So are there more questions? You can always ask Miro later in Discord. Just press Command and K in Discord. And then you see Window and you can enter Invalid there. And then you will see the channel talk. Your name is Invalid. And there, oh, you get already got big applause there. Really great talk, poignant and entertaining. While other people are maybe asking more questions. Last chance. If not, please go. Oh, even more applause in the channel. So if not, please, Miro, go to the channel. Your name is Invalid. And maybe there are more questions there. And in the meantime, we maybe have time for another advertisement. OK. Bye. Thank you. Thank you again, Miro. Oh, that's the first second one.
expressions, and then about names on the web, you have Chinese name, this is not a name. This decides about the encoding. But in this case, define an error. What to do in the case of error? sharp as a variable name in Python. These are all uppercase and lowercase, there is one problem and they will not be the same. So it means that then there is some letters, and then the shuffles have those other characters included at not so doesn't use the specific sorting of the specific unique code characters that belong to some it means that you have to use or work with these last name or first name or somewhere in the They didn't want to accept my name. And my name So if you work with names in some forms, don't last name was typed differently in the serial you just described where you want to take your