Build and control a Python-powered robot.
Spadaro, Antonio
Antonio Spadaro - Build and control a Python-powered robot. During this talk you will see how to make a robot able to recognize people with a Raspberry Pi as main board and Python as language. The talk will cover the hardware and modules, discuss briefly the alternatives, and finally show a live demo. - The robot uses two main modules: - **OpenCV** (_Open Source Computer Vision Library_), an open-source library that includes several hundreds of computer vision algorithms. Usage ranges from interactive art, to mines inspection, stitching maps on the web or through advanced robotics. - **gpiozero**, a simple interface to everyday GPIO components used with Raspberry Pi. The first is used to recognize the people and the object; the second to control the robot.
 introduce you Antonio's Palaro and he's going to talk about how to micro, how to Python power a mobile robot. So please, big hand for Antonio. Okay, thank you. And this talk I will speak about how to build and control a Python powered robot. Begin with something about me. I will be a school students the next year. I'm an enter the first line Linux user and a Python programmer. I live in Italy and I live in Italy and I from Sicily. Okay, this robot is only a little small project because it's too easy to make. This robot, what do we do? This robot to stream the camera. It has a camera, a pi camera. Exactly. It don't hit against the wall. Thanks to ultrasonic sensor. It recognizes and watch humans. So one with the pi camera, where it recognizes a human, it move the pi camera and see the human. So what is the hardware? There is a Raspberry Pi 3. A pi camera for motor DC. Two motor drivers for control the motor DC. Two servo motors. An ultrasonic sensor and a power bank for power. Wait a moment. This is the robot. This is the rescued. This is a little scan of the hardware. At the center we found the Raspberry Pi 3. At the bottom we can find a pi camera and a servo motor. The pi camera send the data for a Raspberry Pi 3. Start to move the servo motor. The ultrasonic sensor, the Raspberry Pi read the ultrasonic sensor, the data and control the motor for don't hit against the wall. A PC controls all motors of the Raspberry Pi and a servo motor. The software is a socket server on the Raspberry Pi. A camera socket server on the Raspberry Pi. Two GUI for control the robot and for receive the frame of the camera. The socket server commands the motors, servo motors and the camera. Directly it controls the servo motors from the camera and the motors from the ultrasonic sensor. The communication protocol used is Wi-Fi. But why Wi-Fi? Why do not I use Bluetooth? Wi-Fi is more speed, it's more range of distance and we can connect more days. So we can send high quality images on low time. We can go to the robot. OpenCV. These are awesome libraries for recognizing the human. It is an acronym of open source computer vision. This library allows to use to manage the image from PC. But how it works? How it recognizes the object, the humans, the animals? First image will be sent from machine learning on OpenCV. Then it will try to find the model and return a predictive model. And it returns the position of the object if it found it. This is a little example of how to work. First we import OpenCV. Then we open the camera for the PC. We connect the camera. Then we load the RcacheCade, the predictive model with the XML file. On the right view we capture the frame. We make a gray scale of the image. We search some face on the frame. This face is returned on an empty array with the position of the face and the width and the height. Now we make a rectangle around it. Then we show the image with the rectangle and all. We found if we have pressed exit and it exit. This is the result. It recognizes the smile, people, the petonians and the drain wings. Because they use a simple algorithm that will try to recognize the similarity from ever this. This is the performance on our Zerryp3. We found a face on an image of 60,040 for 4080. We will be 0 with 52 seconds. On 30,020 for 20,040 image it will be 0,040 for 2080. 0,510,13 seconds. But we can optimize it. We can insert a simple line of code. On this line we specify the minimum size of the face and the maximum size of the face. This is the result of the performance. Instead of 0,520 seconds it will be 0,17 seconds. Instead of 0,15 seconds it will be 0,04 seconds. But we return back. Why I do not tell you that Arduino? Arduino does not support Python. It is more powerful to calculate. And as a Wi-Fi, the price is equal to plus or minus. But it is more small. Because if we take an Arduino, we insert a camera module or shield. It will be bigger than our Zerryp3. So I use a Zerryp3 instead of Arduino. For command robot, I use a socket server with a CIO protocol. This socket server allows a synchronous connection for multiple clients. We can control the hardware. For sender and the frame from Zerryp3 to client, I use another socket server. But the performance is a little problem. The resolution, the big resolution, has a big frame delay. For example, 64,080 image, we can find the video at 4fps. But the grayscale is more speed. If we want a fluid video, we should use a lower resolution with grayscale. Instead, if we want the quality with the colors, we should use a bigger resolution, but it will be slow. Because the color has three channels of color, instead the black and gray has only one color channel. We use a Zerryp3 when we start the robot to make an hotspot. The client can connect to the hotspot. Then we can control it. For Zerryp3, I use a CIO camera socket and a serial interconnection. It is more speed of an USB port. With the CIO server motor, we move the camera to the face. So it could be a very small speed parameter. Okay, this is the robot, a little robot, a little robot. So, at the moment, here we can see the Raspberry Pi. This is the power bank that powered the Raspberry Pi and the motors. And this is the ultrasound sensor. Okay, now it's powered on. So, for control later, use a joystick gamepad. Now, if you have a PC, you can see a new Wi-Fi hotspot called the robot. Okay, now I'm connected to the robot. Okay. We can be more the robot, the servo motors with the joystick. We can rotate the robot, and if we want... Wait a moment. We can control the robot with a simple joystick. We can go back. We can go back, forward, and rotate on itself. We can enable the camera. Now, I can see the output of the camera. Okay. Okay. Okay. This is the streaming. At the moment, it is at the minimum quality. Here, it is in white and blank. It is more speed but white and blank. We can set the more beautiful quality but it will be slower. And the width of the collar is more slow. Okay. Okay. Thank you. Anybody has questions?
 introduce you Antonio's Palaro and he's going to talk about how to  micro, how to Python power a mobile robot. So please, big hand for Antonio.  Okay, thank you. And this talk I will speak about how to build and control a Python powered robot.  Begin with something about me. I will be a school students the next year. I'm an  enter the first line Linux user and a Python programmer. I live in Italy and I live in Italy and I from Sicily.  Okay, this robot is only a little small project because it's too easy to make.  This robot, what do we do? This robot to stream the camera. It has a camera, a pi camera.  Exactly. It don't hit against the wall. Thanks to ultrasonic sensor. It recognizes and  watch humans. So one with the pi camera, where it recognizes a human, it move the  pi camera and see the human. So what is the hardware? There is a Raspberry Pi 3.  A pi camera for motor DC. Two motor drivers for control the motor DC. Two servo motors.  An ultrasonic sensor and a power bank for power.  Wait a moment. This is the robot. This is the rescued. This is a little scan of the hardware.  At the center we found the Raspberry Pi 3. At the bottom we can find a pi camera and a servo motor.  The pi camera send the data for a Raspberry Pi 3. Start to move the servo motor.  The ultrasonic sensor, the Raspberry Pi read the ultrasonic sensor, the data and control the motor for  don't hit against the wall. A PC controls all motors of the Raspberry Pi and a servo motor.  The software is a socket server on the Raspberry Pi. A camera socket server on the Raspberry Pi.  Two GUI for control the robot and for receive the frame of the camera.  The socket server commands the motors, servo motors and the camera.  Directly it controls the servo motors from the camera and the motors from the ultrasonic sensor.  The communication protocol used is Wi-Fi. But why Wi-Fi? Why do not I use Bluetooth?  Wi-Fi is more speed, it's more range of distance and we can connect more days.  So we can send high quality images on low time.  We can go to the robot. OpenCV. These are awesome libraries for recognizing the human.  It is an acronym of open source computer vision. This library allows to use to manage the image from PC.  But how it works? How it recognizes the object, the humans, the animals?  First image will be sent from machine learning on OpenCV.  Then it will try to find the model and return a predictive model.  And it returns the position of the object if it found it. This is a little example of how to work.  First we import OpenCV. Then we open the camera for the PC. We connect the camera.  Then we load the RcacheCade, the predictive model with the XML file.  On the right view we capture the frame. We make a gray scale of the image. We search some face on the frame.  This face is returned on an empty array with the position of the face and the width and the height.  Now we make a rectangle around it. Then we show the image with the rectangle and all.  We found if we have pressed exit and it exit. This is the result. It recognizes the smile, people,  the petonians and the drain wings. Because they use a simple algorithm that will try to recognize the similarity from ever this.  This is the performance on our Zerryp3. We found a face on an image of 60,040 for 4080.  We will be 0 with 52 seconds. On 30,020 for 20,040 image it will be 0,040 for 2080.  0,510,13 seconds. But we can optimize it. We can insert a simple line of code.  On this line we specify the minimum size of the face and the maximum size of the face.  This is the result of the performance. Instead of 0,520 seconds it will be 0,17 seconds.  Instead of 0,15 seconds it will be 0,04 seconds.  But we return back. Why I do not tell you that Arduino? Arduino does not support Python.  It is more powerful to calculate. And as a Wi-Fi, the price is equal to plus or minus.  But it is more small. Because if we take an Arduino, we insert a camera module or shield.  It will be bigger than our Zerryp3. So I use a Zerryp3 instead of Arduino.  For command robot, I use a socket server with a CIO protocol.  This socket server allows a synchronous connection for multiple clients.  We can control the hardware.  For sender and the frame from Zerryp3 to client, I use another socket server.  But the performance is a little problem.  The resolution, the big resolution, has a big frame delay.  For example, 64,080 image, we can find the video at 4fps. But the grayscale is more speed.  If we want a fluid video, we should use a lower resolution with grayscale.  Instead, if we want the quality with the colors, we should use a bigger resolution, but it will be slow.  Because the color has three channels of color, instead the black and gray has only one color channel.  We use a Zerryp3 when we start the robot to make an hotspot.  The client can connect to the hotspot.  Then we can control it.  For Zerryp3, I use a CIO camera socket and a serial interconnection.  It is more speed of an USB port.  With the CIO server motor, we move the camera to the face.  So it could be a very small speed parameter.  Okay, this is the robot, a little robot, a little robot.  So, at the moment, here we can see the Raspberry Pi.  This is the power bank that powered the Raspberry Pi and the motors.  And this is the ultrasound sensor.  Okay, now it's powered on.  So, for control later, use a joystick gamepad.  Now, if you have a PC, you can see a new Wi-Fi hotspot called the robot.  Okay, now I'm connected to the robot.  Okay.  We can be more the robot, the servo motors with the joystick.  We can rotate the robot, and if we want...  Wait a moment.  We can control the robot with a simple joystick.  We can go back.  We can go back, forward, and rotate on itself.  We can enable the camera.  Now, I can see the output of the camera.  Okay.  Okay.  Okay.  This is the streaming.  At the moment, it is at the minimum quality.  Here, it is in white and blank.  It is more speed but white and blank.  We can set the more beautiful quality but it will be slower.  And the width of the collar is more slow.  Okay.  Okay.  Thank you.  Anybody has questions?
Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__0_2159.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__2160_2291.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__2292_3443.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__3444_3971.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__3972_4223.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__4224_8363.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__8364_8975.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__8976_9083.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__9084_9371.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__9372_11315.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__11316_11927.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__11928_12035.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__12036_13487.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__13488_13883.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__13884_16019.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__16020_19391.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__19392_19643.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__19644_30251.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__30252_32747.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__32748_36599.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__36600_37787.png Antonio_Spadaro_-_Build_and_control_a_Python-powered_robot__37788_38099.png
introduce you Antonio's Palaro and he's going to talk about how to micro, how to Python power a mobile robot. So please, big hand for Antonio. Okay, thank you. And this talk I will speak about how to build and control a Python powered robot. Begin with something about me. I will be a school students the next year. I'm an enter the first line Linux user and a Python programmer. I live in Italy and I live in Italy and I from Sicily. Okay, this robot is only a little small project because it's too easy to make. This robot, what do we do? This robot to stream the camera. It has a camera, a pi camera. Exactly. It don't hit against the wall. Thanks to ultrasonic sensor. It recognizes and watch humans. So one with the pi camera, where it recognizes a human, it move the pi camera and see the human. So what is the hardware? There is a Raspberry Pi 3. A pi camera for motor DC. Two motor drivers for control the motor DC. Two servo motors. An ultrasonic sensor and a power bank for power. Wait a moment. This is the robot. This is the rescued. This is a little scan of the hardware. At the center we found the Raspberry Pi 3. At the bottom we can find a pi camera and a servo motor. The pi camera send the data for a Raspberry Pi 3. Start to move the servo motor. The ultrasonic sensor, the Raspberry Pi read the ultrasonic sensor, the data and control the motor for don't hit against the wall. A PC controls all motors of the Raspberry Pi and a servo motor. The software is a socket server on the Raspberry Pi. A camera socket server on the Raspberry Pi. Two GUI for control the robot and for receive the frame of the camera. The socket server commands the motors, servo motors and the camera. Directly it controls the servo motors from the camera and the motors from the ultrasonic sensor. The communication protocol used is Wi-Fi. But why Wi-Fi? Why do not I use Bluetooth? Wi-Fi is more speed, it's more range of distance and we can connect more days. So we can send high quality images on low time. We can go to the robot. OpenCV. These are awesome libraries for recognizing the human. It is an acronym of open source computer vision. This library allows to use to manage the image from PC. But how it works? How it recognizes the object, the humans, the animals? First image will be sent from machine learning on OpenCV. Then it will try to find the model and return a predictive model. And it returns the position of the object if it found it. This is a little example of how to work. First we import OpenCV. Then we open the camera for the PC. We connect the camera. Then we load the RcacheCade, the predictive model with the XML file. On the right view we capture the frame. We make a gray scale of the image. We search some face on the frame. This face is returned on an empty array with the position of the face and the width and the height. Now we make a rectangle around it. Then we show the image with the rectangle and all. We found if we have pressed exit and it exit. This is the result. It recognizes the smile, people, the petonians and the drain wings. Because they use a simple algorithm that will try to recognize the similarity from ever this. This is the performance on our Zerryp3. We found a face on an image of 60,040 for 4080. We will be 0 with 52 seconds. On 30,020 for 20,040 image it will be 0,040 for 2080. 0,510,13 seconds. But we can optimize it. We can insert a simple line of code. On this line we specify the minimum size of the face and the maximum size of the face. This is the result of the performance. Instead of 0,520 seconds it will be 0,17 seconds. Instead of 0,15 seconds it will be 0,04 seconds. But we return back. Why I do not tell you that Arduino? Arduino does not support Python. It is more powerful to calculate. And as a Wi-Fi, the price is equal to plus or minus. But it is more small. Because if we take an Arduino, we insert a camera module or shield. It will be bigger than our Zerryp3. So I use a Zerryp3 instead of Arduino. For command robot, I use a socket server with a CIO protocol. This socket server allows a synchronous connection for multiple clients. We can control the hardware. For sender and the frame from Zerryp3 to client, I use another socket server. But the performance is a little problem. The resolution, the big resolution, has a big frame delay. For example, 64,080 image, we can find the video at 4fps. But the grayscale is more speed. If we want a fluid video, we should use a lower resolution with grayscale. Instead, if we want the quality with the colors, we should use a bigger resolution, but it will be slow. Because the color has three channels of color, instead the black and gray has only one color channel. We use a Zerryp3 when we start the robot to make an hotspot. The client can connect to the hotspot. Then we can control it. For Zerryp3, I use a CIO camera socket and a serial interconnection. It is more speed of an USB port. With the CIO server motor, we move the camera to the face. So it could be a very small speed parameter. Okay, this is the robot, a little robot, a little robot. So, at the moment, here we can see the Raspberry Pi. This is the power bank that powered the Raspberry Pi and the motors. And this is the ultrasound sensor. Okay, now it's powered on. So, for control later, use a joystick gamepad. Now, if you have a PC, you can see a new Wi-Fi hotspot called the robot. Okay, now I'm connected to the robot. Okay. We can be more the robot, the servo motors with the joystick. We can rotate the robot, and if we want... Wait a moment. We can control the robot with a simple joystick. We can go back. We can go back, forward, and rotate on itself. We can enable the camera. Now, I can see the output of the camera. Okay. Okay. Okay. This is the streaming. At the moment, it is at the minimum quality. Here, it is in white and blank. It is more speed but white and blank. We can set the more beautiful quality but it will be slower. And the width of the collar is more slow. Okay. Okay. Thank you. Anybody has questions?
Okay, thank you. And this talk I will speak about how to build and control a Python powered robot. server on the Raspberry Pi. A camera socket camera. The socket server commands the motors, acronym of open source computer vision. This the object if it found it. This is a little use a simple algorithm that will try to recognize the hardware. For sender and the frame from use a Zerryp3 when we start the robot to make an control the robot with a simple joystick. We can