Docker meets Python - A look on the Docker SDK for Python
Wagner, Jan
My talk aims to introduce and have a closer look on the Docker SDK for Python. I will cover: - How and where to get the SDK - How it works and how to use it in general - Possible use-cases like: Processing Container-Logs, Testing with pytest on different Python Versions, Deploy via Python Script, etc.. For my talk, you should know what Docker is and how to use it. A basic idea of pytest and server administration is nice to have, but not necessarily needed to follow my talk.
 Hi. Yeah, thank you. I'm surprised, happy and scared that so many people are here actually. So this is my second conference talk. So in case I collapse and faint, please don't make pictures. Okay, be cool. So welcome to my talk. Dogger meets Python. A look on the Dogger SDK for Python. And what I'm going to talk about is briefly introduce myself. Then the Dogger Demon and Service, what that is. Then of course the Dogger SDK. So that's the topic. All right, so where you can get it and some ideas of possibly use cases. So hi, my name is Jan, 32 years old, not married, but engaged. No kids yet. So that's good for me at the moment. I'm a data science consultant at AxanTech. It's a small consultant company in Germany with different locations all around Germany. And of course, like every company here, we focus on data. So if you're interested in job wise or project wise, just get in touch with me. You can do that via email, social media or GitHub. I will put the slides on GitHub later, but they are already on the conference page. So if you want to look them up. All right. Dogger, who of you has used Dogger? Okay, that's good. So you basically know what it is and how it works. So after you installed Dogger on your system, on your local laptop, I'm assuming just a local laptop, the Demon or the service is reachable per default from within the system, so from local host, or you can configure it to be reachable from outside. And if you have the Demon, of course, you want to talk to it, kind of. And there are different ways to do that. So I separated the command line. You know, it just put in Dogger something, something, or you can use graphical user interfaces like KiteMedic or the Dogger extension for Visual Studio Code, which I personally like a lot. Or you can use Webguys like Portaino. But well, sometimes, depending what you want to do, these options are too big or too small. And maybe not customizable enough for you. So this is where the Dogger SDK comes into place. So just build your own. To get it, it's easy. It's officially available for Python and Golang. There's an awesome documentation available. So it's really just pip, pip3 install Dogger. And for Konda, there's also a package available. Some small code examples. If you go to the website, the code examples are written in Python 2. I don't know why. It is supported, but I wrote them to, yeah, you see it on the print statement in Python 3. On the left side, you see what you would usually type into the command line. And on the right side, you will see the equivalent if you would use a Python script and use the Dogger SDK. So it's really just import Dogger. Then you get, you use some name like client. That would be best practice. Dogger.fromNV without any further parameters. This means you want to connect to your local Dogger service. You can also connect to a remote one. And then you can pretty much do whatever you want. In this example, the print statement is only because Hello World usually, I guess, you would do the Hello World example from Dogger. Only a few. Okay. If you start this, this is just an image starting up a container printing on subtext, stopping the container, and that's it. So if you use the print statement here, you will see what was given out in the standard out. You can put an image. In this example, the Alpine image, it's pretty straightforward. Nevertheless, a hint, if you used to use the command line interface from Dogger, you usually type DoggerImage, DoggerContainer. DoggerImage is an exception that works on the command line. But if you use the SDK, it's always images and containers. And if you use a proper IDE, autocomplete will solve this problem. But if you just go into, I don't know, some easy text editor, this might be a trap. All right. So when I wrote the proposal, I was thinking about some ideas and use cases and some that I use. And one is, yeah, just to use Python as a container starting script. These are two examples more or less like the original on the Dogger Hub site for Postgres. And depending what you want to start, these commands get really, really long and complex. And if you want to change something in the middle, you have to use your back keys. You might save them in the text file and copy paste them to your command line. But that's somehow not so cool. So my first idea was, okay, Python comes batteries included, just import OS. And then with OS.system, you can give a command directly to your systems shell. But I want to show you the Dogger SDK. So this is maybe one example how it could work with the SDK. So you just declare your variables with strings or with a list of strings. And you get again the client object, the Dogger object, yeah, in the variable client. And then you can just put in to the parameters, all your variables. And that way you can easily list up all your parameters. Yeah, more easy, more better to read, maybe easier to change if you have to change them regularly, and I think sometimes this is quite useful. You could also extend it a little bit. So you could, if a container is running at the moment, you could go and get that container and stop it. You can prune it or delete it in that case if you haven't started with the command that it should delete automatically after you've stopped it. Then of course you can build a new image from a Dogger file that is saved somewhere. And you can get the idea and then start again a new container with a new image that you just built. Another idea was to get the logs. You can just stream the logs of one container and write your logs wherever you need them, maybe in a separate text file, maybe directly into some database that you set up or whatever comes to your mind and whatever is helpful for you. Of course you can do this with several containers and not just one. Then another idea that I had when I wrote the proposal was, okay, why not save all your Python files in one folder, use it as mounting point, then spin up different containers with different Python versions and mount your Python files there, and then run PyTest. I have to admit that when I started to prepare the talk and then started to work out on this use case, somewhere in the middle I was like, okay, it can work, but maybe it's not really useful. So in that sense, maybe just write a whole CI CD pipeline. If you're at that point, maybe that's really the better choice. And yeah, I'm noticing I'm way too fast, like really way too fast. This is already a wrap up, but I guess you're not so angry about it to get to the coffee break. So yeah, depending on your task at hand that you want to solve on your laptop, Python might be a better choice than, for example, a best script, especially if you want to maintain it and maybe keep it readable kind of. And the Docker SDK might sometimes be a better choice than import OS or equivalents. And yeah, at this point I hope that you at least have a high level idea about the Docker SDK and some inspiration there for your day to day work. And as a last hint, I can really recommend the Docker documentation. Also for the SDK or for Docker itself, it is usually up to date and really good and a lot of information. So now that I was so fast, we have a lot of time for questions. Thank you. Thank you very much for the fast talk. And as you said, we have now a lot of time for questions. So first comment, I have been doing talks for like 13 years and yesterday I was also early by 10 minutes. So yes, that is not a problem. You created a client by calling Docker.from.mv. What does it do exactly and what are the other options for creating clients? So what it does exactly, exactly, I can't answer. But it does basically create an object that is connected to your local, to your local service, Docker service. And of course you can give it parameters to connect some remote Docker service. You can use it with authentication, with HTTPS and so on and so forth. Thank you. Can you go to the microphone? Hi, thank you. How is your experience with Windows with this SDK? Okay. But is there any issue? Yeah, so if you install Docker on your Windows laptop, it works if you don't have like virtual box installed that does not work in parallel. You have to activate and deactivate what is it called Hyper-V. So that's one point. Another point is if you install Docker on a Windows laptop, you have to give the service explicit resources like memory, CPUs and so on. So it doesn't take that on need. And on the other side, all these resources are locked if you have the demon running without any usage. So it works, but on a Linux system, it works better. At the end is us working with a common line. Yeah. Okay. Thank you. Any further questions? Okay. Thank you very much again and have a good one.
 Hi.  Yeah, thank you.  I'm surprised, happy and scared that so many people are here actually.  So this is my second conference talk.  So in case I collapse and faint, please don't make pictures.  Okay, be cool.  So welcome to my talk.  Dogger meets Python.  A look on the Dogger SDK for Python.  And what I'm going to talk about is briefly introduce myself.  Then the Dogger Demon and Service, what that is.  Then of course the Dogger SDK.  So that's the topic.  All right, so where you can get it and some ideas of possibly use cases.  So hi, my name is Jan, 32 years old, not married, but engaged.  No kids yet.  So that's good for me at the moment.  I'm a data science consultant at AxanTech.  It's a small consultant company in Germany with different locations all around Germany.  And of course, like every company here, we focus on data.  So if you're interested in job wise or project wise, just get in touch with me.  You can do that via email, social media or GitHub.  I will put the slides on GitHub later, but they are already on the conference page.  So if you want to look them up.  All right.  Dogger, who of you has used Dogger?  Okay, that's good.  So you basically know what it is and how it works.  So after you installed Dogger on your system, on your local laptop, I'm assuming just a  local laptop, the Demon or the service is reachable per default from within the system,  so from local host, or you can configure it to be reachable from outside.  And if you have the Demon, of course, you want to talk to it, kind of.  And there are different ways to do that.  So I separated the command line.  You know, it just put in Dogger something, something, or you can use graphical user interfaces  like KiteMedic or the Dogger extension for Visual Studio Code, which I personally like  a lot.  Or you can use Webguys like Portaino.  But well, sometimes, depending what you want to do, these options are too big or too small.  And maybe not customizable enough for you.  So this is where the Dogger SDK comes into place.  So just build your own.  To get it, it's easy.  It's officially available for Python and Golang.  There's an awesome documentation available.  So it's really just pip, pip3 install Dogger.  And for Konda, there's also a package available.  Some small code examples.  If you go to the website, the code examples are written in Python 2.  I don't know why.  It is supported, but I wrote them to, yeah, you see it on the print statement in Python  3.  On the left side, you see what you would usually type into the command line.  And on the right side, you will see the equivalent if you would use a Python script and use the  Dogger SDK.  So it's really just import Dogger.  Then you get, you use some name like client.  That would be best practice.  Dogger.fromNV without any further parameters.  This means you want to connect to your local Dogger service.  You can also connect to a remote one.  And then you can pretty much do whatever you want.  In this example, the print statement is only because Hello World usually, I guess, you  would do the Hello World example from Dogger.  Only a few.  Okay.  If you start this, this is just an image starting up a container printing on subtext, stopping  the container, and that's it.  So if you use the print statement here, you will see what was given out in the standard  out.  You can put an image.  In this example, the Alpine image, it's pretty straightforward.  Nevertheless, a hint, if you used to use the command line interface from Dogger, you usually  type DoggerImage, DoggerContainer.  DoggerImage is an exception that works on the command line.  But if you use the SDK, it's always images and containers.  And if you use a proper IDE, autocomplete will solve this problem.  But if you just go into, I don't know, some easy text editor, this might be a trap.  All right.  So when I wrote the proposal, I was thinking about some ideas and use cases and some that  I use.  And one is, yeah, just to use Python as a container starting script.  These are two examples more or less like the original on the Dogger Hub site for Postgres.  And depending what you want to start, these commands get really, really long and complex.  And if you want to change something in the middle, you have to use your back keys.  You might save them in the text file and copy paste them to your command line.  But that's somehow not so cool.  So my first idea was, okay, Python comes batteries included, just import OS.  And then with OS.system, you can give a command directly to your systems shell.  But I want to show you the Dogger SDK.  So this is maybe one example how it could work with the SDK.  So you just declare your variables with strings or with a list of strings.  And you get again the client object, the Dogger object, yeah, in the variable client.  And then you can just put in to the parameters, all your variables.  And that way you can easily list up all your parameters.  Yeah, more easy, more better to read, maybe easier to change if you have to change them  regularly, and I think sometimes this is quite useful.  You could also extend it a little bit.  So you could, if a container is running at the moment, you could go and get that container  and stop it.  You can prune it or delete it in that case if you haven't started with the command that  it should delete automatically after you've stopped it.  Then of course you can build a new image from a Dogger file that is saved somewhere.  And you can get the idea and then start again a new container with a new image that you  just built.  Another idea was to get the logs.  You can just stream the logs of one container and write your logs wherever you need them,  maybe in a separate text file, maybe directly into some database that you set up or whatever  comes to your mind and whatever is helpful for you.  Of course you can do this with several containers and not just one.  Then another idea that I had when I wrote the proposal was, okay, why not save all your  Python files in one folder, use it as mounting point, then spin up different containers with  different Python versions and mount your Python files there, and then run PyTest.  I have to admit that when I started to prepare the talk and then started to work out on this  use case, somewhere in the middle I was like, okay, it can work, but maybe it's not really  useful.  So in that sense, maybe just write a whole CI CD pipeline.  If you're at that point, maybe that's really the better choice.  And yeah, I'm noticing I'm way too fast, like really way too fast.  This is already a wrap up, but I guess you're not so angry about it to get to the coffee  break.  So yeah, depending on your task at hand that you want to solve on your laptop, Python might  be a better choice than, for example, a best script, especially if you want to maintain  it and maybe keep it readable kind of.  And the Docker SDK might sometimes be a better choice than import OS or equivalents.  And yeah, at this point I hope that you at least have a high level idea about the Docker  SDK and some inspiration there for your day to day work.  And as a last hint, I can really recommend the Docker documentation.  Also for the SDK or for Docker itself, it is usually up to date and really good and a lot  of information.  So now that I was so fast, we have a lot of time for questions.  Thank you.  Thank you very much for the fast talk.  And as you said, we have now a lot of time for questions.  So first comment, I have been doing talks for like 13 years and yesterday I was also  early by 10 minutes.  So yes, that is not a problem.  You created a client by calling Docker.from.mv.  What does it do exactly and what are the other options for creating clients?  So what it does exactly, exactly, I can't answer.  But it does basically create an object that is connected to your local, to your local  service, Docker service.  And of course you can give it parameters to connect some remote Docker service.  You can use it with authentication, with HTTPS and so on and so forth.  Thank you.  Can you go to the microphone?  Hi, thank you.  How is your experience with Windows with this SDK?  Okay.  But is there any issue?  Yeah, so if you install Docker on your Windows laptop, it works if you don't have like virtual  box installed that does not work in parallel.  You have to activate and deactivate what is it called Hyper-V.  So that's one point.  Another point is if you install Docker on a Windows laptop, you have to give the service  explicit resources like memory, CPUs and so on.  So it doesn't take that on need.  And on the other side, all these resources are locked if you have the demon running without  any usage.  So it works, but on a Linux system, it works better.  At the end is us working with a common line.  Yeah.  Okay.  Thank you.  Any further questions?  Okay.  Thank you very much again and have a good one.
Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_0_1823.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_1824_2063.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_2064_2123.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_2124_2171.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_2172_13607.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_13608_15203.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_15204_15347.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_15348_15527.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_15528_15683.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_15684_16763.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_16764_17303.png Jan_Wagner_-_Docker_meets_Python_-_A_look_on_the_Docker_SDK_for_Pyt_17304_19166.png
Hi. Yeah, thank you. I'm surprised, happy and scared that so many people are here actually. So this is my second conference talk. So in case I collapse and faint, please don't make pictures. Okay, be cool. So welcome to my talk. Dogger meets Python. A look on the Dogger SDK for Python. And what I'm going to talk about is briefly introduce myself. Then the Dogger Demon and Service, what that is. Then of course the Dogger SDK. So that's the topic. All right, so where you can get it and some ideas of possibly use cases. So hi, my name is Jan, 32 years old, not married, but engaged. No kids yet. So that's good for me at the moment. I'm a data science consultant at AxanTech. It's a small consultant company in Germany with different locations all around Germany. And of course, like every company here, we focus on data. So if you're interested in job wise or project wise, just get in touch with me. You can do that via email, social media or GitHub. I will put the slides on GitHub later, but they are already on the conference page. So if you want to look them up. All right. Dogger, who of you has used Dogger? Okay, that's good. So you basically know what it is and how it works. So after you installed Dogger on your system, on your local laptop, I'm assuming just a local laptop, the Demon or the service is reachable per default from within the system, so from local host, or you can configure it to be reachable from outside. And if you have the Demon, of course, you want to talk to it, kind of. And there are different ways to do that. So I separated the command line. You know, it just put in Dogger something, something, or you can use graphical user interfaces like KiteMedic or the Dogger extension for Visual Studio Code, which I personally like a lot. Or you can use Webguys like Portaino. But well, sometimes, depending what you want to do, these options are too big or too small. And maybe not customizable enough for you. So this is where the Dogger SDK comes into place. So just build your own. To get it, it's easy. It's officially available for Python and Golang. There's an awesome documentation available. So it's really just pip, pip3 install Dogger. And for Konda, there's also a package available. Some small code examples. If you go to the website, the code examples are written in Python 2. I don't know why. It is supported, but I wrote them to, yeah, you see it on the print statement in Python 3. On the left side, you see what you would usually type into the command line. And on the right side, you will see the equivalent if you would use a Python script and use the Dogger SDK. So it's really just import Dogger. Then you get, you use some name like client. That would be best practice. Dogger.fromNV without any further parameters. This means you want to connect to your local Dogger service. You can also connect to a remote one. And then you can pretty much do whatever you want. In this example, the print statement is only because Hello World usually, I guess, you would do the Hello World example from Dogger. Only a few. Okay. If you start this, this is just an image starting up a container printing on subtext, stopping the container, and that's it. So if you use the print statement here, you will see what was given out in the standard out. You can put an image. In this example, the Alpine image, it's pretty straightforward. Nevertheless, a hint, if you used to use the command line interface from Dogger, you usually type DoggerImage, DoggerContainer. DoggerImage is an exception that works on the command line. But if you use the SDK, it's always images and containers. And if you use a proper IDE, autocomplete will solve this problem. But if you just go into, I don't know, some easy text editor, this might be a trap. All right. So when I wrote the proposal, I was thinking about some ideas and use cases and some that I use. And one is, yeah, just to use Python as a container starting script. These are two examples more or less like the original on the Dogger Hub site for Postgres. And depending what you want to start, these commands get really, really long and complex. And if you want to change something in the middle, you have to use your back keys. You might save them in the text file and copy paste them to your command line. But that's somehow not so cool. So my first idea was, okay, Python comes batteries included, just import OS. And then with OS.system, you can give a command directly to your systems shell. But I want to show you the Dogger SDK. So this is maybe one example how it could work with the SDK. So you just declare your variables with strings or with a list of strings. And you get again the client object, the Dogger object, yeah, in the variable client. And then you can just put in to the parameters, all your variables. And that way you can easily list up all your parameters. Yeah, more easy, more better to read, maybe easier to change if you have to change them regularly, and I think sometimes this is quite useful. You could also extend it a little bit. So you could, if a container is running at the moment, you could go and get that container and stop it. You can prune it or delete it in that case if you haven't started with the command that it should delete automatically after you've stopped it. Then of course you can build a new image from a Dogger file that is saved somewhere. And you can get the idea and then start again a new container with a new image that you just built. Another idea was to get the logs. You can just stream the logs of one container and write your logs wherever you need them, maybe in a separate text file, maybe directly into some database that you set up or whatever comes to your mind and whatever is helpful for you. Of course you can do this with several containers and not just one. Then another idea that I had when I wrote the proposal was, okay, why not save all your Python files in one folder, use it as mounting point, then spin up different containers with different Python versions and mount your Python files there, and then run PyTest. I have to admit that when I started to prepare the talk and then started to work out on this use case, somewhere in the middle I was like, okay, it can work, but maybe it's not really useful. So in that sense, maybe just write a whole CI CD pipeline. If you're at that point, maybe that's really the better choice. And yeah, I'm noticing I'm way too fast, like really way too fast. This is already a wrap up, but I guess you're not so angry about it to get to the coffee break. So yeah, depending on your task at hand that you want to solve on your laptop, Python might be a better choice than, for example, a best script, especially if you want to maintain it and maybe keep it readable kind of. And the Docker SDK might sometimes be a better choice than import OS or equivalents. And yeah, at this point I hope that you at least have a high level idea about the Docker SDK and some inspiration there for your day to day work. And as a last hint, I can really recommend the Docker documentation. Also for the SDK or for Docker itself, it is usually up to date and really good and a lot of information. So now that I was so fast, we have a lot of time for questions. Thank you. Thank you very much for the fast talk. And as you said, we have now a lot of time for questions. So first comment, I have been doing talks for like 13 years and yesterday I was also early by 10 minutes. So yes, that is not a problem. You created a client by calling Docker.from.mv. What does it do exactly and what are the other options for creating clients? So what it does exactly, exactly, I can't answer. But it does basically create an object that is connected to your local, to your local service, Docker service. And of course you can give it parameters to connect some remote Docker service. You can use it with authentication, with HTTPS and so on and so forth. Thank you. Can you go to the microphone? Hi, thank you. How is your experience with Windows with this SDK? Okay. But is there any issue? Yeah, so if you install Docker on your Windows laptop, it works if you don't have like virtual box installed that does not work in parallel. You have to activate and deactivate what is it called Hyper-V. So that's one point. Another point is if you install Docker on a Windows laptop, you have to give the service explicit resources like memory, CPUs and so on. So it doesn't take that on need. And on the other side, all these resources are locked if you have the demon running without any usage. So it works, but on a Linux system, it works better. At the end is us working with a common line. Yeah. Okay. Thank you. Any further questions? Okay. Thank you very much again and have a good one.
Okay, be cool. So welcome to my talk. Dogger meets Python. A look on the Dogger SDK for or GitHub. I will put the slides on GitHub later, you basically know what it is and how it works. you use the SDK, it's always images and you just built. Another idea was to get the logs. with different Python versions and mount your work, but maybe it's not really useful. So in keep it readable kind of. And the Docker SDK very much again and have a good one.