Continuous integration for mobile
Kroell, Sven
Karpouzis, Thanos
Mobile apps are a multi-billion market and one of the fastest growing sectors of technology, but the development of the apps does not meet the quality standards of backend systems. A small agile team can produce prototypes and MVPs efficiently, but once the project scales, tools like beta distribution platforms enable all stakeholders to be engaged and informed, while developers focus on producing high quality software. This (45 mins) talk will outline our experiences with implementing continuous integration and providing meaningful reports, which are essential for decision making. As well as providing an environment the developers which helps to identify problems in the codebase or with the implementation. Another important factor is the distribution of the application to the different departments for test, demo and acceptance. We will show how it is possible that stakeholders are able to create and distribute builds depending on their needs. A project template will be presented, which boilerplates robolectric for android tests, apium for automated UI tests, and a Jenkins setup which orchestrates the build test and distribution, using internal distribution tools such as crashlytics and hockeyapp.
 I hope you all enjoyed DroidCon until now, as much as we are. We are here to talk about continuous integration for mobile. We will deepen in more advanced topics a little bit, like built pipelines, quality gates, and distributed channels. I am Tharnos Carpuzzi, senior Android engineer for bubble. With me on stage is Sven. My name is Sven Koeil, I am a mobile quality engineer at bubble. As Tharnos said, we are going a little bit deeper on the topic. But first we start with a short introduction, what is continuous integration and what is best for this than quoting Martin Fowler. Continuous integration is a software development practice where the members of a team integrate their work frequently. Usually each person integrates at least daily, leading to multiple integrations per day. Each integration is very afraid by an automated build, including the tests to detect integration errors as quickly as possible. So having now a small quote of this, we have to think about the differences between the web world and the mobile world. Because mobile is quite new, so there are differences and big differences. So one thing is definitely the infrastructure. You have to provide a good infrastructure for your tests and for your build because it has to think about devices, which devices have to be tested. Also the staging servers are totally different. So it has to be sometimes coded into the application. It has to be a testing flavor for pointing to a specific staging server. You have to do code signing for the applications. And of course the testing is totally new. It's maybe two or three years ago going bigger and bigger. But it's still a problem to do proper testing for mobile and integrating this into a continued integration is also always a bit complicated. Also the distribution, you have to provide APKs. You have to make it possible that the people care or your testers, your stakeholders can use them properly. So talking about this, we are going a little bit deeper in different systems we are using and we're used to use a bubble and we are still using. So first we are starting with Travis CI. So we start working with Travis CI, which is continuously based on a service, pretty much. It's a Berlin company and that's a very good part of Berlin. It's really easy to use and that was the best part of it. It's a hosted cloud solution. It's free for open source projects. So if you are a small team, if you're doing something open source, that's the way to go. But at the same time, you can have premium accounts if you are a bigger organization and you want to keep your code private. The configuration is super simple. It's just one file, just five lines more or less that you can find on their website or a lot of forums provide some more customizable configurations. And then it gives easy decoration with multiple services out of the box. Classy and tools, GitHub and all the things that you need to get, all the feedback right away within your tools that you are already using within your team. And of course, since it's a cloud solution, it's scalable by default. We have been using that for more than a year, but in our case, for mobile, we wanted to have control over our hardware. We wanted to be able to add different devices for our tests. We wanted to play with some more exotic tools like Watsys and other channels. So at that point, Travis was a limitation for us. So we tried to do something that we control more and we went back to the classics, back to the basics and taking it. And yeah, we switched recently to Jenkins, which is, I think, well-known. It has a lot of plugins. You can integrate most of the stuff you need by plugin. There are plugins for displaying the unit test coverage where we talk about later. It's also scalable because you can self-host it, you can open nodes, you can open everything you need, and you can have a specific node for a specific purpose. So you can have a node for building stuff where the key chains are, which is really secured. You can have a node just for building infrastructure. You can actually spawn new virtual machines with Puppet and with WageRant. And this makes it also really flexible. And you can have, in these virtual machines, you can also have different simulators and emulators started for different test purposes. It has high flexibility on the artifacts because you can store it on the device. You can hand them around. You have a really cool permission matrix, which means you have a job which is publishing to Google. It's not a good practice to that everyone can publish to Google. You have a job where the QA can issue their own builds, which leads to really less work for the developers because they don't have to go to the QA and talk to them. The QA can issue their own builds. And this makes it really, really flexible. And this was one of the biggest reasons besides the possibility to update everything as you want. And the next step, we're talking about our build pipelines, which means it's a really cool working with Jenkins to build pipelines, especially for mobile, because you want to have a big, you want to have fast feedback. You want to have, like, you want to know what's going wrong really fast because mobile tests usually take longer. You don't have jump points like in the web world. You want to have really fast information. You want maybe to paralyze things which are taking longer so you can paralyze different steps of testing. You want to have instant reports of what's going wrong, what's the problem. And one really cool thing we implemented at Bublr, automated build pipeline generation. This means we have a master pipeline for the release. We have a master pipeline for the distribution. We have a master pipeline for the normal develop. And it's possible to just when the developer opens a new branch that everything gets created out of those templates. How does build pipelines look like? So when the development team checks in, everything gets triggered and there are logs for everything you need. So it's a really cool way to get the information quickly and as exact as you need it. So the next topic will be quality gates. Okay. So as we mentioned, we are not talking so much about the configuration and setting up the CI. I guess everybody can spend, like, a day and set it up. Quality gates is the first a little bit more advanced part. It's not just building an APK but going one step before that in the process and ensure that your code is of high quality enough and ensure that it could be expandable and maintainable in an automated way. Of course, we have the first step. The easiest step is also provided by the Android tools is using some linters, going with ProGuard, Minify, and make sure that you don't have garbage in your APK. You just have the resources, the libraries, and the methods that you need. And ProGuard and linters take care of that. Second part is your test coverage. We believe that it's one of the most important KPIs for the health of a project to know to have automated tests. And the easiest way to measure that is play with tools like Zacoco and make sure that you're keeping very high test coverage on your project. It's also really easy to set up that usually takes less than half hour to set it up and plug it in with Jenkins in this case. In means of Travis, usually it's a little bit more complicated to get the reports as you want but Jenkins you have total control and it's really easy to play with. Style guidelines, that's something not so important for most teams, especially if they are small or if they're using the ID code styling to get there. But if you are talking for bigger teams, if you are talking about teams that either are distributed all over the world, you want to ensure that your code looks exactly the same, not based on your developer effort but with automated tools. Also provided as plugins for Jenkins, you can just browse for them. There are multiple of them. You can find the code style that you like most and apply it. And now the most important of all is study code analysis. So we are great developers, we are building great products but we are working fast. We need to have a point of truth that we verify that our code is of high quality. There are multiple tools. We are playing with SonarCube solutions in bubble and with Jenkins it took us, okay, we had a few troubles setting it up. It took us like a week. And still we have problems with that but we get a lot of feedback based on how clean is our code, how much code is duplicate within our projects. We are able really fast to make measurable estimation of how and make decisions on how to move forward based on the numbers on that again. SonarCube provides a lot of tools for Android projects. It's kind of hard to hook it up with Jenkins but you get it there. I think the problem is mostly the Android projects and not so much Jenkins but we had some problems but now it's working. Having that the next step after you make sure that your code is of high quality, you need to make sure that your application still functions as you expect it to be. And that means we need to have some testing. Yes, most teams have dedicated QA departments but that doesn't mean that we should avoid the automated test. That's the basics. We start with unit testing using Roboelectric Android test framework and what's getting interesting is how you hook them up with your CI in order to work with both physical and virtual devices. Also very simple for Jenkins to do so. You just plug in your couple of devices and you have a small script to start your emulators from the CI. And maybe the one thing that you need to focus is put a bit more intelligence on your CI in order to keep the devices not just physical but also real. That means start some other applications. Facebook is a good example. It takes a lot of resources. Facebook Messenger, for example, to see how your application really behaves in a real device. The trouble with CI is that we have a really lab-like environment. Everything is so clean. The emulator is usually completely fresh every time that you start them. Put some effort on your CI infrastructure to make the devices feel real. It doesn't take that much time. Even run a monkey test that is provided by Android Studio just before running your test or right after. It will give you way better feedback. And the unit test and integration test is just the first step. We also have acceptance tests that this man will talk about. As I said before, acceptance tests are the next step which might be also paralyzed. But there are tools out there which handle a lot of acceptance tests like Appium which we are using at Bubble, ColorBush or Expresso. So you can have really meaningful tests and also really good test results for the developers, for the stakeholders, for your product managers, for basically everyone. You can even put it to some monitors at your workplace so everyone can see how great you proceed, how good your test coverage gets. And you can have, as I said, HTML pages published from the Jenkins to Web server or on the Jenkins itself where everybody can see how it looks like, how many tests are there. If they are failing their screenshots or even videos, whether they are developers or other people can see what's going on. Maybe there's a notification too much or there is something wrong in the navigation through the application. So this makes it really cool that it's automated and you get it on your CI and really every stakeholder you need can get it where. And the next big step is the integration with the different cloud services. So we all know there's a big fragmentation in Android. So there are cloud services out there like test right, like test objects like browser stack, et cetera. So there are a number less of options which you can take and which you should consider to if you're, for example, if you're seeing that your application is crashing on a specific device and you don't want to have all those devices in your house, you can use those services to do your integration tests from the Jenkins. They get sent to those. I think there was a talk in the bar camp about this. There will be also talks about unit testing. So we don't dig really deep into this just that you see what is possible. So a test result could look like this. So you see what's the result or it's read, what's going wrong. So you see things are failing. You get a screenshot. You get everything, what you can think of, whoops, what's going wrong. So you as a developer could see what's going wrong, where I can do it, and you get the information as fast as possible that you can fix what's going wrong. And now we have already talked about so many different aspects that provide a lot of feedback. We talk about, we are talking about reports that they're going for many pages. The important part of how you use the CI is that it actually can handle this information and not spam you. You don't have enough time to go through all the results about your linder. You don't have enough time to go through all the results from your cloud testing. You can add all this logic to your CI. You can only have alerts when it's meaningful for you and have different alerts for the different people that are working within your organization for that reason. For example, QA may care about an acceptance test that failed. You may care just for the unit tests. Other people, your stakeholders, may care that the build is ready to be published. The CI can take care of all this overhead that until now was handled mostly by developers and QA engineers. And last but not least, I think it's the most important actually is distribution. It's typical for all of us to at the end have a build in our own machine, our own laptop, and then push it to the different distribution channels. That's fine, but if you are doing it for teams at the scale and they provide multiple builds all the time, why handle it yourself? The CI extracts for now is really simple to hook up with all the different distribution services. We are using cross-litics, for example, and actually it's super easy to hook up with Jenkins because you don't really do that. You are using Gradle and SDK provided by them. So that's done out of the box both for Travis and Jenkins as well. Travis has a few problems with the artifacts provided because you need to have some kind of your own storage system, use either S3 or some kind of your own server to store all the artifacts. Jenkins is in your own machine. You can use this space to hold all the artifacts. Then you can, the thing that you can do about distribution and your CI server is add a level of customization for its build. You can either have some templates that do it all out of the box or you can give your input while you set up a build. So you can select testers. You can select not just testers, you can select everybody who you want to get the build. You can form your change log. You can use actually multiple plugins for that or write your own script, taking the history from your Git log or write it yourself and also select stuff like branches which is pretty easy or if we are talking about multi-flavor app, set which flavors and how. All of this stuff for any of the CI servers out there usually takes less than half hour. And then the most important thing, how we publish to the Google Play. Yes, a lot of people want to have still the control over it, but now last week Google announced the Cloud Test Lab that works with the alpha and beta channels of the Google Play. There is also a very simple API to help you push your APK directly to the store. Let's not talk about the production environment, but let's just focus on the beta channel. Right now, not right now. In a couple of weeks, they said they would make it publicly available that an automated monkey test will run every time that you push something to your beta channel. Why not using it once a day on your nightly builds to make sure that the Google crawlers are able to verify that your build is stable. Again, the provided API by not Google really, yeah, it's by Google, but also has already a lot of scripts ready for you in the community. It took us 10 minutes to integrate it with our CI to have this. You have a blue screen of death. Let's see this password. Yeah. And again, you just spent less than an hour to set up all this thing. So we talk about five add-ons to your basic build. Unfortunately, we don't have enough time to get into deep details about them. We just mentioned what you can do. We are available for you to give you more information on how we did it, which tools we are using. But go ahead, play around. The CI is not just to build something or just run tests. It's so much more things that you can do with that. Yeah. And, yeah, we thank you for your attention. And if you have any questions, I think we have enough time for a talk. We have. We have. Thank you, Thanos. Thank you, Sven. Thank you.
 I hope you all enjoyed DroidCon until now, as much as we are.  We are here to talk about continuous integration for mobile.  We will deepen in more advanced topics a little bit, like built pipelines, quality gates,  and distributed channels.  I am Tharnos Carpuzzi, senior Android engineer for bubble.  With me on stage is Sven.  My name is Sven Koeil, I am a mobile quality engineer at bubble.  As Tharnos said, we are going a little bit deeper on the topic.  But first we start with a short introduction, what is continuous integration and what is  best for this than quoting Martin Fowler.  Continuous integration is a software development practice where the members of a team integrate  their work frequently.  Usually each person integrates at least daily, leading to multiple integrations per day.  Each integration is very afraid by an automated build, including the tests to detect integration  errors as quickly as possible.  So having now a small quote of this, we have to think about the differences between the  web world and the mobile world.  Because mobile is quite new, so there are differences and big differences.  So one thing is definitely the infrastructure.  You have to provide a good infrastructure for your tests and for your build because it  has to think about devices, which devices have to be tested.  Also the staging servers are totally different.  So it has to be sometimes coded into the application.  It has to be a testing flavor for pointing to a specific staging server.  You have to do code signing for the applications.  And of course the testing is totally new.  It's maybe two or three years ago going bigger and bigger.  But it's still a problem to do proper testing for mobile and integrating this into a continued  integration is also always a bit complicated.  Also the distribution, you have to provide APKs.  You have to make it possible that the people care or your testers, your stakeholders can  use them properly.  So talking about this, we are going a little bit deeper in different systems we are using  and we're used to use a bubble and we are still using.  So first we are starting with Travis CI.  So we start working with Travis CI, which is continuously based on a service, pretty  much.  It's a Berlin company and that's a very good part of Berlin.  It's really easy to use and that was the best part of it.  It's a hosted cloud solution.  It's free for open source projects.  So if you are a small team, if you're doing something open source, that's the way to go.  But at the same time, you can have premium accounts if you are a bigger organization  and you want to keep your code private.  The configuration is super simple.  It's just one file, just five lines more or less that you can find on their website or  a lot of forums provide some more customizable configurations.  And then it gives easy decoration with multiple services out of the box.  Classy and tools, GitHub and all the things that you need to get, all the feedback right  away within your tools that you are already using within your team.  And of course, since it's a cloud solution, it's scalable by default.  We have been using that for more than a year, but in our case, for mobile, we wanted to  have control over our hardware.  We wanted to be able to add different devices for our tests.  We wanted to play with some more exotic tools like Watsys and other channels.  So at that point, Travis was a limitation for us.  So we tried to do something that we control more and we went back to the classics, back  to the basics and taking it.  And yeah, we switched recently to Jenkins, which is, I think, well-known.  It has a lot of plugins.  You can integrate most of the stuff you need by plugin.  There are plugins for displaying the unit test coverage where we talk about later.  It's also scalable because you can self-host it, you can open nodes, you can open everything  you need, and you can have a specific node for a specific purpose.  So you can have a node for building stuff where the key chains are, which is really secured.  You can have a node just for building infrastructure.  You can actually spawn new virtual machines with Puppet and with WageRant.  And this makes it also really flexible.  And you can have, in these virtual machines, you can also have different simulators and  emulators started for different test purposes.  It has high flexibility on the artifacts because you can store it on the device.  You can hand them around.  You have a really cool permission matrix, which means you have a job which is publishing  to Google.  It's not a good practice to that everyone can publish to Google.  You have a job where the QA can issue their own builds, which leads to really less work  for the developers because they don't have to go to the QA and talk to them.  The QA can issue their own builds.  And this makes it really, really flexible.  And this was one of the biggest reasons besides the possibility to update everything as you  want.  And the next step, we're talking about our build pipelines, which means it's a really  cool working with Jenkins to build pipelines, especially for mobile, because you want to  have a big, you want to have fast feedback.  You want to have, like, you want to know what's going wrong really fast because mobile tests  usually take longer.  You don't have jump points like in the web world.  You want to have really fast information.  You want maybe to paralyze things which are taking longer so you can paralyze different  steps of testing.  You want to have instant reports of what's going wrong, what's the problem.  And one really cool thing we implemented at Bublr, automated build pipeline generation.  This means we have a master pipeline for the release.  We have a master pipeline for the distribution.  We have a master pipeline for the normal develop.  And it's possible to just when the developer opens a new branch that everything gets created  out of those templates.  How does build pipelines look like?  So when the development team checks in, everything gets triggered and there are logs for everything  you need.  So it's a really cool way to get the information quickly and as exact as you need it.  So the next topic will be quality gates.  Okay.  So as we mentioned, we are not talking so much about the configuration and setting up the  CI.  I guess everybody can spend, like, a day and set it up.  Quality gates is the first a little bit more advanced part.  It's not just building an APK but going one step before that in the process and ensure  that your code is of high quality enough and ensure that it could be expandable and maintainable  in an automated way.  Of course, we have the first step.  The easiest step is also provided by the Android tools is using some linters, going with ProGuard,  Minify, and make sure that you don't have garbage in your APK.  You just have the resources, the libraries, and the methods that you need.  And ProGuard and linters take care of that.  Second part is your test coverage.  We believe that it's one of the most important KPIs for the health of a project to know to  have automated tests.  And the easiest way to measure that is play with tools like Zacoco and make sure that  you're keeping very high test coverage on your project.  It's also really easy to set up that usually takes less than half hour to set it up and  plug it in with Jenkins in this case.  In means of Travis, usually it's a little bit more complicated to get the reports as  you want but Jenkins you have total control and it's really easy to play with.  Style guidelines, that's something not so important for most teams, especially if they  are small or if they're using the ID code styling to get there.  But if you are talking for bigger teams, if you are talking about teams that either are  distributed all over the world, you want to ensure that your code looks exactly the same,  not based on your developer effort but with automated tools.  Also provided as plugins for Jenkins, you can just browse for them.  There are multiple of them.  You can find the code style that you like most and apply it.  And now the most important of all is study code analysis.  So we are great developers, we are building great products but we are working fast.  We need to have a point of truth that we verify that our code is of high quality.  There are multiple tools.  We are playing with SonarCube solutions in bubble and with Jenkins it took us, okay,  we had a few troubles setting it up.  It took us like a week.  And still we have problems with that but we get a lot of feedback based on how clean is  our code, how much code is duplicate within our projects.  We are able really fast to make measurable estimation of how and make decisions on how  to move forward based on the numbers on that again.  SonarCube provides a lot of tools for Android projects.  It's kind of hard to hook it up with Jenkins but you get it there.  I think the problem is mostly the Android projects and not so much Jenkins but we had  some problems but now it's working.  Having that the next step after you make sure that your code is of high quality, you need  to make sure that your application still functions as you expect it to be.  And that means we need to have some testing.  Yes, most teams have dedicated QA departments but that doesn't mean that we should avoid  the automated test.  That's the basics.  We start with unit testing using Roboelectric Android test framework and what's getting  interesting is how you hook them up with your CI in order to work with both physical and  virtual devices.  Also very simple for Jenkins to do so.  You just plug in your couple of devices and you have a small script to start your emulators  from the CI.  And maybe the one thing that you need to focus is put a bit more intelligence on your CI  in order to keep the devices not just physical but also real.  That means start some other applications.  Facebook is a good example.  It takes a lot of resources.  Facebook Messenger, for example, to see how your application really behaves in a real  device.  The trouble with CI is that we have a really lab-like environment.  Everything is so clean.  The emulator is usually completely fresh every time that you start them.  Put some effort on your CI infrastructure to make the devices feel real.  It doesn't take that much time.  Even run a monkey test that is provided by Android Studio just before running your test  or right after.  It will give you way better feedback.  And the unit test and integration test is just the first step.  We also have acceptance tests that this man will talk about.  As I said before, acceptance tests are the next step which might be also paralyzed.  But there are tools out there which handle a lot of acceptance tests like Appium which  we are using at Bubble, ColorBush or Expresso.  So you can have really meaningful tests and also really good test results for the developers,  for the stakeholders, for your product managers, for basically everyone.  You can even put it to some monitors at your workplace so everyone can see how great you  proceed, how good your test coverage gets.  And you can have, as I said, HTML pages published from the Jenkins to Web server or on the Jenkins  itself where everybody can see how it looks like, how many tests are there.  If they are failing their screenshots or even videos, whether they are developers or other  people can see what's going on.  Maybe there's a notification too much or there is something wrong in the navigation through  the application.  So this makes it really cool that it's automated and you get it on your CI and really every  stakeholder you need can get it where.  And the next big step is the integration with the different cloud services.  So we all know there's a big fragmentation in Android.  So there are cloud services out there like test right, like test objects like browser  stack, et cetera.  So there are a number less of options which you can take and which you should consider  to if you're, for example, if you're seeing that your application is crashing on a specific  device and you don't want to have all those devices in your house, you can use those services  to do your integration tests from the Jenkins.  They get sent to those.  I think there was a talk in the bar camp about this.  There will be also talks about unit testing.  So we don't dig really deep into this just that you see what is possible.  So a test result could look like this.  So you see what's the result or it's read, what's going wrong.  So you see things are failing.  You get a screenshot.  You get everything, what you can think of, whoops, what's going wrong.  So you as a developer could see what's going wrong, where I can do it, and you get the  information as fast as possible that you can fix what's going wrong.  And now we have already talked about so many different aspects that provide a lot of feedback.  We talk about, we are talking about reports that they're going for many pages.  The important part of how you use the CI is that it actually can handle this information  and not spam you.  You don't have enough time to go through all the results about your linder.  You don't have enough time to go through all the results from your cloud testing.  You can add all this logic to your CI.  You can only have alerts when it's meaningful for you and have different alerts for the  different people that are working within your organization for that reason.  For example, QA may care about an acceptance test that failed.  You may care just for the unit tests.  Other people, your stakeholders, may care that the build is ready to be published.  The CI can take care of all this overhead that until now was handled mostly by developers  and QA engineers.  And last but not least, I think it's the most important actually is distribution.  It's typical for all of us to at the end have a build in our own machine, our own laptop,  and then push it to the different distribution channels.  That's fine, but if you are doing it for teams at the scale and they provide multiple builds  all the time, why handle it yourself?  The CI extracts for now is really simple to hook up with all the different distribution  services.  We are using cross-litics, for example, and actually it's super easy to hook up with Jenkins  because you don't really do that.  You are using Gradle and SDK provided by them.  So that's done out of the box both for Travis and Jenkins as well.  Travis has a few problems with the artifacts provided because you need to have some kind  of your own storage system, use either S3 or some kind of your own server to store all  the artifacts.  Jenkins is in your own machine.  You can use this space to hold all the artifacts.  Then you can, the thing that you can do about distribution and your CI server is add a level  of customization for its build.  You can either have some templates that do it all out of the box or you can give your  input while you set up a build.  So you can select testers.  You can select not just testers, you can select everybody who you want to get the build.  You can form your change log.  You can use actually multiple plugins for that or write your own script, taking the history  from your Git log or write it yourself and also select stuff like branches which is pretty  easy or if we are talking about multi-flavor app, set which flavors and how.  All of this stuff for any of the CI servers out there usually takes less than half hour.  And then the most important thing, how we publish to the Google Play.  Yes, a lot of people want to have still the control over it, but now last week Google  announced the Cloud Test Lab that works with the alpha and beta channels of the Google  Play.  There is also a very simple API to help you push your APK directly to the store.  Let's not talk about the production environment, but let's just focus on the beta channel.  Right now, not right now.  In a couple of weeks, they said they would make it publicly available that an automated  monkey test will run every time that you push something to your beta channel.  Why not using it once a day on your nightly builds to make sure that the Google crawlers  are able to verify that your build is stable.  Again, the provided API by not Google really, yeah, it's by Google, but also has already  a lot of scripts ready for you in the community.  It took us 10 minutes to integrate it with our CI to have this.  You have a blue screen of death.  Let's see this password.  Yeah.  And again, you just spent less than an hour to set up all this thing.  So we talk about five add-ons to your basic build.  Unfortunately, we don't have enough time to get into deep details about them.  We just mentioned what you can do.  We are available for you to give you more information on how we did it, which tools  we are using.  But go ahead, play around.  The CI is not just to build something or just run tests.  It's so much more things that you can do with that.  Yeah.  And, yeah, we thank you for your attention.  And if you have any questions, I think we have enough time for a talk.  We have.  We have.  Thank you, Thanos.  Thank you, Sven.  Thank you.
_droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_0_503.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_504_551.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_552_599.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_600_1955.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_1956_2363.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_2364_3479.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_3480_5771.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_5772_6911.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_6912_7007.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_7008_8351.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_8352_12875.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_12876_16715.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_16716_18563.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_18564_19691.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_19692_19883.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_19884_21371.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_21372_22751.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_22752_26807.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_26808_27335.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_27336_27383.png _droidconDE_2015__Sven_Kroell___Thanos_Karpouzis___ULEIygk98xw_27384_29467.png
I hope you all enjoyed DroidCon until now, as much as we are. We are here to talk about continuous integration for mobile. We will deepen in more advanced topics a little bit, like built pipelines, quality gates, and distributed channels. I am Tharnos Carpuzzi, senior Android engineer for bubble. With me on stage is Sven. My name is Sven Koeil, I am a mobile quality engineer at bubble. As Tharnos said, we are going a little bit deeper on the topic. But first we start with a short introduction, what is continuous integration and what is best for this than quoting Martin Fowler. Continuous integration is a software development practice where the members of a team integrate their work frequently. Usually each person integrates at least daily, leading to multiple integrations per day. Each integration is very afraid by an automated build, including the tests to detect integration errors as quickly as possible. So having now a small quote of this, we have to think about the differences between the web world and the mobile world. Because mobile is quite new, so there are differences and big differences. So one thing is definitely the infrastructure. You have to provide a good infrastructure for your tests and for your build because it has to think about devices, which devices have to be tested. Also the staging servers are totally different. So it has to be sometimes coded into the application. It has to be a testing flavor for pointing to a specific staging server. You have to do code signing for the applications. And of course the testing is totally new. It's maybe two or three years ago going bigger and bigger. But it's still a problem to do proper testing for mobile and integrating this into a continued integration is also always a bit complicated. Also the distribution, you have to provide APKs. You have to make it possible that the people care or your testers, your stakeholders can use them properly. So talking about this, we are going a little bit deeper in different systems we are using and we're used to use a bubble and we are still using. So first we are starting with Travis CI. So we start working with Travis CI, which is continuously based on a service, pretty much. It's a Berlin company and that's a very good part of Berlin. It's really easy to use and that was the best part of it. It's a hosted cloud solution. It's free for open source projects. So if you are a small team, if you're doing something open source, that's the way to go. But at the same time, you can have premium accounts if you are a bigger organization and you want to keep your code private. The configuration is super simple. It's just one file, just five lines more or less that you can find on their website or a lot of forums provide some more customizable configurations. And then it gives easy decoration with multiple services out of the box. Classy and tools, GitHub and all the things that you need to get, all the feedback right away within your tools that you are already using within your team. And of course, since it's a cloud solution, it's scalable by default. We have been using that for more than a year, but in our case, for mobile, we wanted to have control over our hardware. We wanted to be able to add different devices for our tests. We wanted to play with some more exotic tools like Watsys and other channels. So at that point, Travis was a limitation for us. So we tried to do something that we control more and we went back to the classics, back to the basics and taking it. And yeah, we switched recently to Jenkins, which is, I think, well-known. It has a lot of plugins. You can integrate most of the stuff you need by plugin. There are plugins for displaying the unit test coverage where we talk about later. It's also scalable because you can self-host it, you can open nodes, you can open everything you need, and you can have a specific node for a specific purpose. So you can have a node for building stuff where the key chains are, which is really secured. You can have a node just for building infrastructure. You can actually spawn new virtual machines with Puppet and with WageRant. And this makes it also really flexible. And you can have, in these virtual machines, you can also have different simulators and emulators started for different test purposes. It has high flexibility on the artifacts because you can store it on the device. You can hand them around. You have a really cool permission matrix, which means you have a job which is publishing to Google. It's not a good practice to that everyone can publish to Google. You have a job where the QA can issue their own builds, which leads to really less work for the developers because they don't have to go to the QA and talk to them. The QA can issue their own builds. And this makes it really, really flexible. And this was one of the biggest reasons besides the possibility to update everything as you want. And the next step, we're talking about our build pipelines, which means it's a really cool working with Jenkins to build pipelines, especially for mobile, because you want to have a big, you want to have fast feedback. You want to have, like, you want to know what's going wrong really fast because mobile tests usually take longer. You don't have jump points like in the web world. You want to have really fast information. You want maybe to paralyze things which are taking longer so you can paralyze different steps of testing. You want to have instant reports of what's going wrong, what's the problem. And one really cool thing we implemented at Bublr, automated build pipeline generation. This means we have a master pipeline for the release. We have a master pipeline for the distribution. We have a master pipeline for the normal develop. And it's possible to just when the developer opens a new branch that everything gets created out of those templates. How does build pipelines look like? So when the development team checks in, everything gets triggered and there are logs for everything you need. So it's a really cool way to get the information quickly and as exact as you need it. So the next topic will be quality gates. Okay. So as we mentioned, we are not talking so much about the configuration and setting up the CI. I guess everybody can spend, like, a day and set it up. Quality gates is the first a little bit more advanced part. It's not just building an APK but going one step before that in the process and ensure that your code is of high quality enough and ensure that it could be expandable and maintainable in an automated way. Of course, we have the first step. The easiest step is also provided by the Android tools is using some linters, going with ProGuard, Minify, and make sure that you don't have garbage in your APK. You just have the resources, the libraries, and the methods that you need. And ProGuard and linters take care of that. Second part is your test coverage. We believe that it's one of the most important KPIs for the health of a project to know to have automated tests. And the easiest way to measure that is play with tools like Zacoco and make sure that you're keeping very high test coverage on your project. It's also really easy to set up that usually takes less than half hour to set it up and plug it in with Jenkins in this case. In means of Travis, usually it's a little bit more complicated to get the reports as you want but Jenkins you have total control and it's really easy to play with. Style guidelines, that's something not so important for most teams, especially if they are small or if they're using the ID code styling to get there. But if you are talking for bigger teams, if you are talking about teams that either are distributed all over the world, you want to ensure that your code looks exactly the same, not based on your developer effort but with automated tools. Also provided as plugins for Jenkins, you can just browse for them. There are multiple of them. You can find the code style that you like most and apply it. And now the most important of all is study code analysis. So we are great developers, we are building great products but we are working fast. We need to have a point of truth that we verify that our code is of high quality. There are multiple tools. We are playing with SonarCube solutions in bubble and with Jenkins it took us, okay, we had a few troubles setting it up. It took us like a week. And still we have problems with that but we get a lot of feedback based on how clean is our code, how much code is duplicate within our projects. We are able really fast to make measurable estimation of how and make decisions on how to move forward based on the numbers on that again. SonarCube provides a lot of tools for Android projects. It's kind of hard to hook it up with Jenkins but you get it there. I think the problem is mostly the Android projects and not so much Jenkins but we had some problems but now it's working. Having that the next step after you make sure that your code is of high quality, you need to make sure that your application still functions as you expect it to be. And that means we need to have some testing. Yes, most teams have dedicated QA departments but that doesn't mean that we should avoid the automated test. That's the basics. We start with unit testing using Roboelectric Android test framework and what's getting interesting is how you hook them up with your CI in order to work with both physical and virtual devices. Also very simple for Jenkins to do so. You just plug in your couple of devices and you have a small script to start your emulators from the CI. And maybe the one thing that you need to focus is put a bit more intelligence on your CI in order to keep the devices not just physical but also real. That means start some other applications. Facebook is a good example. It takes a lot of resources. Facebook Messenger, for example, to see how your application really behaves in a real device. The trouble with CI is that we have a really lab-like environment. Everything is so clean. The emulator is usually completely fresh every time that you start them. Put some effort on your CI infrastructure to make the devices feel real. It doesn't take that much time. Even run a monkey test that is provided by Android Studio just before running your test or right after. It will give you way better feedback. And the unit test and integration test is just the first step. We also have acceptance tests that this man will talk about. As I said before, acceptance tests are the next step which might be also paralyzed. But there are tools out there which handle a lot of acceptance tests like Appium which we are using at Bubble, ColorBush or Expresso. So you can have really meaningful tests and also really good test results for the developers, for the stakeholders, for your product managers, for basically everyone. You can even put it to some monitors at your workplace so everyone can see how great you proceed, how good your test coverage gets. And you can have, as I said, HTML pages published from the Jenkins to Web server or on the Jenkins itself where everybody can see how it looks like, how many tests are there. If they are failing their screenshots or even videos, whether they are developers or other people can see what's going on. Maybe there's a notification too much or there is something wrong in the navigation through the application. So this makes it really cool that it's automated and you get it on your CI and really every stakeholder you need can get it where. And the next big step is the integration with the different cloud services. So we all know there's a big fragmentation in Android. So there are cloud services out there like test right, like test objects like browser stack, et cetera. So there are a number less of options which you can take and which you should consider to if you're, for example, if you're seeing that your application is crashing on a specific device and you don't want to have all those devices in your house, you can use those services to do your integration tests from the Jenkins. They get sent to those. I think there was a talk in the bar camp about this. There will be also talks about unit testing. So we don't dig really deep into this just that you see what is possible. So a test result could look like this. So you see what's the result or it's read, what's going wrong. So you see things are failing. You get a screenshot. You get everything, what you can think of, whoops, what's going wrong. So you as a developer could see what's going wrong, where I can do it, and you get the information as fast as possible that you can fix what's going wrong. And now we have already talked about so many different aspects that provide a lot of feedback. We talk about, we are talking about reports that they're going for many pages. The important part of how you use the CI is that it actually can handle this information and not spam you. You don't have enough time to go through all the results about your linder. You don't have enough time to go through all the results from your cloud testing. You can add all this logic to your CI. You can only have alerts when it's meaningful for you and have different alerts for the different people that are working within your organization for that reason. For example, QA may care about an acceptance test that failed. You may care just for the unit tests. Other people, your stakeholders, may care that the build is ready to be published. The CI can take care of all this overhead that until now was handled mostly by developers and QA engineers. And last but not least, I think it's the most important actually is distribution. It's typical for all of us to at the end have a build in our own machine, our own laptop, and then push it to the different distribution channels. That's fine, but if you are doing it for teams at the scale and they provide multiple builds all the time, why handle it yourself? The CI extracts for now is really simple to hook up with all the different distribution services. We are using cross-litics, for example, and actually it's super easy to hook up with Jenkins because you don't really do that. You are using Gradle and SDK provided by them. So that's done out of the box both for Travis and Jenkins as well. Travis has a few problems with the artifacts provided because you need to have some kind of your own storage system, use either S3 or some kind of your own server to store all the artifacts. Jenkins is in your own machine. You can use this space to hold all the artifacts. Then you can, the thing that you can do about distribution and your CI server is add a level of customization for its build. You can either have some templates that do it all out of the box or you can give your input while you set up a build. So you can select testers. You can select not just testers, you can select everybody who you want to get the build. You can form your change log. You can use actually multiple plugins for that or write your own script, taking the history from your Git log or write it yourself and also select stuff like branches which is pretty easy or if we are talking about multi-flavor app, set which flavors and how. All of this stuff for any of the CI servers out there usually takes less than half hour. And then the most important thing, how we publish to the Google Play. Yes, a lot of people want to have still the control over it, but now last week Google announced the Cloud Test Lab that works with the alpha and beta channels of the Google Play. There is also a very simple API to help you push your APK directly to the store. Let's not talk about the production environment, but let's just focus on the beta channel. Right now, not right now. In a couple of weeks, they said they would make it publicly available that an automated monkey test will run every time that you push something to your beta channel. Why not using it once a day on your nightly builds to make sure that the Google crawlers are able to verify that your build is stable. Again, the provided API by not Google really, yeah, it's by Google, but also has already a lot of scripts ready for you in the community. It took us 10 minutes to integrate it with our CI to have this. You have a blue screen of death. Let's see this password. Yeah. And again, you just spent less than an hour to set up all this thing. So we talk about five add-ons to your basic build. Unfortunately, we don't have enough time to get into deep details about them. We just mentioned what you can do. We are available for you to give you more information on how we did it, which tools we are using. But go ahead, play around. The CI is not just to build something or just run tests. It's so much more things that you can do with that. Yeah. And, yeah, we thank you for your attention. And if you have any questions, I think we have enough time for a talk. We have. We have. Thank you, Thanos. Thank you, Sven. Thank you.
continuous integration for mobile. We will deepen integration is a software development practice infrastructure for your tests and for your build possible that the people care or your testers, deeper in different systems we are using and if you are a small team, if you're doing tools, GitHub and all the things that you need to QA can issue their own builds. And this makes it really, really flexible. And this was one of the generation. This means we have a master pipeline the distribution. We have a master pipeline for important KPIs for the health of a project to verify that our code is of high quality. There the unit test and integration test is just the acceptance tests are the next step which might be Expresso. So you can have really meaningful tests developers, for the stakeholders, for your everybody can see how it looks like, how many through the application. So this makes it really integration with the different cloud services. So was a talk in the bar camp about this. There will your stakeholders, may care that the build is Jenkins as well. Travis has a few problems with environment, but let's just focus on the beta available that an automated monkey test will run channel. Why not using it once a day on your crawlers are able to verify that your build is