Release monitoring org
Konečný, Michal
Release-monitoring org is an open source application that let you add any project and checks for new versions of this project. It is used as part of the packager workflow in Fedora. This talk will show you the basic features of this app and how it is integrated in Fedora packager workflow. There will also be some demo of the application itself.
 Hello everyone and welcome to the talk about RISmointering.org. My name is Michal Koneczny and I'm a maintainer of RISmointering.org for Fedora. This is recording because I'm on the road right now and I'm not sure if I should be available in the chat, but I'm not available to present. Okay, so let's start. Here is the agenda. This is an illustration of the RISmointering.org and how I imagine it when I'm writing my blog posts. You can see it's a very nice word and the new hotness is actually something above floating above the new hotness, above Anitya. Okay, so let's start with some basics. What is RISmointering.org? It's containing from two applications. The first one is Anitya. It provides web interface for the users to actually do the things they need. The users could add new projects, watch for the releases. It automatically checks for new releases of the project and it's sending Fedora message when a new version is retrieved. The new hotness is Fedora messaging consumer. For those of you not aware, Fedora messaging is a messaging bus we are using in Fedora as basically for every message that is sent in infrastructure. The new hotness is listening to messages emitted by Anitya. It is only interested in the ones that are about the new release font and there are others. It's great or updates Paxilla which is your issue tracker system when new release is found. So the package who maintains the project, the package in Fedora is notified about new version. It could start scratch builds in your build system if this is configured for the project. Let's start with Anitya. I will talk about it a bit more because it's a really interesting project. Let's start with some magic numbers. These numbers are actually kind of impressive. The first commit for the Anitya was on 28th November 2013. First release was 0.1.0 which was on 29th September 2014. So as you can see the first release was almost a year later after the project was created. I'm the maintainer of the Anitya for two years so I don't have too much knowledge about the history of Anitya but I at least know what it is. Let's see at some contributions so you can see that the project is still alive. There are almost 2,000 commits. We have slightly above 50 contributors. There were almost half thousand issues created and we closed plenty of them. As you can see from this we still have some open and we are trying to work to close them as soon as possible. Current version of the Anitya is 0.18.0. For just to make it more interesting. The number of projects that are currently watched by Riesmoenturing.org are 126,435. I think this number will be much higher because there are new projects added each day. Just for this talk I have here the number of packages that are mapped to opensews distribution and we have 331 packages up to opensews. It is primarily used by Fedora so there are plenty more for Fedora. Here you can see the diagram which represents how the Anitya is working. As you can see there are two kind of users that could interact with it. One is the regular user which could add new project, add new distribution to the Anitya, edit project, flag project, flag is a mechanism that is used to mark the project that there is some issue. Admin usually looks at it and investigates if this is really an issue and solves it and closes it. He could add new mapping, mappings are the mappings between the project and the distribution and the name of the package in the distribution. The admin could do some advanced roles. He could delete projects, delete mappings, delete distribution, he could edit distribution, he could close flags that are solved. He could also ban regular users and create admins from the regular users if needed. So everybody who works with Anitya could become an admin, just say if you want and I could make it happen. The Anitya has two other parts, there are parts of it but they are running as separate application. The version checker which is being run regularly and it's checking every project that is there. If there is new version. There are some rules for the checking, it's not checking every project every run, there are exceptions. The main exception is the GitHub because GitHub is limited by the number of requests you could do in one hour. So in this case the new version is actually limited sometime but most of the projects, if there is new version, we will know about it in a matter of hours. There is also libraries.io consumer. The libraries.io consumer is just a listener for the libraries.io messages. And when it fonts new project that is interesting, is actually hosted on the platform we are interested in, then it reports it to Anitya and creates a new project if this is configured or reports a new version. If the new version is reported, the Anitya is sending the message to federal messaging broker. The Anitya is sending message in almost every change that will happen in the application. So if you want to see the history, you could use our data wrapper application which is actually only wrapper for the messages in the message broker. And you could see what exactly was sent and what changed in the Anitya. Probably there should be a full history of Anitya. So the current situation. Current situation in Anitya is that we are working on the milestone 1.0 because Anitya is major enough. I just stayed with the previous versioning system. Here is a link to the actual milestone. You could see what needs to be done, what is done. Most of the things for the 1.0 are done. I just want to do a few more features and update the documentation to better state because there are plenty of things that are outdated. Because we have October 1st this month, Anitya is part of it. So if you want to help on the Anitya, there are issues that are marked, that are labeled October 1st. So you could do it. It's a nice way to start contributing to it. Next one is the new hotness. The new hotness is floating island in the realm of magic. It's actually the point between Anitya and Fedora. There is monitoring on the Fedora island. Here are some statistics. The first comment was 13 March 2014. First release was 0.1.2, 17 November 2014. I'm not sure why the first release was in minor version because I'm just in the not minor version. I didn't find any attack that will be older than this. So it's possible that there was some areas just not attacked before. There are 140 commits from 22 different distributors. We have almost 150 issues created and we closed most of them. Current version is 0.13.1. I'm currently working as same as Anitya on 0.1.2. But there are plenty of work. This will be released. Maybe in the meantime I will release some bug fix release. The new hotness is actually much more complex. When you take in consideration the number of external applications it needs to communicate with. It's easier as a script. It's much less code based than Anitya. But it communicates with plenty of other projects. It starts, the journey starts with Fedora messaging broker. It actually sends a message that the new hotness wants to consume. In the case that this is the new version, it contacts MDAPI, it contacts PDC, or MDAPI is the metadata API which lets us check if the project is still active, if we should report this. PDC just looks if it's retired or what version is actually there. Then we contact the Pagur. We are hosting our packages sources there. We look at what's on the notification settings. It could be set on the Pagur page. If this all is correct, we have a new version. The project is still alive. We should report it. We contact Baxilla and create a ticket or update the ticket that was created before. In case the user wants to have even the scratch build, we will prepare it and send the Koji information that we want to build it. And then wait on the actual message from the Koji that everything went well. If this is a new mapping for the project in Anitya, we actually contact Anitya and just send the API request to do a check for the project. We need to check if there is a new version because the mapping was changed. The current situation is that Milestone wine20 is currently being pored on. The same as for Anitya, there are much more to be done in the case of new hotness because there are, for most of the things, I need the staging environment for testing because I need to communicate with plenty of external services. I need some testing environment. Right now we finished the Data Center move of Fedora, which was a really big project. And we are setting up the staging environment to have something to work on. So right now I'm blocked on the new hotness work. The Anitya, because it doesn't relies on anything external, it could be work on without issue. Same as Anitya, the new hotness is part of the HectorBerfest event. You could do some small changes. Still you can work on it. It's just that you can't test it with actual systems to connect to, but you could work on it and write a test. The new hotness and Anitya, it both are almost 100% test coverage, so it's nice to work with them. Okay, so how is this integrated to package workflow? This is package workflow for Fedora. It is not integrated to OpenCS package workflow. This is where the magic actually happens and this is how it looks. We have a package. Package requests are repository for new package. This is created in Pagur. He creates a project in Anitya and add new mapping. I would be glad to have this automated in the future, but right now it needs to be done manually by package. There is a script that between Pagur and Buxilla creates the component. It synchronizes the Pagur with Buxilla. It creates the component in the Buxilla, so you could actually have something to report to. Anitya, if it phones a new version, it reports it to the new hotness and new hotness is creating a ticket in Buxilla. In the L case, the package always has up-to-date information about new versions. There are exceptions like projects that are changing the version system, that are hard to guess which version is actually latest. In Terris change I am working on, that will allow Anitya to report not only latest, but everything it founds and the new hotness could actually notify the users. There is a new version that is not latest, but we found a new version, so if you are interested you could work on it. Let's go to the demo. Here is Anitya, this is how it looks. I am actually logged in, you can log in using the federal account or Terris option to use openID or I am not sure which option is the tiered one. I am using the federal account as you can see. As you can see there are more projects than I actually announced, because the number is growing. You can look at the docs, if you want a quick, this is the docs for the actual version, that is deployed. You can look at the projects here. The projects are just a list of the monitored projects. I can look at some of it. Here you can see the latest version that is reported. You can see the version list on the right side. You can see the status of the project. There was no new version found, but this is okay, we just didn't found anything new. You can see the homepage of the project. You can see the backend. The backend I will talk about this more later. The ecosystem is only visible for the admins. This is not something the user should be aware of, it's just for our own info. There is a default version scheme for every project, it's RPM, but if you allow to use Semantic and Calendar. Version check is done. The delete project is only for the admins. Below you can see the mappings. As a simple user you could edit or add new mappings. You can see that this project 0AD has mappings for Fedora, actually 3 mappings I see. 2 for Ubuntu, 2 for Debian, 2 for Magaya, 2 for Arch Linux and 2 for OpenMandriva. Here at the top you can see the flag button which is actually used to report anything. If you flag it you could just do the right array and submit it. As an admin you could look at all the flags that are reported on the Flux page. Currently we have only a few of them, so it's nice, it looks like most of them are solved. Here you can see the list of distribution. Distributions could be added by every user. We just changed the input method because before you could actually add a new distribution through the text field. We have plenty of typos, but we changed it to actual form. If the distribution is already there you could just choose it from the drop-down. The last thing you could do is to add the project. Here you can see what could be added. There is project name, homepage, backend, version scheme. Which could be chosen from 3 options. We have plenty of backend supported. Backend is actually the hosting service. You could look at it and see plenty of them are supported. If it isn't supported you could always use custom, but the custom is somewhat harder to actually set. Then you do a version pattern, which is only for the calendar versioning. The semantic just used the owner project. This one is changed based on the backend. The version prefix is used just to remove anything before the version. If the project is using for instance for example v version, then you could actually add the v to this field. And it will be removed from the sorting mechanism. You could add some mapping from the start. Here you can see the drop-down that you could choose the distribution from. And there you could write the name of the package. The check latest release on submit is when you submit it, it checks for the latest release. This is not needed, but most of the projects are checked until one hour. So it's not needed to do it this way. This will be changed in the future because I want to allow the user to actually test their settings. If the project could be found and everything before submitting it and adding it to the new project. Did I miss something? I don't think so. The last thing you could do is you can search for any project to the search field. It will just show you the projects that are found. At the bottom of the page you could see where the last check was done. And how many projects were checked. How many was okay. How many had some error. How many hit to rate limit. As you can see in the last check there was no rate limit hit. Which means that we didn't had any issue with the GitHub rate limiting. Okay, let's go back to the slides. Okay, here we have references. You can look at the release monitoring blog post. I'm trying to do our regular updates. It's written from a magic word. This is why I had to visit our heads on. There is URL for the release monitoring org. And the repositories URL. So you can look at the Aninja and the new hotness. If you are interested you can help. We are always happy to see any new contributor. So thank you for attending this. And hopefully I'm available in chat to answer any questions.
 Hello everyone and welcome to the talk about RISmointering.org.  My name is Michal Koneczny and I'm a maintainer of RISmointering.org for Fedora.  This is recording because I'm on the road right now and I'm not sure if I should be available in the chat,  but I'm not available to present.  Okay, so let's start.  Here is the agenda.  This is an illustration of the RISmointering.org  and how I imagine it when I'm writing my blog posts.  You can see it's a very nice word  and the new hotness is actually something above floating above the new hotness, above Anitya.  Okay, so let's start with some basics.  What is RISmointering.org?  It's containing from two applications.  The first one is Anitya.  It provides web interface for the users to actually do the things they need.  The users could add new projects, watch for the releases.  It automatically checks for new releases of the project  and it's sending Fedora message when a new version is retrieved.  The new hotness is Fedora messaging consumer.  For those of you not aware, Fedora messaging is a messaging bus we are using in Fedora  as basically for every message that is sent in infrastructure.  The new hotness is listening to messages emitted by Anitya.  It is only interested in the ones that are about the new release font and there are others.  It's great or updates Paxilla which is your issue tracker system when new release is found.  So the package who maintains the project, the package in Fedora is notified about new version.  It could start scratch builds in your build system if this is configured for the project.  Let's start with Anitya. I will talk about it a bit more because it's a really interesting project.  Let's start with some magic numbers.  These numbers are actually kind of impressive.  The first commit for the Anitya was on 28th November 2013.  First release was 0.1.0 which was on 29th September 2014.  So as you can see the first release was almost a year later after the project was created.  I'm the maintainer of the Anitya for two years so I don't have too much knowledge about the history of Anitya  but I at least know what it is.  Let's see at some contributions so you can see that the project is still alive.  There are almost 2,000 commits. We have slightly above 50 contributors.  There were almost half thousand issues created and we closed plenty of them.  As you can see from this we still have some open and we are trying to work to close them as soon as possible.  Current version of the Anitya is 0.18.0.  For just to make it more interesting.  The number of projects that are currently watched by Riesmoenturing.org are 126,435.  I think this number will be much higher because there are new projects added each day.  Just for this talk I have here the number of packages that are mapped to opensews distribution  and we have 331 packages up to opensews.  It is primarily used by Fedora so there are plenty more for Fedora.  Here you can see the diagram which represents how the Anitya is working.  As you can see there are two kind of users that could interact with it.  One is the regular user which could add new project, add new distribution to the Anitya,  edit project, flag project, flag is a mechanism that is used to mark the project that there is some issue.  Admin usually looks at it and investigates if this is really an issue and solves it and closes it.  He could add new mapping, mappings are the mappings between the project and the distribution  and the name of the package in the distribution.  The admin could do some advanced roles.  He could delete projects, delete mappings, delete distribution, he could edit distribution,  he could close flags that are solved.  He could also ban regular users and create admins from the regular users if needed.  So everybody who works with Anitya could become an admin, just say if you want and I could make it happen.  The Anitya has two other parts, there are parts of it but they are running as separate application.  The version checker which is being run regularly and it's checking every project that is there.  If there is new version.  There are some rules for the checking, it's not checking every project every run, there are exceptions.  The main exception is the GitHub because GitHub is limited by the number of requests you could do in one hour.  So in this case the new version is actually limited sometime but most of the projects, if there is new version,  we will know about it in a matter of hours.  There is also libraries.io consumer.  The libraries.io consumer is just a listener for the libraries.io messages.  And when it fonts new project that is interesting, is actually hosted on the platform we are interested in,  then it reports it to Anitya and creates a new project if this is configured or reports a new version.  If the new version is reported, the Anitya is sending the message to federal messaging broker.  The Anitya is sending message in almost every change that will happen in the application.  So if you want to see the history, you could use our data wrapper application which is actually only wrapper for the messages in the message broker.  And you could see what exactly was sent and what changed in the Anitya. Probably there should be a full history of Anitya.  So the current situation.  Current situation in Anitya is that we are working on the milestone 1.0 because Anitya is major enough.  I just stayed with the previous versioning system.  Here is a link to the actual milestone. You could see what needs to be done, what is done.  Most of the things for the 1.0 are done.  I just want to do a few more features and update the documentation to better state because there are plenty of things that are outdated.  Because we have October 1st this month, Anitya is part of it.  So if you want to help on the Anitya, there are issues that are marked, that are labeled October 1st.  So you could do it. It's a nice way to start contributing to it.  Next one is the new hotness.  The new hotness is floating island in the realm of magic. It's actually the point between Anitya and Fedora.  There is monitoring on the Fedora island.  Here are some statistics. The first comment was 13 March 2014.  First release was 0.1.2, 17 November 2014.  I'm not sure why the first release was in minor version because I'm just in the not minor version.  I didn't find any attack that will be older than this.  So it's possible that there was some areas just not attacked before.  There are 140 commits from 22 different distributors.  We have almost 150 issues created and we closed most of them.  Current version is 0.13.1.  I'm currently working as same as Anitya on 0.1.2.  But there are plenty of work. This will be released.  Maybe in the meantime I will release some bug fix release.  The new hotness is actually much more complex.  When you take in consideration the number of external applications it needs to communicate with.  It's easier as a script. It's much less code based than Anitya.  But it communicates with plenty of other projects.  It starts, the journey starts with Fedora messaging broker.  It actually sends a message that the new hotness wants to consume.  In the case that this is the new version, it contacts MDAPI, it contacts PDC,  or MDAPI is the metadata API which lets us check if the project is still active,  if we should report this. PDC just looks if it's retired or what version is actually there.  Then we contact the Pagur. We are hosting our packages sources there.  We look at what's on the notification settings. It could be set on the Pagur page.  If this all is correct, we have a new version. The project is still alive.  We should report it. We contact Baxilla and create a ticket or update the ticket that was created before.  In case the user wants to have even the scratch build, we will prepare it and send the Koji information that we want to build it.  And then wait on the actual message from the Koji that everything went well.  If this is a new mapping for the project in Anitya, we actually contact Anitya and just send the API request to do a check for the project.  We need to check if there is a new version because the mapping was changed.  The current situation is that Milestone wine20 is currently being pored on.  The same as for Anitya, there are much more to be done in the case of new hotness because there are, for most of the things,  I need the staging environment for testing because I need to communicate with plenty of external services.  I need some testing environment. Right now we finished the Data Center move of Fedora, which was a really big project.  And we are setting up the staging environment to have something to work on.  So right now I'm blocked on the new hotness work.  The Anitya, because it doesn't relies on anything external, it could be work on without issue.  Same as Anitya, the new hotness is part of the HectorBerfest event. You could do some small changes.  Still you can work on it. It's just that you can't test it with actual systems to connect to,  but you could work on it and write a test. The new hotness and Anitya, it both are almost 100% test coverage,  so it's nice to work with them.  Okay, so how is this integrated to package workflow?  This is package workflow for Fedora. It is not integrated to OpenCS package workflow.  This is where the magic actually happens and this is how it looks.  We have a package. Package requests are repository for new package.  This is created in Pagur. He creates a project in Anitya and add new mapping.  I would be glad to have this automated in the future, but right now it needs to be done manually by package.  There is a script that between Pagur and Buxilla creates the component.  It synchronizes the Pagur with Buxilla. It creates the component in the Buxilla,  so you could actually have something to report to.  Anitya, if it phones a new version, it reports it to the new hotness and new hotness is creating a ticket in Buxilla.  In the L case, the package always has up-to-date information about new versions.  There are exceptions like projects that are changing the version system,  that are hard to guess which version is actually latest.  In Terris change I am working on, that will allow Anitya to report not only latest,  but everything it founds and the new hotness could actually notify the users.  There is a new version that is not latest, but we found a new version,  so if you are interested you could work on it.  Let's go to the demo.  Here is Anitya, this is how it looks.  I am actually logged in, you can log in using the federal account  or Terris option to use openID or I am not sure which option is the tiered one.  I am using the federal account as you can see.  As you can see there are more projects than I actually announced,  because the number is growing.  You can look at the docs, if you want a quick, this is the docs for the actual version,  that is deployed.  You can look at the projects here.  The projects are just a list of the monitored projects.  I can look at some of it.  Here you can see the latest version that is reported.  You can see the version list on the right side.  You can see the status of the project.  There was no new version found, but this is okay, we just didn't found anything new.  You can see the homepage of the project.  You can see the backend.  The backend I will talk about this more later.  The ecosystem is only visible for the admins.  This is not something the user should be aware of, it's just for our own info.  There is a default version scheme for every project, it's RPM,  but if you allow to use Semantic and Calendar.  Version check is done.  The delete project is only for the admins.  Below you can see the mappings.  As a simple user you could edit or add new mappings.  You can see that this project 0AD has mappings for Fedora,  actually 3 mappings I see.  2 for Ubuntu, 2 for Debian, 2 for Magaya, 2 for Arch Linux and 2 for OpenMandriva.  Here at the top you can see the flag button which is actually used to report anything.  If you flag it you could just do the right array and submit it.  As an admin you could look at all the flags that are reported on the Flux page.  Currently we have only a few of them, so it's nice, it looks like most of them are solved.  Here you can see the list of distribution.  Distributions could be added by every user.  We just changed the input method because before you could actually add a new distribution through the text field.  We have plenty of typos, but we changed it to actual form.  If the distribution is already there you could just choose it from the drop-down.  The last thing you could do is to add the project.  Here you can see what could be added.  There is project name, homepage, backend, version scheme.  Which could be chosen from 3 options.  We have plenty of backend supported.  Backend is actually the hosting service.  You could look at it and see plenty of them are supported.  If it isn't supported you could always use custom, but the custom is somewhat harder to actually set.  Then you do a version pattern, which is only for the calendar versioning.  The semantic just used the owner project.  This one is changed based on the backend.  The version prefix is used just to remove anything before the version.  If the project is using for instance for example v version, then you could actually add the v to this field.  And it will be removed from the sorting mechanism.  You could add some mapping from the start.  Here you can see the drop-down that you could choose the distribution from.  And there you could write the name of the package.  The check latest release on submit is when you submit it, it checks for the latest release.  This is not needed, but most of the projects are checked until one hour.  So it's not needed to do it this way.  This will be changed in the future because I want to allow the user to actually test their settings.  If the project could be found and everything before submitting it and adding it to the new project.  Did I miss something? I don't think so.  The last thing you could do is you can search for any project to the search field.  It will just show you the projects that are found.  At the bottom of the page you could see where the last check was done.  And how many projects were checked.  How many was okay. How many had some error. How many hit to rate limit.  As you can see in the last check there was no rate limit hit.  Which means that we didn't had any issue with the GitHub rate limiting.  Okay, let's go back to the slides.  Okay, here we have references.  You can look at the release monitoring blog post.  I'm trying to do our regular updates.  It's written from a magic word. This is why I had to visit our heads on.  There is URL for the release monitoring org.  And the repositories URL.  So you can look at the Aninja and the new hotness.  If you are interested you can help.  We are always happy to see any new contributor.  So thank you for attending this.  And hopefully I'm available in chat to answer any questions.
Release_monitoring_org_72x_RLmgx48_0_887.png Release_monitoring_org_72x_RLmgx48_888_959.png Release_monitoring_org_72x_RLmgx48_960_1619.png Release_monitoring_org_72x_RLmgx48_1620_4007.png Release_monitoring_org_72x_RLmgx48_4008_4271.png Release_monitoring_org_72x_RLmgx48_4272_5231.png Release_monitoring_org_72x_RLmgx48_5232_5423.png Release_monitoring_org_72x_RLmgx48_5424_8111.png Release_monitoring_org_72x_RLmgx48_8112_14255.png Release_monitoring_org_72x_RLmgx48_14256_16715.png Release_monitoring_org_72x_RLmgx48_16716_18683.png Release_monitoring_org_72x_RLmgx48_18684_21215.png Release_monitoring_org_72x_RLmgx48_21216_23039.png Release_monitoring_org_72x_RLmgx48_23040_25343.png Release_monitoring_org_72x_RLmgx48_25344_25415.png Release_monitoring_org_72x_RLmgx48_25416_25715.png Release_monitoring_org_72x_RLmgx48_25716_26171.png Release_monitoring_org_72x_RLmgx48_26172_29171.png Release_monitoring_org_72x_RLmgx48_29172_29231.png Release_monitoring_org_72x_RLmgx48_29232_30611.png Release_monitoring_org_72x_RLmgx48_30612_30779.png Release_monitoring_org_72x_RLmgx48_30780_31031.png Release_monitoring_org_72x_RLmgx48_31032_33827.png Release_monitoring_org_72x_RLmgx48_33828_34127.png Release_monitoring_org_72x_RLmgx48_34128_34223.png Release_monitoring_org_72x_RLmgx48_34224_34655.png Release_monitoring_org_72x_RLmgx48_34656_34823.png Release_monitoring_org_72x_RLmgx48_34824_34907.png Release_monitoring_org_72x_RLmgx48_34908_35135.png Release_monitoring_org_72x_RLmgx48_35136_35183.png Release_monitoring_org_72x_RLmgx48_35184_35291.png Release_monitoring_org_72x_RLmgx48_35292_35399.png Release_monitoring_org_72x_RLmgx48_35400_35675.png Release_monitoring_org_72x_RLmgx48_35676_35723.png Release_monitoring_org_72x_RLmgx48_35724_40295.png Release_monitoring_org_72x_RLmgx48_40296_41099.png Release_monitoring_org_72x_RLmgx48_41100_41171.png Release_monitoring_org_72x_RLmgx48_41172_41243.png Release_monitoring_org_72x_RLmgx48_41244_42215.png Release_monitoring_org_72x_RLmgx48_42216_42536.png
Hello everyone and welcome to the talk about RISmointering.org. My name is Michal Koneczny and I'm a maintainer of RISmointering.org for Fedora. This is recording because I'm on the road right now and I'm not sure if I should be available in the chat, but I'm not available to present. Okay, so let's start. Here is the agenda. This is an illustration of the RISmointering.org and how I imagine it when I'm writing my blog posts. You can see it's a very nice word and the new hotness is actually something above floating above the new hotness, above Anitya. Okay, so let's start with some basics. What is RISmointering.org? It's containing from two applications. The first one is Anitya. It provides web interface for the users to actually do the things they need. The users could add new projects, watch for the releases. It automatically checks for new releases of the project and it's sending Fedora message when a new version is retrieved. The new hotness is Fedora messaging consumer. For those of you not aware, Fedora messaging is a messaging bus we are using in Fedora as basically for every message that is sent in infrastructure. The new hotness is listening to messages emitted by Anitya. It is only interested in the ones that are about the new release font and there are others. It's great or updates Paxilla which is your issue tracker system when new release is found. So the package who maintains the project, the package in Fedora is notified about new version. It could start scratch builds in your build system if this is configured for the project. Let's start with Anitya. I will talk about it a bit more because it's a really interesting project. Let's start with some magic numbers. These numbers are actually kind of impressive. The first commit for the Anitya was on 28th November 2013. First release was 0.1.0 which was on 29th September 2014. So as you can see the first release was almost a year later after the project was created. I'm the maintainer of the Anitya for two years so I don't have too much knowledge about the history of Anitya but I at least know what it is. Let's see at some contributions so you can see that the project is still alive. There are almost 2,000 commits. We have slightly above 50 contributors. There were almost half thousand issues created and we closed plenty of them. As you can see from this we still have some open and we are trying to work to close them as soon as possible. Current version of the Anitya is 0.18.0. For just to make it more interesting. The number of projects that are currently watched by Riesmoenturing.org are 126,435. I think this number will be much higher because there are new projects added each day. Just for this talk I have here the number of packages that are mapped to opensews distribution and we have 331 packages up to opensews. It is primarily used by Fedora so there are plenty more for Fedora. Here you can see the diagram which represents how the Anitya is working. As you can see there are two kind of users that could interact with it. One is the regular user which could add new project, add new distribution to the Anitya, edit project, flag project, flag is a mechanism that is used to mark the project that there is some issue. Admin usually looks at it and investigates if this is really an issue and solves it and closes it. He could add new mapping, mappings are the mappings between the project and the distribution and the name of the package in the distribution. The admin could do some advanced roles. He could delete projects, delete mappings, delete distribution, he could edit distribution, he could close flags that are solved. He could also ban regular users and create admins from the regular users if needed. So everybody who works with Anitya could become an admin, just say if you want and I could make it happen. The Anitya has two other parts, there are parts of it but they are running as separate application. The version checker which is being run regularly and it's checking every project that is there. If there is new version. There are some rules for the checking, it's not checking every project every run, there are exceptions. The main exception is the GitHub because GitHub is limited by the number of requests you could do in one hour. So in this case the new version is actually limited sometime but most of the projects, if there is new version, we will know about it in a matter of hours. There is also libraries.io consumer. The libraries.io consumer is just a listener for the libraries.io messages. And when it fonts new project that is interesting, is actually hosted on the platform we are interested in, then it reports it to Anitya and creates a new project if this is configured or reports a new version. If the new version is reported, the Anitya is sending the message to federal messaging broker. The Anitya is sending message in almost every change that will happen in the application. So if you want to see the history, you could use our data wrapper application which is actually only wrapper for the messages in the message broker. And you could see what exactly was sent and what changed in the Anitya. Probably there should be a full history of Anitya. So the current situation. Current situation in Anitya is that we are working on the milestone 1.0 because Anitya is major enough. I just stayed with the previous versioning system. Here is a link to the actual milestone. You could see what needs to be done, what is done. Most of the things for the 1.0 are done. I just want to do a few more features and update the documentation to better state because there are plenty of things that are outdated. Because we have October 1st this month, Anitya is part of it. So if you want to help on the Anitya, there are issues that are marked, that are labeled October 1st. So you could do it. It's a nice way to start contributing to it. Next one is the new hotness. The new hotness is floating island in the realm of magic. It's actually the point between Anitya and Fedora. There is monitoring on the Fedora island. Here are some statistics. The first comment was 13 March 2014. First release was 0.1.2, 17 November 2014. I'm not sure why the first release was in minor version because I'm just in the not minor version. I didn't find any attack that will be older than this. So it's possible that there was some areas just not attacked before. There are 140 commits from 22 different distributors. We have almost 150 issues created and we closed most of them. Current version is 0.13.1. I'm currently working as same as Anitya on 0.1.2. But there are plenty of work. This will be released. Maybe in the meantime I will release some bug fix release. The new hotness is actually much more complex. When you take in consideration the number of external applications it needs to communicate with. It's easier as a script. It's much less code based than Anitya. But it communicates with plenty of other projects. It starts, the journey starts with Fedora messaging broker. It actually sends a message that the new hotness wants to consume. In the case that this is the new version, it contacts MDAPI, it contacts PDC, or MDAPI is the metadata API which lets us check if the project is still active, if we should report this. PDC just looks if it's retired or what version is actually there. Then we contact the Pagur. We are hosting our packages sources there. We look at what's on the notification settings. It could be set on the Pagur page. If this all is correct, we have a new version. The project is still alive. We should report it. We contact Baxilla and create a ticket or update the ticket that was created before. In case the user wants to have even the scratch build, we will prepare it and send the Koji information that we want to build it. And then wait on the actual message from the Koji that everything went well. If this is a new mapping for the project in Anitya, we actually contact Anitya and just send the API request to do a check for the project. We need to check if there is a new version because the mapping was changed. The current situation is that Milestone wine20 is currently being pored on. The same as for Anitya, there are much more to be done in the case of new hotness because there are, for most of the things, I need the staging environment for testing because I need to communicate with plenty of external services. I need some testing environment. Right now we finished the Data Center move of Fedora, which was a really big project. And we are setting up the staging environment to have something to work on. So right now I'm blocked on the new hotness work. The Anitya, because it doesn't relies on anything external, it could be work on without issue. Same as Anitya, the new hotness is part of the HectorBerfest event. You could do some small changes. Still you can work on it. It's just that you can't test it with actual systems to connect to, but you could work on it and write a test. The new hotness and Anitya, it both are almost 100% test coverage, so it's nice to work with them. Okay, so how is this integrated to package workflow? This is package workflow for Fedora. It is not integrated to OpenCS package workflow. This is where the magic actually happens and this is how it looks. We have a package. Package requests are repository for new package. This is created in Pagur. He creates a project in Anitya and add new mapping. I would be glad to have this automated in the future, but right now it needs to be done manually by package. There is a script that between Pagur and Buxilla creates the component. It synchronizes the Pagur with Buxilla. It creates the component in the Buxilla, so you could actually have something to report to. Anitya, if it phones a new version, it reports it to the new hotness and new hotness is creating a ticket in Buxilla. In the L case, the package always has up-to-date information about new versions. There are exceptions like projects that are changing the version system, that are hard to guess which version is actually latest. In Terris change I am working on, that will allow Anitya to report not only latest, but everything it founds and the new hotness could actually notify the users. There is a new version that is not latest, but we found a new version, so if you are interested you could work on it. Let's go to the demo. Here is Anitya, this is how it looks. I am actually logged in, you can log in using the federal account or Terris option to use openID or I am not sure which option is the tiered one. I am using the federal account as you can see. As you can see there are more projects than I actually announced, because the number is growing. You can look at the docs, if you want a quick, this is the docs for the actual version, that is deployed. You can look at the projects here. The projects are just a list of the monitored projects. I can look at some of it. Here you can see the latest version that is reported. You can see the version list on the right side. You can see the status of the project. There was no new version found, but this is okay, we just didn't found anything new. You can see the homepage of the project. You can see the backend. The backend I will talk about this more later. The ecosystem is only visible for the admins. This is not something the user should be aware of, it's just for our own info. There is a default version scheme for every project, it's RPM, but if you allow to use Semantic and Calendar. Version check is done. The delete project is only for the admins. Below you can see the mappings. As a simple user you could edit or add new mappings. You can see that this project 0AD has mappings for Fedora, actually 3 mappings I see. 2 for Ubuntu, 2 for Debian, 2 for Magaya, 2 for Arch Linux and 2 for OpenMandriva. Here at the top you can see the flag button which is actually used to report anything. If you flag it you could just do the right array and submit it. As an admin you could look at all the flags that are reported on the Flux page. Currently we have only a few of them, so it's nice, it looks like most of them are solved. Here you can see the list of distribution. Distributions could be added by every user. We just changed the input method because before you could actually add a new distribution through the text field. We have plenty of typos, but we changed it to actual form. If the distribution is already there you could just choose it from the drop-down. The last thing you could do is to add the project. Here you can see what could be added. There is project name, homepage, backend, version scheme. Which could be chosen from 3 options. We have plenty of backend supported. Backend is actually the hosting service. You could look at it and see plenty of them are supported. If it isn't supported you could always use custom, but the custom is somewhat harder to actually set. Then you do a version pattern, which is only for the calendar versioning. The semantic just used the owner project. This one is changed based on the backend. The version prefix is used just to remove anything before the version. If the project is using for instance for example v version, then you could actually add the v to this field. And it will be removed from the sorting mechanism. You could add some mapping from the start. Here you can see the drop-down that you could choose the distribution from. And there you could write the name of the package. The check latest release on submit is when you submit it, it checks for the latest release. This is not needed, but most of the projects are checked until one hour. So it's not needed to do it this way. This will be changed in the future because I want to allow the user to actually test their settings. If the project could be found and everything before submitting it and adding it to the new project. Did I miss something? I don't think so. The last thing you could do is you can search for any project to the search field. It will just show you the projects that are found. At the bottom of the page you could see where the last check was done. And how many projects were checked. How many was okay. How many had some error. How many hit to rate limit. As you can see in the last check there was no rate limit hit. Which means that we didn't had any issue with the GitHub rate limiting. Okay, let's go back to the slides. Okay, here we have references. You can look at the release monitoring blog post. I'm trying to do our regular updates. It's written from a magic word. This is why I had to visit our heads on. There is URL for the release monitoring org. And the repositories URL. So you can look at the Aninja and the new hotness. If you are interested you can help. We are always happy to see any new contributor. So thank you for attending this. And hopefully I'm available in chat to answer any questions.
releases. It automatically checks for new releases of the project and it's sending Fedora the package in Fedora is notified about new as Anitya, the new hotness is part of the workflow? This is package workflow for Fedora. It new versions. There are exceptions like projects the demo. Here is Anitya, this is how it looks. I will just show you the projects that are found.