Managing public data on GitHub: Pay no attention to that git behind the curtain
Reed, Landon
The Atlanta Regional Commission (ARC) continuously solicits feedback on transportation data from local government partners. Historically, this process has taken the form of lots of markings on plotted maps with immeasurable amounts of manual work on the tail end to organize and interpret this feedback. Many tools developed specifically for this process today often fall short of the needs of agencies (such as geospatial presentation and tracking comments), yet the cost to develop or implement custom software is generally out of reach for government agencies.This presentation introduces a case study of the process to develop geospatial collaboration tools for managing transportation data directly hosted on GitHub pages (currently in development at-it/ and-review/). This approach was partially inspired by GitHub's recent features additions that make collaborating on geospatial data simple and elegant. Because these data span both functional and jurisdictional divisions, many of the greatest challenges have been project management related - coordinating stakeholder feedback and project requirements. However, by utilizing the existing git/GitHub infrastructure, many of these requirements can be managed cost effectively. Moreover, the framework allows for direct integration with other application environments via the GitHub API and GDAL Tools, ensuring that local modifications to project data are committed back to the data repository.
 So, if you were here for the presentation just previous to mine, you're in luck because he, Aaron laid a great foundation for my talk. It's actually dovetails perfectly. So, that, I don't know, we weren't colluding. Maybe it's just the great organizers at PhosphorG, I don't know. But my presentation is on managing public data on GitHub or paying no attention to that get behind the curtain. So, it's kind of about, you know, GitHub and government. And sometimes, well, you know, if you read Vlad's talk from Leiflet earlier today, he talked a lot about experimentation and creativity. And I think that's something that is really needed in the government sector and that's something that Code for America is really stepping up and helping out with. But I think there's always room for that. And so, this presentation is really kind of looking at one case where I tried to experiment a little bit with GitHub and managing this process. And it went pretty well. Nothing exploded. So the context. I work at, oh, wow. That's not good. Okay. There we go. I work at the Atlanta Regional Commission which is regional government. So basically, we build consensus with all the smaller local governments, counties and cities. And we have a lot of counties in Georgia. We're notorious for that. So there's a lot of partners to manage relationships with. And so the story behind this project was each census. So every 10 years, the urbanized areas are redrawn according to population change. So people move from Atlanta to San Francisco or, you know, all these different places and they have to redraw the areas. And along with that, the Federal Highway Association asks for updated roadway functional classifications. And if you're familiar with OpenStreetMap and OSM speak, functional classifications are just, you know, the highway tag and then any of those motorway, trunk, primary, secondary, residential. But in FHWA speak, we have things like interstates, other freeways, principal materials, these other classifications that basically mean the same thing. And there's some funding tied to these classifications, but it's really just a way for the feds to keep track of all the roads that exist out there. And so the problem that we were facing was at the beginning of this process, we were like, okay, how are we going to manage all these stakeholders, all these local governments? You know, we have maps we can print out and people can draw on them, get around the table and kind of do a collaborative thing, which is awesome. We have forms that people could fill out and we can send people, well, Microsoft Word docs and have them send them back or scan them. And then we'd have a pile of paper to go through and then basically have an analyst go in and tag all the, you know, try to figure out which road they're talking about, find it on the map, tag it, change all the functional classifications, really kind of a daunting task. And, you know, so it's this process that we didn't really want to manage. And when I was in the meeting, when they talked about it, I was like, there's got to be a better way to do it. But I had to kind of demonstrate that. So we like that first part, but we don't like all the forms and having unstructured data and turning that into data structures. And so, you know, this is really kind of a huge process. It starts at the Federal Highway Administration, which is obviously national in scope, communicating with all the DOTs, the state DOTs, which GDOT is the one that we kind of work with. And then GDOT communicates with all the regions in the state and then all the regions kind of coordinate locally. So it's this huge process. And there's no telling how many different ways people are doing it around the country. So, you know, why not, like I mentioned OSM tags earlier, why not just use OSM? Well, there are great tools for editing OpenStreetMap. And that was one of the, you know, we were really considering doing that. Or I was in my head. But the kind of motivations for not using it outweighed the great reasons for using it. So basically, we have this DOT road network that we really wanted to kind of work with and attach all those new attributes to. We wanted to have build out a custom interface pretty easily. We wanted to track changes with the functional classification numbering system. And we wanted to manage this whole approval process so that, you know, it kind of filter up the hierarchy. And, you know, we first tried to kind of figure out what happened with the last census 10 years ago. But because this process happens every 10 years, you can imagine that the tools are going to be different, the people are going to be different, people probably just forgot how it was done 10 years ago. We don't really know. But this form was kind of proposed as a way to actually manage the process. And I don't know about you, but I would not want to have a pile, like a stack of these forms and kind of going back and forth between a road network and doing all this manual entry. So, you know, we, GitHub was kind of what we used. So why do we use it? So Aaron, you know, talked a lot about some great tools that are out there that I think kind of were inspired by GitHub kind of starting to render GeoJSON. We have GeoJSON.io, which is an amazing editor for, you know, pulling up GeoJSON or other shape data and then really easily editing it. And that's all run on GitHub Pages, this great free hosting that you can do. So GitHub Pages is really the curtain in this talk, right? We're kind of keeping GitHub behind this curtain that is GitHub Pages. It's just a web app that's running on GitHub and taking advantage of the GitHub API. And so I said it's really crucial in Gov because sometimes when you're trying to convey an idea, just using words isn't really enough. And by really quickly getting a demo up onto GitHub Pages, you can convey a message like, this is something that works and, you know, we could possibly use it in this functional classification overhaul. And then issue tracking is another great thing that we want to take advantage of. We have to coordinate with all these local governments and we wanted them all to be aware of what changes other people were proposing. And so getting that integrated communications with the tracking, issue tracking, is really important. And we wanted to possibly share this with other agencies. I mean, that's why I'm here today. That's, it's all open source code. And there's this fork and go mentality which Jessica Lord kind of seemed to have come up with, which is basically, you know, you put something up on GitHub and then GitHub really easily allows you to fork projects and make your own copy to kind of customize for your own situation. So this is a continuum that I created. Just further justification for why we needed this curtain around Git, which is kind of scary, right? You know, on the left-hand side we have familiarity with Git and on the right-hand side we have, you know, your subsequent kind of relationship to it. So the first, at the very top we have what's a version control system, which, you know, I was at about a year and a half ago. We have novice and expert. And so it's kind of like, I guess, stages of grief or something. You have kind of indifference. And then, so you don't really care about Git. Why do I, as a person who can live my life normally, why do I need Git? And confusion, like, I don't really understand what's happening. Fear, panic, discomfort, reliance, and, you know, you ultimately grow to really depend on Git. So I'm kind of in this panic, discomfort stage probably. You know, sometimes I waver depending on what I've done wrong as far as committing data to Git. The originator of Git is probably at the expert level. But the majority of the target audience for this project is at the indifference stage. And if you started talking about Git, it would lead to confusion most likely. And so there's a lot of battles that you kind of have to overcome to get people comfortable with this thing that is really unfamiliar. And so, you know, that's kind of, you know, and there's been a lot of work by GitHub especially to kind of make Git permeate other areas besides just software development. I think that's amazing. But I don't think we're quite to the point where your average Joe is going to recognize what Git is. So the workflow for this project, so we had this G.road network that we wanted to submit changes to. So we got that from them in a shape file version. We did a ogre to ogre conversion to GeoJSON and did some, you know, reprojection. And we, there's a gentleman earlier in the session who asked about file size constraints for GitHub. And so we actually had to split the, we had to drop a lot of roads from the dataset. And then we had to split it by county to keep that file size nice and small because these are huge road networks that we're talking about. And then we load that up onto a GitHub pages web app which I'll show in a second. And then we had government staff, you know, view the road segments and see what the existing conditions are and then submit change requests for given road segments in their jurisdiction. And so, you know, the other reason we split up the GeoJSON road network into counties is because we're dealing with multiple jurisdictions. And there was a lot of concern by, you know, say, for roads that are on the county boundary or that cross multiple jurisdictions. We wanted, you know, our partners to feel comfortable that they were submitting changes only for their jurisdiction and also kind of, you know, just kind of keep that collaboration alive. And so one of the things we had to do was split up the GeoJSON into these counties. So it was kind of a multi-purpose reason for that. And we actually, you'll see this is a screen capture from GitHub and it shows a few of the different counties. We basically created teams that correlated to counties. And so all of the cities or counties that corresponded to a given county, we would assign them to that team. And that's how they kind of had access to the data and then could submit change requests. And so without further ado, I'll just kind of open up this application. So it should be showing up, yeah. So this is the application we built. It's, you can see it's running on this atlregional.github.io. That's the GitHub pages domain. And basically we have a map over here. And it's got map box tiles on underneath it. And you can see this key, this legend is kind of janky because the screen size is off. But we can, I built some basic tile layers to, in tile mail to help people kind of see what was out there. So this is like, you know, really kind of messy, but when you zoom in it looks a little nicer. But it's just showing all the different classifications we have. We have like, you know, city boundaries just to people, so people know where they're working from. We have urbanized area that's not very helpful, but people requested it. And so when you log in, as I'm going to do right now, you're basically assigned to a county or collection of counties if the jurisdiction you're in spans that. So these are Fulton and CAB counties. And that's the data I have access to. And in reality, if I go to the GitHub page and I want to, you know, create a pull request for any of the data that's on that tool or on this repository, I can do that. But our users are not about to open up a command line, interface, and type, get commit or get add and all that stuff. So this is really their way to interact with that data. And so you can click on road segments. And this is just showing kind of general leaflet functionality. And when you want to suggest a change for that road segment, this form pops up where, you know, you give the name of road because that's not really an attribute that the DOT maintains. You, you know, say you want the entire segment that's highlighted or just two and from intersections. You can see the existing functional class. You can see if it has volumes, so like traffic counts for the road, you can see that. And then you can kind of propose a new functional class. And so we also have kind of a list of all the proposed change that has been created by using that same form. And so you can see that this is just kind of like a data table that's searchable, tried to make it as user-friendly as possible. But these are basically just, if you're familiar with GitHub, they're issues on GitHub that we're just using the API to pull all that information, kind of format it in a way that's familiar to people. And when you view it, it loads that road segment, kind of gives all the description and justification that the user submitted. And so it's really just kind of a wrapper around GitHub focused on this particular application. So I'm going to hop back to the presentation here. Here we go. So that's the app. You might have seen when I pulled up one of those issue boxes, this, you know, change status, all these little buttons right here. And that was just really taking advantage of the milestones feature on GitHub. You can set milestones for different, you know, issues and if they're like bug fixes or feature requests or, you know, this issue is going to be in version 1.0, we just re-adapted that for are these road requests, are they approved, are they advancing or in review? So we're kind of adapting the GitHub interface for our own purposes. And then we also built this another tool that allows you to basically compare the existing conditions, which is the map on the left, with all of the different categories of issues. And so right here I think that's, I can't really read, I think that's advancing. So it shows all the road segments that are advancing. And you can do the same kinds of tile overlays to kind of get a picture of what the whole network looks like region wide. So we also wanted to pull data down because inevitably there's going to be some work on the back end to, you know, figure out which requests get approved, which get, you know, kind of push down the line. So we have an export button that just exports all that issues into a CSV. And also if you wanted to actually work with it in like ArcMap, which our agency has, because it's not in a GeoJSON format, you kind of have to, you know, use desktop GitHub or Git pull and pull that down and then re-project it into or change the format into a shape file. And so this is really the project timeline. We're at the tail end of it. So on the left were the y-axis we have the number of change requests and on the x-axis we have date. So you can see the first phase of the project was really training. And that was pretty much for the month of May. So we had a few requests put in in that early stage. And then the bulk of the requests actually came at the very end of the submission period. Surprise, right? Like you send a reminder email and a flood of issues get pulled that very same day. So, you know, at the max we had 257 in a single day. That was like the Thursday before the Friday deadline. And then you see a gap in the weekend, right? And then the review period is kind of what Sage is wearing right now. So it's kind of, you know, reviewing the whole network, all the changes that people have made and figuring out what makes sense, you know, what doesn't. Like is this like local residential road actually need to be upgraded to an interstate? No. It's actually a local road with speed bumps. It's not an interstate. So we're in that review process right now. So, yeah, deadline on Friday. And so the total change request we had was about a thousand, which if we had managed that through a stack of paper forms and trying to convert that all back into spatial data, that would have been really challenging. And this is just a summary of change requests. It's kind of hard to look at. But basically, it shows, you know, the approximate number of changes requested. And we actually, I kind of put this here because we had 20 submissions that were indicating no proposed functional classification. So they just didn't fill out the form all the way. And really that ended up being just a form validation. We weren't doing checking, like, making that a mandatory field. And so we had some errors as a result of that. And the other thing to point out here is we had a huge number of local roads that were proposed to be upgraded. And, you know, from a data standpoint, that's fine. We were like, okay, that's, we'll just manage that process. There's a lot of changes, but that's okay. But actually, we found out later from the DOT that that actually means that there's going to be a lot of additional data collection on their part because they have to go out and, like, you know, figure out what the roadway design is like and do traffic counts. And so it comes with a lot of funding questions and a lot of challenges, too. So the kind of overall challenges, one of the big ones was source data. We had used this data set from the DOT, but it was unfamiliar to us. Some of the segment divisions were really strange. So it was a lot of figuring out what we're actually working with. And I mentioned earlier that we had to pull a lot of roads out of the network in order for GitHub to serve it properly. That's because about 80 percent, somewhere around that, maybe a little less of the entire network are local roads. And so we actually had to pull all that out. And then we ended up just having kind of the next classification up all the way up to Interstate because we didn't want to deal with the question of all those local roads. And so we would have people kind of send us into, like, batch requests to add in local roads, which ended up being a lot of manual work. Users, a lot of these users, they're all government staff and a lot of their agencies are still running on, like, you know, pretty old Windows XP Internet Explorer. And so, you know, trying to get people to download the latest browsers to make this application work was really challenging. Sometimes we did have to show the bones of GitHub, especially when people wanted to make comments. And so that would occasionally confuse people. And also the procrastination issue, I mentioned the deadline of all the change requests coming in. That was, that posed some personnel issues, like trying to manage that whole process and instructing people how to open the application. And, you know, really the volume of change requests continues to be a challenge because we have these additional data collection efforts that are acquired by the DOT and also just kind of figuring out what makes sense from a network perspective, like going through all those, even in a GIS format is challenging. So moving forward, we're continuing to process these change requests. And really, really the goal is to make a cohesive network. Something that makes sense and reflects actual travel patterns. And we're considering the framework for other data sets. For instance, the Transportation Improvement Program, which kind of manages a list of projects that are in the hopper for the, like, five-year period. And, you know, we'd love to hear ideas from others. I think hearing what Aaron is working on is amazing and hopefully others are kind of experimenting with this stuff. But I think that's kind of the crucial theme, experimenting. And, you know, it worked for our process. It might not work for other processes, but we're always curious to hear from others. And this is at the Oregon border. Awesome state to be in. I'm really happy to be here, but happy to take any questions right now. There. Great talk. I was just curious. I'm a little fuzzy on how you're getting the information from GitHub back to the GIS. Like, are all those attributes coming over automatically or are the edits being made in the GIS? And this is just tracking the... Yeah. So basically, we used GitHub's API to read the data from GitHub pages and then to write back to it. So in the GitHub pages branch, we were writing data to that file. And then to get that back into like a GIS environment, we would just pull that, all that data down and convert it into a shape file and then you can kind of work within an ARC map. Yeah. Yeah. We're just kind of like adding attributes to the GeoJSON, all those polylines. Actually, hopefully it helped answer that question. So we actually built... So I worked for Esri. We actually built an open source library called Coup, K-O-O-P. Have you seen it? It's... I have. It hopefully solves this problem. The idea was it's no JS. You point it at like a GitHub repository and it can serve it out as feature services or shape file. So someone can go and live an ARC map and pull data from GitHub or vice versa. If someone has a server, they want to use GeoJSON. You can deploy that as well. So that might help solve that. Keep the data in GitHub, but I want to pull it out into a GIS or other formats and I have to download it and convert it by hand and things like that. Yeah. That was one of the problems we had because everybody has ARC map, but kind of getting GitHub installed on their machines and QGIS, there's a lot of overhead with that, like going through IT and all that. So that's good to know. In your timeline, it looked like the whole review process was just a couple months. How was... How long was the whole process of getting the infrastructure set up for this? So look back at my commit history, but it kind of got started. Like we were kind of informed by GDOT that we needed to get started around, I think, like February around then. And then so I was like, hey, I think I have a solution for this. And everybody was like, well, let's see it. So it was kind of a rapid development cycle. And then so it kind of, I think, started around like end of February and wrapped up in May. So, yeah. Have you gotten interest in this type of project from other city governments or other government organizations? So GDOT was... They were actually really interested in the process. They're not really at liberty to kind of impose any process on other regions in the state. But I think in the future, they had interest in possibly using this or some iteration of it. We haven't really reached out to other MPOs around the country, but I think that's kind of the next step. And it's not really clear what the best form is for that. I am speaking at the Association of MPOs in like October. So I think that's really going to be our chance to share this more broadly with that community. But, yeah. So you mentioned used Mapbox for the map that we saw and the data that we saw inside of the GitHub application. Was that your choice or does GitHub integrate directly with Mapbox in some way? That was our choice because we really wanted the kind of tile overlays in there. And the easiest way to get that up was, for us, was through Tile Mill and then to load those up to Mapbox. So there's not any like direct and because like the code on GitHub pages is just, you know, HTML, JavaScript thrown on there. And so it was just easiest for us to get up and running. I would say thank you for this time. Thanks.
 So, if you were here for the presentation just previous to mine, you're in luck because  he, Aaron laid a great foundation for my talk. It's actually dovetails perfectly. So, that,  I don't know, we weren't colluding. Maybe it's just the great organizers at PhosphorG,  I don't know. But my presentation is on managing public data on GitHub or paying no attention  to that get behind the curtain. So, it's kind of about, you know, GitHub and government.  And sometimes, well, you know, if you read Vlad's talk from Leiflet earlier today, he  talked a lot about experimentation and creativity. And I think that's something that is really  needed in the government sector and that's something that Code for America is really  stepping up and helping out with. But I think there's always room for that. And so, this  presentation is really kind of looking at one case where I tried to experiment a little  bit with GitHub and managing this process. And it went pretty well. Nothing exploded.  So the context. I work at, oh, wow. That's not good. Okay. There we go. I work at the  Atlanta Regional Commission which is regional government. So basically, we build consensus  with all the smaller local governments, counties and cities. And we have a lot of counties  in Georgia. We're notorious for that. So there's a lot of partners to manage relationships  with. And so the story behind this project was each census. So every 10 years, the urbanized  areas are redrawn according to population change. So people move from Atlanta to San  Francisco or, you know, all these different places and they have to redraw the areas.  And along with that, the Federal Highway Association asks for updated roadway functional  classifications. And if you're familiar with OpenStreetMap and OSM speak, functional classifications  are just, you know, the highway tag and then any of those motorway, trunk, primary, secondary,  residential. But in FHWA speak, we have things like interstates, other freeways, principal  materials, these other classifications that basically mean the same thing. And there's  some funding tied to these classifications, but it's really just a way for the feds to  keep track of all the roads that exist out there. And so the problem that we were facing  was at the beginning of this process, we were like, okay, how are we going to manage all  these stakeholders, all these local governments? You know, we have maps we can print out and  people can draw on them, get around the table and kind of do a collaborative thing, which  is awesome. We have forms that people could fill out and we can send people, well, Microsoft  Word docs and have them send them back or scan them. And then we'd have a pile of paper  to go through and then basically have an analyst go in and tag all the, you know, try to figure  out which road they're talking about, find it on the map, tag it, change all the functional  classifications, really kind of a daunting task. And, you know, so it's this process  that we didn't really want to manage. And when I was in the meeting, when they talked  about it, I was like, there's got to be a better way to do it. But I had to kind of  demonstrate that. So we like that first part, but we don't like all the forms and having  unstructured data and turning that into data structures. And so, you know, this is really  kind of a huge process. It starts at the Federal Highway Administration, which is obviously  national in scope, communicating with all the DOTs, the state DOTs, which GDOT is the  one that we kind of work with. And then GDOT communicates with all the regions in the state  and then all the regions kind of coordinate locally. So it's this huge process. And there's  no telling how many different ways people are doing it around the country. So, you know,  why not, like I mentioned OSM tags earlier, why not just use OSM? Well, there are great  tools for editing OpenStreetMap. And that was one of the, you know, we were really considering  doing that. Or I was in my head. But the kind of motivations for not using it outweighed  the great reasons for using it. So basically, we have this DOT road network that we really  wanted to kind of work with and attach all those new attributes to. We wanted to have  build out a custom interface pretty easily. We wanted to track changes with the functional  classification numbering system. And we wanted to manage this whole approval process so that,  you know, it kind of filter up the hierarchy. And, you know, we first tried to kind of figure  out what happened with the last census 10 years ago. But because this process happens  every 10 years, you can imagine that the tools are going to be different, the people are  going to be different, people probably just forgot how it was done 10 years ago. We don't  really know. But this form was kind of proposed as a way to actually manage the process. And  I don't know about you, but I would not want to have a pile, like a stack of these forms  and kind of going back and forth between a road network and doing all this manual entry.  So, you know, we, GitHub was kind of what we used. So why do we use it? So Aaron, you  know, talked a lot about some great tools that are out there that I think kind of were  inspired by GitHub kind of starting to render GeoJSON. We have GeoJSON.io, which is an  amazing editor for, you know, pulling up GeoJSON or other shape data and then really  easily editing it. And that's all run on GitHub Pages, this great free hosting that you can  do. So GitHub Pages is really the curtain in this talk, right? We're kind of keeping  GitHub behind this curtain that is GitHub Pages. It's just a web app that's running on GitHub  and taking advantage of the GitHub API. And so I said it's really crucial in Gov because  sometimes when you're trying to convey an idea, just using words isn't really enough.  And by really quickly getting a demo up onto GitHub Pages, you can convey a message like,  this is something that works and, you know, we could possibly use it in this functional  classification overhaul. And then issue tracking is another great thing that we want to take  advantage of. We have to coordinate with all these local governments and we wanted them  all to be aware of what changes other people were proposing. And so getting that integrated  communications with the tracking, issue tracking, is really important. And we wanted to possibly  share this with other agencies. I mean, that's why I'm here today. That's, it's all open  source code. And there's this fork and go mentality which Jessica Lord kind of seemed  to have come up with, which is basically, you know, you put something up on GitHub and  then GitHub really easily allows you to fork projects and make your own copy to kind of  customize for your own situation. So this is a continuum that I created. Just further  justification for why we needed this curtain around Git, which is kind of scary, right?  You know, on the left-hand side we have familiarity with Git and on the right-hand side we have,  you know, your subsequent kind of relationship to it. So the first, at the very top we have  what's a version control system, which, you know, I was at about a year and a half ago.  We have novice and expert. And so it's kind of like, I guess, stages of grief or something.  You have kind of indifference. And then, so you don't really care about Git. Why do I,  as a person who can live my life normally, why do I need Git? And confusion, like, I  don't really understand what's happening. Fear, panic, discomfort, reliance, and, you  know, you ultimately grow to really depend on Git. So I'm kind of in this panic, discomfort  stage probably. You know, sometimes I waver depending on what I've done wrong as far as  committing data to Git. The originator of Git is probably at the expert level. But the  majority of the target audience for this project is at the indifference stage. And if you started  talking about Git, it would lead to confusion most likely. And so there's a lot of battles  that you kind of have to overcome to get people comfortable with this thing that is really  unfamiliar. And so, you know, that's kind of, you know, and there's been a lot of work  by GitHub especially to kind of make Git permeate other areas besides just software development.  I think that's amazing. But I don't think we're quite to the point where your average  Joe is going to recognize what Git is. So the workflow for this project, so we had this  G.road network that we wanted to submit changes to. So we got that from them in a shape file  version. We did a ogre to ogre conversion to GeoJSON and did some, you know, reprojection.  And we, there's a gentleman earlier in the session who asked about file size constraints  for GitHub. And so we actually had to split the, we had to drop a lot of roads from the  dataset. And then we had to split it by county to keep that file size nice and small because  these are huge road networks that we're talking about. And then we load that up onto a GitHub  pages web app which I'll show in a second. And then we had government staff, you know,  view the road segments and see what the existing conditions are and then submit change requests  for given road segments in their jurisdiction. And so, you know, the other reason we split  up the GeoJSON road network into counties is because we're dealing with multiple jurisdictions.  And there was a lot of concern by, you know, say, for roads that are on the county boundary  or that cross multiple jurisdictions. We wanted, you know, our partners to feel comfortable  that they were submitting changes only for their jurisdiction and also kind of, you know,  just kind of keep that collaboration alive. And so one of the things we had to do was  split up the GeoJSON into these counties. So it was kind of a multi-purpose reason for  that. And we actually, you'll see this is a screen capture from GitHub and it shows a  few of the different counties. We basically created teams that correlated to counties.  And so all of the cities or counties that corresponded to a given county, we would assign  them to that team. And that's how they kind of had access to the data and then could submit  change requests. And so without further ado, I'll just kind of open up this application.  So it should be showing up, yeah. So this is the application we built. It's, you can  see it's running on this atlregional.github.io. That's the GitHub pages domain. And basically  we have a map over here. And it's got map box tiles on underneath it. And you can see  this key, this legend is kind of janky because the screen size is off. But we can, I built  some basic tile layers to, in tile mail to help people kind of see what was out there.  So this is like, you know, really kind of messy, but when you zoom in it looks a little  nicer. But it's just showing all the different classifications we have. We have like, you  know, city boundaries just to people, so people know where they're working from. We  have urbanized area that's not very helpful, but people requested it. And so when you log  in, as I'm going to do right now, you're basically assigned to a county or collection  of counties if the jurisdiction you're in spans that. So these are Fulton and CAB counties.  And that's the data I have access to. And in reality, if I go to the GitHub page and  I want to, you know, create a pull request for any of the data that's on that tool or  on this repository, I can do that. But our users are not about to open up a command line,  interface, and type, get commit or get add and all that stuff. So this is really their  way to interact with that data. And so you can click on road segments. And this is just  showing kind of general leaflet functionality. And when you want to suggest a change for  that road segment, this form pops up where, you know, you give the name of road because  that's not really an attribute that the DOT maintains. You, you know, say you want the  entire segment that's highlighted or just two and from intersections. You can see the  existing functional class. You can see if it has volumes, so like traffic counts for  the road, you can see that. And then you can kind of propose a new functional class. And  so we also have kind of a list of all the proposed change that has been created by using  that same form. And so you can see that this is just kind of like a data table that's searchable,  tried to make it as user-friendly as possible. But these are basically just, if you're familiar  with GitHub, they're issues on GitHub that we're just using the API to pull all that  information, kind of format it in a way that's familiar to people. And when you view it,  it loads that road segment, kind of gives all the description and justification that  the user submitted. And so it's really just kind of a wrapper around GitHub focused on  this particular application. So I'm going to hop back to the presentation here. Here  we go. So that's the app. You might have seen when I pulled up one of those issue boxes,  this, you know, change status, all these little buttons right here. And that was just really  taking advantage of the milestones feature on GitHub. You can set milestones for different,  you know, issues and if they're like bug fixes or feature requests or, you know, this issue  is going to be in version 1.0, we just re-adapted that for are these road requests, are they  approved, are they advancing or in review? So we're kind of adapting the GitHub interface  for our own purposes. And then we also built this another tool that allows you to basically  compare the existing conditions, which is the map on the left, with all of the different  categories of issues. And so right here I think that's, I can't really read, I think  that's advancing. So it shows all the road segments that are advancing. And you can do  the same kinds of tile overlays to kind of get a picture of what the whole network looks  like region wide. So we also wanted to pull data down because inevitably there's going  to be some work on the back end to, you know, figure out which requests get approved, which  get, you know, kind of push down the line. So we have an export button that just exports  all that issues into a CSV. And also if you wanted to actually work with it in like ArcMap,  which our agency has, because it's not in a GeoJSON format, you kind of have to, you  know, use desktop GitHub or Git pull and pull that down and then re-project it into or  change the format into a shape file. And so this is really the project timeline. We're  at the tail end of it. So on the left were the y-axis we have the number of change requests  and on the x-axis we have date. So you can see the first phase of the project was really  training. And that was pretty much for the month of May. So we had a few requests put  in in that early stage. And then the bulk of the requests actually came at the very  end of the submission period. Surprise, right? Like you send a reminder email and a flood  of issues get pulled that very same day. So, you know, at the max we had 257 in a single  day. That was like the Thursday before the Friday deadline. And then you see a gap in  the weekend, right? And then the review period is kind of what Sage is wearing right now.  So it's kind of, you know, reviewing the whole network, all the changes that people have  made and figuring out what makes sense, you know, what doesn't. Like is this like local  residential road actually need to be upgraded to an interstate? No. It's actually a local  road with speed bumps. It's not an interstate. So we're in that review process right now.  So, yeah, deadline on Friday. And so the total change request we had was about a thousand,  which if we had managed that through a stack of paper forms and trying to convert that  all back into spatial data, that would have been really challenging. And this is just  a summary of change requests. It's kind of hard to look at. But basically, it shows,  you know, the approximate number of changes requested. And we actually, I kind of put this  here because we had 20 submissions that were indicating no proposed functional classification.  So they just didn't fill out the form all the way. And really that ended up being just  a form validation. We weren't doing checking, like, making that a mandatory field. And so  we had some errors as a result of that. And the other thing to point out here is we had  a huge number of local roads that were proposed to be upgraded. And, you know, from a data  standpoint, that's fine. We were like, okay, that's, we'll just manage that process. There's  a lot of changes, but that's okay. But actually, we found out later from the DOT that that  actually means that there's going to be a lot of additional data collection on their  part because they have to go out and, like, you know, figure out what the roadway design  is like and do traffic counts. And so it comes with a lot of funding questions and a lot  of challenges, too. So the kind of overall challenges, one of the big ones was source  data. We had used this data set from the DOT, but it was unfamiliar to us. Some of the segment  divisions were really strange. So it was a lot of figuring out what we're actually working  with. And I mentioned earlier that we had to pull a lot of roads out of the network in  order for GitHub to serve it properly. That's because about 80 percent, somewhere around  that, maybe a little less of the entire network are local roads. And so we actually had to  pull all that out. And then we ended up just having kind of the next classification up  all the way up to Interstate because we didn't want to deal with the question of all those  local roads. And so we would have people kind of send us into, like, batch requests to add  in local roads, which ended up being a lot of manual work. Users, a lot of these users,  they're all government staff and a lot of their agencies are still running on, like,  you know, pretty old Windows XP Internet Explorer. And so, you know, trying to get people to  download the latest browsers to make this application work was really challenging. Sometimes  we did have to show the bones of GitHub, especially when people wanted to make comments. And so  that would occasionally confuse people. And also the procrastination issue, I mentioned  the deadline of all the change requests coming in. That was, that posed some personnel issues,  like trying to manage that whole process and instructing people how to open the application.  And, you know, really the volume of change requests continues to be a challenge because  we have these additional data collection efforts that are acquired by the DOT and also just  kind of figuring out what makes sense from a network perspective, like going through  all those, even in a GIS format is challenging. So moving forward, we're continuing to process  these change requests. And really, really the goal is to make a cohesive network. Something  that makes sense and reflects actual travel patterns. And we're considering the framework  for other data sets. For instance, the Transportation Improvement Program, which kind of manages  a list of projects that are in the hopper for the, like, five-year period. And, you know,  we'd love to hear ideas from others. I think hearing what Aaron is working on is amazing  and hopefully others are kind of experimenting with this stuff. But I think that's kind of  the crucial theme, experimenting. And, you know, it worked for our process. It might  not work for other processes, but we're always curious to hear from others. And this is at  the Oregon border. Awesome state to be in. I'm really happy to be here, but happy to  take any questions right now.  There. Great talk. I was just curious. I'm a little fuzzy on how you're getting the information  from GitHub back to the GIS. Like, are all those attributes coming over automatically  or are the edits being made in the GIS? And this is just tracking the...  Yeah. So basically, we used GitHub's API to read the data from GitHub pages and then to  write back to it. So in the GitHub pages branch, we were writing data to that file. And then  to get that back into like a GIS environment, we would just pull that, all that data down  and convert it into a shape file and then you can kind of work within an ARC map.  Yeah. Yeah. We're just kind of like adding attributes to the GeoJSON, all those polylines.  Actually, hopefully it helped answer that question. So we actually built... So I worked  for Esri. We actually built an open source library called Coup, K-O-O-P. Have you seen  it? It's... I have.  It hopefully solves this problem. The idea was it's no JS. You point it at like a GitHub  repository and it can serve it out as feature services or shape file. So someone can go  and live an ARC map and pull data from GitHub or vice versa. If someone has a server, they  want to use GeoJSON. You can deploy that as well. So that might help solve that. Keep  the data in GitHub, but I want to pull it out into a GIS or other formats and I have  to download it and convert it by hand and things like that.  Yeah. That was one of the problems we had because everybody has ARC map, but kind of  getting GitHub installed on their machines and QGIS, there's a lot of overhead with that,  like going through IT and all that. So that's good to know.  In your timeline, it looked like the whole review process was just a couple months. How  was... How long was the whole process of getting the infrastructure set up for this?  So look back at my commit history, but it kind of got started. Like we were kind of  informed by GDOT that we needed to get started around, I think, like February around then.  And then so I was like, hey, I think I have a solution for this. And everybody was like,  well, let's see it. So it was kind of a rapid development cycle. And then so it kind of,  I think, started around like end of February and wrapped up in May. So, yeah.  Have you gotten interest in this type of project from other city governments or other government  organizations? So GDOT was... They were actually really interested  in the process. They're not really at liberty to kind of impose any process on other regions  in the state. But I think in the future, they had interest in possibly using this or some  iteration of it. We haven't really reached out to other MPOs around the country, but  I think that's kind of the next step. And it's not really clear what the best form is  for that. I am speaking at the Association of MPOs in like October. So I think that's  really going to be our chance to share this more broadly with that community. But, yeah.  So you mentioned used Mapbox for the map that we saw and the data that we saw inside of  the GitHub application. Was that your choice or does GitHub integrate directly with Mapbox  in some way? That was our choice because we really wanted  the kind of tile overlays in there. And the easiest way to get that up was, for us, was  through Tile Mill and then to load those up to Mapbox. So there's not any like direct  and because like the code on GitHub pages is just, you know, HTML, JavaScript thrown  on there. And so it was just easiest for us to get up and running.  I would say thank you for this time. Thanks.
05_Landon_Reed_720p_0_107.png 05_Landon_Reed_720p_108_1679.png 05_Landon_Reed_720p_1680_1727.png 05_Landon_Reed_720p_1728_1859.png 05_Landon_Reed_720p_1860_1919.png 05_Landon_Reed_720p_1920_1967.png 05_Landon_Reed_720p_1968_2795.png 05_Landon_Reed_720p_2796_3911.png 05_Landon_Reed_720p_3912_5291.png 05_Landon_Reed_720p_5292_5507.png 05_Landon_Reed_720p_5508_6299.png 05_Landon_Reed_720p_6300_6419.png 05_Landon_Reed_720p_6420_6731.png 05_Landon_Reed_720p_6732_7571.png 05_Landon_Reed_720p_7572_8423.png 05_Landon_Reed_720p_8424_8579.png 05_Landon_Reed_720p_8580_10163.png 05_Landon_Reed_720p_10164_11375.png 05_Landon_Reed_720p_11376_14087.png 05_Landon_Reed_720p_14088_14339.png 05_Landon_Reed_720p_14340_15059.png 05_Landon_Reed_720p_15060_15167.png 05_Landon_Reed_720p_15168_15443.png 05_Landon_Reed_720p_15444_16991.png 05_Landon_Reed_720p_16992_17111.png 05_Landon_Reed_720p_17112_17987.png 05_Landon_Reed_720p_17988_18131.png 05_Landon_Reed_720p_18132_19607.png 05_Landon_Reed_720p_19608_19727.png 05_Landon_Reed_720p_19728_20579.png 05_Landon_Reed_720p_20580_20927.png 05_Landon_Reed_720p_20928_21203.png 05_Landon_Reed_720p_21204_21743.png 05_Landon_Reed_720p_21744_21791.png 05_Landon_Reed_720p_21792_21839.png 05_Landon_Reed_720p_21840_22703.png 05_Landon_Reed_720p_22704_23507.png 05_Landon_Reed_720p_23508_24467.png 05_Landon_Reed_720p_24468_26687.png 05_Landon_Reed_720p_26688_28463.png 05_Landon_Reed_720p_28464_31055.png 05_Landon_Reed_720p_31056_31523.png 05_Landon_Reed_720p_31524_32783.png 05_Landon_Reed_720p_32784_36263.png 05_Landon_Reed_720p_36264_39863.png 05_Landon_Reed_720p_39864_39890.png
So, if you were here for the presentation just previous to mine, you're in luck because he, Aaron laid a great foundation for my talk. It's actually dovetails perfectly. So, that, I don't know, we weren't colluding. Maybe it's just the great organizers at PhosphorG, I don't know. But my presentation is on managing public data on GitHub or paying no attention to that get behind the curtain. So, it's kind of about, you know, GitHub and government. And sometimes, well, you know, if you read Vlad's talk from Leiflet earlier today, he talked a lot about experimentation and creativity. And I think that's something that is really needed in the government sector and that's something that Code for America is really stepping up and helping out with. But I think there's always room for that. And so, this presentation is really kind of looking at one case where I tried to experiment a little bit with GitHub and managing this process. And it went pretty well. Nothing exploded. So the context. I work at, oh, wow. That's not good. Okay. There we go. I work at the Atlanta Regional Commission which is regional government. So basically, we build consensus with all the smaller local governments, counties and cities. And we have a lot of counties in Georgia. We're notorious for that. So there's a lot of partners to manage relationships with. And so the story behind this project was each census. So every 10 years, the urbanized areas are redrawn according to population change. So people move from Atlanta to San Francisco or, you know, all these different places and they have to redraw the areas. And along with that, the Federal Highway Association asks for updated roadway functional classifications. And if you're familiar with OpenStreetMap and OSM speak, functional classifications are just, you know, the highway tag and then any of those motorway, trunk, primary, secondary, residential. But in FHWA speak, we have things like interstates, other freeways, principal materials, these other classifications that basically mean the same thing. And there's some funding tied to these classifications, but it's really just a way for the feds to keep track of all the roads that exist out there. And so the problem that we were facing was at the beginning of this process, we were like, okay, how are we going to manage all these stakeholders, all these local governments? You know, we have maps we can print out and people can draw on them, get around the table and kind of do a collaborative thing, which is awesome. We have forms that people could fill out and we can send people, well, Microsoft Word docs and have them send them back or scan them. And then we'd have a pile of paper to go through and then basically have an analyst go in and tag all the, you know, try to figure out which road they're talking about, find it on the map, tag it, change all the functional classifications, really kind of a daunting task. And, you know, so it's this process that we didn't really want to manage. And when I was in the meeting, when they talked about it, I was like, there's got to be a better way to do it. But I had to kind of demonstrate that. So we like that first part, but we don't like all the forms and having unstructured data and turning that into data structures. And so, you know, this is really kind of a huge process. It starts at the Federal Highway Administration, which is obviously national in scope, communicating with all the DOTs, the state DOTs, which GDOT is the one that we kind of work with. And then GDOT communicates with all the regions in the state and then all the regions kind of coordinate locally. So it's this huge process. And there's no telling how many different ways people are doing it around the country. So, you know, why not, like I mentioned OSM tags earlier, why not just use OSM? Well, there are great tools for editing OpenStreetMap. And that was one of the, you know, we were really considering doing that. Or I was in my head. But the kind of motivations for not using it outweighed the great reasons for using it. So basically, we have this DOT road network that we really wanted to kind of work with and attach all those new attributes to. We wanted to have build out a custom interface pretty easily. We wanted to track changes with the functional classification numbering system. And we wanted to manage this whole approval process so that, you know, it kind of filter up the hierarchy. And, you know, we first tried to kind of figure out what happened with the last census 10 years ago. But because this process happens every 10 years, you can imagine that the tools are going to be different, the people are going to be different, people probably just forgot how it was done 10 years ago. We don't really know. But this form was kind of proposed as a way to actually manage the process. And I don't know about you, but I would not want to have a pile, like a stack of these forms and kind of going back and forth between a road network and doing all this manual entry. So, you know, we, GitHub was kind of what we used. So why do we use it? So Aaron, you know, talked a lot about some great tools that are out there that I think kind of were inspired by GitHub kind of starting to render GeoJSON. We have GeoJSON.io, which is an amazing editor for, you know, pulling up GeoJSON or other shape data and then really easily editing it. And that's all run on GitHub Pages, this great free hosting that you can do. So GitHub Pages is really the curtain in this talk, right? We're kind of keeping GitHub behind this curtain that is GitHub Pages. It's just a web app that's running on GitHub and taking advantage of the GitHub API. And so I said it's really crucial in Gov because sometimes when you're trying to convey an idea, just using words isn't really enough. And by really quickly getting a demo up onto GitHub Pages, you can convey a message like, this is something that works and, you know, we could possibly use it in this functional classification overhaul. And then issue tracking is another great thing that we want to take advantage of. We have to coordinate with all these local governments and we wanted them all to be aware of what changes other people were proposing. And so getting that integrated communications with the tracking, issue tracking, is really important. And we wanted to possibly share this with other agencies. I mean, that's why I'm here today. That's, it's all open source code. And there's this fork and go mentality which Jessica Lord kind of seemed to have come up with, which is basically, you know, you put something up on GitHub and then GitHub really easily allows you to fork projects and make your own copy to kind of customize for your own situation. So this is a continuum that I created. Just further justification for why we needed this curtain around Git, which is kind of scary, right? You know, on the left-hand side we have familiarity with Git and on the right-hand side we have, you know, your subsequent kind of relationship to it. So the first, at the very top we have what's a version control system, which, you know, I was at about a year and a half ago. We have novice and expert. And so it's kind of like, I guess, stages of grief or something. You have kind of indifference. And then, so you don't really care about Git. Why do I, as a person who can live my life normally, why do I need Git? And confusion, like, I don't really understand what's happening. Fear, panic, discomfort, reliance, and, you know, you ultimately grow to really depend on Git. So I'm kind of in this panic, discomfort stage probably. You know, sometimes I waver depending on what I've done wrong as far as committing data to Git. The originator of Git is probably at the expert level. But the majority of the target audience for this project is at the indifference stage. And if you started talking about Git, it would lead to confusion most likely. And so there's a lot of battles that you kind of have to overcome to get people comfortable with this thing that is really unfamiliar. And so, you know, that's kind of, you know, and there's been a lot of work by GitHub especially to kind of make Git permeate other areas besides just software development. I think that's amazing. But I don't think we're quite to the point where your average Joe is going to recognize what Git is. So the workflow for this project, so we had this G.road network that we wanted to submit changes to. So we got that from them in a shape file version. We did a ogre to ogre conversion to GeoJSON and did some, you know, reprojection. And we, there's a gentleman earlier in the session who asked about file size constraints for GitHub. And so we actually had to split the, we had to drop a lot of roads from the dataset. And then we had to split it by county to keep that file size nice and small because these are huge road networks that we're talking about. And then we load that up onto a GitHub pages web app which I'll show in a second. And then we had government staff, you know, view the road segments and see what the existing conditions are and then submit change requests for given road segments in their jurisdiction. And so, you know, the other reason we split up the GeoJSON road network into counties is because we're dealing with multiple jurisdictions. And there was a lot of concern by, you know, say, for roads that are on the county boundary or that cross multiple jurisdictions. We wanted, you know, our partners to feel comfortable that they were submitting changes only for their jurisdiction and also kind of, you know, just kind of keep that collaboration alive. And so one of the things we had to do was split up the GeoJSON into these counties. So it was kind of a multi-purpose reason for that. And we actually, you'll see this is a screen capture from GitHub and it shows a few of the different counties. We basically created teams that correlated to counties. And so all of the cities or counties that corresponded to a given county, we would assign them to that team. And that's how they kind of had access to the data and then could submit change requests. And so without further ado, I'll just kind of open up this application. So it should be showing up, yeah. So this is the application we built. It's, you can see it's running on this atlregional.github.io. That's the GitHub pages domain. And basically we have a map over here. And it's got map box tiles on underneath it. And you can see this key, this legend is kind of janky because the screen size is off. But we can, I built some basic tile layers to, in tile mail to help people kind of see what was out there. So this is like, you know, really kind of messy, but when you zoom in it looks a little nicer. But it's just showing all the different classifications we have. We have like, you know, city boundaries just to people, so people know where they're working from. We have urbanized area that's not very helpful, but people requested it. And so when you log in, as I'm going to do right now, you're basically assigned to a county or collection of counties if the jurisdiction you're in spans that. So these are Fulton and CAB counties. And that's the data I have access to. And in reality, if I go to the GitHub page and I want to, you know, create a pull request for any of the data that's on that tool or on this repository, I can do that. But our users are not about to open up a command line, interface, and type, get commit or get add and all that stuff. So this is really their way to interact with that data. And so you can click on road segments. And this is just showing kind of general leaflet functionality. And when you want to suggest a change for that road segment, this form pops up where, you know, you give the name of road because that's not really an attribute that the DOT maintains. You, you know, say you want the entire segment that's highlighted or just two and from intersections. You can see the existing functional class. You can see if it has volumes, so like traffic counts for the road, you can see that. And then you can kind of propose a new functional class. And so we also have kind of a list of all the proposed change that has been created by using that same form. And so you can see that this is just kind of like a data table that's searchable, tried to make it as user-friendly as possible. But these are basically just, if you're familiar with GitHub, they're issues on GitHub that we're just using the API to pull all that information, kind of format it in a way that's familiar to people. And when you view it, it loads that road segment, kind of gives all the description and justification that the user submitted. And so it's really just kind of a wrapper around GitHub focused on this particular application. So I'm going to hop back to the presentation here. Here we go. So that's the app. You might have seen when I pulled up one of those issue boxes, this, you know, change status, all these little buttons right here. And that was just really taking advantage of the milestones feature on GitHub. You can set milestones for different, you know, issues and if they're like bug fixes or feature requests or, you know, this issue is going to be in version 1.0, we just re-adapted that for are these road requests, are they approved, are they advancing or in review? So we're kind of adapting the GitHub interface for our own purposes. And then we also built this another tool that allows you to basically compare the existing conditions, which is the map on the left, with all of the different categories of issues. And so right here I think that's, I can't really read, I think that's advancing. So it shows all the road segments that are advancing. And you can do the same kinds of tile overlays to kind of get a picture of what the whole network looks like region wide. So we also wanted to pull data down because inevitably there's going to be some work on the back end to, you know, figure out which requests get approved, which get, you know, kind of push down the line. So we have an export button that just exports all that issues into a CSV. And also if you wanted to actually work with it in like ArcMap, which our agency has, because it's not in a GeoJSON format, you kind of have to, you know, use desktop GitHub or Git pull and pull that down and then re-project it into or change the format into a shape file. And so this is really the project timeline. We're at the tail end of it. So on the left were the y-axis we have the number of change requests and on the x-axis we have date. So you can see the first phase of the project was really training. And that was pretty much for the month of May. So we had a few requests put in in that early stage. And then the bulk of the requests actually came at the very end of the submission period. Surprise, right? Like you send a reminder email and a flood of issues get pulled that very same day. So, you know, at the max we had 257 in a single day. That was like the Thursday before the Friday deadline. And then you see a gap in the weekend, right? And then the review period is kind of what Sage is wearing right now. So it's kind of, you know, reviewing the whole network, all the changes that people have made and figuring out what makes sense, you know, what doesn't. Like is this like local residential road actually need to be upgraded to an interstate? No. It's actually a local road with speed bumps. It's not an interstate. So we're in that review process right now. So, yeah, deadline on Friday. And so the total change request we had was about a thousand, which if we had managed that through a stack of paper forms and trying to convert that all back into spatial data, that would have been really challenging. And this is just a summary of change requests. It's kind of hard to look at. But basically, it shows, you know, the approximate number of changes requested. And we actually, I kind of put this here because we had 20 submissions that were indicating no proposed functional classification. So they just didn't fill out the form all the way. And really that ended up being just a form validation. We weren't doing checking, like, making that a mandatory field. And so we had some errors as a result of that. And the other thing to point out here is we had a huge number of local roads that were proposed to be upgraded. And, you know, from a data standpoint, that's fine. We were like, okay, that's, we'll just manage that process. There's a lot of changes, but that's okay. But actually, we found out later from the DOT that that actually means that there's going to be a lot of additional data collection on their part because they have to go out and, like, you know, figure out what the roadway design is like and do traffic counts. And so it comes with a lot of funding questions and a lot of challenges, too. So the kind of overall challenges, one of the big ones was source data. We had used this data set from the DOT, but it was unfamiliar to us. Some of the segment divisions were really strange. So it was a lot of figuring out what we're actually working with. And I mentioned earlier that we had to pull a lot of roads out of the network in order for GitHub to serve it properly. That's because about 80 percent, somewhere around that, maybe a little less of the entire network are local roads. And so we actually had to pull all that out. And then we ended up just having kind of the next classification up all the way up to Interstate because we didn't want to deal with the question of all those local roads. And so we would have people kind of send us into, like, batch requests to add in local roads, which ended up being a lot of manual work. Users, a lot of these users, they're all government staff and a lot of their agencies are still running on, like, you know, pretty old Windows XP Internet Explorer. And so, you know, trying to get people to download the latest browsers to make this application work was really challenging. Sometimes we did have to show the bones of GitHub, especially when people wanted to make comments. And so that would occasionally confuse people. And also the procrastination issue, I mentioned the deadline of all the change requests coming in. That was, that posed some personnel issues, like trying to manage that whole process and instructing people how to open the application. And, you know, really the volume of change requests continues to be a challenge because we have these additional data collection efforts that are acquired by the DOT and also just kind of figuring out what makes sense from a network perspective, like going through all those, even in a GIS format is challenging. So moving forward, we're continuing to process these change requests. And really, really the goal is to make a cohesive network. Something that makes sense and reflects actual travel patterns. And we're considering the framework for other data sets. For instance, the Transportation Improvement Program, which kind of manages a list of projects that are in the hopper for the, like, five-year period. And, you know, we'd love to hear ideas from others. I think hearing what Aaron is working on is amazing and hopefully others are kind of experimenting with this stuff. But I think that's kind of the crucial theme, experimenting. And, you know, it worked for our process. It might not work for other processes, but we're always curious to hear from others. And this is at the Oregon border. Awesome state to be in. I'm really happy to be here, but happy to take any questions right now. There. Great talk. I was just curious. I'm a little fuzzy on how you're getting the information from GitHub back to the GIS. Like, are all those attributes coming over automatically or are the edits being made in the GIS? And this is just tracking the... Yeah. So basically, we used GitHub's API to read the data from GitHub pages and then to write back to it. So in the GitHub pages branch, we were writing data to that file. And then to get that back into like a GIS environment, we would just pull that, all that data down and convert it into a shape file and then you can kind of work within an ARC map. Yeah. Yeah. We're just kind of like adding attributes to the GeoJSON, all those polylines. Actually, hopefully it helped answer that question. So we actually built... So I worked for Esri. We actually built an open source library called Coup, K-O-O-P. Have you seen it? It's... I have. It hopefully solves this problem. The idea was it's no JS. You point it at like a GitHub repository and it can serve it out as feature services or shape file. So someone can go and live an ARC map and pull data from GitHub or vice versa. If someone has a server, they want to use GeoJSON. You can deploy that as well. So that might help solve that. Keep the data in GitHub, but I want to pull it out into a GIS or other formats and I have to download it and convert it by hand and things like that. Yeah. That was one of the problems we had because everybody has ARC map, but kind of getting GitHub installed on their machines and QGIS, there's a lot of overhead with that, like going through IT and all that. So that's good to know. In your timeline, it looked like the whole review process was just a couple months. How was... How long was the whole process of getting the infrastructure set up for this? So look back at my commit history, but it kind of got started. Like we were kind of informed by GDOT that we needed to get started around, I think, like February around then. And then so I was like, hey, I think I have a solution for this. And everybody was like, well, let's see it. So it was kind of a rapid development cycle. And then so it kind of, I think, started around like end of February and wrapped up in May. So, yeah. Have you gotten interest in this type of project from other city governments or other government organizations? So GDOT was... They were actually really interested in the process. They're not really at liberty to kind of impose any process on other regions in the state. But I think in the future, they had interest in possibly using this or some iteration of it. We haven't really reached out to other MPOs around the country, but I think that's kind of the next step. And it's not really clear what the best form is for that. I am speaking at the Association of MPOs in like October. So I think that's really going to be our chance to share this more broadly with that community. But, yeah. So you mentioned used Mapbox for the map that we saw and the data that we saw inside of the GitHub application. Was that your choice or does GitHub integrate directly with Mapbox in some way? That was our choice because we really wanted the kind of tile overlays in there. And the easiest way to get that up was, for us, was through Tile Mill and then to load those up to Mapbox. So there's not any like direct and because like the code on GitHub pages is just, you know, HTML, JavaScript thrown on there. And so it was just easiest for us to get up and running. I would say thank you for this time. Thanks.
And so, this presentation is really kind of little bit with GitHub and managing this process. Okay. There we go. I work at the Atlanta Regional Commission which is regional government. So notorious for that. So there's a lot of partners You know, we have maps we can print out and you can imagine that the tools are going to be there that I think kind of were inspired by taking advantage of the GitHub API. And so I said communications with the tracking, issue tracking, share this with other agencies. I mean, that's something up on GitHub and then GitHub really areas besides just software development. I think tool or on this repository, I can do that. But You can see the existing functional class. You make it as user-friendly as possible. But these on the back end to, you know, figure out which is really the project timeline. We're at the tail into spatial data, that would have been really fill out the form all the way. And really that of the segment divisions were really strange. So to pull a lot of roads out of the network in local roads. And so we actually had to pull all up being a lot of manual work. Users, a lot of application work was really challenging. comments. And so that would occasionally confuse we're considering the framework for other data data from GitHub pages and then to write back to of the problems we had because everybody has ARC for this time. Thanks.