FOSS4G In AWS: Choosing, Deploying And Tuning Open Source Components In AWS
Sayed, Mohamed (Nokia & OpenEgypt & others.)
This presentation will show methods of working with AWS to design, deploy and tune Open source software with an end goal to bring up various geo-oriented full stacks. This includes databases, tile renderers, geocoders, routers with all dependencies. It will cover choosing the components, the deployment posture, prototyping, designing for cloud scalability, performance benchmarking and ongoing maintenance. Most of the concepts will lend themselves well to other public or private cloud situations.
 Speaker is Mohamed Sayed, so I would then invite him to go on the stage and start his presentation. He is from here, which in Dutch is strange because it means he is from here. He lives here, but he is working for a company I guess called here. By the way, I'm not the one on the nerd, so I do want you to switch off your phones, or put them to silence, please, because it's annoying for the speakers. Thank you very much. Hi, good morning. So, yeah, I work for Heel, which is Nokia's location and commerce, but I'm not here for Heel, I'm just here on my own, and I put this presentation together, and I hope that it will be a contribution to the community. So, this is Agenda, and a couple of the screeners, one you already heard and just one more, some goals and motives, and a little bit of historical background, definition of glass computing, so we can talk about the same thing. And some use cases for phosphor G, and then I'll talk about AWS, the components and service at a high level, and if you want to be in the cloud, what you're going to be doing, and how you're going to be doing it, and then some common-foss tasks. If you are building an SDI in the cloud, you're going to have to import some data, you're going to do some rendering, some geocoding, and so on. And then hopefully we'll have time for questions. So, you heard the first one I worked for here, I'm a senior architect in the core platform group for Nokia's location and commerce, but again, this is just personal work, personal effort. I'm also not affiliated with AWS other than being a customer, so I use them to make sense, if somebody else makes more sense, I'll be using them, especially if they give me a bell-a-bell for the buck. This is still work in progress, I hope to be doing this once a month, or at least once a quarter, so you'll knowledge my vary, but I've tried to document it as much as I could. So why did I want to do this? I wanted to maybe validate some ideas with you, and maybe you can validate some ideas with me. I've done quite a few services in the past. I've worked for Yahoo before I went to Nokia, and so I also did www.yahoo.com to adopt my Yahoo. So I have a little bit of background in that area. Maybe I'll get some feedback from you, and we would like to see you try. Hopefully I'll help you save some money. There was a lot of frustration while I was doing this, so hopefully you don't have to go through it, and there's already stuff that some artifacts are produced, so maybe you can use some of that. In the process, I've discovered some problems and issues, and maybe everybody knows about them. I'm not very strongly connected to the PhosphorG community, although I've been an open source guy for a very long time. So I'd like to hear if somebody knows if there's work in progress to address the risks when we bring them up. And why I wanted to do this is basically because I think right now we are at a stage where everybody's talking about open data and geo and location, and I think this is a very good opportunity where you may be called upon to contribute, either in your organization or in your community or university, using open source technologies, maybe save some taxpayers some dollars, and at the same time have some fun. So I think this is kind of my main motivation is that I think this is an opportunity where we can do some disruption with open source in the geo-location space, and I think I can help a little bit. So a little bit of background. So cloud computing a couple of years ago was just a buzzword, and I'm like, yeah, everybody's talking about this, and it's kind of emerged out of that stigma of being a buzzword into a reality where people actually use it and they can deploy services to it. It had, you know, immensely lowered the bar for entry, the barrier to entry for start-up companies, for nonprofit organizations and so on. But this goes back a few years, maybe a little bit over a decade. I think it all started with virtualization. I'm not going to go into the ancient history before VMware and virtualization on the X86, but VMware and PaL used to ship commercial products, and they had some customers, a lot of people used them in lab environments and so on. But it wasn't massively adopted. Solaris, you know, back when Sun existed, they also did Solaris zones and containers, but it wasn't really until Zen before KVM came up, and they came with this power virtualization technique which really helped the performance, and this became a very viable solution to run things in a production environment. And so that was kind of like the disruption. I think Zen really tipped the scale here. At the same time, we were having problems with hardware, so we were not able to deliver any faster processors. Problems with cooling, problems with power, and the solution from the hardware vendors was to go for multicore, so you don't have faster processors, you just have more. So that was one thing, and then they caught on, that if they wanted people to run things on multiple computers, multiple processors, the software at the time was not quite ready, so how do you utilize that? So virtualization was a natural solution, but because of the performance, it was still not quite up to snuff. Some hesitation was there, and the hardware vendors started supporting virtualization into the chipset. So first AMD came up with nested page tables, and then Intel did extended page tables where you can virtualize the page table so the virtual machine doesn't have to contact switch all the time. Later on, there was IOL offloading, so TCP offloading, for example. You can just process things in the neck without having to go up to the main processor anymore. Same thing for storage. And then the storage and network vendors started thinking about how could they support this from an infrastructure standpoint. So there was virtualization in the storage. They kind of called everything that they had done before virtualization, even if it wasn't really new, but volumes became virtual volumes, and slices became virtual slices, and so on. But yes, so it started to gain momentum from there. And then on the consumer side, we started seeing smartphone stabilizers and multi-screens, and people wanting to use services and be able to access the same thing from everywhere. So now it became the idea, okay, we don't want to re-store things on desktops anymore, and maybe we store them on servers, but the servers have to be accessible, and this is where cloud computing kind of crystallized. AWS had really been pushing this for a long time, so they have really done a lot of work there, and they're way ahead of everybody else, as far as I know, in terms at least in breadth of coverage. OpenStack is trying to catch on. I think they do great work as well, but it's just, you know, they deliver software, and now there are companies which are trying to take the software and build infrastructure and services around it. So my definition of cloud computing, and this is just a definition, it's a computing paradigm where it's composed of abstractions, a set of primitives, and some interfaces and tools around them. The idea is that you try to hide the physical stuff, the stuff that's hard to move, the stuff that you don't want to be tied to, so you want to abstract that as much as possible, and then you have a new set of primitives, some of them not necessarily very new, but images, for example, is a primitive. Snapshots, volumes, a region, availability zones, they may have other terms for other providers, but they basically talk about the same thing, they're trying to abstract the data center, the actual computer or the actual hard disk away from you. And then tools and administrative utilities around that. What happens is that once cloud computing kicked off and people started deploying virtual machines and cloud and so on, things spiraled out of control really fast, and it wasn't in a very good shape to begin with, so the tools and automation also really helped set that path, so puppet chef, any other configuration, CF Engine 3, any other configuration management, but the idea is that you have the primitives, you have the abstractions, and you have the tools to manage them. So this is kind of like a block diagram, so at the very bottom you have the physical stuff, and then the primitives sit on top, and you have the tools and APIs at the highest level. We can even go further up, so if you look at things like Heroku, for example, they abstract even more where you just deploy to a platform, so you're very far away from everything else, you just have a command, you run it, you got a service. So this is kind of a clean representation of what it looks like. This is open stack implementation, so there's quite a few areas going back and forth, and this is kind of what it looks like in real life, so this is clock computing, and that machine is very important, because if that gets unplugged, the whole thing goes to shit. Alright, so AWS is a public cloud, so the same kind of diagram, but it would just be a little bit more specific, so we talked about compute as an EC2 instances storage. So EC2 instances is just virtual machines, they have a predefined set of configuration, so you cannot change or tweak the CPU or memory settings, you can just choose one of the models, you can attach drives as you wish. Then a set of storage, so S3 is like a storage over HTTP or HTTPS, and they have elastic block storage, which is kind of a NAS, or a SAM idea, and Glass-Hero is kind of a long-term archival. They have the foundation, you know, the regions, the actual brick-and-mortar implementation, the data centers, the power, the cooling, all the stuff that we don't want to think about. And networking, so we're with 53s DNS service, elastic load balancing, CloudFront is a caching service, and a set of tools around security, so identity management security groups. You go up one level, you see the simple queuing service, or search as a service, or redshift, which is post-glace SQL, and storage, kind of, so if you don't want to run a cluster, and you don't want to manage it, then they will do it for you, and you just put your schema, you connect to it, and you treat it just like a post-guess, or post-gray. Unfortunately, at the minimum, they don't support spatial, so it's only post-gray SQL. And there's more. There's simple email service, simple modification service, and so on. And the management layer is the API, is the outer scale, the CloudFormation, and that configuration, and so on. So what kind of use cases we can do with Phosphor Gene in the Cloud? Well, for start, disaster recovery backup, so it's very simple, very easy to just dump a table, or archive your SQL dump, encrypt it, ship it over to an S3 bucket, get it back when you need it, hopefully you never need it. So this is a very straightforward use case. The other use case is static logic free web publishing. So if you just have some vector data, or roster data, or any kind of static data, where you're not doing any logic, anybody who can make a request can get back that response. You can just publish this using S3 and CloudFront. I'll show you an example, a diagram. You don't have to run a web server, you don't have to run load balancer, you don't have to do anything. You just publish it, and you will pay for the request as they come in, but you're not going to have to maintain any infrastructure. Obviously, online Phosphor Gene, so you can do geocoding, or tiling, or routing, and so on. So any of the software that is available to us under a public license, you can just run it. If you run a GPL license software, you have to make sure that this you're compiling is that license. Data transformation drops. If you have a set of tiles, or a set of data, and you want to just transform them from one format to another, or maybe you have four or five different formats, and you have to do this overnight, you don't need to borrow a bunch of machines and just have them sit for the rest of the day, so you can just fire a job, get it done, and shut them back. Consecuration batch processes, so again, the same kind of concept, if you're collaborating with other people and you want to have some central storage, where they can upload their files, and maybe you can do some processing, put it back, and so on. So this is like a blueprint. If you wanted to do this static logic-free content using AWS, this is your content, and you can just put it to this S3 bucket. You can configure a platform distribution, which will point to this bucket, and you publish it, and you make your DNS-seaning areas to this zone that you are going to configure here, and that's it. Now users will go request your data, they will get seen into the CloudFront zone, and based on some telemetry and some other magic, they will get routed to the closest cache edge to them. If you want to have logs, you can also configure logs to go to an S3 bucket, where you can just retrieve them back later. So how do you build it? If you wanted to do this maybe for a university or for a district or just your company, how do you do that? So I think there are some architectural patterns, and you'll not see this in books, I kind of came up with this overnight, and I just wanted to share them with you, so you're aware of them, they fit some things better than others, sometimes you have to mix them, everything in the world is almost polyglot, so this is not a holy book. So the cookie-cutters, the idea is that you have a machine, the machine has everything that you would need, and you just manufacture them, you just have 10, 20, 100, as many as you need, or as many as you need. They have everything together, so they have the application and they have the data, and they scale horizontally, so if the traffic is actually growing, then you can just scale up, if the traffic is dying down, or on a low point, you can just shrink them. The data is accessible to the machine itself, they are not connected to each other in any way, so if in case they fail, the failures look like. So very simple, scales very well in some use cases, so simplicity is one of the pros, and scales horizontally was laid, and localized failure impact, these are the main points. Problems is that poor support for right-oriented service. If you look at here, if you have so many machines, and everyone has a copy of the data, and the data has to change, then you have to push this back somewhere, somehow. And if the users are allowed to change the data, that's even worse, because now one machine is going to change, and it's going to have to replicate, then that doesn't work very well. It's a coarse grain scalability, so when you scale, you scale everything, or you shrink everything. If you actually have a service where your data layer is very, very fast, but your web service is not so fast, so you need a web service, or web applications, you can't do it, you're going to have to scale the whole thing together. So it's a cookie cutter. The load capacity has vertical scalability issues, so if you have data growing, or if your memory consumption is growing, you're going to hit the ceiling at some point, where you can't just grow anymore within that box. So there is a vertical ceiling, and how much you can process per node. When there's a centrist approach, basically you take the data out, and you let the application run on the nodes, you can have a second copy of the data as a backup disaster recovery, and this is, you know, now the data is centralized in this database, these are the clients of this, and then the user is out there, they're the clients of your web service. Works okay in a lot of cases. Scales well for mid-level loads, so if you're doing 20, 30, 40 new requests per second or so, it probably works. It has some other issues. So the pros first, you know, you can actually scale the web service by itself, or you can scale the database by itself, and that's a big advantage over the other approach. Five minutes. Okay, and we have to run with the slides. So the replicator, the master of colonies, where you can have masters of freckles distributed over the world, and they can replicate to each other as read-only slaves, and that works pretty good for read scalability. You're going to have to do some culture changes. You know, you release engineering, this has to be in really good shape. You have to adopt automation. You really need to think about agility. You need to think about using the primitives that are available to you. You need to make sure that you get a buy-in from the stakeholders. These are very, very key things. Some process changes. And some of the things that you have to remember, the leader implications, don't try to scale in the cloud as you would in a brick-and-mortar situation. So don't try to go for a, let's cluster these machines together and have a big cluster and four or five clusters. Just things fail and just plan for it, and it's okay, and just think about how you could recover as fast as possible. It may take one or two tries. You probably get it right on the third time. But the old approach, you know, trying to connect things and make sure they are going to be reliable. It doesn't work the wrong way. I did some other work in this process. I'll go through it really quick. When I started, I wanted to see if I can profile a renderer, a geocoder, a router in the cloud. And then I hit the first problem. I wanted to get data in. And then I started reading about people taking ten days to get the OSM data set. I thought that was horrible. I didn't want to get a synthetic data set with 200 megs and say, yeah, this works and I'm sure it will heal. So I really wanted to get the OSM data in. And I did. So first I did some tests. I looked at a bunch of countries, small countries. This is the time it took in seconds versus the size of the data set. So up to 3.2 gigs. We are within a 30 minutes or 35 minutes range. I started collecting some stats around this. And I provisioned different infrastructures. So I looked at the local drive, how long it took. I looked at provision IOPS, which guarantees IOPS performance. I looked at SSD, which is very expensive. And so guess how long it took. I went with the SSD one because I wanted to finish as fast as possible. Any guesses how long it took? Ah, you're right. An hour, I wish. It took 35 hours. But this is 250 gig. This is not so bad because a lot of people spend six or seven days to get this done. So this is actually not so bad at all. But guess what I did after I just finished this? No, I made a copy. So I made a copy. I built a RAID zero set over provision IOPS. And I created a logical volume on top. And I kicked off a data copy. And guess how long that took? Wow, it's short. Okay, it took two and a half hours. So actually, this is a file system copy, right? So I just shut down the database and then I copy the volume over. And then I archive it, of course. And this took two and a half hours. So this is what it took to do a provision IOPS to SSD. It took five and a half hours. It took about seven hours to do a SQL dump to SSD. SSD took provision IOPS to two and a half hours. SSD to SQL dump took four and a half hours. OSM to PG SQL to SSD took 35 hours. So guess where the problem is? So this is a profile for OSM to PG SQL. You can see the rank hash nodes get 15%, Y max 10%, copy to table 10%. So there's a lot of things going on before the data actually is stored in the database. And this is part of the problem. And I wanted to talk about that a little bit more. So you can read these notes later. Then I did some profiling with Matta and Matnik. One thread, three threads. So if you run this four threads, if you run this four rendering threads, you actually get six threads. So two threads do the bookkeeping for threads due to actual work. You're going to have to read this unfortunately. I'm going to run to the G-server part a little bit. So G-server, single layer, I took a small country called Finland. And zoom level 15 from 0 to 1 to 15. And this I did around this. And this is about 100 tiles per second. You can do about 100 tiles per second in that kind of setup. And this is kind of a ceiling because this is RAM disk. So it's not going to get any faster. Well, it could, but it would be very expensive. Trankation is very slow. So doing Tranky, your G-web cache, try to publish your data as version data, as version layers. This is much better if you can help it. Stano-known G-web cache will work a lot better. So try to think about just yanking that GW cache out and put some G-web servers behind it. I'll show you a new point. There's some possibly waste conditions in thread writing tiles. And this is kind of an example deployment where you can take the G-web cache out and you put G-servers behind them. And you put a load balancer that can do URL persistence. So these nodes are not coherent. They are incoherent by design. The idea is that you will go to the node that has the tile. If there is no node that has the tile, one will get selected and then from there on, it will be persistent. You can mix the disks so you can mix a fast disk and slow disk in a volume. That will probably give you a very good performance. How much did all this cost? $866 and two weeks. And then I have a backlog where these snapshots to the public. So I actually have the data now in AWS. I'm going to make it public so you can just import it. It should hopefully help out. I'm going to do some G-coding profiling in an OSRM profiler. And I'm open to suggestions as well. Thank you very much. So you took off all of your time. So I'm afraid you don't have time for any questions. I'm sure your slides will go up on the ELO geoplatform.
 Speaker is Mohamed Sayed, so I would then invite him to go on the stage and start his presentation.  He is from here, which in Dutch is strange because it means he is from here.  He lives here, but he is working for a company I guess called here.  By the way, I'm not the one on the nerd, so I do want you to switch off your phones,  or put them to silence, please, because it's annoying for the speakers.  Thank you very much.  Hi, good morning.  So, yeah, I work for Heel, which is Nokia's location and commerce, but I'm not here for Heel,  I'm just here on my own, and I put this presentation together,  and I hope that it will be a contribution to the community.  So, this is Agenda, and a couple of the screeners,  one you already heard and just one more, some goals and motives,  and a little bit of historical background, definition of glass computing,  so we can talk about the same thing.  And some use cases for phosphor G, and then I'll talk about AWS,  the components and service at a high level,  and if you want to be in the cloud, what you're going to be doing,  and how you're going to be doing it, and then some common-foss tasks.  If you are building an SDI in the cloud, you're going to have to import some data,  you're going to do some rendering, some geocoding, and so on.  And then hopefully we'll have time for questions.  So, you heard the first one I worked for here,  I'm a senior architect in the core platform group for Nokia's location and commerce,  but again, this is just personal work, personal effort.  I'm also not affiliated with AWS other than being a customer,  so I use them to make sense, if somebody else makes more sense,  I'll be using them, especially if they give me a bell-a-bell for the buck.  This is still work in progress, I hope to be doing this once a month,  or at least once a quarter, so you'll knowledge my vary,  but I've tried to document it as much as I could.  So why did I want to do this?  I wanted to maybe validate some ideas with you,  and maybe you can validate some ideas with me.  I've done quite a few services in the past.  I've worked for Yahoo before I went to Nokia,  and so I also did www.yahoo.com to adopt my Yahoo.  So I have a little bit of background in that area.  Maybe I'll get some feedback from you,  and we would like to see you try.  Hopefully I'll help you save some money.  There was a lot of frustration while I was doing this,  so hopefully you don't have to go through it,  and there's already stuff that some artifacts are produced,  so maybe you can use some of that.  In the process, I've discovered some problems and issues,  and maybe everybody knows about them.  I'm not very strongly connected to the PhosphorG community,  although I've been an open source guy for a very long time.  So I'd like to hear if somebody knows if there's work in progress  to address the risks when we bring them up.  And why I wanted to do this is basically because I think right now  we are at a stage where everybody's talking about open data  and geo and location,  and I think this is a very good opportunity  where you may be called upon to contribute,  either in your organization or in your community or university,  using open source technologies,  maybe save some taxpayers some dollars,  and at the same time have some fun.  So I think this is kind of my main motivation is that  I think this is an opportunity where we can do some disruption  with open source in the geo-location space,  and I think I can help a little bit.  So a little bit of background.  So cloud computing a couple of years ago was just a buzzword,  and I'm like, yeah, everybody's talking about this,  and it's kind of emerged out of that stigma of being a buzzword  into a reality where people actually use it and they can deploy services to it.  It had, you know, immensely lowered the bar for entry,  the barrier to entry for start-up companies,  for nonprofit organizations and so on.  But this goes back a few years, maybe a little bit over a decade.  I think it all started with virtualization.  I'm not going to go into the ancient history before VMware and virtualization on the X86,  but VMware and PaL used to ship commercial products,  and they had some customers, a lot of people used them in lab environments and so on.  But it wasn't massively adopted.  Solaris, you know, back when Sun existed,  they also did Solaris zones and containers,  but it wasn't really until Zen before KVM came up,  and they came with this power virtualization technique  which really helped the performance,  and this became a very viable solution to run things in a production environment.  And so that was kind of like the disruption.  I think Zen really tipped the scale here.  At the same time, we were having problems with hardware,  so we were not able to deliver any faster processors.  Problems with cooling, problems with power,  and the solution from the hardware vendors was to go for multicore,  so you don't have faster processors, you just have more.  So that was one thing, and then they caught on,  that if they wanted people to run things on multiple computers,  multiple processors, the software at the time was not quite ready,  so how do you utilize that?  So virtualization was a natural solution,  but because of the performance, it was still not quite up to snuff.  Some hesitation was there,  and the hardware vendors started supporting virtualization into the chipset.  So first AMD came up with nested page tables,  and then Intel did extended page tables  where you can virtualize the page table  so the virtual machine doesn't have to contact switch all the time.  Later on, there was IOL offloading, so TCP offloading, for example.  You can just process things in the neck  without having to go up to the main processor anymore.  Same thing for storage.  And then the storage and network vendors started thinking about  how could they support this from an infrastructure standpoint.  So there was virtualization in the storage.  They kind of called everything that they had done before virtualization,  even if it wasn't really new,  but volumes became virtual volumes,  and slices became virtual slices, and so on.  But yes, so it started to gain momentum from there.  And then on the consumer side,  we started seeing smartphone stabilizers and multi-screens,  and people wanting to use services  and be able to access the same thing from everywhere.  So now it became the idea,  okay, we don't want to re-store things on desktops anymore,  and maybe we store them on servers,  but the servers have to be accessible,  and this is where cloud computing kind of crystallized.  AWS had really been pushing this for a long time,  so they have really done a lot of work there,  and they're way ahead of everybody else,  as far as I know, in terms at least in breadth of coverage.  OpenStack is trying to catch on.  I think they do great work as well,  but it's just, you know, they deliver software,  and now there are companies which are trying to take the software  and build infrastructure and services around it.  So my definition of cloud computing,  and this is just a definition,  it's a computing paradigm where it's composed of abstractions,  a set of primitives, and some interfaces and tools around them.  The idea is that you try to hide the physical stuff,  the stuff that's hard to move,  the stuff that you don't want to be tied to,  so you want to abstract that as much as possible,  and then you have a new set of primitives,  some of them not necessarily very new,  but images, for example, is a primitive.  Snapshots, volumes, a region, availability zones,  they may have other terms for other providers,  but they basically talk about the same thing,  they're trying to abstract the data center,  the actual computer or the actual hard disk away from you.  And then tools and administrative utilities around that.  What happens is that once cloud computing kicked off  and people started deploying virtual machines and cloud and so on,  things spiraled out of control really fast,  and it wasn't in a very good shape to begin with,  so the tools and automation also really helped set that path,  so puppet chef, any other configuration, CF Engine 3,  any other configuration management,  but the idea is that you have the primitives,  you have the abstractions,  and you have the tools to manage them.  So this is kind of like a block diagram,  so at the very bottom you have the physical stuff,  and then the primitives sit on top,  and you have the tools and APIs at the highest level.  We can even go further up,  so if you look at things like Heroku, for example,  they abstract even more where you just deploy to a platform,  so you're very far away from everything else,  you just have a command, you run it, you got a service.  So this is kind of a clean representation of what it looks like.  This is open stack implementation,  so there's quite a few areas going back and forth,  and this is kind of what it looks like in real life,  so this is clock computing,  and that machine is very important,  because if that gets unplugged, the whole thing goes to shit.  Alright, so AWS is a public cloud,  so the same kind of diagram,  but it would just be a little bit more specific,  so we talked about compute as an EC2 instances storage.  So EC2 instances is just virtual machines,  they have a predefined set of configuration,  so you cannot change or tweak the CPU or memory settings,  you can just choose one of the models,  you can attach drives as you wish.  Then a set of storage, so S3 is like a storage over HTTP or HTTPS,  and they have elastic block storage,  which is kind of a NAS, or a SAM idea,  and Glass-Hero is kind of a long-term archival.  They have the foundation,  you know, the regions, the actual brick-and-mortar implementation,  the data centers, the power, the cooling,  all the stuff that we don't want to think about.  And networking, so we're with 53s DNS service,  elastic load balancing,  CloudFront is a caching service,  and a set of tools around security,  so identity management security groups.  You go up one level, you see the simple queuing service,  or search as a service, or redshift,  which is post-glace SQL,  and storage, kind of,  so if you don't want to run a cluster,  and you don't want to manage it,  then they will do it for you,  and you just put your schema, you connect to it,  and you treat it just like a post-guess, or post-gray.  Unfortunately, at the minimum, they don't support spatial,  so it's only post-gray SQL.  And there's more.  There's simple email service,  simple modification service, and so on.  And the management layer is the API,  is the outer scale, the CloudFormation,  and that configuration, and so on.  So what kind of use cases we can do with Phosphor Gene  in the Cloud?  Well, for start, disaster recovery backup,  so it's very simple, very easy to just dump a table,  or archive your SQL dump,  encrypt it, ship it over to an S3 bucket,  get it back when you need it,  hopefully you never need it.  So this is a very straightforward use case.  The other use case is static logic free web publishing.  So if you just have some vector data, or roster data,  or any kind of static data,  where you're not doing any logic,  anybody who can make a request can get back that response.  You can just publish this using S3 and CloudFront.  I'll show you an example, a diagram.  You don't have to run a web server,  you don't have to run load balancer,  you don't have to do anything.  You just publish it,  and you will pay for the request as they come in,  but you're not going to have to maintain any infrastructure.  Obviously, online Phosphor Gene,  so you can do geocoding, or tiling, or routing, and so on.  So any of the software that is available to us  under a public license, you can just run it.  If you run a GPL license software,  you have to make sure that this you're compiling is that license.  Data transformation drops.  If you have a set of tiles,  or a set of data, and you want to just transform them  from one format to another,  or maybe you have four or five different formats,  and you have to do this overnight,  you don't need to borrow a bunch of machines  and just have them sit for the rest of the day,  so you can just fire a job, get it done, and shut them back.  Consecuration batch processes,  so again, the same kind of concept,  if you're collaborating with other people  and you want to have some central storage,  where they can upload their files,  and maybe you can do some processing, put it back, and so on.  So this is like a blueprint.  If you wanted to do this static logic-free content using AWS,  this is your content,  and you can just put it to this S3 bucket.  You can configure a platform distribution,  which will point to this bucket,  and you publish it,  and you make your DNS-seaning areas to this zone  that you are going to configure here,  and that's it.  Now users will go request your data,  they will get seen into the CloudFront zone,  and based on some telemetry and some other magic,  they will get routed to the closest cache edge to them.  If you want to have logs,  you can also configure logs to go to an S3 bucket,  where you can just retrieve them back later.  So how do you build it?  If you wanted to do this maybe for a university or for a district  or just your company, how do you do that?  So I think there are some architectural patterns,  and you'll not see this in books,  I kind of came up with this overnight,  and I just wanted to share them with you,  so you're aware of them,  they fit some things better than others,  sometimes you have to mix them,  everything in the world is almost polyglot,  so this is not a holy book.  So the cookie-cutters,  the idea is that you have a machine,  the machine has everything that you would need,  and you just manufacture them,  you just have 10, 20, 100, as many as you need,  or as many as you need.  They have everything together,  so they have the application and they have the data,  and they scale horizontally,  so if the traffic is actually growing,  then you can just scale up,  if the traffic is dying down,  or on a low point, you can just shrink them.  The data is accessible to the machine itself,  they are not connected to each other in any way,  so if in case they fail,  the failures look like.  So very simple,  scales very well in some use cases,  so simplicity is one of the pros,  and scales horizontally was laid,  and localized failure impact,  these are the main points.  Problems is that poor support for right-oriented service.  If you look at here,  if you have so many machines,  and everyone has a copy of the data,  and the data has to change,  then you have to push this back somewhere, somehow.  And if the users are allowed to change the data,  that's even worse,  because now one machine is going to change,  and it's going to have to replicate,  then that doesn't work very well.  It's a coarse grain scalability,  so when you scale, you scale everything,  or you shrink everything.  If you actually have a service where your data layer is very, very fast,  but your web service is not so fast,  so you need a web service,  or web applications,  you can't do it, you're going to have to scale the whole thing together.  So it's a cookie cutter.  The load capacity has vertical scalability issues,  so if you have data growing,  or if your memory consumption is growing,  you're going to hit the ceiling at some point,  where you can't just grow anymore within that box.  So there is a vertical ceiling,  and how much you can process per node.  When there's a centrist approach,  basically you take the data out,  and you let the application run on the nodes,  you can have a second copy of the data  as a backup disaster recovery,  and this is, you know,  now the data is centralized in this database,  these are the clients of this,  and then the user is out there,  they're the clients of your web service.  Works okay in a lot of cases.  Scales well for mid-level loads,  so if you're doing 20, 30, 40 new requests per second or so,  it probably works.  It has some other issues.  So the pros first,  you know, you can actually scale the web service by itself,  or you can scale the database by itself,  and that's a big advantage over the other approach.  Five minutes.  Okay, and we have to run with the slides.  So the replicator, the master of colonies,  where you can have masters of freckles  distributed over the world,  and they can replicate to each other as read-only slaves,  and that works pretty good for read scalability.  You're going to have to do some culture changes.  You know, you release engineering,  this has to be in really good shape.  You have to adopt automation.  You really need to think about agility.  You need to think about using the primitives  that are available to you.  You need to make sure that you get a buy-in from the stakeholders.  These are very, very key things.  Some process changes.  And some of the things that you have to remember,  the leader implications,  don't try to scale in the cloud as you would in a brick-and-mortar situation.  So don't try to go for a,  let's cluster these machines together  and have a big cluster and four or five clusters.  Just things fail and just plan for it,  and it's okay,  and just think about how you could recover as fast as possible.  It may take one or two tries.  You probably get it right on the third time.  But the old approach, you know,  trying to connect things and make sure they are going to be reliable.  It doesn't work the wrong way.  I did some other work in this process.  I'll go through it really quick.  When I started, I wanted to see if I can profile a renderer, a geocoder,  a router in the cloud.  And then I hit the first problem.  I wanted to get data in.  And then I started reading about people taking ten days to get the OSM data set.  I thought that was horrible.  I didn't want to get a synthetic data set with 200 megs  and say, yeah, this works and I'm sure it will heal.  So I really wanted to get the OSM data in.  And I did.  So first I did some tests.  I looked at a bunch of countries, small countries.  This is the time it took in seconds versus the size of the data set.  So up to 3.2 gigs.  We are within a 30 minutes or 35 minutes range.  I started collecting some stats around this.  And I provisioned different infrastructures.  So I looked at the local drive, how long it took.  I looked at provision IOPS, which guarantees IOPS performance.  I looked at SSD, which is very expensive.  And so guess how long it took.  I went with the SSD one because I wanted to finish as fast as possible.  Any guesses how long it took?  Ah, you're right.  An hour, I wish.  It took 35 hours.  But this is 250 gig.  This is not so bad because a lot of people spend six or seven days to get this done.  So this is actually not so bad at all.  But guess what I did after I just finished this?  No, I made a copy.  So I made a copy.  I built a RAID zero set over provision IOPS.  And I created a logical volume on top.  And I kicked off a data copy.  And guess how long that took?  Wow, it's short.  Okay, it took two and a half hours.  So actually, this is a file system copy, right?  So I just shut down the database and then I copy the volume over.  And then I archive it, of course.  And this took two and a half hours.  So this is what it took to do a provision IOPS to SSD.  It took five and a half hours.  It took about seven hours to do a SQL dump to SSD.  SSD took provision IOPS to two and a half hours.  SSD to SQL dump took four and a half hours.  OSM to PG SQL to SSD took 35 hours.  So guess where the problem is?  So this is a profile for OSM to PG SQL.  You can see the rank hash nodes get 15%, Y max 10%, copy to table 10%.  So there's a lot of things going on before the data actually is stored in the database.  And this is part of the problem.  And I wanted to talk about that a little bit more.  So you can read these notes later.  Then I did some profiling with Matta and Matnik.  One thread, three threads.  So if you run this four threads, if you run this four rendering threads, you actually get six threads.  So two threads do the bookkeeping for threads due to actual work.  You're going to have to read this unfortunately.  I'm going to run to the G-server part a little bit.  So G-server, single layer, I took a small country called Finland.  And zoom level 15 from 0 to 1 to 15.  And this I did around this.  And this is about 100 tiles per second.  You can do about 100 tiles per second in that kind of setup.  And this is kind of a ceiling because this is RAM disk.  So it's not going to get any faster.  Well, it could, but it would be very expensive.  Trankation is very slow.  So doing Tranky, your G-web cache, try to publish your data as version data, as version layers.  This is much better if you can help it.  Stano-known G-web cache will work a lot better.  So try to think about just yanking that GW cache out and put some G-web servers behind it.  I'll show you a new point. There's some possibly waste conditions in thread writing tiles.  And this is kind of an example deployment where you can take the G-web cache out  and you put G-servers behind them.  And you put a load balancer that can do URL persistence.  So these nodes are not coherent.  They are incoherent by design.  The idea is that you will go to the node that has the tile.  If there is no node that has the tile, one will get selected and then from there on, it will be persistent.  You can mix the disks so you can mix a fast disk and slow disk in a volume.  That will probably give you a very good performance.  How much did all this cost? $866 and two weeks.  And then I have a backlog where these snapshots to the public.  So I actually have the data now in AWS.  I'm going to make it public so you can just import it.  It should hopefully help out.  I'm going to do some G-coding profiling in an OSRM profiler.  And I'm open to suggestions as well.  Thank you very much.  So you took off all of your time.  So I'm afraid you don't have time for any questions.  I'm sure your slides will go up on the ELO geoplatform.
FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__0_1427.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__1428_2711.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__2712_3827.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__3828_7055.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__7056_13703.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__13704_13751.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__13752_16019.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__16020_17003.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__17004_17255.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__17256_17519.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__17520_20591.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__20592_24023.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__24024_25391.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__25392_25691.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__25692_26411.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__26412_27803.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__27804_28259.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__28260_28751.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__28752_29927.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__29928_31319.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__31320_31847.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__31848_31919.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__31920_32327.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__32328_32903.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__32904_32963.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__32964_34127.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__34128_34199.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__34200_35159.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__35160_35747.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__35748_36143.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__36144_36659.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__36660_37175.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__37176_37655.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__37656_38099.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__38100_38903.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__38904_39455.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__39456_39563.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__39564_39635.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__39636_40247.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__40248_41891.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__41892_42815.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__42816_43043.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__43044_43451.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__43452_43871.png FOSS4G_In_AWS__Choosing__Deploying_And_Tuning_Open_Source_Components_In_AWS__720p__43872_44011.png
Speaker is Mohamed Sayed, so I would then invite him to go on the stage and start his presentation. He is from here, which in Dutch is strange because it means he is from here. He lives here, but he is working for a company I guess called here. By the way, I'm not the one on the nerd, so I do want you to switch off your phones, or put them to silence, please, because it's annoying for the speakers. Thank you very much. Hi, good morning. So, yeah, I work for Heel, which is Nokia's location and commerce, but I'm not here for Heel, I'm just here on my own, and I put this presentation together, and I hope that it will be a contribution to the community. So, this is Agenda, and a couple of the screeners, one you already heard and just one more, some goals and motives, and a little bit of historical background, definition of glass computing, so we can talk about the same thing. And some use cases for phosphor G, and then I'll talk about AWS, the components and service at a high level, and if you want to be in the cloud, what you're going to be doing, and how you're going to be doing it, and then some common-foss tasks. If you are building an SDI in the cloud, you're going to have to import some data, you're going to do some rendering, some geocoding, and so on. And then hopefully we'll have time for questions. So, you heard the first one I worked for here, I'm a senior architect in the core platform group for Nokia's location and commerce, but again, this is just personal work, personal effort. I'm also not affiliated with AWS other than being a customer, so I use them to make sense, if somebody else makes more sense, I'll be using them, especially if they give me a bell-a-bell for the buck. This is still work in progress, I hope to be doing this once a month, or at least once a quarter, so you'll knowledge my vary, but I've tried to document it as much as I could. So why did I want to do this? I wanted to maybe validate some ideas with you, and maybe you can validate some ideas with me. I've done quite a few services in the past. I've worked for Yahoo before I went to Nokia, and so I also did www.yahoo.com to adopt my Yahoo. So I have a little bit of background in that area. Maybe I'll get some feedback from you, and we would like to see you try. Hopefully I'll help you save some money. There was a lot of frustration while I was doing this, so hopefully you don't have to go through it, and there's already stuff that some artifacts are produced, so maybe you can use some of that. In the process, I've discovered some problems and issues, and maybe everybody knows about them. I'm not very strongly connected to the PhosphorG community, although I've been an open source guy for a very long time. So I'd like to hear if somebody knows if there's work in progress to address the risks when we bring them up. And why I wanted to do this is basically because I think right now we are at a stage where everybody's talking about open data and geo and location, and I think this is a very good opportunity where you may be called upon to contribute, either in your organization or in your community or university, using open source technologies, maybe save some taxpayers some dollars, and at the same time have some fun. So I think this is kind of my main motivation is that I think this is an opportunity where we can do some disruption with open source in the geo-location space, and I think I can help a little bit. So a little bit of background. So cloud computing a couple of years ago was just a buzzword, and I'm like, yeah, everybody's talking about this, and it's kind of emerged out of that stigma of being a buzzword into a reality where people actually use it and they can deploy services to it. It had, you know, immensely lowered the bar for entry, the barrier to entry for start-up companies, for nonprofit organizations and so on. But this goes back a few years, maybe a little bit over a decade. I think it all started with virtualization. I'm not going to go into the ancient history before VMware and virtualization on the X86, but VMware and PaL used to ship commercial products, and they had some customers, a lot of people used them in lab environments and so on. But it wasn't massively adopted. Solaris, you know, back when Sun existed, they also did Solaris zones and containers, but it wasn't really until Zen before KVM came up, and they came with this power virtualization technique which really helped the performance, and this became a very viable solution to run things in a production environment. And so that was kind of like the disruption. I think Zen really tipped the scale here. At the same time, we were having problems with hardware, so we were not able to deliver any faster processors. Problems with cooling, problems with power, and the solution from the hardware vendors was to go for multicore, so you don't have faster processors, you just have more. So that was one thing, and then they caught on, that if they wanted people to run things on multiple computers, multiple processors, the software at the time was not quite ready, so how do you utilize that? So virtualization was a natural solution, but because of the performance, it was still not quite up to snuff. Some hesitation was there, and the hardware vendors started supporting virtualization into the chipset. So first AMD came up with nested page tables, and then Intel did extended page tables where you can virtualize the page table so the virtual machine doesn't have to contact switch all the time. Later on, there was IOL offloading, so TCP offloading, for example. You can just process things in the neck without having to go up to the main processor anymore. Same thing for storage. And then the storage and network vendors started thinking about how could they support this from an infrastructure standpoint. So there was virtualization in the storage. They kind of called everything that they had done before virtualization, even if it wasn't really new, but volumes became virtual volumes, and slices became virtual slices, and so on. But yes, so it started to gain momentum from there. And then on the consumer side, we started seeing smartphone stabilizers and multi-screens, and people wanting to use services and be able to access the same thing from everywhere. So now it became the idea, okay, we don't want to re-store things on desktops anymore, and maybe we store them on servers, but the servers have to be accessible, and this is where cloud computing kind of crystallized. AWS had really been pushing this for a long time, so they have really done a lot of work there, and they're way ahead of everybody else, as far as I know, in terms at least in breadth of coverage. OpenStack is trying to catch on. I think they do great work as well, but it's just, you know, they deliver software, and now there are companies which are trying to take the software and build infrastructure and services around it. So my definition of cloud computing, and this is just a definition, it's a computing paradigm where it's composed of abstractions, a set of primitives, and some interfaces and tools around them. The idea is that you try to hide the physical stuff, the stuff that's hard to move, the stuff that you don't want to be tied to, so you want to abstract that as much as possible, and then you have a new set of primitives, some of them not necessarily very new, but images, for example, is a primitive. Snapshots, volumes, a region, availability zones, they may have other terms for other providers, but they basically talk about the same thing, they're trying to abstract the data center, the actual computer or the actual hard disk away from you. And then tools and administrative utilities around that. What happens is that once cloud computing kicked off and people started deploying virtual machines and cloud and so on, things spiraled out of control really fast, and it wasn't in a very good shape to begin with, so the tools and automation also really helped set that path, so puppet chef, any other configuration, CF Engine 3, any other configuration management, but the idea is that you have the primitives, you have the abstractions, and you have the tools to manage them. So this is kind of like a block diagram, so at the very bottom you have the physical stuff, and then the primitives sit on top, and you have the tools and APIs at the highest level. We can even go further up, so if you look at things like Heroku, for example, they abstract even more where you just deploy to a platform, so you're very far away from everything else, you just have a command, you run it, you got a service. So this is kind of a clean representation of what it looks like. This is open stack implementation, so there's quite a few areas going back and forth, and this is kind of what it looks like in real life, so this is clock computing, and that machine is very important, because if that gets unplugged, the whole thing goes to shit. Alright, so AWS is a public cloud, so the same kind of diagram, but it would just be a little bit more specific, so we talked about compute as an EC2 instances storage. So EC2 instances is just virtual machines, they have a predefined set of configuration, so you cannot change or tweak the CPU or memory settings, you can just choose one of the models, you can attach drives as you wish. Then a set of storage, so S3 is like a storage over HTTP or HTTPS, and they have elastic block storage, which is kind of a NAS, or a SAM idea, and Glass-Hero is kind of a long-term archival. They have the foundation, you know, the regions, the actual brick-and-mortar implementation, the data centers, the power, the cooling, all the stuff that we don't want to think about. And networking, so we're with 53s DNS service, elastic load balancing, CloudFront is a caching service, and a set of tools around security, so identity management security groups. You go up one level, you see the simple queuing service, or search as a service, or redshift, which is post-glace SQL, and storage, kind of, so if you don't want to run a cluster, and you don't want to manage it, then they will do it for you, and you just put your schema, you connect to it, and you treat it just like a post-guess, or post-gray. Unfortunately, at the minimum, they don't support spatial, so it's only post-gray SQL. And there's more. There's simple email service, simple modification service, and so on. And the management layer is the API, is the outer scale, the CloudFormation, and that configuration, and so on. So what kind of use cases we can do with Phosphor Gene in the Cloud? Well, for start, disaster recovery backup, so it's very simple, very easy to just dump a table, or archive your SQL dump, encrypt it, ship it over to an S3 bucket, get it back when you need it, hopefully you never need it. So this is a very straightforward use case. The other use case is static logic free web publishing. So if you just have some vector data, or roster data, or any kind of static data, where you're not doing any logic, anybody who can make a request can get back that response. You can just publish this using S3 and CloudFront. I'll show you an example, a diagram. You don't have to run a web server, you don't have to run load balancer, you don't have to do anything. You just publish it, and you will pay for the request as they come in, but you're not going to have to maintain any infrastructure. Obviously, online Phosphor Gene, so you can do geocoding, or tiling, or routing, and so on. So any of the software that is available to us under a public license, you can just run it. If you run a GPL license software, you have to make sure that this you're compiling is that license. Data transformation drops. If you have a set of tiles, or a set of data, and you want to just transform them from one format to another, or maybe you have four or five different formats, and you have to do this overnight, you don't need to borrow a bunch of machines and just have them sit for the rest of the day, so you can just fire a job, get it done, and shut them back. Consecuration batch processes, so again, the same kind of concept, if you're collaborating with other people and you want to have some central storage, where they can upload their files, and maybe you can do some processing, put it back, and so on. So this is like a blueprint. If you wanted to do this static logic-free content using AWS, this is your content, and you can just put it to this S3 bucket. You can configure a platform distribution, which will point to this bucket, and you publish it, and you make your DNS-seaning areas to this zone that you are going to configure here, and that's it. Now users will go request your data, they will get seen into the CloudFront zone, and based on some telemetry and some other magic, they will get routed to the closest cache edge to them. If you want to have logs, you can also configure logs to go to an S3 bucket, where you can just retrieve them back later. So how do you build it? If you wanted to do this maybe for a university or for a district or just your company, how do you do that? So I think there are some architectural patterns, and you'll not see this in books, I kind of came up with this overnight, and I just wanted to share them with you, so you're aware of them, they fit some things better than others, sometimes you have to mix them, everything in the world is almost polyglot, so this is not a holy book. So the cookie-cutters, the idea is that you have a machine, the machine has everything that you would need, and you just manufacture them, you just have 10, 20, 100, as many as you need, or as many as you need. They have everything together, so they have the application and they have the data, and they scale horizontally, so if the traffic is actually growing, then you can just scale up, if the traffic is dying down, or on a low point, you can just shrink them. The data is accessible to the machine itself, they are not connected to each other in any way, so if in case they fail, the failures look like. So very simple, scales very well in some use cases, so simplicity is one of the pros, and scales horizontally was laid, and localized failure impact, these are the main points. Problems is that poor support for right-oriented service. If you look at here, if you have so many machines, and everyone has a copy of the data, and the data has to change, then you have to push this back somewhere, somehow. And if the users are allowed to change the data, that's even worse, because now one machine is going to change, and it's going to have to replicate, then that doesn't work very well. It's a coarse grain scalability, so when you scale, you scale everything, or you shrink everything. If you actually have a service where your data layer is very, very fast, but your web service is not so fast, so you need a web service, or web applications, you can't do it, you're going to have to scale the whole thing together. So it's a cookie cutter. The load capacity has vertical scalability issues, so if you have data growing, or if your memory consumption is growing, you're going to hit the ceiling at some point, where you can't just grow anymore within that box. So there is a vertical ceiling, and how much you can process per node. When there's a centrist approach, basically you take the data out, and you let the application run on the nodes, you can have a second copy of the data as a backup disaster recovery, and this is, you know, now the data is centralized in this database, these are the clients of this, and then the user is out there, they're the clients of your web service. Works okay in a lot of cases. Scales well for mid-level loads, so if you're doing 20, 30, 40 new requests per second or so, it probably works. It has some other issues. So the pros first, you know, you can actually scale the web service by itself, or you can scale the database by itself, and that's a big advantage over the other approach. Five minutes. Okay, and we have to run with the slides. So the replicator, the master of colonies, where you can have masters of freckles distributed over the world, and they can replicate to each other as read-only slaves, and that works pretty good for read scalability. You're going to have to do some culture changes. You know, you release engineering, this has to be in really good shape. You have to adopt automation. You really need to think about agility. You need to think about using the primitives that are available to you. You need to make sure that you get a buy-in from the stakeholders. These are very, very key things. Some process changes. And some of the things that you have to remember, the leader implications, don't try to scale in the cloud as you would in a brick-and-mortar situation. So don't try to go for a, let's cluster these machines together and have a big cluster and four or five clusters. Just things fail and just plan for it, and it's okay, and just think about how you could recover as fast as possible. It may take one or two tries. You probably get it right on the third time. But the old approach, you know, trying to connect things and make sure they are going to be reliable. It doesn't work the wrong way. I did some other work in this process. I'll go through it really quick. When I started, I wanted to see if I can profile a renderer, a geocoder, a router in the cloud. And then I hit the first problem. I wanted to get data in. And then I started reading about people taking ten days to get the OSM data set. I thought that was horrible. I didn't want to get a synthetic data set with 200 megs and say, yeah, this works and I'm sure it will heal. So I really wanted to get the OSM data in. And I did. So first I did some tests. I looked at a bunch of countries, small countries. This is the time it took in seconds versus the size of the data set. So up to 3.2 gigs. We are within a 30 minutes or 35 minutes range. I started collecting some stats around this. And I provisioned different infrastructures. So I looked at the local drive, how long it took. I looked at provision IOPS, which guarantees IOPS performance. I looked at SSD, which is very expensive. And so guess how long it took. I went with the SSD one because I wanted to finish as fast as possible. Any guesses how long it took? Ah, you're right. An hour, I wish. It took 35 hours. But this is 250 gig. This is not so bad because a lot of people spend six or seven days to get this done. So this is actually not so bad at all. But guess what I did after I just finished this? No, I made a copy. So I made a copy. I built a RAID zero set over provision IOPS. And I created a logical volume on top. And I kicked off a data copy. And guess how long that took? Wow, it's short. Okay, it took two and a half hours. So actually, this is a file system copy, right? So I just shut down the database and then I copy the volume over. And then I archive it, of course. And this took two and a half hours. So this is what it took to do a provision IOPS to SSD. It took five and a half hours. It took about seven hours to do a SQL dump to SSD. SSD took provision IOPS to two and a half hours. SSD to SQL dump took four and a half hours. OSM to PG SQL to SSD took 35 hours. So guess where the problem is? So this is a profile for OSM to PG SQL. You can see the rank hash nodes get 15%, Y max 10%, copy to table 10%. So there's a lot of things going on before the data actually is stored in the database. And this is part of the problem. And I wanted to talk about that a little bit more. So you can read these notes later. Then I did some profiling with Matta and Matnik. One thread, three threads. So if you run this four threads, if you run this four rendering threads, you actually get six threads. So two threads do the bookkeeping for threads due to actual work. You're going to have to read this unfortunately. I'm going to run to the G-server part a little bit. So G-server, single layer, I took a small country called Finland. And zoom level 15 from 0 to 1 to 15. And this I did around this. And this is about 100 tiles per second. You can do about 100 tiles per second in that kind of setup. And this is kind of a ceiling because this is RAM disk. So it's not going to get any faster. Well, it could, but it would be very expensive. Trankation is very slow. So doing Tranky, your G-web cache, try to publish your data as version data, as version layers. This is much better if you can help it. Stano-known G-web cache will work a lot better. So try to think about just yanking that GW cache out and put some G-web servers behind it. I'll show you a new point. There's some possibly waste conditions in thread writing tiles. And this is kind of an example deployment where you can take the G-web cache out and you put G-servers behind them. And you put a load balancer that can do URL persistence. So these nodes are not coherent. They are incoherent by design. The idea is that you will go to the node that has the tile. If there is no node that has the tile, one will get selected and then from there on, it will be persistent. You can mix the disks so you can mix a fast disk and slow disk in a volume. That will probably give you a very good performance. How much did all this cost? $866 and two weeks. And then I have a backlog where these snapshots to the public. So I actually have the data now in AWS. I'm going to make it public so you can just import it. It should hopefully help out. I'm going to do some G-coding profiling in an OSRM profiler. And I'm open to suggestions as well. Thank you very much. So you took off all of your time. So I'm afraid you don't have time for any questions. I'm sure your slides will go up on the ELO geoplatform.
talk about AWS, the components and service at a effort. I'm also not affiliated with AWS other bell-a-bell for the buck. This is still work in your community or university, using open source so AWS is a public cloud, so the same kind of using S3 and CloudFront. I'll show you an and so on. So any of the software that is work very well. It's a coarse grain scalability, by design. The idea is that you will go to the performance. How much did all this cost? $866 and any questions. I'm sure your slides will go up on