Managing update of tiles of dynamic data
Assink, Herman (IDgis B.V.)
Orbons, Erik (IDgis bv)
Tiling is currently the best solution to achieve high performance and throughput for serving map images. However, because tile images have to be prepared, tiling is often only used for relatively static data. The Dutch national facility for presenting governmental spatial plans, www.ruimtelijkeplannen.nl, is used intensively (app. 15.000 plans) and has high performance demands and therefore wanted to make use of tiling. Because plans often change and are added and deleted from the central database, a special solution was developed to manage the daily update of tiles. The presentation will concentrate on the solution to manage the daily update of tiles, the Tiling Manager. The Tiling Manager software queries the audit trail of plan updates, executes tiling tasks in collaboration with GeoWebCache and monitors progress. We had to deal with several challenges to realize the requirements, such as monitoring progress of tiling tasks in GeoWebCache, run-time generation of GeoWebCache configuration, optimizing WMS performance and assuring that the services will never present old tiles if new plans are available. In addition to the solutions to these challenges the presentation will show the technical architecture of the Tiling Manager.
 I'm trying to make it keep it brief. Good afternoon everybody. Just after 12. My name is Rob Stecklenburg. I'm working for Ideages in the Netherlands, software developer. Ideages is a small company in the Netherlands, a geo company that has clients among mainly the governmental organizations, provinces, councils, police. And we were once involved in a project, a national project where we had to use tiling, but where the map data was updated very regularly. And we had to meet and solve some challenges in that respect and develop special software to do what we wanted to keep up with the goals of the program. The Dutch government has a national facility for showing spatial plans at all levels, council, governmental, provincial, all kinds of spatial plans that are useful for public use and professional use. And at the moment we have around 15,000 plans in the system and it's growing daily, it's really taking off. It's publicly available through a website and besides the website there are facilities for professional use as well. You see, we have 15,000 plans, every dot is a plan and it's almost the view of the Netherlands is coming out through all the small dots. Users have to be able to zoom in to plans, to pan and to zoom out. So we need a good performance and that was one of the demands that the project had. I show you an example, a plan, more in detail from the hometown of our company, Risen in the east of the Netherlands. And the website gives wonderful means of searching. I just found this plan in our town by typing in the address of our company's headquarters. And it's a small plan that says something about how to develop the center of this town, all the documents that belong to it, all the official documents can be found as well. And this is a really common local plan, so to say. It's difficult to translate the specific items into English, but you can say this is a local plan and each local plan has about four to five map layers, including borders, including an overview layer. And these five layers are used for the majority of plans in this facility. And they all constitute the whole service area of the Netherlands. Maybe a few percent of the total number of plans is different. They have often have a larger service area. They are more in the national area or the provincial areas of spatial plans. And they have their own specific layers with them. And that makes one of the challenges in this case. The topography, the map of the town and surroundings that you see is pre-tiled because it changes maybe once a year or even less. So we do that on separate hardware. In this spatial plans project, plans are updated daily because councils and everybody who is using it from the government really wants to be able to present the new data as soon as it is ready. So we have from a few plans a day to a few hundred plans a day per day that are updated in the system and have to be shown on the map as soon as possible. We provide for a number of zoom levels from the overview of the Netherlands up to the utmost detail of a certain plan. And we need to do that for different map layers as well. Most plans contain several layers like the plan I just showed before contains five layers. And some of these special plans contain ten or more layers as well. And we have to provide for that in total we have one known, they said, seven hundred map layers in the system that we have to provide to users. To give you an example of how many tiles that could be for the topography layer, I have been counting the number of tiles for the whole Netherlands for all these zoom levels. And that's for one layer only five and a half million tiles. And we have seven hundred layers in total. And this means that if we would not use tiling it would result in high demand on the WMS. Because at the daytime the map server is also used by professionals as well. And it serves the map data for the website. So we had to come with a solution. And we had to use tiling to diminish the demand on the services. But what we only wanted to do is tile parts of the layer that were changed. We can't change the whole layers every time. And the customer also had said that they wanted to be ready with tiling before office hours. So there would not be any stress on the WMS during office hours. We come up with a solution that we call the tiling manager. Static data is pre-tiled like topography. And dynamic data which can change per day is tiled before office hours. We use the web cache which is a perfectly sound solution for tiling, for static tiling that is. So in this case we're not smiling yet. We do need something extra. And that is the tiling manager which sits between the planned database of the whole project and the web cache. And it queries the planned database on a daily basis and then makes tasks that it can issue to the web cache. And we had to change to adapt the web cache as well to be able to communicate it with it in a way that we wanted and to be able to monitor the progress of the web cache. We faced a number of challenges. As I said the monitoring of the web cache was wanted and we also wanted to be able to stop the web cache because the tiling manager can get a command to stop everything it is doing at that moment. So it has to stop the geocache instance as well with tasks at hand. And we had to extend the rest interface of geobab cache. The tiling manager itself has a simple information page which shows the plans at hand, all the layers per plan, the zoom level to which the layers have to be tiled. And during tiling it gives a percentage of the tiles ready and also an estimate of the time still to expect the tiling will need to go on. And because we have several plans that have their own layers, the geo web cache has a configuration file with all the map layers inside and we only have five of these common layers that are common for a lot of plans. But there are some plans that use specific layers which we can't predict beforehand. So we have to generate a configuration for geo web cache using a template and the tiling manager knows about the plans, it knows the template, it fills in all the details and it makes a proper configuration for geo web cache that it needs. We also did some work on optimizing the WMS performance mainly by using some extra caching so that the database requests were diminished, the number of them. And we also found it very handy to be able to shrink the PNGs that were produced, 24-bit PNGs to 8-bit using PNG quant which is a standalone program that we use for every tile and the transformation is also built into geo web cache. And it means that I think the size of these tiles is indeed going from 100% to 50% or 30% size. So it's really helpful to get less transport over the internet. And what was the process that we had up-to-date tiles? So when a council changes a plan today and they upload it to the site, they really want the map and the view in the next day to be in order. We can't have old tiles running around or still being there. So up-to-date tiles was very important and this was ensured by the cycle that we made, we built into this tiling manager. We have the plan database and the tiling manager takes all the plans that were updated, deleted or added. It finds all the layers in all these plans. It makes a task for each layer, a geo web cache task and all these tasks are visible in the information page of the tiling manager. And the cycle starts with generating a new configuration. So for all the plans that we have in the morning, so to say, we generate a new configuration for geo web cache up-to-date. Then we're going to truncate to delete all the tiles of the changed plans of the day before. So we erase the old situation, delete all the tiles that were changed. Next we send the new configuration that we generated here. We send it to geo web cache and we reset geo web cache because we have to make sure that geo web cache can read up this new configuration file. This was the most safe option we had. Then after that, and geo web cache is back online, we give it seed tasks. So it makes the tiles based on the new configuration and it makes tiles which represent the new plans and in the map later on. And all these tasks in this info page are being done bit by bit until the list is empty and then we're ready for the day. And hopefully before office hours. A small idea about the architecture. There is one tiling manager and there can be multiple geo web cache instances and they can each run in, they run in a tomcat and they can run all in the same tomcat or in a separate tomcat or on separate servers. Communication is via HTTP. So there is a flexible way that you can deploy the tiling manager and the geo web cache instances. You can have many geo web cache instances if you want. But they all do get the same configuration. And the same configuration means that they all send their requests to the same WMS to get and produce the tiles. The tiling manager is trying to find when it goes through his list, through the cycle and through the list of tasks, it finds a geo web cache that is doing nothing at the moment and send a task to it so it can do the tiles for that plan, for that layer. And then finds the next geo web cache that is doing nothing and send another layer to that one and so forth. And if they're all busy, it just waits and monitors progress and waits until one of the geo web cache is free or the list of tasks is empty. And all that is configured in a properties file for the tiling manager. It's a bit bad. It's not readable really, but from a software perspective, there is a tiling manager class that accepts HTTP requests from users, translates that into calls to a manager service. And the manager service has a pool of that holds W, no, geo web cache instances. In the pool, we have several geo web caches representing geo web caches and the status of geo web caches is also monitored in a special class, in a special object, all is written in Java. If I show a very short overview of the database scheme, there is a table that contains the task, the cycle of the day, so to say, a row for the cycle of the day. There is a big table that contains all the status items that are important for the task at hand, the plan, the layer type, the zoom level we need to go to, etc. Then there is a table that combines the type of plan with the layers that the plan is normally containing. And we have for the pre-tiling, for example, when we want to pre-tile topography, it does not make really sense to take the bounding box of the Netherlands because we would also tile a lot of the North Sea, which is not really useful. So we made a table with seed boxes, bounding boxes around the Netherlands in a more efficient way so that most of the land of the Netherlands is covered that way. And the pre-tiling of topography in other layers is a lot faster. Well, this is not readable also, but there is a configuration file where you can really adapt the number of geowab caches, the connection to the database, passwords for Tomcat, etc., etc. And I'm really going fast. This was it already. Okay. Thank you. Any questions? I did. I did. And that was some time ago. And the comments were something like, for example, the PNG 24-bit to 8-bit conversion. They didn't like the way we did it. And for other items, they didn't take it up because they found that the classes we used, we adapted in geowab cache, would not be their choice of using our adaptations. So they didn't take our work less. No. I'm not sure that the talk will be distributed sometime. Yes, I will do so because you have 10 seconds to write down this link. Okay. I won't share it. I won't do it. I'll upload it and... Okay. Yes, but I don't. I really had to go. I really had to go. I have to catch your flight. Sorry. The main link is wwwidgs.nl.
 I'm trying to make it keep it brief. Good afternoon everybody. Just after 12. My name  is Rob Stecklenburg. I'm working for Ideages in the Netherlands, software developer. Ideages  is a small company in the Netherlands, a geo company that has clients among mainly the  governmental organizations, provinces, councils, police. And we were once involved in a project,  a national project where we had to use tiling, but where the map data was updated very regularly.  And we had to meet and solve some challenges in that respect and develop special software to do  what we wanted to keep up with the goals of the program. The Dutch government has a national  facility for showing spatial plans at all levels, council, governmental, provincial,  all kinds of spatial plans that are useful for public use and professional use. And at the  moment we have around 15,000 plans in the system and it's growing daily, it's really taking off.  It's publicly available through a website and besides the website there are facilities for  professional use as well. You see, we have 15,000 plans, every dot is a plan and it's almost  the view of the Netherlands is coming out through all the small dots. Users have to be able to  zoom in to plans, to pan and to zoom out. So we need a good performance and that was one  of the demands that the project had. I show you an example, a plan, more in detail from the  hometown of our company, Risen in the east of the Netherlands. And the website gives wonderful  means of searching. I just found this plan in our town by typing in the address of our  company's headquarters. And it's a small plan that says something about how to develop the  center of this town, all the documents that belong to it, all the official documents can be  found as well. And this is a really common local plan, so to say. It's difficult to translate  the specific items into English, but you can say this is a local plan and each local plan has  about four to five map layers, including borders, including an overview layer. And these five layers  are used for the majority of plans in this facility. And they all constitute the whole service  area of the Netherlands. Maybe a few percent of the total number of plans is different. They have  often have a larger service area. They are more in the national area or the provincial areas of  spatial plans. And they have their own specific layers with them. And that makes one of the  challenges in this case. The topography, the map of the town and surroundings that you see is  pre-tiled because it changes maybe once a year or even less. So we do that on separate hardware. In  this spatial plans project, plans are updated daily because councils and everybody who is using it  from the government really wants to be able to present the new data as soon as it is ready. So  we have from a few plans a day to a few hundred plans a day per day that are updated in the system  and have to be shown on the map as soon as possible. We provide for a number of zoom levels from the  overview of the Netherlands up to the utmost detail of a certain plan. And we need to do that for  different map layers as well. Most plans contain several layers like the plan I just showed before  contains five layers. And some of these special plans contain ten or more layers as well. And we  have to provide for that in total we have one known, they said, seven hundred map layers in the system  that we have to provide to users. To give you an example of how many tiles that could be for the  topography layer, I have been counting the number of tiles for the whole Netherlands for all these  zoom levels. And that's for one layer only five and a half million tiles. And we have seven hundred  layers in total. And this means that if we would not use tiling it would result in high demand on  the WMS. Because at the daytime the map server is also used by professionals as well. And it serves  the map data for the website. So we had to come with a solution. And we had to use tiling to  diminish the demand on the services. But what we only wanted to do is tile parts of the layer that  were changed. We can't change the whole layers every time. And the customer also had said that  they wanted to be ready with tiling before office hours. So there would not be any stress on the  WMS during office hours. We come up with a solution that we call the tiling manager. Static data is  pre-tiled like topography. And dynamic data which can change per day is tiled before office hours.  We use the web cache which is a perfectly sound solution for tiling, for static tiling that is.  So in this case we're not smiling yet. We do need something extra. And that is the tiling manager  which sits between the planned database of the whole project and the web cache. And it queries  the planned database on a daily basis and then makes tasks that it can issue to the web cache.  And we had to change to adapt the web cache as well to be able to communicate it with it in a way  that we wanted and to be able to monitor the progress of the web cache. We faced a number of  challenges. As I said the monitoring of the web cache was wanted and we also wanted to be able  to stop the web cache because the tiling manager can get a command to stop everything it is doing  at that moment. So it has to stop the geocache instance as well with tasks at hand. And we had  to extend the rest interface of geobab cache. The tiling manager itself has a simple information  page which shows the plans at hand, all the layers per plan, the zoom level to which the layers have  to be tiled. And during tiling it gives a percentage of the tiles ready and also an estimate of the  time still to expect the tiling will need to go on. And because we have several plans that have  their own layers, the geo web cache has a configuration file with all the map layers inside  and we only have five of these common layers that are common for a lot of plans. But there are some  plans that use specific layers which we can't predict beforehand. So we have to generate a  configuration for geo web cache using a template and the tiling manager knows about the plans,  it knows the template, it fills in all the details and it makes a proper configuration for  geo web cache that it needs. We also did some work on optimizing the WMS performance mainly by  using some extra caching so that the database requests were diminished, the number of them.  And we also found it very handy to be able to shrink the PNGs that were produced, 24-bit  PNGs to 8-bit using PNG quant which is a standalone program that we use for every  tile and the transformation is also built into geo web cache. And it means that I think the  size of these tiles is indeed going from 100% to 50% or 30% size. So it's really helpful to get less  transport over the internet. And what was the process that we had up-to-date tiles? So when a  council changes a plan today and they upload it to the site, they really want the map and the view  in the next day to be in order. We can't have old tiles running around or still being there. So  up-to-date tiles was very important and this was ensured by the cycle that we made, we built into  this tiling manager. We have the plan database and the tiling manager takes all the plans that  were updated, deleted or added. It finds all the layers in all these plans. It makes a task for  each layer, a geo web cache task and all these tasks are visible in the information page of the  tiling manager. And the cycle starts with generating a new configuration. So for all the plans that  we have in the morning, so to say, we generate a new configuration for geo web cache up-to-date.  Then we're going to truncate to delete all the tiles of the changed plans of the day before. So  we erase the old situation, delete all the tiles that were changed. Next we send the new  configuration that we generated here. We send it to geo web cache and we reset geo web cache  because we have to make sure that geo web cache can read up this new configuration file. This was  the most safe option we had. Then after that, and geo web cache is back online, we give it  seed tasks. So it makes the tiles based on the new configuration and it makes tiles which represent  the new plans and in the map later on. And all these tasks in this info page are being done  bit by bit until the list is empty and then we're ready for the day. And hopefully before office  hours. A small idea about the architecture. There is one tiling manager and there can be  multiple geo web cache instances and they can each run in, they run in a tomcat and they can run  all in the same tomcat or in a separate tomcat or on separate servers. Communication is via HTTP.  So there is a flexible way that you can deploy the tiling manager and the geo web cache instances.  You can have many geo web cache instances if you want. But they all do get the same configuration.  And the same configuration means that they all send their requests to the same WMS to get  and produce the tiles. The tiling manager is trying to find when it goes through his list,  through the cycle and through the list of tasks, it finds a geo web cache that is doing nothing  at the moment and send a task to it so it can do the tiles for that plan, for that layer.  And then finds the next geo web cache that is doing nothing and send another layer to that one  and so forth. And if they're all busy, it just waits and monitors progress and waits until one  of the geo web cache is free or the list of tasks is empty. And all that is configured  in a properties file for the tiling manager. It's a bit bad. It's not readable really,  but from a software perspective, there is a tiling manager class that accepts HTTP requests  from users, translates that into calls to a manager service. And the manager service has a pool  of that holds W, no, geo web cache instances. In the pool, we have several geo web caches  representing geo web caches and the status of geo web caches is also monitored in a special  class, in a special object, all is written in Java. If I show a very short overview of the  database scheme, there is a table that contains the task, the cycle of the day, so to say, a row  for the cycle of the day. There is a big table that contains all the status items that are important  for the task at hand, the plan, the layer type, the zoom level we need to go to, etc. Then there  is a table that combines the type of plan with the layers that the plan is normally containing.  And we have for the pre-tiling, for example, when we want to pre-tile topography, it does not make  really sense to take the bounding box of the Netherlands because we would also tile a lot of  the North Sea, which is not really useful. So we made a table with seed boxes, bounding boxes  around the Netherlands in a more efficient way so that most of the land of the Netherlands is  covered that way. And the pre-tiling of topography in other layers is a lot faster. Well, this is  not readable also, but there is a configuration file where you can really adapt the number of  geowab caches, the connection to the database, passwords for Tomcat, etc., etc. And I'm really  going fast. This was it already. Okay. Thank you. Any questions?  I did. I did. And that was some time ago. And the comments were something like, for example,  the PNG 24-bit to 8-bit conversion. They didn't like the way we did it. And for other items,  they didn't take it up because they found that the classes we used, we adapted in geowab cache,  would not be their choice of using our adaptations. So they didn't take our work less. No.  I'm not sure that the talk will be distributed sometime. Yes, I will do so because you have  10 seconds to write down this link. Okay. I won't share it. I won't do it. I'll upload  it and... Okay. Yes, but I don't. I really had to go. I really had to go. I have to catch  your flight. Sorry. The main link is wwwidgs.nl.
Managing_update_of_tiles_of_dynamic_data__480p__0_1307.png Managing_update_of_tiles_of_dynamic_data__480p__1308_1475.png Managing_update_of_tiles_of_dynamic_data__480p__1476_3227.png Managing_update_of_tiles_of_dynamic_data__480p__3228_6347.png Managing_update_of_tiles_of_dynamic_data__480p__6348_9203.png Managing_update_of_tiles_of_dynamic_data__480p__9204_9983.png Managing_update_of_tiles_of_dynamic_data__480p__9984_10751.png Managing_update_of_tiles_of_dynamic_data__480p__10752_11663.png Managing_update_of_tiles_of_dynamic_data__480p__11664_11759.png Managing_update_of_tiles_of_dynamic_data__480p__11760_16439.png Managing_update_of_tiles_of_dynamic_data__480p__16440_19367.png Managing_update_of_tiles_of_dynamic_data__480p__19368_21911.png Managing_update_of_tiles_of_dynamic_data__480p__21912_23159.png Managing_update_of_tiles_of_dynamic_data__480p__23160_25091.png Managing_update_of_tiles_of_dynamic_data__480p__25092_25667.png Managing_update_of_tiles_of_dynamic_data__480p__25668_25715.png Managing_update_of_tiles_of_dynamic_data__480p__25716_27503.png Managing_update_of_tiles_of_dynamic_data__480p__27504_27671.png Managing_update_of_tiles_of_dynamic_data__480p__27672_27719.png Managing_update_of_tiles_of_dynamic_data__480p__27720_27767.png Managing_update_of_tiles_of_dynamic_data__480p__27768_27815.png Managing_update_of_tiles_of_dynamic_data__480p__27816_27863.png Managing_update_of_tiles_of_dynamic_data__480p__27864_28480.png
I'm trying to make it keep it brief. Good afternoon everybody. Just after 12. My name is Rob Stecklenburg. I'm working for Ideages in the Netherlands, software developer. Ideages is a small company in the Netherlands, a geo company that has clients among mainly the governmental organizations, provinces, councils, police. And we were once involved in a project, a national project where we had to use tiling, but where the map data was updated very regularly. And we had to meet and solve some challenges in that respect and develop special software to do what we wanted to keep up with the goals of the program. The Dutch government has a national facility for showing spatial plans at all levels, council, governmental, provincial, all kinds of spatial plans that are useful for public use and professional use. And at the moment we have around 15,000 plans in the system and it's growing daily, it's really taking off. It's publicly available through a website and besides the website there are facilities for professional use as well. You see, we have 15,000 plans, every dot is a plan and it's almost the view of the Netherlands is coming out through all the small dots. Users have to be able to zoom in to plans, to pan and to zoom out. So we need a good performance and that was one of the demands that the project had. I show you an example, a plan, more in detail from the hometown of our company, Risen in the east of the Netherlands. And the website gives wonderful means of searching. I just found this plan in our town by typing in the address of our company's headquarters. And it's a small plan that says something about how to develop the center of this town, all the documents that belong to it, all the official documents can be found as well. And this is a really common local plan, so to say. It's difficult to translate the specific items into English, but you can say this is a local plan and each local plan has about four to five map layers, including borders, including an overview layer. And these five layers are used for the majority of plans in this facility. And they all constitute the whole service area of the Netherlands. Maybe a few percent of the total number of plans is different. They have often have a larger service area. They are more in the national area or the provincial areas of spatial plans. And they have their own specific layers with them. And that makes one of the challenges in this case. The topography, the map of the town and surroundings that you see is pre-tiled because it changes maybe once a year or even less. So we do that on separate hardware. In this spatial plans project, plans are updated daily because councils and everybody who is using it from the government really wants to be able to present the new data as soon as it is ready. So we have from a few plans a day to a few hundred plans a day per day that are updated in the system and have to be shown on the map as soon as possible. We provide for a number of zoom levels from the overview of the Netherlands up to the utmost detail of a certain plan. And we need to do that for different map layers as well. Most plans contain several layers like the plan I just showed before contains five layers. And some of these special plans contain ten or more layers as well. And we have to provide for that in total we have one known, they said, seven hundred map layers in the system that we have to provide to users. To give you an example of how many tiles that could be for the topography layer, I have been counting the number of tiles for the whole Netherlands for all these zoom levels. And that's for one layer only five and a half million tiles. And we have seven hundred layers in total. And this means that if we would not use tiling it would result in high demand on the WMS. Because at the daytime the map server is also used by professionals as well. And it serves the map data for the website. So we had to come with a solution. And we had to use tiling to diminish the demand on the services. But what we only wanted to do is tile parts of the layer that were changed. We can't change the whole layers every time. And the customer also had said that they wanted to be ready with tiling before office hours. So there would not be any stress on the WMS during office hours. We come up with a solution that we call the tiling manager. Static data is pre-tiled like topography. And dynamic data which can change per day is tiled before office hours. We use the web cache which is a perfectly sound solution for tiling, for static tiling that is. So in this case we're not smiling yet. We do need something extra. And that is the tiling manager which sits between the planned database of the whole project and the web cache. And it queries the planned database on a daily basis and then makes tasks that it can issue to the web cache. And we had to change to adapt the web cache as well to be able to communicate it with it in a way that we wanted and to be able to monitor the progress of the web cache. We faced a number of challenges. As I said the monitoring of the web cache was wanted and we also wanted to be able to stop the web cache because the tiling manager can get a command to stop everything it is doing at that moment. So it has to stop the geocache instance as well with tasks at hand. And we had to extend the rest interface of geobab cache. The tiling manager itself has a simple information page which shows the plans at hand, all the layers per plan, the zoom level to which the layers have to be tiled. And during tiling it gives a percentage of the tiles ready and also an estimate of the time still to expect the tiling will need to go on. And because we have several plans that have their own layers, the geo web cache has a configuration file with all the map layers inside and we only have five of these common layers that are common for a lot of plans. But there are some plans that use specific layers which we can't predict beforehand. So we have to generate a configuration for geo web cache using a template and the tiling manager knows about the plans, it knows the template, it fills in all the details and it makes a proper configuration for geo web cache that it needs. We also did some work on optimizing the WMS performance mainly by using some extra caching so that the database requests were diminished, the number of them. And we also found it very handy to be able to shrink the PNGs that were produced, 24-bit PNGs to 8-bit using PNG quant which is a standalone program that we use for every tile and the transformation is also built into geo web cache. And it means that I think the size of these tiles is indeed going from 100% to 50% or 30% size. So it's really helpful to get less transport over the internet. And what was the process that we had up-to-date tiles? So when a council changes a plan today and they upload it to the site, they really want the map and the view in the next day to be in order. We can't have old tiles running around or still being there. So up-to-date tiles was very important and this was ensured by the cycle that we made, we built into this tiling manager. We have the plan database and the tiling manager takes all the plans that were updated, deleted or added. It finds all the layers in all these plans. It makes a task for each layer, a geo web cache task and all these tasks are visible in the information page of the tiling manager. And the cycle starts with generating a new configuration. So for all the plans that we have in the morning, so to say, we generate a new configuration for geo web cache up-to-date. Then we're going to truncate to delete all the tiles of the changed plans of the day before. So we erase the old situation, delete all the tiles that were changed. Next we send the new configuration that we generated here. We send it to geo web cache and we reset geo web cache because we have to make sure that geo web cache can read up this new configuration file. This was the most safe option we had. Then after that, and geo web cache is back online, we give it seed tasks. So it makes the tiles based on the new configuration and it makes tiles which represent the new plans and in the map later on. And all these tasks in this info page are being done bit by bit until the list is empty and then we're ready for the day. And hopefully before office hours. A small idea about the architecture. There is one tiling manager and there can be multiple geo web cache instances and they can each run in, they run in a tomcat and they can run all in the same tomcat or in a separate tomcat or on separate servers. Communication is via HTTP. So there is a flexible way that you can deploy the tiling manager and the geo web cache instances. You can have many geo web cache instances if you want. But they all do get the same configuration. And the same configuration means that they all send their requests to the same WMS to get and produce the tiles. The tiling manager is trying to find when it goes through his list, through the cycle and through the list of tasks, it finds a geo web cache that is doing nothing at the moment and send a task to it so it can do the tiles for that plan, for that layer. And then finds the next geo web cache that is doing nothing and send another layer to that one and so forth. And if they're all busy, it just waits and monitors progress and waits until one of the geo web cache is free or the list of tasks is empty. And all that is configured in a properties file for the tiling manager. It's a bit bad. It's not readable really, but from a software perspective, there is a tiling manager class that accepts HTTP requests from users, translates that into calls to a manager service. And the manager service has a pool of that holds W, no, geo web cache instances. In the pool, we have several geo web caches representing geo web caches and the status of geo web caches is also monitored in a special class, in a special object, all is written in Java. If I show a very short overview of the database scheme, there is a table that contains the task, the cycle of the day, so to say, a row for the cycle of the day. There is a big table that contains all the status items that are important for the task at hand, the plan, the layer type, the zoom level we need to go to, etc. Then there is a table that combines the type of plan with the layers that the plan is normally containing. And we have for the pre-tiling, for example, when we want to pre-tile topography, it does not make really sense to take the bounding box of the Netherlands because we would also tile a lot of the North Sea, which is not really useful. So we made a table with seed boxes, bounding boxes around the Netherlands in a more efficient way so that most of the land of the Netherlands is covered that way. And the pre-tiling of topography in other layers is a lot faster. Well, this is not readable also, but there is a configuration file where you can really adapt the number of geowab caches, the connection to the database, passwords for Tomcat, etc., etc. And I'm really going fast. This was it already. Okay. Thank you. Any questions? I did. I did. And that was some time ago. And the comments were something like, for example, the PNG 24-bit to 8-bit conversion. They didn't like the way we did it. And for other items, they didn't take it up because they found that the classes we used, we adapted in geowab cache, would not be their choice of using our adaptations. So they didn't take our work less. No. I'm not sure that the talk will be distributed sometime. Yes, I will do so because you have 10 seconds to write down this link. Okay. I won't share it. I won't do it. I'll upload it and... Okay. Yes, but I don't. I really had to go. I really had to go. I have to catch your flight. Sorry. The main link is wwwidgs.nl.
and develop special software to do what we wanted Dutch government has a national facility for around 15,000 plans in the system and it's performance and that was one of the demands that layer. And these five layers are used for the spatial plans project, plans are updated daily present the new data as soon as it is ready. So from the overview of the Netherlands up to the would result in high demand on the WMS. Because solution. And we had to use tiling to diminish the demand on the services. But what we only wanted to do is tile parts of the layer that were And it queries the planned database on a daily the web cache. And we had to change to adapt the will need to go on. And because we have several work on optimizing the WMS performance mainly by to be in order. We can't have old tiles running deleted or added. It finds all the layers in all geo web cache task and all these tasks are new configuration. So for all the plans that we because we have to make sure that geo web cache the new plans and in the map later on. And all produce the tiles. The tiling manager is trying monitors progress and waits until one of the geo plan with the layers that the plan is normally