QGIS and OGC APIs - how do they work together?
Kralidis, Tom
Genuchten, Paul van
QGIS demo as a generic Desktop capability. Alongside the various OGC API server implementations, clients are in the process of being set up to interact with the OGC API services. In this talk we present some new capabilities of QGIS and GDAL to interact with OGC API’s. Additionally to the various OGC API server implementations, more and more clients are being set up to interact with the OGC API services. This talk will focus on on QGIS and some new capabilities of QGIS and GDAL to interact with OGC API’s. - The WFS provider in QGIS has been extended to support OGC API Features. The functionality builds on top of the WFS provider. - The QGIS Metasearch plugin is in the process of being extended to support OGC API - Records a dataset search plugin for QGIS. Metasearch uses internally OWSLib, a python library with extended OGC API client support. - Also GDAL, a swiss army knife for spatial data, has been extended to interact with various OGC API's. In case there is news to share on client support for OGC API Maps, Coverages, Tiles and Styles, you’ll hear of it during the presentation.
 He is currently an STI specialist at ISRIQ. And he's been very involved also in phosphorgy projects like GeoNetwork and PyGEO API. So without further ado, I give you the room. I don't know who is starting. Maybe I can start. Let's see. I'm wondering if I'll share the screen. So maybe Paul, I'll drive and then you can just cue me. Is that okay? Okay. I see you're on mute, but I see that you agree. So let's go. Yes. Okay. Oops. Sorry about that. I muted. I didn't know that. Hello. Sorry, I don't know if you were trying to say something. Yeah, I was. Yeah, fine. Go ahead, Tom. Cool. Okay. Maybe I'll start off with a couple of notes. I guess this presentation is on how QGIS is increasingly supporting the OGC API efforts. And we'll talk a little bit about how they work together. I'll provide a small update on OGC API efforts. And then Paul will get into some of the details around QGIS and we're happy to answer any questions or comments. So OGC API. Hopefully you all have heard of what OGC API is. So going back in time a bit, we've started from in the mid to late 90s around doing web services with a lot of XML and basically remote procedure call over the web. And we moved on from their sort of web 2.0 type things. There are some realities associated with having those OGC, well, with specifications designed in that certain way in terms of overloading the HTTP specification as well as some difficulties in having things crawlable or searchable on mass market search engines and so on. So excuse me, in 2017, W3C came up with the spatial data on the web as practice. And a lot of efforts started to sort of culminate around the concepts of REST as well as JSON and OpenAPI. OGC had an API white paper in 2017 if I'm not mistaken, which basically called for a lot of created the opportunity to create these new OGC API standards which focused on JSON and HTML as first class and making things mass market friendly and web developer friendly. A web developer should not have to download a 400 page document to implement the specification whether it's a server or client or what have you. Mind you, I don't think anybody should have to download a 400 page document. But here we are. You can see a number of the specifications down below, whether they're maps, styles, coverages, features and so on. Most of them are successors to the first generation of OGC API standards. These standards are developed interactively. So these standards are developed now on GitHub. If you go to github.com slash open geospatial you'll see a number of the standards there. You'll see OGCAPI.org. You'll find all the information on all the APIs and all their specification documents fully available there in HTML. There's GitterChat so that you can easily interact with the community around the specification development and questions around compliance and performance and testing for that matter. And as mentioned in the previous presentation by Clemens, Peter and myself, there's really a building block model to all of these specifications that will allow you to plug and play and put those Lego bricks together to be able to come up with something that meets your needs. With that I'm going to turn it over to Paul. Thank you Tom. So I use it. So where to start? So OGCAPI features and QGIS. A lot of people contributed to QGIS over time. I'm not in a position here to mention everybody that contributed but hopefully I give everybody that contributed enough credits. Here they are. I want to drive in quickly since we only have 15 minutes to see some functionality that is available these days. So starting with OGCAPI features, it was also the first one that kind of got adopted within the OGC or started the movement. This was implemented by Ivan Ruo and funded by Planet and it was implemented as an extension to the existing WFS provider. So I think it's landed in somewhere 314, 316, I don't know. You will find this functionality under the add WFS layer and then instead of putting there the WFS capabilities URL, you put the OGCAPI features URL and from then on it kind of works in the same way that it fetches the layers from the service and you select the layer and you add it to the map. There's some special parameters that you can add here, for example the pagination size. So in that sense we have the same challenges as WFS that if you have a very big page size that the service may actually load quite slow, if you make a smaller page size it will be faster but you will only see a partial result. This is always something to keep in mind. So what is also good to know here is that it uses behind the scene the same architecture as WFS, it means that QGIS caches the features in a local cache. So if you zoom in and zoom out outside the area, it will still have the previous features in its cache. So it doesn't need to go back to the server to fetch them again. Next slide Tom. So from Ivan we discussed a bit before this meeting, some learning points from his side is that the fact that the items in the collection most of the times are untyped. So in theory you could add them, add the definition of the item to the open API document but the specification doesn't necessarily need you to do so. So if you have there an item say a tree, if it has a property branches and size that could be specified in the open API document but it's not always required. It means that a client like QGIS has to get the first page of results to see what is actually returned and which properties are available on the object. It's a bit awkward for a client and especially people that are used to build clients against open API specifications will be a bit surprised about this outcome. On the other hand for the server implementers this is easy because they don't need to go into each object to see what properties it has and advertise it in the open API document. Another point that we came up with discussion is that if you have hierarchical data like for example I have a soil profile in the soil profile is a horizon and in a horizon has a certain clay percentage and a certain magnesium value to query the magnesium concentration over the 12 micrograms for the second horizon in profiles within a bee box you need quite a dedicated filter expression capability and this is currently not yet fully crystallized how that will work out but it is something that we maybe we have some time this week to discuss that topic. Next one Tom. OGCI records I give the floor to you Tom for this one. You muted. Okay great thank you Paul. So for OGCI API records as Paul mentioned QGIS is a long standing project with a number of contributors and a long time solid support of the OGCI standard so we can see WMS, WFS, WCS and so on Paul just talked about OGCI features now let's talk about search. So in QGIS which Paul and I are presenting on tomorrow if anybody else is interested in seeing that presentation I won't go into too many details here but in QGIS is a long standing catalog client called Metasearch which has always worked with the OGCI catalog service for the web specification. Well in the last year alongside of working in the OGCI standards working group we also implemented extended support in the Metasearch plugin which is a core part of QGIS so when you download QGIS you will have the Metasearch plugin installed by default where it's all almost merged but as you can see in the lower left part of the screen but we implemented OGCI API records support in Metasearch and Metasearch internally uses the OWS live Python client OWS client library so that client library itself has been extended to support OGCI API features records and coverages that's the current support and we're extending it as time goes on and more OGCI API specifications are ratified but we do have a pull request in with some working functionality against OGCI API records so there you can see a screenshot there that's hitting a note that could hit an OGCI API records endpoint in the same way that it would a CSW endpoint so we've extended the client to be able to do that so when that's merged you'll be able to say okay add my OGCI API records client you do have to specify that it's an API records endpoint or a service as opposed to a CSW service and then after that all of the functionality is abstracted away so you don't really know whether you're working with a CSW or an OGCI API records client and that's the goal of the beauty and the value of the Metasearch plugin so in theory we're able to add any type of search service but here we're concentrating on QGIS on OGCI API records for that matter and in that work we actually did some digging in Metasearch and abstracted the search capability to make it very very easy to add more APIs over time if we need them or profiles of OGCI API records for that matter so like I mentioned it's currently a pull request we're just have some final issues to deal with with regards to packaging and we hope to have that merged very soon maybe even this week there's a code sprint on Saturday so let's see what can be done with regards to that some issues or some discussion plans that came up with OGCI API records link types I mean this has always been an issue in in catalogs and search APIs in discovering actionable links so how do I know that a link is a WMS link or a WCS link or WFS link well we've always had that issue and we have a way of working with that in QGIS which also requires some cooperation from the metadata providers but now we have a new specification with not only having the requirement to articulate the link types of the first generation OGCI web services but now we have we introduce link types of the new OGCI API standards and the link types in the OGCI API records case is going to be a lot easier for Metasearch because we have the ability to crawl down a certain level of an OGCI API record endpoint so we can get to just that collection because of the hierarchical and the restful approach of the way the standards are designed so current so you won't have to open Metasearch find the data set add the WMS and go back and do a get capabilities again and do that round trip and find the layer now you will be able to connect to the OGCI records endpoint find the find the matching record and whatever the link type underneath is which when it's an OGCI API link you can you'll be able to load that directly in QGIS so I think that's going to be a very valuable feature as as more and more of these things get get put online and server data providers make their metadata discovery metadata available through OGCI API records and the new link types so there is change management involved but that's a W you know OGCI web services and OGCI API change management thing that I think will be here for a while and this is what that some of what that change management looks like here in OGCI API records and what that means for the for in QGIS. Other other issues with regards to QGIS there are other OGCI APIs as we've mentioned we're not aware of any current current initiatives at this point in time that implement the other APIs but I would imagine over time that we'll start to see updates on clients for maps, tiles coverage, coverages and processes as those specifications mature and they you know they're formalized by OGCI and and you know clients want to have that functionality implemented. The OGCI does have an API roadmap which which is which will be valuable to I think is valuable for folks to look at if they want to see where what the the timelines of the specifications are and that'll give the QGIS community a better idea of when's a good time to dive into the standard and and implement it so I think this is a you know a great advancement in the OGCI API standards and QGIS is a you know is a very powerful GIS client and we're you know we're we've demonstrated that we're able to implement these standards with a relatively low barrier low barrier workflow. Paul? Yeah let me take that one so so while discussing with Ivan of course he introduced the aspect that the GDAL supports the full OGCI API actually a bit more than than QGIS itself by now so there's support for maps and and tiles and it's actually possible to to add those to QGIS also via this MOAW I don't know you say more file I don't know so it's kind of a processing file where you say okay load this data via GDAL so that's that's already an option in QGIS so if you really want to have your OGCI API maps in QGIS you can do it via such a file that's that's described on that endpoint. Next one yeah so so then dive a bit into QGIS server of course QGIS is mainly a desktop application but there has been some initiatives to have also QGIS as a server so you run it as a Docker instance and request it as a WMS or WFS and within that next one Tom within that area there was also initiative to add OGCI API features to the QGIS server and it works actually out of the box the only thing you need to be aware of is that once you before you upload your your QGIS project to the QGIS server you have to tick a box in the in the server properties of the layer saying that it should be exposed as a WFS layer it will then automatically be advertised also as an OGCI API features layer so that that's really useful and as far as I know no support for maps and tiles also on that side yet but yeah I hope that that work starts soon. Next one yeah so so that's about it do you have a QGIS there Tom or shall I maybe Joanna maybe you can add my screen to the that's a good idea but it's also maybe you can already prepare some some answers to some of the questions Tom because we entered the discussion phase of this presentation so I have here a QGIS instance with focused on the Netherlands with some of the provinces and those provinces are actually coming from the OGCI API features so if I add a layer of type in this case I have to go add WFS layer that that's the one that includes the OGCI API features here is the first you have to set up a server connection if you edit that you will notice that it you just put here the the endpoint of the in this case is a PyG API running on a sandbox environment and here you have those options like putting the page size and you can detect here and says oh it's an OGCI API features and then from that it will fetch from the collections page all the collections that are available in this service and you can then add that to the map so it works exactly like a WFS service and I wanted to add this aspect also related to the OGCI API very records is that this implementation also helps us at the server side to develop our server because it gives such an easy client experience that we always use QGIS for testing our server implementation and that's also a benefit because we make sure that it works on QGIS first before any other client unfortunately I cannot show the OGCI API records on my Mitter search because I had a problem with the OE or WSLip sorry Tom that's okay I mean we are also running out of time that's a yeah so we better go into the questions yeah thank you thank you very much okay so we have two questions here so the first one is speaking of caching caching is also needed for WMTS XYZ raster tiles QGIS QGIS is not responsible for the largest data load of the free map raster tiles according to the OSM or maintainers hope that becomes better with OGCI API it's a question yes I don't know the internals of how how this is managed in QGIS but I know that I am one of those users that that add local layers of street map almost every day so so I can imagine that it's very popular I mean having the white screen with a with a simple shape file on top just looks bad you always want a background and open street map is also my favorite one but yeah it's a I think providing these with these background servers background tile servers is really a point of attention like a geo network geo server open layers in all their examples they're using the open street map tiles which is a community initiative also so we should be careful I hope at some point OSGO board for example may set up a tile server that we can use for this type of demos and it doesn't need to go up till level skill 18 or 20 if it's for this type of demos skills 12 is just fine so that that's my idea and that I don't know what do you think yeah I don't know much very much about the internals but yeah I'm with you Paul okay so the next question is do you think we will need not only a QGIS OGC API section but maybe thinking in the future a different QGIS platform that support and display the requested information think on sparkle queries it will be possible to gather not only maps geocal localized data but also sounds measurements features papers videos etc I think that's a good question my my my initial reaction to that would be QGIS provides you know a strong mechanism for extensibility so you can develop your own for example your own Python plugins so you can I could see an ecosystem of plugins being made to satisfy those those work cases and those those use cases in fact metasarch itself is written in Python not not c++ so I can't see you know I can see an ecosystem of these plugins being built just the same to deal with sparkle and maybe maybe some other things I don't know Paul what do you think it's it's a when I look at for example to the 3d world in a 3d world view you you interact with what you see around you and this is also a bit the paradigm of QGIS that is layer oriented and it's really hard to make links from from like we see here the city cities of the Netherlands and then there's the open street map in the back but it's really hard to make a make an interaction between the two and that that's related to the QGIS and and all all most of the key of the the here's desktop clients that but in a 3d world you it's far more common to click to follow layers and see interactions between a traffic light in the street that it's connected to and and and and sound that it makes and then when I when I see a thing like sparkle typically is also object oriented you go through a network of objects and then I think QGIS by design is not really a tool fit for for such spatial navigation so that's a bit where I'm heading on one hand the little layer paradigm brought us a lot but now it hinders us to advance in a direction and I think at some point we need new clients that engage with data in a more object oriented way okay thank you thank you very much Paul and Tom let's give a virtual clap to the speakers and before moving to our next presentation so we're going to be talking about the modular OGC API workflows for processing and visualization.
 He is currently an STI specialist at ISRIQ.  And he's been very involved also in phosphorgy projects like GeoNetwork and PyGEO API.  So without further ado, I give you the room.  I don't know who is starting.  Maybe I can start.  Let's see.  I'm wondering if I'll share the screen.  So maybe Paul, I'll drive and then you can just cue me.  Is that okay?  Okay.  I see you're on mute, but I see that you agree.  So let's go.  Yes.  Okay.  Oops.  Sorry about that.  I muted.  I didn't know that.  Hello.  Sorry, I don't know if you were trying to say something.  Yeah, I was.  Yeah, fine.  Go ahead, Tom.  Cool.  Okay.  Maybe I'll start off with a couple of notes.  I guess this presentation is on how QGIS is increasingly supporting the OGC API efforts.  And we'll talk a little bit about how they work together.  I'll provide a small update on OGC API efforts.  And then Paul will get into some of the details around QGIS and we're happy to answer any  questions or comments.  So OGC API.  Hopefully you all have heard of what OGC API is.  So going back in time a bit, we've started from in the mid to late 90s around doing web  services with a lot of XML and basically remote procedure call over the web.  And we moved on from their sort of web 2.0 type things.  There are some realities associated with having those OGC, well, with specifications designed  in that certain way in terms of overloading the HTTP specification as well as some difficulties  in having things crawlable or searchable on mass market search engines and so on.  So excuse me, in 2017, W3C came up with the spatial data on the web as practice.  And a lot of efforts started to sort of culminate around the concepts of REST as well as JSON  and OpenAPI.  OGC had an API white paper in 2017 if I'm not mistaken, which basically called for a  lot of created the opportunity to create these new OGC API standards which focused on JSON  and HTML as first class and making things mass market friendly and web developer friendly.  A web developer should not have to download a 400 page document to implement the specification  whether it's a server or client or what have you.  Mind you, I don't think anybody should have to download a 400 page document.  But here we are.  You can see a number of the specifications down below, whether they're maps, styles,  coverages, features and so on.  Most of them are successors to the first generation of OGC API standards.  These standards are developed interactively.  So these standards are developed now on GitHub.  If you go to github.com slash open geospatial you'll see a number of the standards there.  You'll see OGCAPI.org.  You'll find all the information on all the APIs and all their specification documents  fully available there in HTML.  There's GitterChat so that you can easily interact with the community around the specification  development and questions around compliance and performance and testing for that matter.  And as mentioned in the previous presentation by Clemens, Peter and myself, there's really  a building block model to all of these specifications that will allow you to plug and play and put  those Lego bricks together to be able to come up with something that meets your needs.  With that I'm going to turn it over to Paul.  Thank you Tom.  So I use it.  So where to start?  So OGCAPI features and QGIS.  A lot of people contributed to QGIS over time.  I'm not in a position here to mention everybody that contributed but hopefully I give everybody  that contributed enough credits.  Here they are.  I want to drive in quickly since we only have 15 minutes to see some functionality that  is available these days.  So starting with OGCAPI features, it was also the first one that kind of got adopted within  the OGC or started the movement.  This was implemented by Ivan Ruo and funded by Planet and it was implemented as an extension  to the existing WFS provider.  So I think it's landed in somewhere 314, 316, I don't know.  You will find this functionality under the add WFS layer and then instead of putting  there the WFS capabilities URL, you put the OGCAPI features URL and from then on it kind  of works in the same way that it fetches the layers from the service and you select the  layer and you add it to the map.  There's some special parameters that you can add here, for example the pagination size.  So in that sense we have the same challenges as WFS that if you have a very big page size  that the service may actually load quite slow, if you make a smaller page size it will be  faster but you will only see a partial result.  This is always something to keep in mind.  So what is also good to know here is that it uses behind the scene the same architecture  as WFS, it means that QGIS caches the features in a local cache.  So if you zoom in and zoom out outside the area, it will still have the previous features  in its cache.  So it doesn't need to go back to the server to fetch them again.  Next slide Tom.  So from Ivan we discussed a bit before this meeting, some learning points from his side  is that the fact that the items in the collection most of the times are untyped.  So in theory you could add them, add the definition of the item to the open API document but the  specification doesn't necessarily need you to do so.  So if you have there an item say a tree, if it has a property branches and size that could  be specified in the open API document but it's not always required.  It means that a client like QGIS has to get the first page of results to see what is actually  returned and which properties are available on the object.  It's a bit awkward for a client and especially people that are used to build clients against  open API specifications will be a bit surprised about this outcome.  On the other hand for the server implementers this is easy because they don't need to go  into each object to see what properties it has and advertise it in the open API document.  Another point that we came up with discussion is that if you have hierarchical data like  for example I have a soil profile in the soil profile is a horizon and in a horizon has a  certain clay percentage and a certain magnesium value to query the magnesium concentration  over the 12 micrograms for the second horizon in profiles within a bee box you need quite  a dedicated filter expression capability and this is currently not yet fully crystallized  how that will work out but it is something that we maybe we have some time this week  to discuss that topic.  Next one Tom.  OGCI records I give the floor to you Tom for this one.  You muted.  Okay great thank you Paul.  So for OGCI API records as Paul mentioned QGIS is a long standing project with a number  of contributors and a long time solid support of the OGCI standard so we can see WMS, WFS,  WCS and so on Paul just talked about OGCI features now let's talk about search.  So in QGIS which Paul and I are presenting on tomorrow if anybody else is interested  in seeing that presentation I won't go into too many details here but in QGIS is a long  standing catalog client called Metasearch which has always worked with the OGCI catalog  service for the web specification.  Well in the last year alongside of working in the OGCI standards working group we also  implemented extended support in the Metasearch plugin which is a core part of QGIS so when  you download QGIS you will have the Metasearch plugin installed by default where it's all  almost merged but as you can see in the lower left part of the screen but we implemented  OGCI API records support in Metasearch and Metasearch internally uses the OWS live Python  client OWS client library so that client library itself has been extended to support  OGCI API features records and coverages that's the current support and we're extending it  as time goes on and more OGCI API specifications are ratified but we do have a pull request  in with some working functionality against OGCI API records so there you can see a screenshot  there that's hitting a note that could hit an OGCI API records endpoint in the same way  that it would a CSW endpoint so we've extended the client to be able to do that so when that's  merged you'll be able to say okay add my OGCI API records client you do have to specify  that it's an API records endpoint or a service as opposed to a CSW service and then after  that all of the functionality is abstracted away so you don't really know whether you're  working with a CSW or an OGCI API records client and that's the goal of the beauty and  the value of the Metasearch plugin so in theory we're able to add any type of search service  but here we're concentrating on QGIS on OGCI API records for that matter and in that work  we actually did some digging in Metasearch and abstracted the search capability to make  it very very easy to add more APIs over time if we need them or profiles of OGCI API records  for that matter so like I mentioned it's currently a pull request we're just have some final  issues to deal with with regards to packaging and we hope to have that merged very soon maybe  even this week there's a code sprint on Saturday so let's see what can be done with regards to that  some issues or some discussion plans that came up with OGCI API records link types I mean this  has always been an issue in in catalogs and search APIs in discovering actionable links so how do I  know that a link is a WMS link or a WCS link or WFS link well we've always had that issue and we  have a way of working with that in QGIS which also requires some cooperation from the metadata  providers but now we have a new specification with not only having the requirement to articulate  the link types of the first generation OGCI web services but now we have we introduce link types  of the new OGCI API standards and the link types in the OGCI API records case is going to be a lot  easier for Metasearch because we have the ability to crawl down a certain level of an OGCI API  record endpoint so we can get to just that collection because of the hierarchical and the  restful approach of the way the standards are designed so current so you won't have to open  Metasearch find the data set add the WMS and go back and do a get capabilities again and do that  round trip and find the layer now you will be able to connect to the OGCI records endpoint find  the find the matching record and whatever the link type underneath is which when it's an OGCI  API link you can you'll be able to load that directly in QGIS so I think that's going to be a  very valuable feature as as more and more of these things get get put online and server data  providers make their metadata discovery metadata available through OGCI API records and the new  link types so there is change management involved but that's a W you know OGCI web services and  OGCI API change management thing that I think will be here for a while and this is what that some  of what that change management looks like here in OGCI API records and what that means for the  for in QGIS. Other other issues with regards to QGIS there are other OGCI APIs as we've mentioned  we're not aware of any current current initiatives at this point in time that implement the other  APIs but I would imagine over time that we'll start to see updates on clients for maps,  tiles coverage, coverages and processes as those specifications mature and they you know  they're formalized by OGCI and and you know clients want to have that functionality implemented.  The OGCI does have an API roadmap which which is which will be valuable to I think is valuable  for folks to look at if they want to see where what the the timelines of the specifications are  and that'll give the QGIS community a better idea of when's a good time to dive into the standard  and and implement it so I think this is a you know a great advancement in the OGCI API standards  and QGIS is a you know is a very powerful GIS client and we're you know we're we've demonstrated  that we're able to implement these standards with a relatively low barrier low barrier workflow.  Paul? Yeah let me take that one so so while discussing with Ivan of course he introduced  the aspect that the GDAL supports the full OGCI API actually a bit more than than QGIS  itself by now so there's support for maps and and tiles and it's actually possible to to add  those to QGIS also via this MOAW I don't know you say more file I don't know so it's kind of a  processing file where you say okay load this data via GDAL so that's that's already an option in QGIS  so if you really want to have your OGCI API maps in QGIS you can do it via such a file that's  that's described on that endpoint. Next one yeah so so then dive a bit into QGIS server of course  QGIS is mainly a desktop application but there has been some initiatives to have also QGIS as a  server so you run it as a Docker instance and request it as a WMS or WFS and within that next  one Tom within that area there was also initiative to add OGCI API features to the QGIS server and  it works actually out of the box the only thing you need to be aware of is that once you before  you upload your your QGIS project to the QGIS server you have to tick a box in the in the  server properties of the layer saying that it should be exposed as a WFS layer it will then  automatically be advertised also as an OGCI API features layer so that that's really useful and  as far as I know no support for maps and tiles also on that side yet but yeah I hope that that  work starts soon. Next one yeah so so that's about it do you have a QGIS there Tom or shall I maybe  Joanna maybe you can add my screen to the that's a good idea but it's also maybe you can already  prepare some some answers to some of the questions Tom because we entered the discussion phase of  this presentation so I have here a QGIS instance with focused on the Netherlands with some of the  provinces and those provinces are actually coming from the OGCI API features so if I add a layer of  type in this case I have to go add WFS layer that that's the one that includes the OGCI API features  here is the first you have to set up a server connection if you edit that you will notice that  it you just put here the the endpoint of the in this case is a PyG API running on a sandbox  environment and here you have those options like putting the page size and you can detect here  and says oh it's an OGCI API features and then from that it will fetch from the collections page  all the collections that are available in this service and you can then add that to the map so  it works exactly like a WFS service and I wanted to add this aspect also related to the OGCI  API very records is that this implementation also helps us at the server side to develop our  server because it gives such an easy client experience that we always use QGIS for testing  our server implementation and that's also a benefit because we make sure that it works on  QGIS first before any other client unfortunately I cannot show the OGCI API records on my Mitter  search because I had a problem with the OE or WSLip sorry Tom that's okay I mean we are also  running out of time that's a yeah so we better go into the questions yeah thank you thank you very  much okay so we have two questions here so the first one is speaking of caching caching is also  needed for WMTS XYZ raster tiles QGIS QGIS is not responsible for the largest data load of the free  map raster tiles according to the OSM or maintainers hope that becomes better with OGCI API it's a  question yes I don't know the internals of how how this is managed in QGIS but I know that I am  one of those users that that add local layers of street map almost every day so so I can imagine  that it's very popular I mean having the white screen with a with a simple shape file on top just  looks bad you always want a background and open street map is also my favorite one but yeah it's  a I think providing these with these background servers background tile servers is really a point  of attention like a geo network geo server open layers in all their examples they're using the  open street map tiles which is a community initiative also so we should be careful I hope at  some point OSGO board for example may set up a tile server that we can use for this type of  demos and it doesn't need to go up till level skill 18 or 20 if it's for this type of demos skills  12 is just fine so that that's my idea and that I don't know what do you think yeah I don't know  much very much about the internals but yeah I'm with you Paul okay so the next question is do you  think we will need not only a QGIS OGC API section but maybe thinking in the future a different QGIS  platform that support and display the requested information think on sparkle queries it will  be possible to gather not only maps geocal localized data but also sounds measurements  features papers videos etc I think that's a good question my my my initial reaction to  that would be QGIS provides you know a strong mechanism for extensibility so you can develop  your own for example your own Python plugins so you can I could see an ecosystem of plugins  being made to satisfy those those work cases and those those use cases in fact metasarch itself is  written in Python not not c++ so I can't see you know I can see an ecosystem of these plugins  being built just the same to deal with sparkle and maybe maybe some other things I don't know Paul  what do you think it's it's a when I look at for example to the 3d world in a 3d world view  you you interact with what you see around you and this is also a bit the paradigm of QGIS  that is layer oriented and it's really hard to make links from from like we see here the city  cities of the Netherlands and then there's the open street map in the back but it's really hard  to make a make an interaction between the two and that that's related to the QGIS and and all  all most of the key of the the here's desktop clients that but in a 3d world you it's far  more common to click to follow layers and see interactions between a traffic light in the  street that it's connected to and and and and sound that it makes and then when I when I see a thing  like sparkle typically is also object oriented you go through a network of objects and then I think  QGIS by design is not really a tool fit for for such spatial navigation so that's a bit where I'm  heading on one hand the little layer paradigm brought us a lot but now it hinders us to advance  in a direction and I think at some point we need new clients that engage with data in a more  object oriented way okay thank you thank you very much Paul and Tom let's give a virtual  clap to the speakers and before moving to our next presentation so we're going to be talking about  the modular OGC API workflows for processing and visualization.
FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_0_131.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_132_551.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_552_779.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_780_1211.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_1212_1307.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_1308_1391.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_1392_1511.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_1512_2195.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_2196_2639.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_2640_2699.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_2700_5543.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_5544_6863.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_6864_6983.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_6984_7091.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_7092_7139.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_7140_11351.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_11352_14771.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_14772_19859.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_19860_23087.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_23088_24863.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_24864_26567.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_26568_27119.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_27120_28535.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_28536_28979.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_28980_29195.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_29196_29387.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_29388_29459.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_29460_29507.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_29508_29699.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_29700_29747.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_29748_29987.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_29988_30467.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_30468_30647.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_30648_31883.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_31884_39791.png FOSS4G_-_QGIS_and_OGC_APIs_-_how_do_they_work_toge_BQXrPtsMbGg_39792_39942.png
He is currently an STI specialist at ISRIQ. And he's been very involved also in phosphorgy projects like GeoNetwork and PyGEO API. So without further ado, I give you the room. I don't know who is starting. Maybe I can start. Let's see. I'm wondering if I'll share the screen. So maybe Paul, I'll drive and then you can just cue me. Is that okay? Okay. I see you're on mute, but I see that you agree. So let's go. Yes. Okay. Oops. Sorry about that. I muted. I didn't know that. Hello. Sorry, I don't know if you were trying to say something. Yeah, I was. Yeah, fine. Go ahead, Tom. Cool. Okay. Maybe I'll start off with a couple of notes. I guess this presentation is on how QGIS is increasingly supporting the OGC API efforts. And we'll talk a little bit about how they work together. I'll provide a small update on OGC API efforts. And then Paul will get into some of the details around QGIS and we're happy to answer any questions or comments. So OGC API. Hopefully you all have heard of what OGC API is. So going back in time a bit, we've started from in the mid to late 90s around doing web services with a lot of XML and basically remote procedure call over the web. And we moved on from their sort of web 2.0 type things. There are some realities associated with having those OGC, well, with specifications designed in that certain way in terms of overloading the HTTP specification as well as some difficulties in having things crawlable or searchable on mass market search engines and so on. So excuse me, in 2017, W3C came up with the spatial data on the web as practice. And a lot of efforts started to sort of culminate around the concepts of REST as well as JSON and OpenAPI. OGC had an API white paper in 2017 if I'm not mistaken, which basically called for a lot of created the opportunity to create these new OGC API standards which focused on JSON and HTML as first class and making things mass market friendly and web developer friendly. A web developer should not have to download a 400 page document to implement the specification whether it's a server or client or what have you. Mind you, I don't think anybody should have to download a 400 page document. But here we are. You can see a number of the specifications down below, whether they're maps, styles, coverages, features and so on. Most of them are successors to the first generation of OGC API standards. These standards are developed interactively. So these standards are developed now on GitHub. If you go to github.com slash open geospatial you'll see a number of the standards there. You'll see OGCAPI.org. You'll find all the information on all the APIs and all their specification documents fully available there in HTML. There's GitterChat so that you can easily interact with the community around the specification development and questions around compliance and performance and testing for that matter. And as mentioned in the previous presentation by Clemens, Peter and myself, there's really a building block model to all of these specifications that will allow you to plug and play and put those Lego bricks together to be able to come up with something that meets your needs. With that I'm going to turn it over to Paul. Thank you Tom. So I use it. So where to start? So OGCAPI features and QGIS. A lot of people contributed to QGIS over time. I'm not in a position here to mention everybody that contributed but hopefully I give everybody that contributed enough credits. Here they are. I want to drive in quickly since we only have 15 minutes to see some functionality that is available these days. So starting with OGCAPI features, it was also the first one that kind of got adopted within the OGC or started the movement. This was implemented by Ivan Ruo and funded by Planet and it was implemented as an extension to the existing WFS provider. So I think it's landed in somewhere 314, 316, I don't know. You will find this functionality under the add WFS layer and then instead of putting there the WFS capabilities URL, you put the OGCAPI features URL and from then on it kind of works in the same way that it fetches the layers from the service and you select the layer and you add it to the map. There's some special parameters that you can add here, for example the pagination size. So in that sense we have the same challenges as WFS that if you have a very big page size that the service may actually load quite slow, if you make a smaller page size it will be faster but you will only see a partial result. This is always something to keep in mind. So what is also good to know here is that it uses behind the scene the same architecture as WFS, it means that QGIS caches the features in a local cache. So if you zoom in and zoom out outside the area, it will still have the previous features in its cache. So it doesn't need to go back to the server to fetch them again. Next slide Tom. So from Ivan we discussed a bit before this meeting, some learning points from his side is that the fact that the items in the collection most of the times are untyped. So in theory you could add them, add the definition of the item to the open API document but the specification doesn't necessarily need you to do so. So if you have there an item say a tree, if it has a property branches and size that could be specified in the open API document but it's not always required. It means that a client like QGIS has to get the first page of results to see what is actually returned and which properties are available on the object. It's a bit awkward for a client and especially people that are used to build clients against open API specifications will be a bit surprised about this outcome. On the other hand for the server implementers this is easy because they don't need to go into each object to see what properties it has and advertise it in the open API document. Another point that we came up with discussion is that if you have hierarchical data like for example I have a soil profile in the soil profile is a horizon and in a horizon has a certain clay percentage and a certain magnesium value to query the magnesium concentration over the 12 micrograms for the second horizon in profiles within a bee box you need quite a dedicated filter expression capability and this is currently not yet fully crystallized how that will work out but it is something that we maybe we have some time this week to discuss that topic. Next one Tom. OGCI records I give the floor to you Tom for this one. You muted. Okay great thank you Paul. So for OGCI API records as Paul mentioned QGIS is a long standing project with a number of contributors and a long time solid support of the OGCI standard so we can see WMS, WFS, WCS and so on Paul just talked about OGCI features now let's talk about search. So in QGIS which Paul and I are presenting on tomorrow if anybody else is interested in seeing that presentation I won't go into too many details here but in QGIS is a long standing catalog client called Metasearch which has always worked with the OGCI catalog service for the web specification. Well in the last year alongside of working in the OGCI standards working group we also implemented extended support in the Metasearch plugin which is a core part of QGIS so when you download QGIS you will have the Metasearch plugin installed by default where it's all almost merged but as you can see in the lower left part of the screen but we implemented OGCI API records support in Metasearch and Metasearch internally uses the OWS live Python client OWS client library so that client library itself has been extended to support OGCI API features records and coverages that's the current support and we're extending it as time goes on and more OGCI API specifications are ratified but we do have a pull request in with some working functionality against OGCI API records so there you can see a screenshot there that's hitting a note that could hit an OGCI API records endpoint in the same way that it would a CSW endpoint so we've extended the client to be able to do that so when that's merged you'll be able to say okay add my OGCI API records client you do have to specify that it's an API records endpoint or a service as opposed to a CSW service and then after that all of the functionality is abstracted away so you don't really know whether you're working with a CSW or an OGCI API records client and that's the goal of the beauty and the value of the Metasearch plugin so in theory we're able to add any type of search service but here we're concentrating on QGIS on OGCI API records for that matter and in that work we actually did some digging in Metasearch and abstracted the search capability to make it very very easy to add more APIs over time if we need them or profiles of OGCI API records for that matter so like I mentioned it's currently a pull request we're just have some final issues to deal with with regards to packaging and we hope to have that merged very soon maybe even this week there's a code sprint on Saturday so let's see what can be done with regards to that some issues or some discussion plans that came up with OGCI API records link types I mean this has always been an issue in in catalogs and search APIs in discovering actionable links so how do I know that a link is a WMS link or a WCS link or WFS link well we've always had that issue and we have a way of working with that in QGIS which also requires some cooperation from the metadata providers but now we have a new specification with not only having the requirement to articulate the link types of the first generation OGCI web services but now we have we introduce link types of the new OGCI API standards and the link types in the OGCI API records case is going to be a lot easier for Metasearch because we have the ability to crawl down a certain level of an OGCI API record endpoint so we can get to just that collection because of the hierarchical and the restful approach of the way the standards are designed so current so you won't have to open Metasearch find the data set add the WMS and go back and do a get capabilities again and do that round trip and find the layer now you will be able to connect to the OGCI records endpoint find the find the matching record and whatever the link type underneath is which when it's an OGCI API link you can you'll be able to load that directly in QGIS so I think that's going to be a very valuable feature as as more and more of these things get get put online and server data providers make their metadata discovery metadata available through OGCI API records and the new link types so there is change management involved but that's a W you know OGCI web services and OGCI API change management thing that I think will be here for a while and this is what that some of what that change management looks like here in OGCI API records and what that means for the for in QGIS. Other other issues with regards to QGIS there are other OGCI APIs as we've mentioned we're not aware of any current current initiatives at this point in time that implement the other APIs but I would imagine over time that we'll start to see updates on clients for maps, tiles coverage, coverages and processes as those specifications mature and they you know they're formalized by OGCI and and you know clients want to have that functionality implemented. The OGCI does have an API roadmap which which is which will be valuable to I think is valuable for folks to look at if they want to see where what the the timelines of the specifications are and that'll give the QGIS community a better idea of when's a good time to dive into the standard and and implement it so I think this is a you know a great advancement in the OGCI API standards and QGIS is a you know is a very powerful GIS client and we're you know we're we've demonstrated that we're able to implement these standards with a relatively low barrier low barrier workflow. Paul? Yeah let me take that one so so while discussing with Ivan of course he introduced the aspect that the GDAL supports the full OGCI API actually a bit more than than QGIS itself by now so there's support for maps and and tiles and it's actually possible to to add those to QGIS also via this MOAW I don't know you say more file I don't know so it's kind of a processing file where you say okay load this data via GDAL so that's that's already an option in QGIS so if you really want to have your OGCI API maps in QGIS you can do it via such a file that's that's described on that endpoint. Next one yeah so so then dive a bit into QGIS server of course QGIS is mainly a desktop application but there has been some initiatives to have also QGIS as a server so you run it as a Docker instance and request it as a WMS or WFS and within that next one Tom within that area there was also initiative to add OGCI API features to the QGIS server and it works actually out of the box the only thing you need to be aware of is that once you before you upload your your QGIS project to the QGIS server you have to tick a box in the in the server properties of the layer saying that it should be exposed as a WFS layer it will then automatically be advertised also as an OGCI API features layer so that that's really useful and as far as I know no support for maps and tiles also on that side yet but yeah I hope that that work starts soon. Next one yeah so so that's about it do you have a QGIS there Tom or shall I maybe Joanna maybe you can add my screen to the that's a good idea but it's also maybe you can already prepare some some answers to some of the questions Tom because we entered the discussion phase of this presentation so I have here a QGIS instance with focused on the Netherlands with some of the provinces and those provinces are actually coming from the OGCI API features so if I add a layer of type in this case I have to go add WFS layer that that's the one that includes the OGCI API features here is the first you have to set up a server connection if you edit that you will notice that it you just put here the the endpoint of the in this case is a PyG API running on a sandbox environment and here you have those options like putting the page size and you can detect here and says oh it's an OGCI API features and then from that it will fetch from the collections page all the collections that are available in this service and you can then add that to the map so it works exactly like a WFS service and I wanted to add this aspect also related to the OGCI API very records is that this implementation also helps us at the server side to develop our server because it gives such an easy client experience that we always use QGIS for testing our server implementation and that's also a benefit because we make sure that it works on QGIS first before any other client unfortunately I cannot show the OGCI API records on my Mitter search because I had a problem with the OE or WSLip sorry Tom that's okay I mean we are also running out of time that's a yeah so we better go into the questions yeah thank you thank you very much okay so we have two questions here so the first one is speaking of caching caching is also needed for WMTS XYZ raster tiles QGIS QGIS is not responsible for the largest data load of the free map raster tiles according to the OSM or maintainers hope that becomes better with OGCI API it's a question yes I don't know the internals of how how this is managed in QGIS but I know that I am one of those users that that add local layers of street map almost every day so so I can imagine that it's very popular I mean having the white screen with a with a simple shape file on top just looks bad you always want a background and open street map is also my favorite one but yeah it's a I think providing these with these background servers background tile servers is really a point of attention like a geo network geo server open layers in all their examples they're using the open street map tiles which is a community initiative also so we should be careful I hope at some point OSGO board for example may set up a tile server that we can use for this type of demos and it doesn't need to go up till level skill 18 or 20 if it's for this type of demos skills 12 is just fine so that that's my idea and that I don't know what do you think yeah I don't know much very much about the internals but yeah I'm with you Paul okay so the next question is do you think we will need not only a QGIS OGC API section but maybe thinking in the future a different QGIS platform that support and display the requested information think on sparkle queries it will be possible to gather not only maps geocal localized data but also sounds measurements features papers videos etc I think that's a good question my my my initial reaction to that would be QGIS provides you know a strong mechanism for extensibility so you can develop your own for example your own Python plugins so you can I could see an ecosystem of plugins being made to satisfy those those work cases and those those use cases in fact metasarch itself is written in Python not not c++ so I can't see you know I can see an ecosystem of these plugins being built just the same to deal with sparkle and maybe maybe some other things I don't know Paul what do you think it's it's a when I look at for example to the 3d world in a 3d world view you you interact with what you see around you and this is also a bit the paradigm of QGIS that is layer oriented and it's really hard to make links from from like we see here the city cities of the Netherlands and then there's the open street map in the back but it's really hard to make a make an interaction between the two and that that's related to the QGIS and and all all most of the key of the the here's desktop clients that but in a 3d world you it's far more common to click to follow layers and see interactions between a traffic light in the street that it's connected to and and and and sound that it makes and then when I when I see a thing like sparkle typically is also object oriented you go through a network of objects and then I think QGIS by design is not really a tool fit for for such spatial navigation so that's a bit where I'm heading on one hand the little layer paradigm brought us a lot but now it hinders us to advance in a direction and I think at some point we need new clients that engage with data in a more object oriented way okay thank you thank you very much Paul and Tom let's give a virtual clap to the speakers and before moving to our next presentation so we're going to be talking about the modular OGC API workflows for processing and visualization.
came up with the spatial data on the web as to the first generation of OGC API standards. existing WFS provider. So I think it's landed in talk about search. So in QGIS which Paul and I support in the Metasearch plugin which is a core Metasearch and Metasearch internally uses the OWS client library itself has been extended to search capability to make it very very easy to very valuable feature as as more and more of to QGIS there are other OGCI APIs as we've to have that functionality implemented. The OGCI will be valuable to I think is valuable for folks to set up a server connection if you edit that endpoint of the in this case is a PyG API running you you interact with what you see around you and at some point we need new clients that engage