The OGC's Standards Process and the Role of Reference Implementations
Trakas, Athina
Simonis, Ingo
This presentation focuses on the role of reference implementations for the OGC standards process and gives an insight in the related OGC Compliance Program. OGC Standards are developed in the OGC Standards Program and follow rules and guidlines that have been set by the OGC's members. Before an OGC standard is adopted by the OGC membership, a public review period is required where non-members can also contribute. Increasingly, some standards working groups also decide to work more openly in the public (like the GeoPackage work group did) to be more inclusive. OGC interface standards also come with reference implementations. The OGC rules state that these have to be "free and publicly available for testing". Some well known OSGeo projects like GeoServer, MapServer, deegree and others are reference implementations of OGC standards. A Reference Implementation is a -> fully functional, licensed copy of a tested, branded software that has passed the test for an associated conformance class in a version of an Implementation Standard and that -> is free and publicly available for testing via a web service or download.
 Hello. Good morning on the Thursday. Yeah, you probably had another session before this one. This is my first. So, yeah, there's always a, when you look at the program, it's kind of, you kind of puzzle when how did the organization make the, put the different presentations together. Well, this one is clearly around the OGC standards. So, yeah, this morning we have three presentations about OGC related standards. First one is Athena, Long History in OGC. And I think no need to introduce you. Good luck. Thank you. Welcome everyone to the session. Yeah, my name is Athena Dracas. I work for OGC and I'm OGC's representative in Europe. So, whenever you have questions. And I brought a co-speaker. That's Dirk Stegger from Latteland. And so what we are doing, I do a first more theoretical introduction and then we go practical with Dirk. And I have to say, so compliance doesn't, that's my colleague, Luis Bermules. So, he is our compliance person. And actually, he provided most of the input for my slides. So, just one slide about the OGC. What do we do? We, that is our members, they develop open standards and associated standards, best practices. And the OGC as an entire organization, we want to serve for, as a global forum for the communication and collaboration of users, software developers, scientists in the geospatial domain. What we are also doing, we provide an agile and innovative development. Actually, we are bringing people together so they can test their software. If they can, against a real world scenario, they can use the standards and the data. And then this all goes back into the various working groups and the information is shared. And so, we try to improve. We are over 525 member organizations worldwide. And we have an internal portal for the members and that's nearly 7,000 users. And our members have developed more than 60 standards. Most of, or many of them are also ISO standards. Because one aspect we are doing, we collaborate with a lot of other standards organizations or general organizations in the domain, also with OSGEO. Today, I want to talk about the compliance testing. And the compliance programs deals actually with various activities. And the key topic is, of course, it oversees the compliance test activities with the standards working groups and software developers to improve the testing tools and the tests of the standards. And the compliance program, and in person, Luis Bermunes, he is managing the process for certification of software product. And what does this mean? I will try to explain in the next few slides. Why do we need reference implementations? So, we have standards, there is software, but people might want to look, they need more information about, you know, does the standard really does this job? And so, there are three ideas or there's a process. First of all, we need to validate implementations. That is, we test and prove that the software does what it has to do according to the definitions in the standard. Then we need a validator for this. That's a machine that tests the software that does the job of the testing. And then we need implementations to verify that the test makes sense. And this is an iterative process, but at one point we stopped with that. So, we had at Phosphor G, CEE, Central European Central Europe, Central and Eastern Europe 2014, there was a benchmark session that was in Bremen on the WPS. And the results, so you can see various projects that are implementing the WPS standard assigned individuals from their community to participate in this benchmark. And you see Gerard Verneuil mentioned, he is here and 5 to N, they have a booth there. So if you want to learn more details about that, you can visit these people or approach the people. But stuff like this is really important and really valuable for OGC because we get feedback on the standards, but it's also very valuable for OSG for the open source software community. So, we need to have a validator that tests and tools that run the test. So before we can go to the certification program or to the certification of the software, we have to test, they have to define a test and check if the test is right. So there are various testing tools, there's tools, communities and support. We use team engine and as you can see, team engine is on GitHub, so it's open source software. There are code contributors and the second one is Luis Bermudez, he has contributed the most code, but there are more people that participate to the code of the test engine. Then we have a public forum for support and of course, communities or communication facilities. The idea is that if you want to see what the test does, you can download the software, the code from GitHub, but you can also go directly to the OCC website, so you don't have to store it locally on your computer. Once the test has been approved and is fixed, then and you want to go for OCC certification, you have to do this on the OCC website, that's a service, but on the one side it's a service, but if you want to have an OCC certificate, you have to do it on the website, but it's transparent, it's open source, so you can also download the source code. Anybody can run the test and see if it does the work correctly, so this means not only members of the OCC can do that, but also non-members and you register and you don't have to install it on your computer, you can just use the OCC website. There's a selection of tests, we have various tests, not only one, we have, I don't know how many exactly, so you select with standards you want to test and then the test suite, you enter a lot of some additional information and then you can start the new test session. The team engine will provide various windows where you can select options, so it's very step by step, it's nothing, it's not rocket science, it's nothing complicated, the test guides you through step by step through the entire process and then there comes a result and here you can see that they have tested the WMS 1.1.1 test suite and there are two processes that failed and if you can click here on the view details to see why this failed and then you can identify is the test not mature enough, do we have to work on the test, we're still in the phase of testing the test for the standard or you can see if the software is not correct and then of course you would provide feedback to Lewis or the group so they can of course improve either the test or you can look if it's in your software. We have various types of tests, encodings like sensor ML, KML, GML, metadata and XML and more, then servers, there are many tests out there and by the way all this you can read this in more detail on the OTC website, there is a compliance test website and we have one client test for WMS 1.3. So now we have identified, well we have tested and validated the test and have said okay now the test is fixed, secure, it's mature enough, now we need implementations to verify that the test makes sense. But that's, we have a process on OTC so if you say okay the test is mature enough, we go through a formal process in the OTC and well I don't go through all the details here but the members, they approve that the test is okay and once it's approved by the members we can go for the next step and start with the reference implementations. Here another example, the catalog services standard, we have two reference implementations that's PYCSW and I saw Angelos Sotsas is here and maybe later in the question answering session because he has implemented the reference implementation with a group for the catalog service and as for geoportal service another reference implementation and the reference implementation for an OTC standard has to be freely and publicly available for testing via a web service or download so this is a precondition, this really has to be like that. Okay so how can you use now the test once it has been approved and in the process a test for a special standard? Well if you're a software developer you can get OTC certified and we certify the implementations then you get a badge with your name on it which tests you have taken and you can use this for your website. Basically we have 851 implementations and 203 compliant products, this means that they are 851 implementations that say we are implemented as special standard but they have not taken the compliance test and 203 products has taken the compliance test so they can use the logo and say okay we are OTC compliant. There might be some analogies with the OSGEO incorporation process but the OTC process is purely technical. Yes? So why is this difference? Okay, for some it's okay they say we have implemented it, we don't want to do it to the very detail. It has to be said that also if you take the certification and you want to use the logo you have to pay a license fee for that so your software might be 100% implementing the standard but if you say okay I don't want to pay the fee for that then you don't go for this certification. So for users, if you want to show that your implementation really works you can take the test and then you can use that for marketing material or for tenders you can say okay see I have run the test on the OTC website so this might help you to get more business to have a well be a step ahead of some of your competitors and for users it might be also interesting if they are looking for software and don't know exactly what software should I take they can go on the OTC website and find the entire list of implementing products and certified products. So if you want, if you need for your architecture a special standard and want to use software please have a look on our website because there you can find a lot of information, a lot of products that are implementing or even have been certified as OTC compliant. For implementers it's also interesting you know if you have the logo you can say I'm OTC compliant and there is no doubt that your software does not implement the standard so it's very straightforward. You can promote it in your products on your website so that's the marketing part of the work. And here this is kind of a summary how it works again actually it's more or less you know you take the test, the approved test now that is secure that is mature, you take the test you go through the entire process and then it comes if there's no fail coming back you can say okay I want to go the next step I enter some more data so OTC stuff that's me or in this case we check all the details and then we give you the OTC compliance logo and you can use that. So very very very very important conformant equals compliant equals passing the test plus the license fee. You cannot say that you're conformant or compliant if you have not the OTC certification logo and please don't do it because that makes the job easier because if you say you're compliant you have not passed the test I have to call you if you're based in Europe and then we have to go through the entire process again so yeah I would be very appreciative if you don't do that. If not you still can say we're implement OTC standards. Good that's a list of available tests you can go through that that's the plans for expected tests that come out in 2016 and then for those who have to do tenders have you published tenders we have developed a guide, a white paper for a compliant guide for software acquisition white paper so if you want if you have to write a tender and you're looking for the right correct language then you can find that on our website. So that was the theoretical background and I would like to hand over to Dirk from Ladlund because he is working very closely with Louis Bemundes. Hello I'm Dirk Stenger from Ladlund and currently I'm working together with Louis and currently as part of the project testbed 12 or the testbed 12 initiative we are implementing a WFS2-0 reference implementation. But first I want to give you a short introduction as I already said I'm working at Ladlund. My main areas of work of course focus on OTC standards and we also developed the degree software which is providing OTC standards like WMS, WFS, CSW on an open source basis. Also a member of the side team which is led by Louis and my part is that I'm the technical lead of the WMS, WFS and WCS test suites. ETS stands for executable test suite and so if someone is reporting a bug to one of those test suites I'm taking a look at it and try to solve it or give you advice and of course I'm also maintaining those test suites. This is just a slide of what the OTC testbed 12 initiative is. So in general sponsors document their needs and create projects out of that then participants can apply for those projects and we are one of those participants. Now we are implementing the use case which was defined by the sponsor and of course the sponsor also pays money for that. That we are doing the implementation. For example participants implement existing OTC standards or develop new prototypes. OTC testbed 12 ends at the end of the year. Our project of testbed 12 that we developed the WFS2.0 reference implementation. The main objective is that our implementation should be comply with following conformance classes and you can see a list there. As you can see there are really advanced features there. For example feature learning or managed as thought queries is hardly implemented yet. When the project is finished we will pass all those conformance classes. There is also a corresponding project which advances the WFS2.0 test suite. This is also part of the testbed 12 initiative. This project concentrates on the same conformance classes but not the reference implementation is created but the corresponding test suite. The process of implementing the reference implementation has a lot of communication and coordination with it. The other participant is advancing the test suite and then we can enhance our implementation with this improvements. Of course this leads to a lot of communication between us two. We develop our own integration tests with SOPAI and all those tests concentrate on the WFS2.0 specification. Also as I already said the main part is the discussion with other members and we have questions like how clear is the specification or does the specification allow multiple ways of implementing a new feature. Sometimes the specification is not clear enough that everyone knows how should the test look like and the reference implementation. Of course we also detect bugs in the test suite and we report them and this also leads to better quality of the test suite. So are there any questions from the audience? I don't know. There is some special conditions for compliance if you are open source and the reference implementation because the fees between something 60 and 11,000 euros I don't know what it was. I think you did the PyCSW implementation. Can you say something on this? So for PyCSW we did the reference implementation following the catalog 3 version of the standard. So in our case we didn't have to pay any fee for that because we were also involved in the process of testing the test suite. So it was like we were implementing the standard but also the test suite was in the process of being implemented. So we were following the beta versions and every time the beta version of the test was released we were also testing our beta version and we were able to find problems, issues, bugs in both the site test and our software. And also this is important why we have two reference implementations because if there is a conflict between those two reference implementations then the test will not be specific to one reference implement because there are some aspects that one has to take a technical decision and having two reference implementations is very, very important so we have more consensus. But yeah, it didn't require any fees from our point because it was an open source project. It was released actually the day that Odyssey released the final version of the standard. We released the same day because we were, the slide that Athena saw before has this process that takes long to release the standard so we were waiting when the standard is going to be released to release actually our software. So yeah, but it was very, very important for us to be able to work with the site team to be able to comply with the test early. And thank you. We have room for one more question. Is that somebody? Not really. So as a software developer I really appreciate the work here because as a software developer I don't like to read thick documents but what I do know is test-driven development and this allows me to do test-driven development. I can just do a bit of development and test against this and know that it works. So very thanks for your work. Also, Clemens, the next speaker will also talk about testing specifications. So please bring back your slides. Thank you very much. It's a huge topic and we can only have a spotlight into that. And yeah, thanks for the opportunity to speak here.
 Hello. Good morning on the Thursday. Yeah, you probably had another session before this  one. This is my first. So, yeah, there's always a, when you look at the program, it's  kind of, you kind of puzzle when how did the organization make the, put the different  presentations together. Well, this one is clearly around the OGC standards. So, yeah,  this morning we have three presentations about OGC related standards. First one is Athena,  Long History in OGC. And I think no need to introduce you. Good luck.  Thank you. Welcome everyone to the session. Yeah, my name is Athena Dracas. I work for  OGC and I'm OGC's representative in Europe. So, whenever you have questions. And I brought  a co-speaker. That's Dirk Stegger from Latteland. And so what we are doing, I do a first more  theoretical introduction and then we go practical with Dirk. And I have to say, so compliance  doesn't, that's my colleague, Luis Bermules. So, he is our compliance person. And actually,  he provided most of the input for my slides. So, just one slide about the OGC. What do  we do? We, that is our members, they develop open standards and associated standards, best  practices. And the OGC as an entire organization, we want to serve for, as a global forum for  the communication and collaboration of users, software developers, scientists in the geospatial  domain. What we are also doing, we provide an agile and innovative development. Actually,  we are bringing people together so they can test their software. If they can, against  a real world scenario, they can use the standards and the data. And then this all goes back  into the various working groups and the information is shared. And so, we try to improve. We are  over 525 member organizations worldwide. And we have an internal portal for the members  and that's nearly 7,000 users. And our members have developed more than 60 standards. Most  of, or many of them are also ISO standards. Because one aspect we are doing, we collaborate  with a lot of other standards organizations or general organizations in the domain, also  with OSGEO. Today, I want to talk about the compliance testing. And the compliance programs  deals actually with various activities. And the key topic is, of course, it oversees the  compliance test activities with the standards working groups and software developers to  improve the testing tools and the tests of the standards. And the compliance program,  and in person, Luis Bermunes, he is managing the process for certification of software  product. And what does this mean? I will try to explain in the next few slides. Why do  we need reference implementations? So, we have standards, there is software, but people  might want to look, they need more information about, you know, does the standard really  does this job? And so, there are three ideas or there's a process. First of all, we need  to validate implementations. That is, we test and prove that the software does what it has  to do according to the definitions in the standard. Then we need a validator for this.  That's a machine that tests the software that does the job of the testing. And then we need  implementations to verify that the test makes sense. And this is an iterative process, but  at one point we stopped with that. So, we had at Phosphor G, CEE, Central European Central  Europe, Central and Eastern Europe 2014, there was a benchmark session that was in Bremen  on the WPS. And the results, so you can see various projects that are implementing the  WPS standard assigned individuals from their community to participate in this benchmark.  And you see Gerard Verneuil mentioned, he is here and 5 to N, they have a booth there.  So if you want to learn more details about that, you can visit these people or approach  the people. But stuff like this is really important and really valuable for OGC because  we get feedback on the standards, but it's also very valuable for OSG for the open source  software community. So, we need to have a validator that tests and tools that run the  test. So before we can go to the certification program or to the certification of the software,  we have to test, they have to define a test and check if the test is right. So there are  various testing tools, there's tools, communities and support. We use team engine and as you  can see, team engine is on GitHub, so it's open source software. There are code contributors  and the second one is Luis Bermudez, he has contributed the most code, but there are more  people that participate to the code of the test engine. Then we have a public forum for  support and of course, communities or communication facilities. The idea is that if you want to  see what the test does, you can download the software, the code from GitHub, but you can  also go directly to the OCC website, so you don't have to store it locally on your computer.  Once the test has been approved and is fixed, then and you want to go for OCC certification,  you have to do this on the OCC website, that's a service, but on the one side it's a service,  but if you want to have an OCC certificate, you have to do it on the website, but it's  transparent, it's open source, so you can also download the source code.  Anybody can run the test and see if it does the work correctly, so this means not only  members of the OCC can do that, but also non-members and you register and you don't have to install  it on your computer, you can just use the OCC website. There's a selection of tests,  we have various tests, not only one, we have, I don't know how many exactly, so you select  with standards you want to test and then the test suite, you enter a lot of some additional  information and then you can start the new test session.  The team engine will provide various windows where you can select options, so it's very  step by step, it's nothing, it's not rocket science, it's nothing complicated, the test  guides you through step by step through the entire process and then there comes a result  and here you can see that they have tested the WMS 1.1.1 test suite and there are two  processes that failed and if you can click here on the view details to see why this failed  and then you can identify is the test not mature enough, do we have to work on the test,  we're still in the phase of testing the test for the standard or you can see if the software  is not correct and then of course you would provide feedback to Lewis or the group so  they can of course improve either the test or you can look if it's in your software.  We have various types of tests, encodings like sensor ML, KML, GML, metadata and XML  and more, then servers, there are many tests out there and by the way all this you can  read this in more detail on the OTC website, there is a compliance test website and we  have one client test for WMS 1.3.  So now we have identified, well we have tested and validated the test and have said okay  now the test is fixed, secure, it's mature enough, now we need implementations to verify  that the test makes sense.  But that's, we have a process on OTC so if you say okay the test is mature enough, we  go through a formal process in the OTC and well I don't go through all the details here  but the members, they approve that the test is okay and once it's approved by the members  we can go for the next step and start with the reference implementations.  Here another example, the catalog services standard, we have two reference implementations  that's PYCSW and I saw Angelos Sotsas is here and maybe later in the question answering  session because he has implemented the reference implementation with a group for the catalog  service and as for geoportal service another reference implementation and the reference  implementation for an OTC standard has to be freely and publicly available for testing  via a web service or download so this is a precondition, this really has to be like  that.  Okay so how can you use now the test once it has been approved and in the process a test  for a special standard?  Well if you're a software developer you can get OTC certified and we certify the implementations  then you get a badge with your name on it which tests you have taken and you can use  this for your website.  Basically we have 851 implementations and 203 compliant products, this means that they  are 851 implementations that say we are implemented as special standard but they have not taken  the compliance test and 203 products has taken the compliance test so they can use the logo  and say okay we are OTC compliant.  There might be some analogies with the OSGEO incorporation process but the OTC process  is purely technical.  Yes?  So why is this difference?  Okay, for some it's okay they say we have implemented it, we don't want to do it to the very detail.  It has to be said that also if you take the certification and you want to use the logo  you have to pay a license fee for that so your software might be 100% implementing the  standard but if you say okay I don't want to pay the fee for that then you don't go  for this certification.  So for users, if you want to show that your implementation really works you can take the  test and then you can use that for marketing material or for tenders you can say okay see  I have run the test on the OTC website so this might help you to get more business to  have a well be a step ahead of some of your competitors and for users it might be also  interesting if they are looking for software and don't know exactly what software should  I take they can go on the OTC website and find the entire list of implementing products  and certified products.  So if you want, if you need for your architecture a special standard and want to use software  please have a look on our website because there you can find a lot of information, a  lot of products that are implementing or even have been certified as OTC compliant.  For implementers it's also interesting you know if you have the logo you can say I'm  OTC compliant and there is no doubt that your software does not implement the standard  so it's very straightforward.  You can promote it in your products on your website so that's the marketing part of the  work.  And here this is kind of a summary how it works again actually it's more or less you  know you take the test, the approved test now that is secure that is mature, you take  the test you go through the entire process and then it comes if there's no fail coming  back you can say okay I want to go the next step I enter some more data so OTC stuff that's  me or in this case we check all the details and then we give you the OTC compliance logo  and you can use that.  So very very very very important conformant equals compliant equals passing the test plus  the license fee.  You cannot say that you're conformant or compliant if you have not the OTC certification logo  and please don't do it because that makes the job easier because if you say you're  compliant you have not passed the test I have to call you if you're based in Europe and  then we have to go through the entire process again so yeah I would be very appreciative  if you don't do that.  If not you still can say we're implement OTC standards.  Good that's a list of available tests you can go through that that's the plans for expected  tests that come out in 2016 and then for those who have to do tenders have you published  tenders we have developed a guide, a white paper for a compliant guide for software acquisition  white paper so if you want if you have to write a tender and you're looking for the  right correct language then you can find that on our website.  So that was the theoretical background and I would like to hand over to Dirk from Ladlund  because he is working very closely with Louis Bemundes.  Hello I'm Dirk Stenger from Ladlund and currently I'm working together with Louis  and currently as part of the project testbed 12 or the testbed 12 initiative we are implementing  a WFS2-0 reference implementation.  But first I want to give you a short introduction as I already said I'm working at Ladlund.  My main areas of work of course focus on OTC standards and we also developed the degree  software which is providing OTC standards like WMS, WFS, CSW on an open source basis.  Also a member of the side team which is led by Louis and my part is that I'm the technical  lead of the WMS, WFS and WCS test suites.  ETS stands for executable test suite and so if someone is reporting a bug to one of those  test suites I'm taking a look at it and try to solve it or give you advice and of course  I'm also maintaining those test suites.  This is just a slide of what the OTC testbed 12 initiative is.  So in general sponsors document their needs and create projects out of that then participants  can apply for those projects and we are one of those participants.  Now we are implementing the use case which was defined by the sponsor and of course the  sponsor also pays money for that.  That we are doing the implementation.  For example participants implement existing OTC standards or develop new prototypes.  OTC testbed 12 ends at the end of the year.  Our project of testbed 12 that we developed the WFS2.0 reference implementation.  The main objective is that our implementation should be comply with following conformance  classes and you can see a list there.  As you can see there are really advanced features there.  For example feature learning or managed as thought queries is hardly implemented yet.  When the project is finished we will pass all those conformance classes.  There is also a corresponding project which advances the WFS2.0 test suite.  This is also part of the testbed 12 initiative.  This project concentrates on the same conformance classes but not the reference implementation  is created but the corresponding test suite.  The process of implementing the reference implementation has a lot of communication and coordination  with it.  The other participant is advancing the test suite and then we can enhance our implementation  with this improvements.  Of course this leads to a lot of communication between us two.  We develop our own integration tests with SOPAI and all those tests concentrate on  the WFS2.0 specification.  Also as I already said the main part is the discussion with other members and we have  questions like how clear is the specification or does the specification allow multiple ways  of implementing a new feature.  Sometimes the specification is not clear enough that everyone knows how should the test look  like and the reference implementation.  Of course we also detect bugs in the test suite and we report them and this also leads  to better quality of the test suite.  So are there any questions from the audience?  I don't know.  There is some special conditions for compliance if you are open source and the reference implementation  because the fees between something 60 and 11,000 euros I don't know what it was.  I think you did the PyCSW implementation.  Can you say something on this?  So for PyCSW we did the reference implementation following the catalog 3 version of the standard.  So in our case we didn't have to pay any fee for that because we were also involved in  the process of testing the test suite.  So it was like we were implementing the standard but also the test suite was in the process  of being implemented.  So we were following the beta versions and every time the beta version of the test was  released we were also testing our beta version and we were able to find problems, issues,  bugs in both the site test and our software.  And also this is important why we have two reference implementations because if there  is a conflict between those two reference implementations then the test will not be  specific to one reference implement because there are some aspects that one has to take  a technical decision and having two reference implementations is very, very important so  we have more consensus.  But yeah, it didn't require any fees from our point because it was an open source project.  It was released actually the day that Odyssey released the final version of the standard.  We released the same day because we were, the slide that Athena saw before has this  process that takes long to release the standard so we were waiting when the standard is going  to be released to release actually our software.  So yeah, but it was very, very important for us to be able to work with the site team to  be able to comply with the test early.  And thank you.  We have room for one more question.  Is that somebody?  Not really.  So as a software developer I really appreciate the work here because as a software developer  I don't like to read thick documents but what I do know is test-driven development and  this allows me to do test-driven development.  I can just do a bit of development and test against this and know that it works.  So very thanks for your work.  Also, Clemens, the next speaker will also talk about testing specifications.  So please bring back your slides.  Thank you very much.  It's a huge topic and we can only have a spotlight into that.  And yeah, thanks for the opportunity to speak here.
foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_0_215.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_216_1895.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_1896_2135.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_2136_2219.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_2220_3575.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_3576_5027.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_5028_11063.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_11064_14567.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_14568_14687.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_14688_14819.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_14820_14951.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_14952_19667.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_19668_19739.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_19740_20543.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_20544_20591.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_20592_21443.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_21444_22919.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_22920_23015.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_23016_24155.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_24156_24971.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_24972_25955.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_25956_26687.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_26688_30083.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_30084_30215.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_30216_30911.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_30912_32375.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_32376_33323.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_33324_33419.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_33420_38123.png foss4g-2016-1327-the_ogc_s_standards_process_and_the_role_of_reference_implementations-hd_38124_38251.png
Hello. Good morning on the Thursday. Yeah, you probably had another session before this one. This is my first. So, yeah, there's always a, when you look at the program, it's kind of, you kind of puzzle when how did the organization make the, put the different presentations together. Well, this one is clearly around the OGC standards. So, yeah, this morning we have three presentations about OGC related standards. First one is Athena, Long History in OGC. And I think no need to introduce you. Good luck. Thank you. Welcome everyone to the session. Yeah, my name is Athena Dracas. I work for OGC and I'm OGC's representative in Europe. So, whenever you have questions. And I brought a co-speaker. That's Dirk Stegger from Latteland. And so what we are doing, I do a first more theoretical introduction and then we go practical with Dirk. And I have to say, so compliance doesn't, that's my colleague, Luis Bermules. So, he is our compliance person. And actually, he provided most of the input for my slides. So, just one slide about the OGC. What do we do? We, that is our members, they develop open standards and associated standards, best practices. And the OGC as an entire organization, we want to serve for, as a global forum for the communication and collaboration of users, software developers, scientists in the geospatial domain. What we are also doing, we provide an agile and innovative development. Actually, we are bringing people together so they can test their software. If they can, against a real world scenario, they can use the standards and the data. And then this all goes back into the various working groups and the information is shared. And so, we try to improve. We are over 525 member organizations worldwide. And we have an internal portal for the members and that's nearly 7,000 users. And our members have developed more than 60 standards. Most of, or many of them are also ISO standards. Because one aspect we are doing, we collaborate with a lot of other standards organizations or general organizations in the domain, also with OSGEO. Today, I want to talk about the compliance testing. And the compliance programs deals actually with various activities. And the key topic is, of course, it oversees the compliance test activities with the standards working groups and software developers to improve the testing tools and the tests of the standards. And the compliance program, and in person, Luis Bermunes, he is managing the process for certification of software product. And what does this mean? I will try to explain in the next few slides. Why do we need reference implementations? So, we have standards, there is software, but people might want to look, they need more information about, you know, does the standard really does this job? And so, there are three ideas or there's a process. First of all, we need to validate implementations. That is, we test and prove that the software does what it has to do according to the definitions in the standard. Then we need a validator for this. That's a machine that tests the software that does the job of the testing. And then we need implementations to verify that the test makes sense. And this is an iterative process, but at one point we stopped with that. So, we had at Phosphor G, CEE, Central European Central Europe, Central and Eastern Europe 2014, there was a benchmark session that was in Bremen on the WPS. And the results, so you can see various projects that are implementing the WPS standard assigned individuals from their community to participate in this benchmark. And you see Gerard Verneuil mentioned, he is here and 5 to N, they have a booth there. So if you want to learn more details about that, you can visit these people or approach the people. But stuff like this is really important and really valuable for OGC because we get feedback on the standards, but it's also very valuable for OSG for the open source software community. So, we need to have a validator that tests and tools that run the test. So before we can go to the certification program or to the certification of the software, we have to test, they have to define a test and check if the test is right. So there are various testing tools, there's tools, communities and support. We use team engine and as you can see, team engine is on GitHub, so it's open source software. There are code contributors and the second one is Luis Bermudez, he has contributed the most code, but there are more people that participate to the code of the test engine. Then we have a public forum for support and of course, communities or communication facilities. The idea is that if you want to see what the test does, you can download the software, the code from GitHub, but you can also go directly to the OCC website, so you don't have to store it locally on your computer. Once the test has been approved and is fixed, then and you want to go for OCC certification, you have to do this on the OCC website, that's a service, but on the one side it's a service, but if you want to have an OCC certificate, you have to do it on the website, but it's transparent, it's open source, so you can also download the source code. Anybody can run the test and see if it does the work correctly, so this means not only members of the OCC can do that, but also non-members and you register and you don't have to install it on your computer, you can just use the OCC website. There's a selection of tests, we have various tests, not only one, we have, I don't know how many exactly, so you select with standards you want to test and then the test suite, you enter a lot of some additional information and then you can start the new test session. The team engine will provide various windows where you can select options, so it's very step by step, it's nothing, it's not rocket science, it's nothing complicated, the test guides you through step by step through the entire process and then there comes a result and here you can see that they have tested the WMS 1.1.1 test suite and there are two processes that failed and if you can click here on the view details to see why this failed and then you can identify is the test not mature enough, do we have to work on the test, we're still in the phase of testing the test for the standard or you can see if the software is not correct and then of course you would provide feedback to Lewis or the group so they can of course improve either the test or you can look if it's in your software. We have various types of tests, encodings like sensor ML, KML, GML, metadata and XML and more, then servers, there are many tests out there and by the way all this you can read this in more detail on the OTC website, there is a compliance test website and we have one client test for WMS 1.3. So now we have identified, well we have tested and validated the test and have said okay now the test is fixed, secure, it's mature enough, now we need implementations to verify that the test makes sense. But that's, we have a process on OTC so if you say okay the test is mature enough, we go through a formal process in the OTC and well I don't go through all the details here but the members, they approve that the test is okay and once it's approved by the members we can go for the next step and start with the reference implementations. Here another example, the catalog services standard, we have two reference implementations that's PYCSW and I saw Angelos Sotsas is here and maybe later in the question answering session because he has implemented the reference implementation with a group for the catalog service and as for geoportal service another reference implementation and the reference implementation for an OTC standard has to be freely and publicly available for testing via a web service or download so this is a precondition, this really has to be like that. Okay so how can you use now the test once it has been approved and in the process a test for a special standard? Well if you're a software developer you can get OTC certified and we certify the implementations then you get a badge with your name on it which tests you have taken and you can use this for your website. Basically we have 851 implementations and 203 compliant products, this means that they are 851 implementations that say we are implemented as special standard but they have not taken the compliance test and 203 products has taken the compliance test so they can use the logo and say okay we are OTC compliant. There might be some analogies with the OSGEO incorporation process but the OTC process is purely technical. Yes? So why is this difference? Okay, for some it's okay they say we have implemented it, we don't want to do it to the very detail. It has to be said that also if you take the certification and you want to use the logo you have to pay a license fee for that so your software might be 100% implementing the standard but if you say okay I don't want to pay the fee for that then you don't go for this certification. So for users, if you want to show that your implementation really works you can take the test and then you can use that for marketing material or for tenders you can say okay see I have run the test on the OTC website so this might help you to get more business to have a well be a step ahead of some of your competitors and for users it might be also interesting if they are looking for software and don't know exactly what software should I take they can go on the OTC website and find the entire list of implementing products and certified products. So if you want, if you need for your architecture a special standard and want to use software please have a look on our website because there you can find a lot of information, a lot of products that are implementing or even have been certified as OTC compliant. For implementers it's also interesting you know if you have the logo you can say I'm OTC compliant and there is no doubt that your software does not implement the standard so it's very straightforward. You can promote it in your products on your website so that's the marketing part of the work. And here this is kind of a summary how it works again actually it's more or less you know you take the test, the approved test now that is secure that is mature, you take the test you go through the entire process and then it comes if there's no fail coming back you can say okay I want to go the next step I enter some more data so OTC stuff that's me or in this case we check all the details and then we give you the OTC compliance logo and you can use that. So very very very very important conformant equals compliant equals passing the test plus the license fee. You cannot say that you're conformant or compliant if you have not the OTC certification logo and please don't do it because that makes the job easier because if you say you're compliant you have not passed the test I have to call you if you're based in Europe and then we have to go through the entire process again so yeah I would be very appreciative if you don't do that. If not you still can say we're implement OTC standards. Good that's a list of available tests you can go through that that's the plans for expected tests that come out in 2016 and then for those who have to do tenders have you published tenders we have developed a guide, a white paper for a compliant guide for software acquisition white paper so if you want if you have to write a tender and you're looking for the right correct language then you can find that on our website. So that was the theoretical background and I would like to hand over to Dirk from Ladlund because he is working very closely with Louis Bemundes. Hello I'm Dirk Stenger from Ladlund and currently I'm working together with Louis and currently as part of the project testbed 12 or the testbed 12 initiative we are implementing a WFS2-0 reference implementation. But first I want to give you a short introduction as I already said I'm working at Ladlund. My main areas of work of course focus on OTC standards and we also developed the degree software which is providing OTC standards like WMS, WFS, CSW on an open source basis. Also a member of the side team which is led by Louis and my part is that I'm the technical lead of the WMS, WFS and WCS test suites. ETS stands for executable test suite and so if someone is reporting a bug to one of those test suites I'm taking a look at it and try to solve it or give you advice and of course I'm also maintaining those test suites. This is just a slide of what the OTC testbed 12 initiative is. So in general sponsors document their needs and create projects out of that then participants can apply for those projects and we are one of those participants. Now we are implementing the use case which was defined by the sponsor and of course the sponsor also pays money for that. That we are doing the implementation. For example participants implement existing OTC standards or develop new prototypes. OTC testbed 12 ends at the end of the year. Our project of testbed 12 that we developed the WFS2.0 reference implementation. The main objective is that our implementation should be comply with following conformance classes and you can see a list there. As you can see there are really advanced features there. For example feature learning or managed as thought queries is hardly implemented yet. When the project is finished we will pass all those conformance classes. There is also a corresponding project which advances the WFS2.0 test suite. This is also part of the testbed 12 initiative. This project concentrates on the same conformance classes but not the reference implementation is created but the corresponding test suite. The process of implementing the reference implementation has a lot of communication and coordination with it. The other participant is advancing the test suite and then we can enhance our implementation with this improvements. Of course this leads to a lot of communication between us two. We develop our own integration tests with SOPAI and all those tests concentrate on the WFS2.0 specification. Also as I already said the main part is the discussion with other members and we have questions like how clear is the specification or does the specification allow multiple ways of implementing a new feature. Sometimes the specification is not clear enough that everyone knows how should the test look like and the reference implementation. Of course we also detect bugs in the test suite and we report them and this also leads to better quality of the test suite. So are there any questions from the audience? I don't know. There is some special conditions for compliance if you are open source and the reference implementation because the fees between something 60 and 11,000 euros I don't know what it was. I think you did the PyCSW implementation. Can you say something on this? So for PyCSW we did the reference implementation following the catalog 3 version of the standard. So in our case we didn't have to pay any fee for that because we were also involved in the process of testing the test suite. So it was like we were implementing the standard but also the test suite was in the process of being implemented. So we were following the beta versions and every time the beta version of the test was released we were also testing our beta version and we were able to find problems, issues, bugs in both the site test and our software. And also this is important why we have two reference implementations because if there is a conflict between those two reference implementations then the test will not be specific to one reference implement because there are some aspects that one has to take a technical decision and having two reference implementations is very, very important so we have more consensus. But yeah, it didn't require any fees from our point because it was an open source project. It was released actually the day that Odyssey released the final version of the standard. We released the same day because we were, the slide that Athena saw before has this process that takes long to release the standard so we were waiting when the standard is going to be released to release actually our software. So yeah, but it was very, very important for us to be able to work with the site team to be able to comply with the test early. And thank you. We have room for one more question. Is that somebody? Not really. So as a software developer I really appreciate the work here because as a software developer I don't like to read thick documents but what I do know is test-driven development and this allows me to do test-driven development. I can just do a bit of development and test against this and know that it works. So very thanks for your work. Also, Clemens, the next speaker will also talk about testing specifications. So please bring back your slides. Thank you very much. It's a huge topic and we can only have a spotlight into that. And yeah, thanks for the opportunity to speak here.
presentations about OGC related standards. First test activities with the standards working groups need reference implementations? So, we have various projects that are implementing the WPS code of the test engine. Then we have a public mature enough, do we have to work on the test, have identified, well we have tested and once it's approved by the members we can go for reference implementation with a group for the reference implementation for an OTC standard has to be freely and publicly available for testing via a web service or download so this is a been approved and in the process a test for a and we also developed the degree software which test suite. The process of implementing the test look like and the reference implementation. on this? So for PyCSW we did the reference Odyssey released the final version of the