Leveraging documentation power for better web APIs
Sicard, Rudy
Rudy Sicard - Leveraging documentation power for better web APIs Web APIs that are easier to understand, develop, test and use, is a popular subject. "An API is only as good as its documentation". We decided to play with this proverb and leverage the power of documentation. We propose to use the code documentation and the type system to provide lots of free features: explorable APIs, better error messages, automatic testing. Python is perfect to explore code and documentation dynamically. We'll demonstrate what we came up to and the lessons we've learned. - 'Rich' web APIs that are easier to understand, develop, test and use, is a popular subject. There are a lot of new specification languages (e.g. swagger, apiblueprint ...) and libraries (django-rest-framework, drf ...) that provide features in this direction. Following the old proverb "An API is only as good as its documentation", we decided to play with these ideas and focus on leveraging the power of documentation. We propose to use the code documentation and the type system to provide: - browsable APIs, that are easy to interact with and visualize, reducing the need to provide custom UIs - verification of inputs/outputs along with precise error message if needed - automatic [de]-serialization of inputs/ouputs outside of the domain code - smart exception handling, e.g. exceptions that are not documented are automatically converted into internal errors - automatic testing, e.g. input, output and result including exceptions are tested ensuring the code works and the documentation is up-to-date. This use case is one of the rare situation where introspection is desirable and unavoidable. And Python is a good language to explore/exploit code and documentation dynamically. The perfect excuse to spend some time on meta coding a first implementation while being at work. We'll demonstrate what we came up to, the advantages and limitations compared to other approaches. And we'll share the lessons we learned from this experiment.
 Welcome. We have Rudy here, showing us how to leverage documentation power for better APIs. Give him some clap. Hello. Hello. Hello, I'm Rudy. I'm working at Criteo, a company that delivers personalized ads at scale. My team is building monitoring tools for our large software and hardware inventory. And sometimes, some time ago in this team, we decided to make monitoring as a service to experiment different ideas around web service and browser APIs. So this presentation gives a glimpse of what we have done. And this is a joint work with my colleague, Hugo Robour, which is there. So what is a browser API? Obviously, it's composed of two words, the browser word. So probably it's related to a browser. And more importantly, it's related to exploration and experimentation. And I will add to that more later. And there is a word API. We'll go to formal talk. So an API is a kind of contract. It can be very explicit, like if you're declaring swagger specification five, or it can be implicit, like in the documentation. The contract defines the valid use of the service that is linked to the API. Although, the operation you can do with this service, and also what are the valid inputs for operation in the service. So why is it interesting? Because it brings a lot of features for your service. So the first feature I will speak about is why we rely on the documentation to describe the API. Then I will speak about the basic feature that you have in the browser API. So the first basic feature is the explorability. You can go to the API and ask what the operation is possible on this API. So here you have a view of an API with different operations. And then you can just click on an operation. And you see the documentation of that operation, the parameter types, and what are the possible results. And then, more than that, you can play with an operation. So you have to tip the input of the operation and call it. And so you can see what is the response and explore the behavior of this operation. But still, typing the input, like a JSON input, can be difficult. So we probably can do something better. Another really important feature of an API is to enforce the validation of the input and the outputs. By the way, this talk focuses on JSON inputs. You don't want an operation to be used in an adapted way. That's why you should validate all the inputs of your service. If a list is transmitted instead of a string, nobody knows how the code will behave in that case. And even for valid JSON inputs, you may want to have a conversion between the JSON input and between your Python types. Okay, this one is an obvious feature. Like I said before, it can be really painful to enter JSON data because it's a verbose format. So having forms make you save times. And it brings client-side input validation on the table. Another really interesting feature is that you have that kind of API contract and you have your service implementation. And you want to be sure that they perfectly match. So that would be really cool to just ask for automatic tests for checking that they match. And I will also present other features that makes a browser API more user-friendly. And last, just to mention it, you may want to generate, even if you use the doc, more formal specification contract like in Swagger because Swagger has a lot of amazing features that you can later use. So, and this is the plan of the talk, in fact. So I'm starting now. So imagine you want to add a new function to your API. What do you do? Okay, it's easy. You document the functions and you add the published decorator on it. So this decorator, you will resist the function as an operation in the API. The function will be left unmodified. So you can still use it as before when the decorator was not there. And behind the scenes, the function will be wrapped in the API where all the data about HTTP will be handled. So, all is done. Documentation will be parsed so that we have information about input, output, and exceptions. This gives you type consents about inputs, outputs, and possible error. And also, the signature of the functions give you default value for inputs. So here we have type consents. That's what's in the red boxes. Type consents, they are more than just type names. They can be parameterized for iterable and mapping types. Like here we have a parameterized list that is parameterized by mytip b. Or it can also contain several type names, like the first parameter, it's either an int or a swing. And from that type consents, we can find all the inputs should be validated and translated from JSON data structure to Python object. For that, we convert that type of thread to a JSON schema. So for those people that don't really know what is JSON and JSON formats, JSON object can be seen as a subset of the built-in Python tabs. So often the conversion between the two is free. But for your services, you may need to do additional validation. And this is done by a JSON schema. So what is JSON schema? A JSON schema, it describes the structure of the expected JSON object, like its types or its minimum value, what it contains, like the name of the field, the one that are optional, so it really describes what you are expecting as an input. And there are also situations where you want real conversion to open. So you have, like, a JSON dictionary and you want it to be converted to your own Python class. So for these two cases, when you want to have additional validation and custom conversion, we introduce two classes that helps to attach the conversion and the JSON schema to your regular class. So from that, we are able to retrieve a JSON schema from any type of constraint. So all of the validation and the conversion happened. When the service is called and the input are received, there is a first validation, for most basic validation, we rely on the type constraint in the documentation. And for more complex one, we rely on this JSON schema. So there is a library called JSON schema that you can use to validate any data structure with your JSON schema. After that validation, there is the conversion step that is either implicit for most basic type or uncrafted using the JSON classes. And then the real function is called. And from that, there can be two outcomes. You can have an exception. And so what we do is that we check that the exceptions is respecting the contract. If it's not, the content or the exception is hidden because we don't want the user of the service to see anything that is not accepted by the contract. And at last, if we get a result from that function called, we validate it based on its type. So what else can we do with JSON schema? After all, it's a sonda. So you can use JSON editor. It's able to use a JSON schema to generate a form, which is also an editor. And it's able to validate the form and give you back the validate JSON value. So you just have to plug in a call to JSON editor in your HTML form template. And you have a cool feature for almost nothing. So here you have a view of what you had before. You had to type the JSON content in the play with me text area. And now using JSON editor, you just have a form. So it's really easy to create an input. So I thought that there are similar features in Django REST framework, but you should also, in this case, declare things manually, like the serializer for the form, just like you have to write a JSON for a schema here. So what else can we do with JSON schema? I don't know if you know a package named hypothesis. It's a package that helps you to do property-based testing. So it's composed of strategies that are able to generate random inputs for many usual types. And it's a really fantastic tool. I encourage you to use it. So let's add the generation strategy for JSON schema. With that, we can generate tests to verify that all the operations of our service respect the contract described by the documentation. So here I've put two examples. So I've taken a valid code that respect the contract of the API, and I've modified it. So in the first case, I changed the code just before. It was rising an exception. I added a type row. I had an internet string. And from the automatic test, we have this message where first, the type row is discovered, and is shown to the developer with a proper stack trace. And second, the programmer is also warned that one of the exceptions that is described in the contract is never occurred with this operation. So it shows that what is described in the documentation could be audited, for instance. And then I've put another example where I modified the valid code and make it return a bad value, like a list of strings for a function that is expected to return something else. And so what we see is that it's also detected automatically. So we have something that is able to take your whole API and automatically check that what you have described in the documentation is on par with the current implementation, which is a very cool feature. So what can I do next with JSON schema? I don't know. We stopped there. So we already have an easy way to enter JSON with forms. Can we have something useful to visualize the JSON response? So taking inspiration from Django RAS framework that already does the coloring of the JSON response, we decided to go further and to add something that we called pretty view. So a pretty view, it helps you to define custom visualization of the JSON response. So here, for instance, so you have the overview above and the pretty view in the raw part of the screen. And you can see that the JSON contains a timestamp. And in the pretty view, the timestamp is converted directly to a human-readable date. And there's also another example where you have an integer value in the JSON and it has a particular semantic. And the pretty view is able to show that semantic. So here it's the value zero, which is an OK flag. So it really makes the JSON response easier to read for humans. And another thing that we added to the pretty view is the ability to add extra information. Like here, the pretty view says something about active downtime that are not in the initial JSON response. And moreover, it adds a link near that extra information. And that link, which we called an input link, it's a link that is able to go to another operation and pre-fill all the information in the form. So you can navigate between different features in the API. And you don't have to tip anything. So here when you click on the red link, you will go to the operation page, to some other operation page, and every field will be already pre-filled. So what's next? We plan to open source all the things that we presented there. So you can watch the Kriteo GitHub repository. We will add the different contributions there. We will also contribute to hypothesis with the JSON schema strategy. One thing that we didn't do and could be very interesting, it's to support NM natively, because it's a really interesting type of place. And to integrate more with PyTest, because the thing that I showed you that gives you a diagnostic for your API, like with the warning and the error about mismatch between the contract and the implementation, currently you have to run it by hand and it will be better if we integrated the standard testing package. Then there is the SwaggerSpec generation. So we already have a lot of cool features, so why would we want that? Because it's Swagger has the ability to generate a client code library. That means you code your service in Python and then you want people to have automatic access in Java. You just write the Swagger file and it will generate the client for you in Java. So that's really a good thing. So if you have to only keep a simple message from this presentation, they are really interesting stuff because they are very user friendly and you can discover the content and explore and test and it's very good. The documentation and the types, they can be really used to add a lot of features to your services. So here I give some examples with the feature I present, but there are a lot more things to do. And also the configuration, JSON and JSON schema, they are very handy, they are standard, there are a lot of tools around them. So don't hesitate to use them. I think it will help you to easily have new features in your code. And that's all. If you have questions. Questions? Thank you for coming.
 Welcome.  We have Rudy here, showing us how to leverage documentation  power for better APIs.  Give him some clap.  Hello.  Hello.  Hello, I'm Rudy.  I'm working at Criteo, a company that delivers personalized ads at scale.  My team is building monitoring tools for our large software and hardware inventory.  And sometimes, some time ago in this team, we decided to make monitoring as a service  to experiment different ideas around web service and browser APIs.  So this presentation gives a glimpse of what we have done.  And this is a joint work with my colleague, Hugo Robour, which is there.  So what is a browser API?  Obviously, it's composed of two words, the browser word.  So probably it's related to a browser.  And more importantly, it's related to exploration and experimentation.  And I will add to that more later.  And there is a word API.  We'll go to formal talk.  So an API is a kind of contract.  It can be very explicit, like if you're declaring swagger specification five, or it can be  implicit, like in the documentation.  The contract defines the valid use of the service that is linked to the API.  Although, the operation you can do with this service, and also what are the valid inputs  for operation in the service.  So why is it interesting?  Because it brings a lot of features for your service.  So the first feature I will speak about is why we rely on the documentation to describe  the API.  Then I will speak about the basic feature that you have in the browser API.  So the first basic feature is the explorability.  You can go to the API and ask what the operation is possible on this API.  So here you have a view of an API with different operations.  And then you can just click on an operation.  And you see the documentation of that operation, the parameter types, and what are the possible  results.  And then, more than that, you can play with an operation.  So you have to tip the input of the operation and call it.  And so you can see what is the response and explore the behavior of this operation.  But still, typing the input, like a JSON input, can be difficult.  So we probably can do something better.  Another really important feature of an API is to enforce the validation of the input  and the outputs.  By the way, this talk focuses on JSON inputs.  You don't want an operation to be used in an adapted way.  That's why you should validate all the inputs of your service.  If a list is transmitted instead of a string, nobody knows how the code will behave in that  case.  And even for valid JSON inputs, you may want to have a conversion between the JSON input  and between your Python types.  Okay, this one is an obvious feature.  Like I said before, it can be really painful to enter JSON data because it's a verbose  format.  So having forms make you save times.  And it brings client-side input validation on the table.  Another really interesting feature is that you have that kind of API contract and you  have your service implementation.  And you want to be sure that they perfectly match.  So that would be really cool to just ask for automatic tests for checking that they match.  And I will also present other features that makes a browser API more user-friendly.  And last, just to mention it, you may want to generate, even if you use the doc, more  formal specification contract like in Swagger because Swagger has a lot of amazing features  that you can later use.  So, and this is the plan of the talk, in fact.  So I'm starting now.  So imagine you want to add a new function to your API.  What do you do?  Okay, it's easy.  You document the functions and you add the published decorator on it.  So this decorator, you will resist the function as an operation in the API.  The function will be left unmodified.  So you can still use it as before when the decorator was not there.  And behind the scenes, the function will be wrapped in the API where all the data about  HTTP will be handled.  So, all is done.  Documentation will be parsed so that we have information about input, output, and exceptions.  This gives you type consents about inputs, outputs, and possible error.  And also, the signature of the functions give you default value for inputs.  So here we have type consents.  That's what's in the red boxes.  Type consents, they are more than just type names.  They can be parameterized for iterable and mapping types.  Like here we have a parameterized list that is parameterized by mytip b.  Or it can also contain several type names, like the first parameter, it's either an int or a swing.  And from that type consents, we can find all the inputs should be validated and translated  from JSON data structure to Python object.  For that, we convert that type of thread to a JSON schema.  So for those people that don't really know what is JSON and JSON formats,  JSON object can be seen as a subset of the built-in Python tabs.  So often the conversion between the two is free.  But for your services, you may need to do additional validation.  And this is done by a JSON schema.  So what is JSON schema?  A JSON schema, it describes the structure of the expected JSON object,  like its types or its minimum value, what it contains, like the name of the field,  the one that are optional, so it really describes what you are expecting as an input.  And there are also situations where you want real conversion to open.  So you have, like, a JSON dictionary and you want it to be converted to your own Python class.  So for these two cases, when you want to have additional validation and custom conversion,  we introduce two classes that helps to attach the conversion and the JSON schema to your regular class.  So from that, we are able to retrieve a JSON schema from any type of constraint.  So all of the validation and the conversion happened.  When the service is called and the input are received,  there is a first validation, for most basic validation, we rely on the type constraint in the documentation.  And for more complex one, we rely on this JSON schema.  So there is a library called JSON schema that you can use to validate any data structure with your JSON schema.  After that validation, there is the conversion step that is either implicit for most basic type or uncrafted using the JSON classes.  And then the real function is called.  And from that, there can be two outcomes.  You can have an exception.  And so what we do is that we check that the exceptions is respecting the contract.  If it's not, the content or the exception is hidden because we don't want the user of the service to see anything that is not accepted by the contract.  And at last, if we get a result from that function called, we validate it based on its type.  So what else can we do with JSON schema?  After all, it's a sonda.  So you can use JSON editor.  It's able to use a JSON schema to generate a form, which is also an editor.  And it's able to validate the form and give you back the validate JSON value.  So you just have to plug in a call to JSON editor in your HTML form template.  And you have a cool feature for almost nothing.  So here you have a view of what you had before.  You had to type the JSON content in the play with me text area.  And now using JSON editor, you just have a form.  So it's really easy to create an input.  So I thought that there are similar features in Django REST framework,  but you should also, in this case, declare things manually, like the serializer for the form,  just like you have to write a JSON for a schema here.  So what else can we do with JSON schema?  I don't know if you know a package named hypothesis.  It's a package that helps you to do property-based testing.  So it's composed of strategies that are able to generate random inputs for many usual types.  And it's a really fantastic tool. I encourage you to use it.  So let's add the generation strategy for JSON schema.  With that, we can generate tests to verify that all the operations of our service  respect the contract described by the documentation.  So here I've put two examples.  So I've taken a valid code that respect the contract of the API, and I've modified it.  So in the first case, I changed the code just before.  It was rising an exception. I added a type row.  I had an internet string.  And from the automatic test, we have this message where first, the type row is discovered,  and is shown to the developer with a proper stack trace.  And second, the programmer is also warned that one of the exceptions that is described in the contract  is never occurred with this operation.  So it shows that what is described in the documentation could be audited, for instance.  And then I've put another example where I modified the valid code and make it return a bad value,  like a list of strings for a function that is expected to return something else.  And so what we see is that it's also detected automatically.  So we have something that is able to take your whole API and automatically check that what you have described  in the documentation is on par with the current implementation,  which is a very cool feature.  So what can I do next with JSON schema?  I don't know. We stopped there.  So we already have an easy way to enter JSON with forms.  Can we have something useful to visualize the JSON response?  So taking inspiration from Django RAS framework that already does the coloring of the JSON response,  we decided to go further and to add something that we called pretty view.  So a pretty view, it helps you to define custom visualization of the JSON response.  So here, for instance, so you have the overview above and the pretty view in the raw part of the screen.  And you can see that the JSON contains a timestamp.  And in the pretty view, the timestamp is converted directly to a human-readable date.  And there's also another example where you have an integer value in the JSON and it has a particular semantic.  And the pretty view is able to show that semantic.  So here it's the value zero, which is an OK flag.  So it really makes the JSON response easier to read for humans.  And another thing that we added to the pretty view is the ability to add extra information.  Like here, the pretty view says something about active downtime that are not in the initial JSON response.  And moreover, it adds a link near that extra information.  And that link, which we called an input link, it's a link that is able to go to another operation  and pre-fill all the information in the form.  So you can navigate between different features in the API.  And you don't have to tip anything.  So here when you click on the red link, you will go to the operation page, to some other operation page,  and every field will be already pre-filled.  So what's next?  We plan to open source all the things that we presented there.  So you can watch the Kriteo GitHub repository.  We will add the different contributions there.  We will also contribute to hypothesis with the JSON schema strategy.  One thing that we didn't do and could be very interesting, it's to support NM natively,  because it's a really interesting type of place.  And to integrate more with PyTest, because the thing that I showed you that gives you a diagnostic for your API,  like with the warning and the error about mismatch between the contract and the implementation,  currently you have to run it by hand and it will be better if we integrated the standard testing package.  Then there is the SwaggerSpec generation.  So we already have a lot of cool features, so why would we want that?  Because it's Swagger has the ability to generate a client code library.  That means you code your service in Python and then you want people to have automatic access in Java.  You just write the Swagger file and it will generate the client for you in Java.  So that's really a good thing.  So if you have to only keep a simple message from this presentation,  they are really interesting stuff because they are very user friendly and you can discover the content and explore and test and it's very good.  The documentation and the types, they can be really used to add a lot of features to your services.  So here I give some examples with the feature I present, but there are a lot more things to do.  And also the configuration, JSON and JSON schema, they are very handy, they are standard, there are a lot of tools around them.  So don't hesitate to use them. I think it will help you to easily have new features in your code.  And that's all. If you have questions.  Questions?  Thank you for coming.
Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_0_83.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_84_167.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_168_323.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_324_551.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_552_623.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_624_923.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_924_1823.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_1824_2183.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_2184_4043.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_4044_4151.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_4152_5795.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_5796_7847.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_7848_8003.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_8004_9251.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_9252_14339.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_14340_18263.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_18264_18323.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_18324_23363.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_23364_23627.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_23628_25427.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_25428_27191.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_27192_28475.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_28476_29123.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_29124_29327.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_29328_32303.png Rudy_Sicard_-_Leveraging_documentation_power_for_better_web_APIs_32304_34175.png
Welcome. We have Rudy here, showing us how to leverage documentation power for better APIs. Give him some clap. Hello. Hello. Hello, I'm Rudy. I'm working at Criteo, a company that delivers personalized ads at scale. My team is building monitoring tools for our large software and hardware inventory. And sometimes, some time ago in this team, we decided to make monitoring as a service to experiment different ideas around web service and browser APIs. So this presentation gives a glimpse of what we have done. And this is a joint work with my colleague, Hugo Robour, which is there. So what is a browser API? Obviously, it's composed of two words, the browser word. So probably it's related to a browser. And more importantly, it's related to exploration and experimentation. And I will add to that more later. And there is a word API. We'll go to formal talk. So an API is a kind of contract. It can be very explicit, like if you're declaring swagger specification five, or it can be implicit, like in the documentation. The contract defines the valid use of the service that is linked to the API. Although, the operation you can do with this service, and also what are the valid inputs for operation in the service. So why is it interesting? Because it brings a lot of features for your service. So the first feature I will speak about is why we rely on the documentation to describe the API. Then I will speak about the basic feature that you have in the browser API. So the first basic feature is the explorability. You can go to the API and ask what the operation is possible on this API. So here you have a view of an API with different operations. And then you can just click on an operation. And you see the documentation of that operation, the parameter types, and what are the possible results. And then, more than that, you can play with an operation. So you have to tip the input of the operation and call it. And so you can see what is the response and explore the behavior of this operation. But still, typing the input, like a JSON input, can be difficult. So we probably can do something better. Another really important feature of an API is to enforce the validation of the input and the outputs. By the way, this talk focuses on JSON inputs. You don't want an operation to be used in an adapted way. That's why you should validate all the inputs of your service. If a list is transmitted instead of a string, nobody knows how the code will behave in that case. And even for valid JSON inputs, you may want to have a conversion between the JSON input and between your Python types. Okay, this one is an obvious feature. Like I said before, it can be really painful to enter JSON data because it's a verbose format. So having forms make you save times. And it brings client-side input validation on the table. Another really interesting feature is that you have that kind of API contract and you have your service implementation. And you want to be sure that they perfectly match. So that would be really cool to just ask for automatic tests for checking that they match. And I will also present other features that makes a browser API more user-friendly. And last, just to mention it, you may want to generate, even if you use the doc, more formal specification contract like in Swagger because Swagger has a lot of amazing features that you can later use. So, and this is the plan of the talk, in fact. So I'm starting now. So imagine you want to add a new function to your API. What do you do? Okay, it's easy. You document the functions and you add the published decorator on it. So this decorator, you will resist the function as an operation in the API. The function will be left unmodified. So you can still use it as before when the decorator was not there. And behind the scenes, the function will be wrapped in the API where all the data about HTTP will be handled. So, all is done. Documentation will be parsed so that we have information about input, output, and exceptions. This gives you type consents about inputs, outputs, and possible error. And also, the signature of the functions give you default value for inputs. So here we have type consents. That's what's in the red boxes. Type consents, they are more than just type names. They can be parameterized for iterable and mapping types. Like here we have a parameterized list that is parameterized by mytip b. Or it can also contain several type names, like the first parameter, it's either an int or a swing. And from that type consents, we can find all the inputs should be validated and translated from JSON data structure to Python object. For that, we convert that type of thread to a JSON schema. So for those people that don't really know what is JSON and JSON formats, JSON object can be seen as a subset of the built-in Python tabs. So often the conversion between the two is free. But for your services, you may need to do additional validation. And this is done by a JSON schema. So what is JSON schema? A JSON schema, it describes the structure of the expected JSON object, like its types or its minimum value, what it contains, like the name of the field, the one that are optional, so it really describes what you are expecting as an input. And there are also situations where you want real conversion to open. So you have, like, a JSON dictionary and you want it to be converted to your own Python class. So for these two cases, when you want to have additional validation and custom conversion, we introduce two classes that helps to attach the conversion and the JSON schema to your regular class. So from that, we are able to retrieve a JSON schema from any type of constraint. So all of the validation and the conversion happened. When the service is called and the input are received, there is a first validation, for most basic validation, we rely on the type constraint in the documentation. And for more complex one, we rely on this JSON schema. So there is a library called JSON schema that you can use to validate any data structure with your JSON schema. After that validation, there is the conversion step that is either implicit for most basic type or uncrafted using the JSON classes. And then the real function is called. And from that, there can be two outcomes. You can have an exception. And so what we do is that we check that the exceptions is respecting the contract. If it's not, the content or the exception is hidden because we don't want the user of the service to see anything that is not accepted by the contract. And at last, if we get a result from that function called, we validate it based on its type. So what else can we do with JSON schema? After all, it's a sonda. So you can use JSON editor. It's able to use a JSON schema to generate a form, which is also an editor. And it's able to validate the form and give you back the validate JSON value. So you just have to plug in a call to JSON editor in your HTML form template. And you have a cool feature for almost nothing. So here you have a view of what you had before. You had to type the JSON content in the play with me text area. And now using JSON editor, you just have a form. So it's really easy to create an input. So I thought that there are similar features in Django REST framework, but you should also, in this case, declare things manually, like the serializer for the form, just like you have to write a JSON for a schema here. So what else can we do with JSON schema? I don't know if you know a package named hypothesis. It's a package that helps you to do property-based testing. So it's composed of strategies that are able to generate random inputs for many usual types. And it's a really fantastic tool. I encourage you to use it. So let's add the generation strategy for JSON schema. With that, we can generate tests to verify that all the operations of our service respect the contract described by the documentation. So here I've put two examples. So I've taken a valid code that respect the contract of the API, and I've modified it. So in the first case, I changed the code just before. It was rising an exception. I added a type row. I had an internet string. And from the automatic test, we have this message where first, the type row is discovered, and is shown to the developer with a proper stack trace. And second, the programmer is also warned that one of the exceptions that is described in the contract is never occurred with this operation. So it shows that what is described in the documentation could be audited, for instance. And then I've put another example where I modified the valid code and make it return a bad value, like a list of strings for a function that is expected to return something else. And so what we see is that it's also detected automatically. So we have something that is able to take your whole API and automatically check that what you have described in the documentation is on par with the current implementation, which is a very cool feature. So what can I do next with JSON schema? I don't know. We stopped there. So we already have an easy way to enter JSON with forms. Can we have something useful to visualize the JSON response? So taking inspiration from Django RAS framework that already does the coloring of the JSON response, we decided to go further and to add something that we called pretty view. So a pretty view, it helps you to define custom visualization of the JSON response. So here, for instance, so you have the overview above and the pretty view in the raw part of the screen. And you can see that the JSON contains a timestamp. And in the pretty view, the timestamp is converted directly to a human-readable date. And there's also another example where you have an integer value in the JSON and it has a particular semantic. And the pretty view is able to show that semantic. So here it's the value zero, which is an OK flag. So it really makes the JSON response easier to read for humans. And another thing that we added to the pretty view is the ability to add extra information. Like here, the pretty view says something about active downtime that are not in the initial JSON response. And moreover, it adds a link near that extra information. And that link, which we called an input link, it's a link that is able to go to another operation and pre-fill all the information in the form. So you can navigate between different features in the API. And you don't have to tip anything. So here when you click on the red link, you will go to the operation page, to some other operation page, and every field will be already pre-filled. So what's next? We plan to open source all the things that we presented there. So you can watch the Kriteo GitHub repository. We will add the different contributions there. We will also contribute to hypothesis with the JSON schema strategy. One thing that we didn't do and could be very interesting, it's to support NM natively, because it's a really interesting type of place. And to integrate more with PyTest, because the thing that I showed you that gives you a diagnostic for your API, like with the warning and the error about mismatch between the contract and the implementation, currently you have to run it by hand and it will be better if we integrated the standard testing package. Then there is the SwaggerSpec generation. So we already have a lot of cool features, so why would we want that? Because it's Swagger has the ability to generate a client code library. That means you code your service in Python and then you want people to have automatic access in Java. You just write the Swagger file and it will generate the client for you in Java. So that's really a good thing. So if you have to only keep a simple message from this presentation, they are really interesting stuff because they are very user friendly and you can discover the content and explore and test and it's very good. The documentation and the types, they can be really used to add a lot of features to your services. So here I give some examples with the feature I present, but there are a lot more things to do. And also the configuration, JSON and JSON schema, they are very handy, they are standard, there are a lot of tools around them. So don't hesitate to use them. I think it will help you to easily have new features in your code. And that's all. If you have questions. Questions? Thank you for coming.
Welcome. We have Rudy here, showing us how to leverage documentation power for better APIs. and hardware inventory. And sometimes, some time ago in this team, we decided to make monitoring as a service to experiment different ideas around web service and browser APIs. So this We'll go to formal talk. So an API is a kind of declaring swagger specification five, or it can operation you can do with this service, and also can play with an operation. So you have to tip a string, nobody knows how the code will behave information about input, output, and exceptions. outputs, and possible error. And also, the you may need to do additional validation. And to your own Python class. So for these two cases, and the input are received, there is a first exception. And so what we do is that we check the contract. And at last, if we get a result encourage you to use it. So let's add the from the automatic test, we have this message where first, the type row is discovered, and is one of the exceptions that is described in the API and automatically check that what you have described in the documentation is on par with the have something useful to visualize the JSON response easier to read for humans. And another that are not in the initial JSON response. And operation page, to some other operation page, and standard testing package. Then there is the you have to only keep a simple message from this discover the content and explore and test and it's very good. The documentation and the types, are standard, there are a lot of tools around will help you to easily have new features in your