State of Plone back-end development today
Derstappen, Maik
A brief introduction into the state of the art Plone back-end development, using modern tools as "plonecli", "bobtemplates.plone" and "Visual Studio Code - Plone Snippets". Building Content Types, Vocabularries, Behaviors and Rest-API Services with plonecli and using VS Code snippets fill-in field definitions and configuration.
 Okay, I want to talk today a bit about the back end development. Now, how it is today. Even though everybody talks all the time about headless and front end. But yeah, you still need a back end. So how are we going to do that? A small overview. We have things like the clone CLI. Some of you might have heard about it. I'm pretty sure a lot of people are using the clone API, which is the Python API, which makes things simpler. We also have snippets for several editors. Also for VS code. Where you can add some code with the editor. And also, of course, you have to clone REST API where you can then express these contents to use it on the client side at the end. So the back end part there will be to customize the REST API to deliver the content as they are needed for the front end. Yeah, of course, if you didn't, you should use the clone CLI. I think it's useful also for not newcomers. I like to use it. Of course, I build it. But it really saves a lot of time for boring stuff. You can focus on your code. Yeah, it helps you to enhance your products, your add-ons step by step. So if you need a functionality, you can add it if clone CLI or BOP templates is providing it. This is just a small demo. So we have a lot of templates already. You can always check the version number with the minus capital V so that you see which version of clone CLI and BOP templates you are using. And then you can create your package. Since we need support for Python 3, Python 2, you can set which version you want to use for clone CLI. It's mainly used for creating local virtual environments for testing. And you can always change that and just run clone CLI build and it will recreate the virtual environment, install the requirements, run build out, and then your setup is on Python 2 or Python 3. You can see the build command does several steps. The last step is to build out and you don't have to type this manually. But you still can, of course. Yeah, there's a small configuration file. This is also the first step when you want to migrate an older package to be compatible with clone CLI. You just create this file. This is the way how clone CLI finds the package root. But you have also some settings there. And this you can use to, or this clone CLI and the add-ons and the sub templates will use to make decisions later. One example, a content type. That was actually the first template I was working on when I changed to this more modular approach because it was not possible before to create multiple content types. We just had one big template which were having one example content type. So you can create a content type. You give it a name. You have the option of container an item. If it's global, addable or not. And then you have it. If you want to create another content type, for example, this time we choose item as a base class. This time it's not globally addable. So when we continue, then it will ask for parent content type, which in this case is the task list. And this clone CLI will use to generate the tests that they are working so that it makes sure that the parent content type exists and then they try to add the content type there. And also the wiring in the FDI settings so that you don't have to go there and say in this container I'm allowed the task to be added. You can see the structure. It creates a Python file for every content type and XML file. Depending on the question of if you want to use the XML file or not, we could clean that up but it doesn't hurt anybody. You can delete it if you don't want to go that way. Or if you say you want to use the supermodel, then you can basically write it in XML or do it through the web and export the XML after you created your schema. We also have template for rest API service. For those of you who don't know what the service is, that's the components you have, which for example provides the breadcrumbs, the navigation, so additional data for your current context. When you're on a page or your custom content type and you want additional data, maybe based on tagging or whatever, so you make a catalog query for that. You can create a service for that and make this information available. With the expand feature of the rest API, you can tell the rest API, please also give me the data with the run request I did for the actual context. You just have to customize this area and it will return, Jason, the rest is more or less boilerplate. There are two things you decide. One is the class name of the service and one is the actual endpoint name, so the service name and these things are used to add this generation. There's another template coming for Zeralizers. Are you using to change the way how the rest API returns your values? For example, for vocabulary I did that once because earlier versions of the rest API were just giving you the token and you want the title and the token and not asked again for this from the client, so you can change the ways how things are returned to you. There's another one which is fairly new, so there might be some bugs, I just found one, but this will help you to easily generate upgrade steps. What we're doing here is basically generate a profile for every upgrade step. This is similar to what blown up upgrades in the core package. We will also provide the Python file and we will also auto increment the version number, so you don't forget to update or increment the version number and the metadata XML. If you don't need something, you can always ignore it. If you don't have Python code to write for your upgrade step because the generic setup import is enough, then you're fine. If not, then you can use the Python file and do, for example, re-indexing after you imported the profile. Basically, you get a file like this for every upgrade step automatically. This includes a folder where you can put profiles like registry XML or catalog XML like I do here in the example. Like I will do now. This way you can easily create upgrade steps and there's no excuse to not do it. If you create more upgrade steps, you can see now when we're doing this, it will generate similar structures and we'll go and update the version number. Whatever version number you have in your existing package, we'll just read it from the metadata and then increment it and that's it. We have a Git integration, so every command you make will automatically commit it to your repository and when you did manually changes like I did with the catalog XML, then it will tell me maybe you want to commit it before you use templates because the templates from Blown CLI or from Mr. Bob, they're not asking you before they overwrite anything. If you have conflicting configurations, you might lose your changes. Always use the Git commit before and then you're fine. It helps you to revert the whole process. You can just go one commit back and nothing happened. One other reason is if you use Blown CLI, all the packages are created by this will have the same structure. That makes it easy to find things. The ideas here are everything has its own folder. If you create behaviors, they are in behaviors folder. If you create content, like content types, they are content indexer folder and the upgrades are in upgrades and so forth. There might be some changes like I choose view list and views to put also in separate folders on the top level. This is a little bit different to the traditional way in having a browser folder. Otherwise, where would you put view list and views? You could put them in browser folder, like a sub folder, but where's the meaning of that? If you think about newcomers, they don't know the browser folder. They have no idea what this is for. There's no real reason to really name it that way. There was at some point in the history, but it's not really that important. That's why the decision was made there. It's also more robust. For one thing, you have one place and you generate usually one new file. For a new view, you generate a new file. You don't update Python files with the generator because when you start this, that can get messy. It shouldn't usually break anything of your code. You would get the test coverage right from the start. Of course, not all the templates have the same amount of tests. Sometimes it's just not easy to generate them as a generic test. Sometimes I just wait for you to add some. There's a bit room for improvement, but the whole structure is there. You can go and also bring your opinions in and say, we should use this way. Recently, we decided to rewrite some tests so that we use more the clone API instead of using getMultiAdapter to get the view, to test the view, things like that. This makes it easier for newcomers to understand what's happened because the clone API is more meaningful in how you use it. By default, you have in the packages you generate the talk setup. I said earlier that we have the option to change the Python version. It's easy. It takes a bit of time. If you created a database and you played with it, you have to delete the database because it will not run with Python 2 and 4 in the same way. What talks does is basically it allows you to have all versions you care about and all combinations of Python and clone be tested without changing anything. You can pick a single environment and say, okay, I just want to test now the Python 3.7 with clone 5.2. If you don't care about your add-on for clone 4, then just go in the talk scene and remove the entry for that. You can run all these things in parallel too. If you run talks in the parallel mode, it will use all your CPUs and burn it down. It goes as fast as your CPU power is. At the end, you have a summary of that. You see that. Here's an example of the upper part of the talk scene. This is what you want to customize. It's basically saying, let's say the second line, for example, it says this is only for Python 2.7. Then there will be generated two environments. One is clone 4.3 and one is 5.1. From the third line, we will have a version with Python 2.7 with clone 5.2 and a version with 3.7 and clone 2. If you go and say, list the environments, then you see these different versions. You can rewrite that so that you have just less and you don't have to test clone 4 if your add-on will never run on clone 4. That should be okay. There's another command. If you use talks-a, it will show you a little bit more environments. These are environments that are defined more or less as a helper. You can use this to update translations and things like that. Or to apply the ISOD. I personally use this in VS Code, so I don't need this, but some people work that way, so it's okay. You can configure the clone CLI or the Mr. Bob for both for your own taste. If you don't want the git questions, then you just say, okay, just do it. It's a good idea. Always commit and maybe use the clone version as default. You have basically two areas. The first is the variables. Things you put there. You will never see the question. If you put an answer there, it will be always answered. Not ask anymore. If you put it in default, you see the question, you see your default. If you press enter, it will just save, but you can change it. The variable names you can find in BobTemplatesplawn where the templates are defined. There in every template folder you have a BobInnify. This BobInnify is just a normal Windows style. You have the variables defined there. This name you copy and this way you can do that. We have a little helper in clone CLI, but we need to update to allow more settings. It will write this configuration, at least for the basic stuff that works already. The author name and things like that, it will set. We could extend this to make more decisions so that you don't have to look for the variables and things like that. That would be a nice thing to spread on. The whole thing is extendable. You can write your own BobTemplates packages and register them for clone CLI. This way, clone CLI will just list them and can use them. It's not hard to do that. This is useful if you have ideas for packages which are maybe customer related or your company way of doing things is different and you want to have some of the templates in your own flavor. Or if you do things which are not necessary for the main development but you have to think for migrations. We can create multiple packages and make them all work with clone CLI. It's a little example of how that works. It's basically using Python entry points. In your setup py, you have something like this and you point to your package or in this case it's BobTemplates clone. There is a Python module, BobRegistry and you just have the key there. This is just a little helper object and an entry in that file looks like this. This is the main clone add-on in this case. You have a clone CLI alias. We don't want to type all the time clone add-on because we're using the clone CLI so it's clear we want something related to clone not to permit. We can skip that. In the context of Mr. Bob, you could use the global name. You can always say this template depends on clone add-on because all the sub templates expect a certain structure and some files to be there. Usually you create your add-on with the add-on template and after that you can use all the sub commands. It's not that hard to upgrade your existing. If you at some point created your content types, I just did that with the easy newsletter for example. We have a chapter to upgrade existing codes in the documentation of BobTemplates clone. If you have a question there just ping me or one of the others and we will fulfill this. I mentioned a small plugin for VS Code. You can just install it like every other VS Code add-on with the extensions. Then you just write clone and after that you write something like text line field and then you can jump around, fill it out and make the settings for requirement. We have basically all common fields in Python and also for the supermodel. You can use this in the XML or in the Python file. There are also some other settings for the registry XML. For example, some of these big chunks of registry thingies. There are also good things to put in the snippets. There is still one bug here because it should replace the name when you start writing. It doesn't do that when you have multiple times. I already opened the issue there. I mentioned clone API. It has a really good documentation. It really makes sense. If you have people to onboard and start with, we really should use this as much as we can. Just because you can read it and you just understand what's happening. This helps a lot. The other reason is when there will be a change in the future under the hood, it's not your problem. You just use the clone API and the clone API will use the new way of doing things. Your code is probably also more robust than using some internal methods from CMM clone or from whatever. Some ideas for the future. There are some things like views. For example, you can register a view to just appear on certain contexts or under certain interfaces. This is by default already there. You have to go in the.cml file and change the interface. It would be nice to have some common options like all the standard content types could be listed there. You can say, okay, I want this on pages by default. Your own content types, which you created earlier, could be listed there or something like that. For REST API, I work already on the Zerializer. Another thing would be the Dyserializer, which is the other way around when you send data to the REST API in a certain way. You want the REST API to understand your data better. Then you can create that. Maybe you have some special data from an application. You cannot really change or it's just not that common so that you want to do some conversions there. Yes, Graphical UI, but just a minimal. For selecting content types or interfaces, also give me the parent content type. We could just select all the content types from standard blown and from our package. This would be already helpful. You can just select it instead of remember and retype it. To do that, we need to change the user interface. That means we have to change Mr. Bob itself. I have ideas for that. There are some libraries which work cross-platform and allow you just in that moment to have a drop-down or things like that. But for that, we have to go upstream before we can use this and Bob template's blown and blown CLI. For VS Code, I wish we would have a better blown API auto completion at least. It's not working that great when you import blown API like normally people would do from blown import API and then use it like API content find. VS Code is not that smart with the namespaces there. There are some problems. Also, VS Code by default, of course, doesn't know the build-out structure and things like that. There's a bit of a recipe though. You can create some additional configuration for VS Code. VS Code will read that and then should at least read all your Python paths without doing anything else. Another way to do this is using the ZOPI Python interpreter you have in the build-out usually. Then VS Code also has all the paths in there. But even if you have that, it's not perfect with auto completion. I would like to have more contributions. There is nothing set in stone as Philip said before. Don't be shy. There's no grand jury who makes decisions and because it's there and it looks like this, it has to be like this. That's not the case. If you have ideas or complaints, open an issue, let's discuss this. If you find a way to test things better, just add a test or write a bit of documentation would be really helpful. If you have other ideas and you're not sure if that makes sense to put this in Bob-Temps-Lates-plon, then create other Bob-Temps-Lates-Packages. Your own would be helpful for you internally but also to make the whole system a bit more robust. There were already some contributions. I guess there are three people from time to time doing something on it. It's nice to see that in use. I would be at the sprints at least Saturday and maybe Sunday morning. If you want to dive into that, it's a good task for a short sprint because there's not that much to learn. You can achieve some things. If you want to work on your custom template and do some stuff, I can just help you if you have questions. Any questions so far? Is there a Bob-Temps-Lates-Package that can create a Bob-Temps-Lates? I don't know if it makes sense. It should be possible because what you usually do is you create one Python file and one folder and a bit of structure. We could do that. You have to write a bit of Python for sure because what we are doing with configuration files, usually we pass them, the XML files or XML files, we just pass and to keep the format, we don't write them out as XML because the formatting would be different then. So, the XML files usually have a line like earlier, like your custom stuff comes here, so that's a marker. It should stay there. But we pass XML, so we find out if we need to add something and if we have to, we insert it after the marker. So, that's the way and this is like some Python functions that get called from Mr. Bob. And usually depending on what template you build, you have to adjust the parts, the variables you create or define in the pre-rendera section. So, there's method pre-rendera, it's called and there's some stuff that's always the same but things like, I don't know, you have the question, upgrade step name or upgrade step title and you want to use this somehow as a variable then you do your initialization there and later in the templates or in the file names or folder names, you can use that. So, that's the work you have always to do but the structure might be useful. Any other question? So, can I customize the command line interface or sub commands? In what way? Like register, maybe additional commands or overwrite them commands, their behavior? I mean, clone CLI itself is just a Python package so if you have ideas which are generally useful then we can add them. I mean, if you create your own Bob templates and you register them with the entry points, they will pop up automatically in clone CLI so this add command has an auto completion feature which shows you the list of all templates and you can run them. So, these sub commands for the templates, this is already there, this is customizable, the rest is it's a Python package, everything what's possible there. Any other one? Then I have one question, who of you is beginner, like in the beginning phase? Who is onboarding people from time to time? So, if you are using already clone CLI for that or most of them not, so give it a try. I think it's easier to understand and to, you don't have to know all the things, you just create what you want to do. Somebody told you you have to create a view for that and good luck by trying to find the documentation that's sometimes really hurtful even though you know what to search for. But it's not easy and with a tool like this, you can just start and then look okay, it works and then adjust it. It's way better for beginners. When I started way back then, we had something like this with ZOB scale and archetypes support. So, you could do similar things and this was really helpful. So, give it a try and please also give feedback how that went and what we can do to improve it. Okay, thank you. Thank you.
 Okay, I want to talk today a bit about  the back end development. Now, how it is today. Even though everybody talks all the time about  headless and front end. But yeah, you still need a back end. So how are we going to do that?  A small overview. We have things like the clone CLI. Some of you might have heard about it. I'm pretty  sure a lot of people are using the clone API, which is the Python API, which makes things simpler.  We also have snippets for several editors. Also for VS code. Where you can add some code with the editor.  And also, of course, you have to clone REST API where you can then express these contents to use it on the client  side at the end. So the back end part there will be to customize the REST API to deliver the content as  they are needed for the front end. Yeah, of course, if you didn't, you should use the clone CLI. I think it's  useful also for not newcomers. I like to use it. Of course, I build it. But it really saves a lot of time  for boring stuff. You can focus on your code. Yeah, it helps you to enhance your products, your add-ons  step by step. So if you need a functionality, you can add it if clone CLI or BOP templates is providing it.  This is just a small demo. So we have a lot of templates already. You can always check the version number with the  minus capital V so that you see which version of clone CLI and BOP templates you are using. And then you can  create your package. Since we need support for Python 3, Python 2, you can set which version you want to use for  clone CLI. It's mainly used for creating local virtual environments for testing. And you can always  change that and just run clone CLI build and it will recreate the virtual environment, install the  requirements, run build out, and then your setup is on Python 2 or Python 3. You can see the build command  does several steps. The last step is to build out and you don't have to type this manually. But you still can, of course.  Yeah, there's a small configuration file. This is also the first step when you want to migrate an older package  to be compatible with clone CLI. You just create this file. This is the way how clone CLI finds the package root.  But you have also some settings there. And this you can use to, or this clone CLI and the add-ons and the  sub templates will use to make decisions later. One example, a content type. That was actually the first template  I was working on when I changed to this more modular approach because it was not possible before to create  multiple content types. We just had one big template which were having one example content type. So you can create  a content type. You give it a name. You have the option of container an item. If it's global, addable or not.  And then you have it. If you want to create another content type, for example, this time we choose item as a base class.  This time it's not globally addable. So when we continue, then it will ask for parent content type, which in this  case is the task list. And this clone CLI will use to generate the tests that they are working so that it makes sure  that the parent content type exists and then they try to add the content type there. And also the wiring in the  FDI settings so that you don't have to go there and say in this container I'm allowed the task to be added.  You can see the structure. It creates a Python file for every content type and XML file. Depending on the question  of if you want to use the XML file or not, we could clean that up but it doesn't hurt anybody. You can delete it  if you don't want to go that way. Or if you say you want to use the supermodel, then you can basically write it in XML or do it  through the web and export the XML after you created your schema.  We also have template for rest API service. For those of you who don't know what the service is, that's the  components you have, which for example provides the breadcrumbs, the navigation, so additional data for your  current context. When you're on a page or your custom content type and you want additional data, maybe based on  tagging or whatever, so you make a catalog query for that. You can create a service for that and make this information  available. With the expand feature of the rest API, you can tell the rest API, please also give me the data with the  run request I did for the actual context. You just have to customize this area and it will return, Jason, the rest is  more or less boilerplate. There are two things you decide. One is the class name of the service and one is the  actual endpoint name, so the service name and these things are used to add this generation. There's another template  coming for Zeralizers. Are you using to change the way how the rest API returns your values? For example, for  vocabulary I did that once because earlier versions of the rest API were just giving you the token and you want  the title and the token and not asked again for this from the client, so you can change the ways how things are  returned to you. There's another one which is fairly new, so there might be some bugs, I just found one, but this will  help you to easily generate upgrade steps. What we're doing here is basically generate a profile for every  upgrade step. This is similar to what blown up upgrades in the core package. We will also provide the Python file  and we will also auto increment the version number, so you don't forget to update or increment the version number  and the metadata XML. If you don't need something, you can always ignore it. If you don't have Python code to write  for your upgrade step because the generic setup import is enough, then you're fine. If not, then you can use the  Python file and do, for example, re-indexing after you imported the profile. Basically, you get a file like this for  every upgrade step automatically. This includes a folder where you can put profiles like registry XML or  catalog XML like I do here in the example. Like I will do now. This way you can easily create  upgrade steps and there's no excuse to not do it. If you create more upgrade steps, you can see now when we're doing this,  it will generate similar structures and we'll go and update the version number. Whatever version number you have in your existing  package, we'll just read it from the metadata and then increment it and that's it. We have a Git integration, so every  command you make will automatically commit it to your repository and when you did manually changes like I did with the  catalog XML, then it will tell me maybe you want to commit it before you use templates because the templates from  Blown CLI or from Mr. Bob, they're not asking you before they overwrite anything. If you have conflicting  configurations, you might lose your changes. Always use the Git commit before and then you're fine. It helps you  to revert the whole process. You can just go one commit back and nothing happened. One other reason is if you use  Blown CLI, all the packages are created by this will have the same structure. That makes it easy to find things.  The ideas here are everything has its own folder. If you create behaviors, they are in behaviors folder. If you  create content, like content types, they are content indexer folder and the upgrades are in upgrades and so forth.  There might be some changes like I choose view list and views to put also in separate folders on the top level.  This is a little bit different to the traditional way in having a browser folder. Otherwise, where would you put  view list and views? You could put them in browser folder, like a sub folder, but where's the meaning of that?  If you think about newcomers, they don't know the browser folder. They have no idea what this is for. There's no real reason  to really name it that way. There was at some point in the history, but it's not really that important. That's why the  decision was made there. It's also more robust. For one thing, you have one place and you generate usually one new  file. For a new view, you generate a new file. You don't update Python files with the generator because when you  start this, that can get messy. It shouldn't usually break anything of your code. You would get the test coverage  right from the start. Of course, not all the templates have the same amount of tests. Sometimes it's just not easy to  generate them as a generic test. Sometimes I just wait for you to add some. There's a bit room for improvement, but  the whole structure is there. You can go and also bring your opinions in and say, we should use this way.  Recently, we decided to rewrite some tests so that we use more the clone API instead of using getMultiAdapter to get  the view, to test the view, things like that. This makes it easier for newcomers to understand what's happened  because the clone API is more meaningful in how you use it. By default, you have in the packages you generate the  talk setup. I said earlier that we have the option to change the Python version. It's easy. It takes a bit  of time. If you created a database and you played with it, you have to delete the database because it will not run with  Python 2 and 4 in the same way. What talks does is basically it allows you to have all versions you care about and  all combinations of Python and clone be tested without changing anything. You can pick a single environment and say,  okay, I just want to test now the Python 3.7 with clone 5.2. If you don't care about your add-on for clone 4,  then just go in the talk scene and remove the entry for that. You can run all these things in parallel too.  If you run talks in the parallel mode, it will use all your CPUs and burn it down. It goes as fast as your CPU power is.  At the end, you have a summary of that. You see that. Here's an example of the upper part of the talk scene.  This is what you want to customize. It's basically saying, let's say the second line, for example, it says this is only for  Python 2.7. Then there will be generated two environments. One is clone 4.3 and one is 5.1. From the third line,  we will have a version with Python 2.7 with clone 5.2 and a version with 3.7 and clone 2. If you go and say,  list the environments, then you see these different versions. You can rewrite that so that you have just less and you don't have to test  clone 4 if your add-on will never run on clone 4. That should be okay.  There's another command. If you use talks-a, it will show you a little bit more environments. These are environments that are defined  more or less as a helper. You can use this to update translations and things like that.  Or to apply the ISOD. I personally use this in VS Code, so I don't need this, but some people work that way, so it's okay.  You can configure the clone CLI or the Mr. Bob for both for your own taste. If you don't want the git questions,  then you just say, okay, just do it. It's a good idea. Always commit and maybe use the clone version as default.  You have basically two areas. The first is the variables. Things you put there. You will never see the question.  If you put an answer there, it will be always answered. Not ask anymore. If you put it in default, you see the question,  you see your default. If you press enter, it will just save, but you can change it.  The variable names you can find in BobTemplatesplawn where the templates are defined.  There in every template folder you have a BobInnify. This BobInnify is just a normal Windows style.  You have the variables defined there. This name you copy and this way you can do that.  We have a little helper in clone CLI, but we need to update to allow more settings. It will write this configuration,  at least for the basic stuff that works already. The author name and things like that, it will set.  We could extend this to make more decisions so that you don't have to look for the variables and things like that.  That would be a nice thing to spread on.  The whole thing is extendable. You can write your own BobTemplates packages and register them for clone CLI.  This way, clone CLI will just list them and can use them. It's not hard to do that.  This is useful if you have ideas for packages which are maybe customer related or your company way of doing things is different  and you want to have some of the templates in your own flavor.  Or if you do things which are not necessary for the main development but you have to think for migrations.  We can create multiple packages and make them all work with clone CLI.  It's a little example of how that works. It's basically using Python entry points.  In your setup py, you have something like this and you point to your package or in this case it's BobTemplates clone.  There is a Python module, BobRegistry and you just have the key there.  This is just a little helper object and an entry in that file looks like this.  This is the main clone add-on in this case. You have a clone CLI alias.  We don't want to type all the time clone add-on because we're using the clone CLI so it's clear we want something related to clone not to permit.  We can skip that.  In the context of Mr. Bob, you could use the global name.  You can always say this template depends on clone add-on because all the sub templates expect a certain structure and some files to be there.  Usually you create your add-on with the add-on template and after that you can use all the sub commands.  It's not that hard to upgrade your existing. If you at some point created your content types, I just did that with the easy newsletter for example.  We have a chapter to upgrade existing codes in the documentation of BobTemplates clone.  If you have a question there just ping me or one of the others and we will fulfill this.  I mentioned a small plugin for VS Code. You can just install it like every other VS Code add-on with the extensions.  Then you just write clone and after that you write something like text line field and then you can jump around, fill it out and make the settings for requirement.  We have basically all common fields in Python and also for the supermodel. You can use this in the XML or in the Python file.  There are also some other settings for the registry XML. For example, some of these big chunks of registry thingies.  There are also good things to put in the snippets.  There is still one bug here because it should replace the name when you start writing. It doesn't do that when you have multiple times.  I already opened the issue there.  I mentioned clone API. It has a really good documentation. It really makes sense.  If you have people to onboard and start with, we really should use this as much as we can.  Just because you can read it and you just understand what's happening. This helps a lot.  The other reason is when there will be a change in the future under the hood, it's not your problem.  You just use the clone API and the clone API will use the new way of doing things.  Your code is probably also more robust than using some internal methods from CMM clone or from whatever.  Some ideas for the future.  There are some things like views. For example, you can register a view to just appear on certain contexts or under certain interfaces.  This is by default already there. You have to go in the.cml file and change the interface.  It would be nice to have some common options like all the standard content types could be listed there.  You can say, okay, I want this on pages by default.  Your own content types, which you created earlier, could be listed there or something like that.  For REST API, I work already on the Zerializer.  Another thing would be the Dyserializer, which is the other way around when you send data to the REST API in a certain way.  You want the REST API to understand your data better.  Then you can create that. Maybe you have some special data from an application.  You cannot really change or it's just not that common so that you want to do some conversions there.  Yes, Graphical UI, but just a minimal.  For selecting content types or interfaces, also give me the parent content type. We could just select all the content types from standard blown and from our package.  This would be already helpful. You can just select it instead of remember and retype it.  To do that, we need to change the user interface. That means we have to change Mr. Bob itself.  I have ideas for that. There are some libraries which work cross-platform and allow you just in that moment to have a drop-down or things like that.  But for that, we have to go upstream before we can use this and Bob template's blown and blown CLI.  For VS Code, I wish we would have a better blown API auto completion at least.  It's not working that great when you import blown API like normally people would do from blown import API and then use it like API content find.  VS Code is not that smart with the namespaces there. There are some problems.  Also, VS Code by default, of course, doesn't know the build-out structure and things like that.  There's a bit of a recipe though. You can create some additional configuration for VS Code.  VS Code will read that and then should at least read all your Python paths without doing anything else.  Another way to do this is using the ZOPI Python interpreter you have in the build-out usually.  Then VS Code also has all the paths in there.  But even if you have that, it's not perfect with auto completion.  I would like to have more contributions. There is nothing set in stone as Philip said before. Don't be shy.  There's no grand jury who makes decisions and because it's there and it looks like this, it has to be like this.  That's not the case. If you have ideas or complaints, open an issue, let's discuss this.  If you find a way to test things better, just add a test or write a bit of documentation would be really helpful.  If you have other ideas and you're not sure if that makes sense to put this in Bob-Temps-Lates-plon,  then create other Bob-Temps-Lates-Packages. Your own would be helpful for you internally but also to make the whole system a bit more robust.  There were already some contributions. I guess there are three people from time to time doing something on it. It's nice to see that in use.  I would be at the sprints at least Saturday and maybe Sunday morning.  If you want to dive into that, it's a good task for a short sprint because there's not that much to learn.  You can achieve some things. If you want to work on your custom template and do some stuff, I can just help you if you have questions.  Any questions so far?  Is there a Bob-Temps-Lates-Package that can create a Bob-Temps-Lates?  I don't know if it makes sense. It should be possible because what you usually do is you create one Python file and one folder and a bit of structure.  We could do that. You have to write a bit of Python for sure because what we are doing with configuration files,  usually we pass them, the XML files or XML files, we just pass and to keep the format, we don't write them out as XML because the formatting would be different then.  So, the XML files usually have a line like earlier, like your custom stuff comes here, so that's a marker. It should stay there. But we pass XML, so we find out if we need to add something and if we have to, we insert it after the marker.  So, that's the way and this is like some Python functions that get called from Mr. Bob. And usually depending on what template you build, you have to adjust the parts, the variables you create or define in the pre-rendera section.  So, there's method pre-rendera, it's called and there's some stuff that's always the same but things like, I don't know, you have the question,  upgrade step name or upgrade step title and you want to use this somehow as a variable then you do your initialization there and later in the templates or in the file names or folder names, you can use that.  So, that's the work you have always to do but the structure might be useful.  Any other question?  So, can I customize the command line interface or sub commands? In what way? Like register, maybe additional commands or overwrite them commands, their behavior?  I mean, clone CLI itself is just a Python package so if you have ideas which are generally useful then we can add them.  I mean, if you create your own Bob templates and you register them with the entry points, they will pop up automatically in clone CLI so this add command has an auto completion feature which shows you the list of all templates and you can run them.  So, these sub commands for the templates, this is already there, this is customizable, the rest is it's a Python package, everything what's possible there.  Any other one?  Then I have one question, who of you is beginner, like in the beginning phase? Who is onboarding people from time to time?  So, if you are using already clone CLI for that or most of them not, so give it a try.  I think it's easier to understand and to, you don't have to know all the things, you just create what you want to do.  Somebody told you you have to create a view for that and good luck by trying to find the documentation that's sometimes really hurtful even though you know what to search for.  But it's not easy and with a tool like this, you can just start and then look okay, it works and then adjust it.  It's way better for beginners. When I started way back then, we had something like this with ZOB scale and archetypes support.  So, you could do similar things and this was really helpful.  So, give it a try and please also give feedback how that went and what we can do to improve it.  Okay, thank you. Thank you.
Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_0_263.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_264_311.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_312_3671.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_3672_4991.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_4992_7115.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_7116_7163.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_7164_7679.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_7680_11723.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_11724_13715.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_13716_14507.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_14508_15119.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_15120_17075.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_17076_23003.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_23004_35831.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_35832_48983.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_48984_50399.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_50400_50675.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_50676_50891.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_50892_52559.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_52560_53711.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_53712_53759.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_53760_53915.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_53916_53963.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_53964_54011.png Maik_Derstappen_-_State_of_Plone_back-end_developm_t9NLzYGyBfU_54012_54041.png
Okay, I want to talk today a bit about the back end development. Now, how it is today. Even though everybody talks all the time about headless and front end. But yeah, you still need a back end. So how are we going to do that? A small overview. We have things like the clone CLI. Some of you might have heard about it. I'm pretty sure a lot of people are using the clone API, which is the Python API, which makes things simpler. We also have snippets for several editors. Also for VS code. Where you can add some code with the editor. And also, of course, you have to clone REST API where you can then express these contents to use it on the client side at the end. So the back end part there will be to customize the REST API to deliver the content as they are needed for the front end. Yeah, of course, if you didn't, you should use the clone CLI. I think it's useful also for not newcomers. I like to use it. Of course, I build it. But it really saves a lot of time for boring stuff. You can focus on your code. Yeah, it helps you to enhance your products, your add-ons step by step. So if you need a functionality, you can add it if clone CLI or BOP templates is providing it. This is just a small demo. So we have a lot of templates already. You can always check the version number with the minus capital V so that you see which version of clone CLI and BOP templates you are using. And then you can create your package. Since we need support for Python 3, Python 2, you can set which version you want to use for clone CLI. It's mainly used for creating local virtual environments for testing. And you can always change that and just run clone CLI build and it will recreate the virtual environment, install the requirements, run build out, and then your setup is on Python 2 or Python 3. You can see the build command does several steps. The last step is to build out and you don't have to type this manually. But you still can, of course. Yeah, there's a small configuration file. This is also the first step when you want to migrate an older package to be compatible with clone CLI. You just create this file. This is the way how clone CLI finds the package root. But you have also some settings there. And this you can use to, or this clone CLI and the add-ons and the sub templates will use to make decisions later. One example, a content type. That was actually the first template I was working on when I changed to this more modular approach because it was not possible before to create multiple content types. We just had one big template which were having one example content type. So you can create a content type. You give it a name. You have the option of container an item. If it's global, addable or not. And then you have it. If you want to create another content type, for example, this time we choose item as a base class. This time it's not globally addable. So when we continue, then it will ask for parent content type, which in this case is the task list. And this clone CLI will use to generate the tests that they are working so that it makes sure that the parent content type exists and then they try to add the content type there. And also the wiring in the FDI settings so that you don't have to go there and say in this container I'm allowed the task to be added. You can see the structure. It creates a Python file for every content type and XML file. Depending on the question of if you want to use the XML file or not, we could clean that up but it doesn't hurt anybody. You can delete it if you don't want to go that way. Or if you say you want to use the supermodel, then you can basically write it in XML or do it through the web and export the XML after you created your schema. We also have template for rest API service. For those of you who don't know what the service is, that's the components you have, which for example provides the breadcrumbs, the navigation, so additional data for your current context. When you're on a page or your custom content type and you want additional data, maybe based on tagging or whatever, so you make a catalog query for that. You can create a service for that and make this information available. With the expand feature of the rest API, you can tell the rest API, please also give me the data with the run request I did for the actual context. You just have to customize this area and it will return, Jason, the rest is more or less boilerplate. There are two things you decide. One is the class name of the service and one is the actual endpoint name, so the service name and these things are used to add this generation. There's another template coming for Zeralizers. Are you using to change the way how the rest API returns your values? For example, for vocabulary I did that once because earlier versions of the rest API were just giving you the token and you want the title and the token and not asked again for this from the client, so you can change the ways how things are returned to you. There's another one which is fairly new, so there might be some bugs, I just found one, but this will help you to easily generate upgrade steps. What we're doing here is basically generate a profile for every upgrade step. This is similar to what blown up upgrades in the core package. We will also provide the Python file and we will also auto increment the version number, so you don't forget to update or increment the version number and the metadata XML. If you don't need something, you can always ignore it. If you don't have Python code to write for your upgrade step because the generic setup import is enough, then you're fine. If not, then you can use the Python file and do, for example, re-indexing after you imported the profile. Basically, you get a file like this for every upgrade step automatically. This includes a folder where you can put profiles like registry XML or catalog XML like I do here in the example. Like I will do now. This way you can easily create upgrade steps and there's no excuse to not do it. If you create more upgrade steps, you can see now when we're doing this, it will generate similar structures and we'll go and update the version number. Whatever version number you have in your existing package, we'll just read it from the metadata and then increment it and that's it. We have a Git integration, so every command you make will automatically commit it to your repository and when you did manually changes like I did with the catalog XML, then it will tell me maybe you want to commit it before you use templates because the templates from Blown CLI or from Mr. Bob, they're not asking you before they overwrite anything. If you have conflicting configurations, you might lose your changes. Always use the Git commit before and then you're fine. It helps you to revert the whole process. You can just go one commit back and nothing happened. One other reason is if you use Blown CLI, all the packages are created by this will have the same structure. That makes it easy to find things. The ideas here are everything has its own folder. If you create behaviors, they are in behaviors folder. If you create content, like content types, they are content indexer folder and the upgrades are in upgrades and so forth. There might be some changes like I choose view list and views to put also in separate folders on the top level. This is a little bit different to the traditional way in having a browser folder. Otherwise, where would you put view list and views? You could put them in browser folder, like a sub folder, but where's the meaning of that? If you think about newcomers, they don't know the browser folder. They have no idea what this is for. There's no real reason to really name it that way. There was at some point in the history, but it's not really that important. That's why the decision was made there. It's also more robust. For one thing, you have one place and you generate usually one new file. For a new view, you generate a new file. You don't update Python files with the generator because when you start this, that can get messy. It shouldn't usually break anything of your code. You would get the test coverage right from the start. Of course, not all the templates have the same amount of tests. Sometimes it's just not easy to generate them as a generic test. Sometimes I just wait for you to add some. There's a bit room for improvement, but the whole structure is there. You can go and also bring your opinions in and say, we should use this way. Recently, we decided to rewrite some tests so that we use more the clone API instead of using getMultiAdapter to get the view, to test the view, things like that. This makes it easier for newcomers to understand what's happened because the clone API is more meaningful in how you use it. By default, you have in the packages you generate the talk setup. I said earlier that we have the option to change the Python version. It's easy. It takes a bit of time. If you created a database and you played with it, you have to delete the database because it will not run with Python 2 and 4 in the same way. What talks does is basically it allows you to have all versions you care about and all combinations of Python and clone be tested without changing anything. You can pick a single environment and say, okay, I just want to test now the Python 3.7 with clone 5.2. If you don't care about your add-on for clone 4, then just go in the talk scene and remove the entry for that. You can run all these things in parallel too. If you run talks in the parallel mode, it will use all your CPUs and burn it down. It goes as fast as your CPU power is. At the end, you have a summary of that. You see that. Here's an example of the upper part of the talk scene. This is what you want to customize. It's basically saying, let's say the second line, for example, it says this is only for Python 2.7. Then there will be generated two environments. One is clone 4.3 and one is 5.1. From the third line, we will have a version with Python 2.7 with clone 5.2 and a version with 3.7 and clone 2. If you go and say, list the environments, then you see these different versions. You can rewrite that so that you have just less and you don't have to test clone 4 if your add-on will never run on clone 4. That should be okay. There's another command. If you use talks-a, it will show you a little bit more environments. These are environments that are defined more or less as a helper. You can use this to update translations and things like that. Or to apply the ISOD. I personally use this in VS Code, so I don't need this, but some people work that way, so it's okay. You can configure the clone CLI or the Mr. Bob for both for your own taste. If you don't want the git questions, then you just say, okay, just do it. It's a good idea. Always commit and maybe use the clone version as default. You have basically two areas. The first is the variables. Things you put there. You will never see the question. If you put an answer there, it will be always answered. Not ask anymore. If you put it in default, you see the question, you see your default. If you press enter, it will just save, but you can change it. The variable names you can find in BobTemplatesplawn where the templates are defined. There in every template folder you have a BobInnify. This BobInnify is just a normal Windows style. You have the variables defined there. This name you copy and this way you can do that. We have a little helper in clone CLI, but we need to update to allow more settings. It will write this configuration, at least for the basic stuff that works already. The author name and things like that, it will set. We could extend this to make more decisions so that you don't have to look for the variables and things like that. That would be a nice thing to spread on. The whole thing is extendable. You can write your own BobTemplates packages and register them for clone CLI. This way, clone CLI will just list them and can use them. It's not hard to do that. This is useful if you have ideas for packages which are maybe customer related or your company way of doing things is different and you want to have some of the templates in your own flavor. Or if you do things which are not necessary for the main development but you have to think for migrations. We can create multiple packages and make them all work with clone CLI. It's a little example of how that works. It's basically using Python entry points. In your setup py, you have something like this and you point to your package or in this case it's BobTemplates clone. There is a Python module, BobRegistry and you just have the key there. This is just a little helper object and an entry in that file looks like this. This is the main clone add-on in this case. You have a clone CLI alias. We don't want to type all the time clone add-on because we're using the clone CLI so it's clear we want something related to clone not to permit. We can skip that. In the context of Mr. Bob, you could use the global name. You can always say this template depends on clone add-on because all the sub templates expect a certain structure and some files to be there. Usually you create your add-on with the add-on template and after that you can use all the sub commands. It's not that hard to upgrade your existing. If you at some point created your content types, I just did that with the easy newsletter for example. We have a chapter to upgrade existing codes in the documentation of BobTemplates clone. If you have a question there just ping me or one of the others and we will fulfill this. I mentioned a small plugin for VS Code. You can just install it like every other VS Code add-on with the extensions. Then you just write clone and after that you write something like text line field and then you can jump around, fill it out and make the settings for requirement. We have basically all common fields in Python and also for the supermodel. You can use this in the XML or in the Python file. There are also some other settings for the registry XML. For example, some of these big chunks of registry thingies. There are also good things to put in the snippets. There is still one bug here because it should replace the name when you start writing. It doesn't do that when you have multiple times. I already opened the issue there. I mentioned clone API. It has a really good documentation. It really makes sense. If you have people to onboard and start with, we really should use this as much as we can. Just because you can read it and you just understand what's happening. This helps a lot. The other reason is when there will be a change in the future under the hood, it's not your problem. You just use the clone API and the clone API will use the new way of doing things. Your code is probably also more robust than using some internal methods from CMM clone or from whatever. Some ideas for the future. There are some things like views. For example, you can register a view to just appear on certain contexts or under certain interfaces. This is by default already there. You have to go in the.cml file and change the interface. It would be nice to have some common options like all the standard content types could be listed there. You can say, okay, I want this on pages by default. Your own content types, which you created earlier, could be listed there or something like that. For REST API, I work already on the Zerializer. Another thing would be the Dyserializer, which is the other way around when you send data to the REST API in a certain way. You want the REST API to understand your data better. Then you can create that. Maybe you have some special data from an application. You cannot really change or it's just not that common so that you want to do some conversions there. Yes, Graphical UI, but just a minimal. For selecting content types or interfaces, also give me the parent content type. We could just select all the content types from standard blown and from our package. This would be already helpful. You can just select it instead of remember and retype it. To do that, we need to change the user interface. That means we have to change Mr. Bob itself. I have ideas for that. There are some libraries which work cross-platform and allow you just in that moment to have a drop-down or things like that. But for that, we have to go upstream before we can use this and Bob template's blown and blown CLI. For VS Code, I wish we would have a better blown API auto completion at least. It's not working that great when you import blown API like normally people would do from blown import API and then use it like API content find. VS Code is not that smart with the namespaces there. There are some problems. Also, VS Code by default, of course, doesn't know the build-out structure and things like that. There's a bit of a recipe though. You can create some additional configuration for VS Code. VS Code will read that and then should at least read all your Python paths without doing anything else. Another way to do this is using the ZOPI Python interpreter you have in the build-out usually. Then VS Code also has all the paths in there. But even if you have that, it's not perfect with auto completion. I would like to have more contributions. There is nothing set in stone as Philip said before. Don't be shy. There's no grand jury who makes decisions and because it's there and it looks like this, it has to be like this. That's not the case. If you have ideas or complaints, open an issue, let's discuss this. If you find a way to test things better, just add a test or write a bit of documentation would be really helpful. If you have other ideas and you're not sure if that makes sense to put this in Bob-Temps-Lates-plon, then create other Bob-Temps-Lates-Packages. Your own would be helpful for you internally but also to make the whole system a bit more robust. There were already some contributions. I guess there are three people from time to time doing something on it. It's nice to see that in use. I would be at the sprints at least Saturday and maybe Sunday morning. If you want to dive into that, it's a good task for a short sprint because there's not that much to learn. You can achieve some things. If you want to work on your custom template and do some stuff, I can just help you if you have questions. Any questions so far? Is there a Bob-Temps-Lates-Package that can create a Bob-Temps-Lates? I don't know if it makes sense. It should be possible because what you usually do is you create one Python file and one folder and a bit of structure. We could do that. You have to write a bit of Python for sure because what we are doing with configuration files, usually we pass them, the XML files or XML files, we just pass and to keep the format, we don't write them out as XML because the formatting would be different then. So, the XML files usually have a line like earlier, like your custom stuff comes here, so that's a marker. It should stay there. But we pass XML, so we find out if we need to add something and if we have to, we insert it after the marker. So, that's the way and this is like some Python functions that get called from Mr. Bob. And usually depending on what template you build, you have to adjust the parts, the variables you create or define in the pre-rendera section. So, there's method pre-rendera, it's called and there's some stuff that's always the same but things like, I don't know, you have the question, upgrade step name or upgrade step title and you want to use this somehow as a variable then you do your initialization there and later in the templates or in the file names or folder names, you can use that. So, that's the work you have always to do but the structure might be useful. Any other question? So, can I customize the command line interface or sub commands? In what way? Like register, maybe additional commands or overwrite them commands, their behavior? I mean, clone CLI itself is just a Python package so if you have ideas which are generally useful then we can add them. I mean, if you create your own Bob templates and you register them with the entry points, they will pop up automatically in clone CLI so this add command has an auto completion feature which shows you the list of all templates and you can run them. So, these sub commands for the templates, this is already there, this is customizable, the rest is it's a Python package, everything what's possible there. Any other one? Then I have one question, who of you is beginner, like in the beginning phase? Who is onboarding people from time to time? So, if you are using already clone CLI for that or most of them not, so give it a try. I think it's easier to understand and to, you don't have to know all the things, you just create what you want to do. Somebody told you you have to create a view for that and good luck by trying to find the documentation that's sometimes really hurtful even though you know what to search for. But it's not easy and with a tool like this, you can just start and then look okay, it works and then adjust it. It's way better for beginners. When I started way back then, we had something like this with ZOB scale and archetypes support. So, you could do similar things and this was really helpful. So, give it a try and please also give feedback how that went and what we can do to improve it. Okay, thank you. Thank you.
simpler. We also have snippets for several that once because earlier versions of the rest Always commit and maybe use the clone version as main development but you have to think for the time clone add-on because we're using the Usually you create your add-on with the add-on configuration for VS Code. VS Code will read that