How to design powerful DSLs for users
Mercouris, John
How can we use DSLs in our applications as a replacement for databases? CSVs? configuration files? In this talk I will explain how we designed and developed several DSLs for usage in the Nyxt browser. I will also explain how we designed our DSLs and what our constraints were. What were we trying to achieve with our DSLs? How can you too leverage DSLs in your applications to empower your users?
 you Hello everyone my name is John McCurris and I'm going to give you a brief explanation of how we designed our DSLs for the Nix browser. I'll explain to you some of our design goals, what we were thinking about matters and hopefully you can get some tips and you can use these exact same techniques in your own applications to make them more powerful for your users. So the first things first what is the Nix browser and who am I? Well the Nix browser is a it's a list based power browser we often call it the hackers browser it's a totally completely customizable and it can do anything you want that's that's the premise of it you can tweak it in any way. So the second question is who am I? Well my name is John I'm a computer scientist I'm a Nix browser developer obviously you can see my email here if you want to reach out some questions you have anything John or Atlas that engineer and you can also check on my website http call slash slash John dot McCurris dot online. So before we get into how you can use DSLs in your program to make them better for your users the question is what is a DSL? So a DSL according to Wikipedia it's an interestingly recursive definition they say a domain specific language is a computer language specialized to a particular application domain that's kind of wonderful because it's self-self-referential it doesn't really say anything so why is that? Well that's because actually a DSL is a really ambiguous thing it's actually any small language that is tailored to a specific purpose it's a language that is limited in scope it doesn't have a lot of bells and whistles maybe it has a small standard library maybe it only has three or four functions exposed it's incredibly tiny that's the key of a DSL to be very tiny to solve a specific problem DSLs are often employed by end users or when I say users I mean engineers programmers sometimes even actually non-technical users as well though they do some people do try to make DSLs simple enough that literally anyone without any formal training in programming language usage compilers etc can use them so why do these languages exist well the first one obviously is they're incredibly simple right so that makes them really good for configuration files things that the user is touching directly or things that the user is going to look at right they're going to see okay we have this specific settings file let me tweak some some parameters here maybe there will be some settings file for a game where they can edit some arguments and change the behavior another thing that you may be more familiar with as an engineer is document markup I'm sure many of you have used markdown or org mode which is actually what we're looking at right now so this is a org mode document with its own DSL and this is extremely useful and of course the the most famous HTML so okay we know what they do we know who uses them now the question is why would we use a DSL instead of a general purpose programming language why are we kind of restricting ourselves to a DSL well number one they're incredibly simple when we make a more complex language when we expose more complexity they become harder to learn and harder to use because we can design them ourselves for our application usually we can tailor the syntax and make it very straightforward specifically for what we're trying to do next is a reality is that we don't typically want to ship a complete compiler bundled with our application this would be pretty dangerous for a lot of reasons if you can compile arbitrary code within your application you open up gigantic attack surface for people to exploit and additionally we don't want to embed a compiler within our program because it's a gigantic undertaking compilers are very complex things compiler tool chains are also very complex and gooding from a piece of source to a binary is a really non-trivial thing to do so make a DSL we don't have to make a crazy compiler we don't have to make you know we don't have to add a lexer tokenizer make a STs and do all this crazy stuff we just look for some key value pairs and about it being about a boom we have ourselves a DSL that's perfectly tailored for our application okay it sounds good so far I can understand why people are doing these things and this is the product that comes out of it is something exactly like the python I9 files so it's just a simple almost key value store where we have some simple string substitution going on we have maybe some sections and this is great for 99% of use cases this is amazing right we've allowed the user we've exposed a whole bunch of you know extra fancy switches to them without having to make a complicated UI in fact this also scales infinitely because with the simple text editor you could search for a particular argument or or value rather you could search for a particular variable and do whatever you need to do however if you want to do something a little bit more complex than assignment that's when things start to get a little bit more tricky and when you're making the you know or rather when you're trying to make the world's most hackable browser now we start to think hmm maybe we would be limiting ourselves a little bit if all we could do was was set the values of variables that might be a little bit tricky so let's let's talk about it what is the radical approach so we had a couple constraints that we discussed that were based on the models of typical applications now because we're you know in a very atypical application our constraints are significantly different so what if we have a compiler right what if we have the ability to compile any form within our our compiled program so next could come with a compiler right why not it's not too hard to do that in this okay fair enough so the next thing is well what if we trusted the user right what if we gave them the power the responsibility to evaluate any arbitrary list what would that be like what could they do and what if we didn't have to invent a new syntax I mean lots of people have already spent a lot of time perfecting syntaxes and thinking about it why do we have to invent yet another one why can't we just reuse the existing work of someone else well maybe we can so let's kind of go through these points a little bit more in depth so the first kind of sub point is about using list syntax so rather than coming up with a whole new syntax let's just use list syntax right because list syntax allows you to represent any arbitrary data and any arbitrary program so we've covered everything that we could possibly want to do right if we want to save data or I mean technically everything is data or code we can do that no problem we don't have to add any anything and additionally one of our kind of desires of making a DSL is incredibly simple syntax and I don't think it gets any simpler than list so what is really list syntax like well it's just you know a bunch of s expressions and an s expression is just an atom and then you can think of it as just a form a form is composed of atoms or other s expressions wrapped in parenz so this is kind of another recursive definition but it's it's it's only two rules and just to clarify something an atom is you can think of it there's a couple of things that can be atoms but simplified for your for your thoughts think okay we have maybe a string maybe we have an integer maybe we have a symbol these can be atoms and they can be nested infinitely with expressions so we have a simple syntax right so the next step is a compiler and I'll kind of explain how this works and again purpose of this presentation is to show you our approach and help you understand it I'm simplifying a lot of things and I'm not necessarily talking with the you know the most correct list terminology for the sake of effective communication so you can think of it like this we have a list program running and then we just take a snapshot of everything okay and we save that snapshot that's what compiling a list program for distribution is we start with the compiler we go compile this compile this compile this compile this compile this compiles a whole bunch of things and then we save the whole world including the compiler in a specific file so now we have a list compiler this is pretty standard okay there are ways to not have the compiler in the list image there are ways to remove things there's tree shakers we're not gonna cover that but suffice is safe for next we include the compiler in our list image because we get it for free for no extra effort on our part so the final point and this is the most difficult one to swallow right you're trying to make a secure application or trying to make a reliable X application remember one that's not gonna suffer from so many potential exploits right you don't want to open up a huge gigantic surface area I get it but in next we trust user input it's their computer they're allowed to do what they want to do with it if they're gonna do all sorts of mistakes that's fine ultimately we give them the power of course they could choose to do nothing with their user input files and you know reduce their risk or they could do whatever they want they could copy and paste blindly without examining snippets of code from the internet nonetheless let's let's just consider that for our application we trust user input and let's see what happens you know what did we gain from all of this so every single configuration file in next is literally just Lisp everything and you can embed any arbitrary list code you want and any configuration file so in the right hand well emacs window regular person frame we have an example of a configuration file so rather than just allowing the user to set arbitrary keys and values we let the user write literally anything so here I have a sample where I've defined a command this is a Lisp expression that we could have also written to next and now I can run the command hello and it'll prompt me for something and I'll print it right this wouldn't be possible if it was just an I and I file we can't anticipate all the potential use cases for next we cannot anticipate how someone is gonna need to mold it to their workflow so every single literally every single configuration file is Lisp and that extends to everything so I provided here a little sample from our bookmarks our bookmarks file is just Lisp again if you wanted to you could make a macro that does all sorts of expansion and generates a whole bunch of bookmarks you could do that you could have another macro that fetches a whole bunch of URLs from the internet and populates your bookmarks so let's go a little bit more practical beyond just the initialization file so the initialization file is an example of someplace so the initialization files the init out this file here on the right the user configuration so the init file is an example of something that typically would have been an I and I file that's a pretty pretty common use case people can understand that right they have a emacs init file where they can arbitrate execute arbitrary Lisp that's fine that's fine but there are use cases outside of the init file for example we have a mode in next called auto mode and auto mode it's effectively a predicate that enables and disables different modes for different URLs or hosts or whatever because we don't know how the user wants to enable and disable modes we allow arbitrary list and this sample that I provided here we have you know tomato cucumber salmon these are exact matches so when we visit the page cucumber it's going to enable dark mode the wikipedia page on cucumbers but there are more cases so a great example of this is actually we had one user who wanted to enable proxy mode for onion addresses right because we didn't limit them because we didn't say to them you have to follow this format or whatever they have provided they made their own predicate they used regex in this case so match regex and then they said then then start my proxy and then they can do all sorts of you know things like disabling force HTTPS or whatever so that's the power of allowing the user to execute arbitrary Lisp in files outside of just the initialization file so and I wrote this here to emphasize the user can write any arbitrary predicate using the complete list language amazing at least I think it's amazing so what's the whole point number one we didn't have to write a compiler we use the compiler SPCL specifically it's already fast it's secure it's efficient secondly or maybe thirdly our users don't don't find themselves hitting a dead end with a particular DSL because of something we didn't anticipate they can use all of Lisp to do whatever they want to do whenever they want to do wherever they want to so I don't know I think that's kind of kind of important personally when you again when you're trying to make a you know the most hackable browser out there anyways the next thing is we don't invent a new language that the user has to learn you know they may already be familiar with S expressions another Lisp guile scheme something like that it's a simple syntax it's the same thing it takes minutes to learn and it's incredibly powerful and finally and most importantly for our context the users are fully in control of their system so I hope that you know that was useful for you I hope you kind of understand why we didn't try to reinvent the wheel why we decided okay why make another DSL when we can just use this amazing wonderful compiler already built into our system maybe it makes sense for you to maybe you can use a a general purpose language as well for your configuration or for your data store or whatever because you saw we're using it also for our bookmarks we're using it also for our auto mode rules all sorts of actually even our history is a is a Lisp file anyways the point is hopefully that's interesting to you hopefully it gives you some ideas I hope you enjoyed thanks for listening thank you so much for your attention and for your time and I will catch you later bye bye
 you  Hello everyone my name is John McCurris and I'm going to give you a brief  explanation of how we designed our DSLs for the Nix browser. I'll explain to you  some of our design goals, what we were thinking about matters and hopefully you  can get some tips and you can use these exact same techniques in your own  applications to make them more powerful for your users. So the first things first  what is the Nix browser and who am I? Well the Nix browser is a it's a list  based power browser we often call it the hackers browser it's a totally  completely customizable and it can do anything you want that's that's the  premise of it you can tweak it in any way. So the second question is who am I?  Well my name is John I'm a computer scientist I'm a Nix browser developer  obviously you can see my email here if you want to reach out some questions you  have anything John or Atlas that engineer and you can also check on my website  http call slash slash John dot McCurris dot online. So before we get into how you  can use DSLs in your program to make them better for your users the question  is what is a DSL? So a DSL according to Wikipedia it's an interestingly recursive  definition they say a domain specific language is a computer language  specialized to a particular application domain that's kind of wonderful  because it's self-self-referential it doesn't really say anything so why is  that? Well that's because actually a DSL is a really ambiguous thing it's  actually any small language that is tailored to a specific purpose it's a  language that is limited in scope it doesn't have a lot of bells and whistles  maybe it has a small standard library maybe it only has three or four functions  exposed it's incredibly tiny that's the key of a DSL to be very tiny to solve a  specific problem DSLs are often employed by end users or when I say users I mean  engineers programmers sometimes even actually non-technical users as well  though they do some people do try to make DSLs simple enough that literally  anyone without any formal training in programming language usage compilers  etc can use them so why do these languages exist well the first one  obviously is they're incredibly simple right so that makes them really good for  configuration files things that the user is touching directly or things that the  user is going to look at right they're going to see okay we have this specific  settings file let me tweak some some parameters here maybe there will be some  settings file for a game where they can edit some arguments and change the  behavior another thing that you may be more familiar with as an engineer is  document markup I'm sure many of you have used markdown or org mode which is  actually what we're looking at right now so this is a org mode document with its  own DSL and this is extremely useful and of course the the most famous HTML so  okay we know what they do we know who uses them now the question is why would  we use a DSL instead of a general purpose programming language why are we  kind of restricting ourselves to a DSL well number one they're incredibly  simple when we make a more complex language when we expose more complexity  they become harder to learn and harder to use because we can design them ourselves  for our application usually we can tailor the syntax and make it very  straightforward specifically for what we're trying to do next is a reality is  that we don't typically want to ship a complete compiler bundled with our  application this would be pretty dangerous for a lot of reasons if you can  compile arbitrary code within your application you open up gigantic attack  surface for people to exploit and additionally we don't want to embed a  compiler within our program because it's a gigantic undertaking compilers are  very complex things compiler tool chains are also very complex and gooding  from a piece of source to a binary is a really non-trivial thing to do so make a  DSL we don't have to make a crazy compiler we don't have to make you know we  don't have to add a lexer tokenizer make a STs and do all this crazy stuff we  just look for some key value pairs and about it being about a boom we have  ourselves a DSL that's perfectly tailored for our application okay it  sounds good so far I can understand why people are doing these things and this  is the product that comes out of it is something exactly like the python I9  files so it's just a simple almost key value store where we have some simple  string substitution going on we have maybe some sections and this is great  for 99% of use cases this is amazing right we've allowed the user we've  exposed a whole bunch of you know extra fancy switches to them without having to  make a complicated UI in fact this also scales infinitely because with the  simple text editor you could search for a particular argument or or value  rather you could search for a particular variable and do whatever you need to do  however if you want to do something a little bit more complex than assignment  that's when things start to get a little bit more tricky and when you're  making the you know or rather when you're trying to make the world's most  hackable browser now we start to think hmm maybe we would be limiting ourselves  a little bit if all we could do was was set the values of variables that might  be a little bit tricky so let's let's talk about it what is the radical  approach so we had a couple constraints that we discussed that were based on  the models of typical applications now because we're you know in a very  atypical application our constraints are significantly different so what if we  have a compiler right what if we have the ability to compile any form within  our our compiled program so next could come with a compiler right why not it's  not too hard to do that in this okay fair enough so the next thing is well  what if we trusted the user right what if we gave them the power the  responsibility to evaluate any arbitrary list what would that be like what  could they do and what if we didn't have to invent a new syntax I mean lots of  people have already spent a lot of time perfecting syntaxes and thinking about  it why do we have to invent yet another one why can't we just reuse the existing  work of someone else well maybe we can so let's kind of go through these points  a little bit more in depth so the first kind of sub point is about using list  syntax so rather than coming up with a whole new syntax let's just use list  syntax right because list syntax allows you to represent any arbitrary data and  any arbitrary program so we've covered everything that we could possibly want to  do right if we want to save data or I mean technically everything is data or  code we can do that no problem we don't have to add any anything and  additionally one of our kind of desires of making a DSL is incredibly simple  syntax and I don't think it gets any simpler than list so what is really list  syntax like well it's just you know a bunch of s expressions and an s  expression is just an atom and then you can think of it as just a form a form is  composed of atoms or other s expressions wrapped in parenz so this is kind of  another recursive definition but it's it's it's only two rules and just to  clarify something an atom is you can think of it there's a couple of things  that can be atoms but simplified for your for your thoughts think okay we have  maybe a string maybe we have an integer maybe we have a symbol these can be atoms  and they can be nested infinitely with expressions so we have a simple syntax  right so the next step is a compiler and I'll kind of explain how this works and  again purpose of this presentation is to show you our approach and help you  understand it I'm simplifying a lot of things and I'm not necessarily talking  with the you know the most correct list terminology for the sake of effective  communication so you can think of it like this we have a list program running  and then we just take a snapshot of everything okay and we save that snapshot  that's what compiling a list program for distribution is we start with the  compiler we go compile this compile this compile this compile this compile this  compiles a whole bunch of things and then we save the whole world including  the compiler in a specific file so now we have a list compiler this is pretty  standard okay there are ways to not have the compiler in the list image there are  ways to remove things there's tree shakers we're not gonna cover that but  suffice is safe for next we include the compiler in our list image because we get  it for free for no extra effort on our part so the final point and this is the  most difficult one to swallow right you're trying to make a secure application  or trying to make a reliable X application remember one that's not  gonna suffer from so many potential exploits right you don't want to open  up a huge gigantic surface area I get it but in next we trust user input it's  their computer they're allowed to do what they want to do with it if they're  gonna do all sorts of mistakes that's fine ultimately we give them the power  of course they could choose to do nothing with their user input files and you  know reduce their risk or they could do whatever they want they could copy and  paste blindly without examining snippets of code from the internet  nonetheless let's let's just consider that for our application we trust user  input and let's see what happens you know what did we gain from all of this so  every single configuration file in next is literally just Lisp everything and  you can embed any arbitrary list code you want and any configuration file so in  the right hand well emacs window regular person frame we have an example of a  configuration file so rather than just allowing the user to set arbitrary keys  and values we let the user write literally anything so here I have a  sample where I've defined a command this is a Lisp expression that we could have  also written to next and now I can run the command hello and it'll prompt me for  something and I'll print it right this wouldn't be possible if it was just an  I and I file we can't anticipate all the potential use cases for next we cannot  anticipate how someone is gonna need to mold it to their workflow so every  single literally every single configuration file is Lisp and that  extends to everything so I provided here a little sample from our bookmarks our  bookmarks file is just Lisp again if you wanted to you could make a macro that  does all sorts of expansion and generates a whole bunch of bookmarks you  could do that you could have another macro that fetches a whole bunch of URLs  from the internet and populates your bookmarks so let's go a little bit more  practical beyond just the initialization file so the initialization file is an  example of someplace so the initialization files the init out this  file here on the right the user configuration so the init file is an  example of something that typically would have been an I and I file that's a  pretty pretty common use case people can understand that right they have a emacs  init file where they can arbitrate execute arbitrary Lisp that's fine  that's fine but there are use cases outside of the init file for example we  have a mode in next called auto mode and auto mode it's effectively a predicate  that enables and disables different modes for different URLs or hosts or  whatever because we don't know how the user wants to enable and disable modes  we allow arbitrary list and this sample that I provided here we have you know  tomato cucumber salmon these are exact matches so when we visit the page  cucumber it's going to enable dark mode the wikipedia page on cucumbers but there  are more cases so a great example of this is actually we had one user who wanted  to enable proxy mode for onion addresses right because we didn't limit them  because we didn't say to them you have to follow this format or whatever they  have provided they made their own predicate they used regex in this case  so match regex and then they said then then start my proxy and then they can do  all sorts of you know things like disabling force HTTPS or whatever so  that's the power of allowing the user to execute arbitrary Lisp in files outside  of just the initialization file so and I wrote this here to emphasize the user  can write any arbitrary predicate using the complete list language amazing at  least I think it's amazing so what's the whole point number one we didn't have  to write a compiler we use the compiler SPCL specifically it's already fast  it's secure it's efficient secondly or maybe thirdly our users don't don't find  themselves hitting a dead end with a particular DSL because of something we  didn't anticipate they can use all of Lisp to do whatever they want to do  whenever they want to do wherever they want to so I don't know I think that's  kind of kind of important personally when you again when you're trying to make a  you know the most hackable browser out there anyways the next thing is we don't  invent a new language that the user has to learn you know they may already be  familiar with S expressions another Lisp guile scheme something like that it's a  simple syntax it's the same thing it takes minutes to learn and it's  incredibly powerful and finally and most importantly for our context the users  are fully in control of their system so I hope that you know that was useful for  you I hope you kind of understand why we didn't try to reinvent the wheel why we  decided okay why make another DSL when we can just use this amazing wonderful  compiler already built into our system maybe it makes sense for you to maybe you  can use a a general purpose language as well for your configuration or for your  data store or whatever because you saw we're using it also for our bookmarks  we're using it also for our auto mode rules all sorts of actually even our  history is a is a Lisp file anyways the point is hopefully that's interesting to  you hopefully it gives you some ideas I hope you enjoyed thanks for listening  thank you so much for your attention and for your time and I will catch you  later bye bye
designdsl_0_155.png designdsl_156_935.png designdsl_936_1799.png designdsl_1800_3059.png designdsl_3060_5087.png designdsl_5088_6515.png designdsl_6516_6851.png designdsl_6852_9227.png designdsl_9228_11099.png designdsl_11100_12779.png designdsl_12780_13775.png designdsl_13776_15155.png designdsl_15156_16907.png designdsl_16908_18239.png designdsl_18240_22679.png designdsl_22680_22727.png designdsl_22728_23327.png designdsl_23328_23399.png designdsl_23400_23495.png designdsl_23496_23555.png designdsl_23556_23603.png designdsl_23604_24251.png designdsl_24252_26051.png designdsl_26052_26135.png designdsl_26136_26219.png designdsl_26220_27539.png designdsl_27540_27815.png designdsl_27816_27948.png
you Hello everyone my name is John McCurris and I'm going to give you a brief explanation of how we designed our DSLs for the Nix browser. I'll explain to you some of our design goals, what we were thinking about matters and hopefully you can get some tips and you can use these exact same techniques in your own applications to make them more powerful for your users. So the first things first what is the Nix browser and who am I? Well the Nix browser is a it's a list based power browser we often call it the hackers browser it's a totally completely customizable and it can do anything you want that's that's the premise of it you can tweak it in any way. So the second question is who am I? Well my name is John I'm a computer scientist I'm a Nix browser developer obviously you can see my email here if you want to reach out some questions you have anything John or Atlas that engineer and you can also check on my website http call slash slash John dot McCurris dot online. So before we get into how you can use DSLs in your program to make them better for your users the question is what is a DSL? So a DSL according to Wikipedia it's an interestingly recursive definition they say a domain specific language is a computer language specialized to a particular application domain that's kind of wonderful because it's self-self-referential it doesn't really say anything so why is that? Well that's because actually a DSL is a really ambiguous thing it's actually any small language that is tailored to a specific purpose it's a language that is limited in scope it doesn't have a lot of bells and whistles maybe it has a small standard library maybe it only has three or four functions exposed it's incredibly tiny that's the key of a DSL to be very tiny to solve a specific problem DSLs are often employed by end users or when I say users I mean engineers programmers sometimes even actually non-technical users as well though they do some people do try to make DSLs simple enough that literally anyone without any formal training in programming language usage compilers etc can use them so why do these languages exist well the first one obviously is they're incredibly simple right so that makes them really good for configuration files things that the user is touching directly or things that the user is going to look at right they're going to see okay we have this specific settings file let me tweak some some parameters here maybe there will be some settings file for a game where they can edit some arguments and change the behavior another thing that you may be more familiar with as an engineer is document markup I'm sure many of you have used markdown or org mode which is actually what we're looking at right now so this is a org mode document with its own DSL and this is extremely useful and of course the the most famous HTML so okay we know what they do we know who uses them now the question is why would we use a DSL instead of a general purpose programming language why are we kind of restricting ourselves to a DSL well number one they're incredibly simple when we make a more complex language when we expose more complexity they become harder to learn and harder to use because we can design them ourselves for our application usually we can tailor the syntax and make it very straightforward specifically for what we're trying to do next is a reality is that we don't typically want to ship a complete compiler bundled with our application this would be pretty dangerous for a lot of reasons if you can compile arbitrary code within your application you open up gigantic attack surface for people to exploit and additionally we don't want to embed a compiler within our program because it's a gigantic undertaking compilers are very complex things compiler tool chains are also very complex and gooding from a piece of source to a binary is a really non-trivial thing to do so make a DSL we don't have to make a crazy compiler we don't have to make you know we don't have to add a lexer tokenizer make a STs and do all this crazy stuff we just look for some key value pairs and about it being about a boom we have ourselves a DSL that's perfectly tailored for our application okay it sounds good so far I can understand why people are doing these things and this is the product that comes out of it is something exactly like the python I9 files so it's just a simple almost key value store where we have some simple string substitution going on we have maybe some sections and this is great for 99% of use cases this is amazing right we've allowed the user we've exposed a whole bunch of you know extra fancy switches to them without having to make a complicated UI in fact this also scales infinitely because with the simple text editor you could search for a particular argument or or value rather you could search for a particular variable and do whatever you need to do however if you want to do something a little bit more complex than assignment that's when things start to get a little bit more tricky and when you're making the you know or rather when you're trying to make the world's most hackable browser now we start to think hmm maybe we would be limiting ourselves a little bit if all we could do was was set the values of variables that might be a little bit tricky so let's let's talk about it what is the radical approach so we had a couple constraints that we discussed that were based on the models of typical applications now because we're you know in a very atypical application our constraints are significantly different so what if we have a compiler right what if we have the ability to compile any form within our our compiled program so next could come with a compiler right why not it's not too hard to do that in this okay fair enough so the next thing is well what if we trusted the user right what if we gave them the power the responsibility to evaluate any arbitrary list what would that be like what could they do and what if we didn't have to invent a new syntax I mean lots of people have already spent a lot of time perfecting syntaxes and thinking about it why do we have to invent yet another one why can't we just reuse the existing work of someone else well maybe we can so let's kind of go through these points a little bit more in depth so the first kind of sub point is about using list syntax so rather than coming up with a whole new syntax let's just use list syntax right because list syntax allows you to represent any arbitrary data and any arbitrary program so we've covered everything that we could possibly want to do right if we want to save data or I mean technically everything is data or code we can do that no problem we don't have to add any anything and additionally one of our kind of desires of making a DSL is incredibly simple syntax and I don't think it gets any simpler than list so what is really list syntax like well it's just you know a bunch of s expressions and an s expression is just an atom and then you can think of it as just a form a form is composed of atoms or other s expressions wrapped in parenz so this is kind of another recursive definition but it's it's it's only two rules and just to clarify something an atom is you can think of it there's a couple of things that can be atoms but simplified for your for your thoughts think okay we have maybe a string maybe we have an integer maybe we have a symbol these can be atoms and they can be nested infinitely with expressions so we have a simple syntax right so the next step is a compiler and I'll kind of explain how this works and again purpose of this presentation is to show you our approach and help you understand it I'm simplifying a lot of things and I'm not necessarily talking with the you know the most correct list terminology for the sake of effective communication so you can think of it like this we have a list program running and then we just take a snapshot of everything okay and we save that snapshot that's what compiling a list program for distribution is we start with the compiler we go compile this compile this compile this compile this compile this compiles a whole bunch of things and then we save the whole world including the compiler in a specific file so now we have a list compiler this is pretty standard okay there are ways to not have the compiler in the list image there are ways to remove things there's tree shakers we're not gonna cover that but suffice is safe for next we include the compiler in our list image because we get it for free for no extra effort on our part so the final point and this is the most difficult one to swallow right you're trying to make a secure application or trying to make a reliable X application remember one that's not gonna suffer from so many potential exploits right you don't want to open up a huge gigantic surface area I get it but in next we trust user input it's their computer they're allowed to do what they want to do with it if they're gonna do all sorts of mistakes that's fine ultimately we give them the power of course they could choose to do nothing with their user input files and you know reduce their risk or they could do whatever they want they could copy and paste blindly without examining snippets of code from the internet nonetheless let's let's just consider that for our application we trust user input and let's see what happens you know what did we gain from all of this so every single configuration file in next is literally just Lisp everything and you can embed any arbitrary list code you want and any configuration file so in the right hand well emacs window regular person frame we have an example of a configuration file so rather than just allowing the user to set arbitrary keys and values we let the user write literally anything so here I have a sample where I've defined a command this is a Lisp expression that we could have also written to next and now I can run the command hello and it'll prompt me for something and I'll print it right this wouldn't be possible if it was just an I and I file we can't anticipate all the potential use cases for next we cannot anticipate how someone is gonna need to mold it to their workflow so every single literally every single configuration file is Lisp and that extends to everything so I provided here a little sample from our bookmarks our bookmarks file is just Lisp again if you wanted to you could make a macro that does all sorts of expansion and generates a whole bunch of bookmarks you could do that you could have another macro that fetches a whole bunch of URLs from the internet and populates your bookmarks so let's go a little bit more practical beyond just the initialization file so the initialization file is an example of someplace so the initialization files the init out this file here on the right the user configuration so the init file is an example of something that typically would have been an I and I file that's a pretty pretty common use case people can understand that right they have a emacs init file where they can arbitrate execute arbitrary Lisp that's fine that's fine but there are use cases outside of the init file for example we have a mode in next called auto mode and auto mode it's effectively a predicate that enables and disables different modes for different URLs or hosts or whatever because we don't know how the user wants to enable and disable modes we allow arbitrary list and this sample that I provided here we have you know tomato cucumber salmon these are exact matches so when we visit the page cucumber it's going to enable dark mode the wikipedia page on cucumbers but there are more cases so a great example of this is actually we had one user who wanted to enable proxy mode for onion addresses right because we didn't limit them because we didn't say to them you have to follow this format or whatever they have provided they made their own predicate they used regex in this case so match regex and then they said then then start my proxy and then they can do all sorts of you know things like disabling force HTTPS or whatever so that's the power of allowing the user to execute arbitrary Lisp in files outside of just the initialization file so and I wrote this here to emphasize the user can write any arbitrary predicate using the complete list language amazing at least I think it's amazing so what's the whole point number one we didn't have to write a compiler we use the compiler SPCL specifically it's already fast it's secure it's efficient secondly or maybe thirdly our users don't don't find themselves hitting a dead end with a particular DSL because of something we didn't anticipate they can use all of Lisp to do whatever they want to do whenever they want to do wherever they want to so I don't know I think that's kind of kind of important personally when you again when you're trying to make a you know the most hackable browser out there anyways the next thing is we don't invent a new language that the user has to learn you know they may already be familiar with S expressions another Lisp guile scheme something like that it's a simple syntax it's the same thing it takes minutes to learn and it's incredibly powerful and finally and most importantly for our context the users are fully in control of their system so I hope that you know that was useful for you I hope you kind of understand why we didn't try to reinvent the wheel why we decided okay why make another DSL when we can just use this amazing wonderful compiler already built into our system maybe it makes sense for you to maybe you can use a a general purpose language as well for your configuration or for your data store or whatever because you saw we're using it also for our bookmarks we're using it also for our auto mode rules all sorts of actually even our history is a is a Lisp file anyways the point is hopefully that's interesting to you hopefully it gives you some ideas I hope you enjoyed thanks for listening thank you so much for your attention and for your time and I will catch you later bye bye
we designed our DSLs for the Nix browser. I'll techniques in your own applications to make them how you can use DSLs in your program to make them better for your users the question is what is a configuration files things that the user is what we're trying to do next is a reality is that compiler bundled with our application this would application our constraints are significantly and for your time and I will catch you later bye