Bridging the stepping stones: using pieces of NixOS without full commitment
Raskin, Michael
The talk explains use of NixOS code as a library instead of a framework: the present, the possible even better future, and some of the payoff. Nix package manager has a lot of useful properties, and NixOS permits to expand the use of such properties beyond just installing packages. However, while Nixpkgs behaves mostly like a library, with overrides sometimes used to change even the fundamental assumptions if these are not used by some packages (see, e.g., pkgsMusl), NixOS is typically perceived closer to a framework. Using NixOS service management means committing to the module system (but Nixpkgs overrides are still useful), nonatomic /etc switch, systemd, NixOS driver management, etc. This creates a leap of faith, as installing Nix side-by-side breaks only storage quotas, but installing NixOS breaks everything; and leads to some duplication with nix-darwin and similar projects. In the talk I will tell what and how to reuse from NixOS now, what NixOS changes could simplify use of NixOS as a shared knowledge collection about running services between different projects with different commitment level, and how a bit of commitment to dumping the core assumptions turns some features from a weird dream into table stakes.
 concludes our introduction and we'll be moving into our first talk which is bridging the stepping stones using pieces of NixOS without full commitment. So our speaker for this talk is going to be Michael Raskin and an interesting thing about Michael is you may have a hard time finding his GitHub account because it is just completely just like a random like hex dump of just random hex bytes so I think I believe it's like 7c6f434c and yeah I always have a really hard time trying to figure out okay what is his like GitHub again because it's like just random. Okay a little background about Michael is that he's one of the few people who stopped using mainline NixOS but actually still uses Nix and Nix packages kernels app like for more than 10 years I believe and he moved to NixOS from Linux from scratch and he was actually using a separate union FS slice for each package in that distribution and he's a post doc in computer science I believe it's theoretical to apply computer science and he started using NixOS in 2007. Okay I believe Michael cannot take it away. Hello I'm Michael Raskin and I will tell you about bridging the stepping stones or how to use pieces of NixOS without full commitment to the full NixOS and without lips of faith. So what pieces of the ecosystem you are expected to use? Well first Nix package manager yes you should use it and it leaves well side by side with anything so the only cost is maybe doubling the space you need for installing software. Next Nix package collection you most likely want to use it and what's the cost maybe a few gigabytes of clones lying around and standard environments build which you would build in a slightly more space efficient way I guess and then there is NixOS operating system distribution well maybe you also want to use that after all what's the cost only changing all of your habits about how to manage an operating system installation. Of course NixOS has a lot of nice features and many of them are listed on the home page but basically it boils down to three bullet points. Your system is Nix package and you happen to have Nix around 2.0. If you boot up to stage 2.0 you are very likely to boot successfully and into a consistent state and you have a declarative config which is just a single expression which gets instantiated from zero to complete and can be versioned and whatever. These are really cool features so now you might be asking yourself can you now finally safely experiment with all the cool stuff like in-game systems, operating system kernels and services which you can override to your liking and also can you finally escape all these complicated interactions between the things that get installed imperatively. Unfortunately the situation is a bit more complicated. Well first of all NixOS hardcodes quite a few things. First of all it is written around a Linux kernel and systemd as the init system and then the configuration of NixOS, this declarative thing is done by a model system which is nice for simple things and propagates your preferences across the configuration correctly. But when you do complicated things you notice that there are a lot of moving parts and there is a global namespace and they sometimes touch the same place in the namespace. Of course this gets resolved automatically but the interaction complexity is back here and also when you want to play with overriding services it turns out that models are less overriding than NixPackages packages. And of course NixOS hardcodes how you describe the core of your system like you have to use this NixOS specific DSL to describe which file systems to mount on boot. On the one hand it looks like not a lot of hard coding because what distribution doesn't hard code such things. On the other hand it turns out that inside of all these are trapped some less opinionated things like configuration generators for multiple demons and also start flag knowledge for these demons. And as an illustration that indeed there is less opinionated code that is kind of trapped inside, this knowledge is duplicated in NixDarvin even though for a different service management system and HomeManager which sometimes has a duplication of functionality with NixOS with slightly different approaches in a confusing way and of course Nix process management which you will hear about later today. Of course one can also ask is this code really trapped? I invoke the law of headlines ending in the question mark to hear the answer. Well it is complicated. There is a strategy how to use all this code outside the mainline NixOS installation. You just evaluate NixOS as a function as a Nix expression with configuration which is be minimal beyond minimal. It is not enough to build a bootable NixOS or maybe even a container. It only talks about the service itself and then you grab the parts you care about like most likely the configuration files in the ATC and also a nice part of NixOS system-day union generation functionality is possibility to export to the contents of exact stuff or to a runnable script. Thanks for that it's really useful in many cases. By the way I use both things I have described here in on my own system to grab CupsRunning and to get XOR configuration. So currently all of these functionality I use is online which allows you to grab service script by service name and NixOS configuration and also ATC file or a bunch of files in ATC. And all of this is online and has been online for some time. So what are the implications and limitations of these approaches? I claim that the main value of NixOS which would take the most time to replicate even based on Nix is a large database of configuration generator for many many demos and programs and many services are actually already reusable which is nice if you know how to do it and you have a use case. Of course there are some catches you need to pay attention because for example many services are configured not just under their namespace but all over the place and also some services are too complicated for a working runner script to be generated automatically by the generic code. You might be able to grab the parts of the unit and slap together the correct runner script on a case-by-case basis and then of course some services have configs that do not go to slash DC and also do not get an option name for the content. They just get reference inside the service starter unit and they are annoying to grab. So you see that I said that I can replace the NixOS boot script so not to run the full NixOS but still use the services but I said something about the model system and what would I use instead. I would use something mimicking Nix-Picogas overlays so make extensible all the things and define a small core system and then the core system adapts by reading some parameters from self like list of services which might be empty by default and in overlay lays you overwrite whatever you want to reconfigure and of course you want to put as much as possible into this ether set and not into let instances because you want everything to be inspectable and maybe even modifiable. On the other hand I can say that what if not the model system is the wrong question. The other question is can we make it not matter. The question is that services could be like packages they could be packages with parameters and they could provide rich pass-through and they could request and inspect other service instances and then its users choice how to ensure that everything gets passed and that every service gets as parameter correctly configured instance of dependencies and of course model system would be the default would be the only thing supported by mainlining XOS but it would be easier to extract it and replace it and it would be a well defined top-level layer with clean separation of responsibilities and another thing that makes us kind of hardcodes is the bootloader handling because of course Nick says assumes that it's the only thing that could want to configure bootloader based on system generations managed by Nick and naturally it assumes that all the layout is the Nick's as layout which leads to some kind of unfortunate things some unmodularity like that the single Perl script generating the group entries must know what how Nick's handles Zen and stuff like that and all of this means that if you have a different system which creates system generations it won't probably be compatible with Nick's OS unless I take everything that Nick's OS does exactly and that's why I'm actually too lazy to do a boot with Nick's OS because I would need to integrate the two lines of bootloader narration so if I dream what I would dream about I would like to see Nick's OS services as Nick's like package collection with services configured by argument of passing and argument overrides and maybe overlays I would love to see model system then on top of that as one of the ways to connect things even if the main one and then I would love to see multiple independence options for core like for boot scripts and stuff arise while still sharing the service database in a efficient way and then to make it do all booting such options more convenient between each other maybe Nick's OS bootloader generator would collect bootloader configuration snippets from each system generation of course that means that you need to provide snippets at once for all the bootloaders you theoretically support and probably you would want to fail unless a special force flag is passed if booted system does not support the loader you are trying to configure for the new system but still it could be a step forward and if I'm dreaming anyway I would like Nick's OS to gain support for atomic slash etc switching it's not completely infeasible I do use it on my system which is kind of similar to Nick's OS and it's a sim link into store and so one sim link change and that's it I promise to talk about a bit about why did I even bother replacing the Nick's OS boot script well first of all I like that my virtual terminals are not owned by system D which makes it easier to do my custom tricks around launching XORC and also it makes it easier to write stuff like power of command is privileged not by password by buy a check of physical presence sorry no screencast because it's very annoying to use XORC screencasting software with a lot of virtual terminal switches you know and then my system is managed by custom return custom command list them and mainly and for example it integrates automatic and is jail rotpers so everything I want can be run in jail because I don't know nobody runs browsers outside containers in 2020 right and of course only handpicked things have sound access which means that my user doesn't actually have sound access only the things that are run in jails which are specifically granted sound and then my own boot mounts are a mess which is for me easier to manage as a straightforward shell script then inside Nick's DSL and maybe contrary to its assumptions and then just for fun and ease of debugging I have full versions of everything I have any trim FS included there so LVM 2 means full LVM 2 yes we full glip see dependency yes it takes space but it only takes space and in memory until I actually switch to the main system so who cares and I like to have my services started via explicitly specified nice traceable scripts so if something goes wrong it's easier to debug and what you can take out of this if you want commitment neither to Nick's as approach nor to my approach well you can take rotpers for pieces extraction out of Nick's as I mentioned and then a roper to create an isolated debuff session instead a container for something that wants the debuff session but doesn't deserve access to your main one if you even have the main debuff session because I don't maybe you might be interested in Firefox empty profile builder based on launching Firefox once in Xdami and then there is some code for converting true type fonts to bitmap format for Linux console because monospace phones often come with very good hinting and font forge is pretty good at converting fonts between formats thanks for your attention are the questions okay hello everyone that was Michael Raskin bridging stepping stones using pieces of nix of us without full commitment and I did get news from the channel that we did start a bit earlier than expected so I think it was supposed to start at 715 so I am sorry about that that may make it such that we don't have a lot of questions or on the Q&A portion let me look into the pad and see if we have any questions Michael yeah I do not actually see any questions I will also manually check the channel next con Q&A if anyone has a question you can ask there yeah I do not see a question yeah I am sorry about that Michael yeah happens whatever on the other hand maybe it means that the talk was clear enough yeah that is also a possibility so I would say you could um I would direct you to like the breakout room which is bridging steps bridging dash steps and I'm sure that if people watch your talk later or like go back into the broadcast they can see that and they can ask you questions from there I think it's Puck says that two questions appear just now I will just read them from the chat Michael so we have one from SRHB and she said is abstracting the service generation worth it in NixOS proper maybe Michael I believe you're muted I'm sorry let me let me work for the word proper I worth it in NixOS proper maybe well I mean I don't know exactly how much you want to say that NixOS is the default set of options I believe that you want well I believe it's valuable to have a service database that we can reuse and that we can work on collectively the regardless of our specific choices I don't know maybe what subset of the options should be called NixOS proper and what subsets of options should be called using the services from the NixOS database on a different system I think that can be discussed once we have that call once we have things to compare I'm not sure like well it's okay if NixOS is an opinionated set of choices in that regard it's just a question of what we want to allow next to NixOS and then we can decide what to reintegrate okay I asked that was pretty clear to me actually so you have a question from Nixer 86 and they say it was not obvious I'm gonna talk or I missed it but do you still use system D? Well it turns out that I ended up not using system D at all and I'm not against using Nix system D in some container or something but just never needed it enough I definitely don't want to have system D own anything on my top level namespace and yeah well as I don't run it in smaller namespaces and I don't want to the top level I ended up without using system D at all well of course I have everything linked against system D libraries but that's another story okay we have another question from Hyperfect oh I know you hey how does the usability of the module system and your extensibility mechanism compare in practice? Well in a sense it's hard to say because for my needs this extensibility via overlays has better transparency and easier debugability for my specific use case but that's another story in a sense I never tried to use model system to have multiple people make multiple people build vastly different configurations and I cannot exclude for many people it is a very good choice to use model system because it propagates your preferences across the system well well at least as long as you are inside the expectations so it's I don't know I didn't try to see for who for who I believe that there are situations where for some people and for some settings different approaches are more usable more debugable more inspectable than the model system and as I said I am not sure it might be that it's still the best choice for the mainline most popular version of NixOS okay we have a question from Ryan TM he says it sounds like you're suggesting to move modules to packages is maybe the opposite of Elko's proposal yesterday what do you think haha well I think the following well first of all as of me not addressing at all any of Elko's points kind of sorry but world of peace can confirm that my recording has been finished before I had any chance to see Elko's talk yesterday and on the other hand yeah I think the following I do believe that well when I looked yesterday for example at the slides of Elko's talk I was a bit worried then you say okay and you can overwrite the configs of the module you extend and then you I saw the config overwrite and it was just global overwrite and so you know it's complicated so I thought that okay so what are the scoping rules I think that for putting modules inside Nix things like scoping rules not to stop being you know purely functional programming language which gives you at least an option to have full referential transparency not only formally but also really and to avoid global namespaces if you want to avoid the global namespaces and so on I think Nix being a purely functional programming language was very valuable from the very beginning and is still valuable now and so I believe yeah as I said I believe that module system is a good thing at some levels but I believe there are layers that are better done as pure functions and fully extend and fully expectable plain data structures okay I'm looking at the pad again for more questions I do not see any I think what we did I very cast what's the what's my in it well as for my in its system well it turns out that if you have a small small enough desktop system and you actually want to see and control what is there and how it runs it turns out that if you have just a few things and you immediately observe whether they are working or not and you restart them for unrelated reasons anyway like I restart my local bin to when I change the network I'm connected to or just so that it reconfigures itself a bit I it turns out that in this situation I don't even need and it's a real in its system in a sense so I do have P1 obviously and P1 is intended as an in it system and it is static a neat as a neat from suck less I don't like everything here a suck less does in well in the sense I don't find useful for me all the tools suck less produce but as a neat I looked at it and it does exactly what I wanted it to do and so yeah my in it is a sin it and then I have as I said I have this jumper demon which manages my system in particular it launches the few demons I want to be to have running on my system so and that's it I'm all this and somehow it turns out that on a simple enough system with enough RAM and they happen to have enough RAM you know cups doesn't crash being doesn't crash on its own XORC doesn't crash on its own and then what is even there for a true system supervisor to monitor okay great whoop you another question is there something else apart from NS jail that I could check to learn more about your jail setup it sounds like you're going to prison well I mean I assume I hope at some point yeah I will at some point publish the slides I guess and I hope that is recorded will be there there is well most of my setup because of course there are some small things which are very local like some of the specific host names and stuff which might not be fully included but generally you know I have most of my setup online in my github account as long as and yeah well you can look at this but basically it's a lot well basically it's just using NS jail and then there is a lot of things that you need to tell to an as jail to comfortably rob some application I need to tell an as jail what directories to provide to the application what of them to provide read-only or read-write I also run an as jail under a specific sub a specific user ID which is unique for every application during a single session and stuff like that and I said the environment variables of course because of course you need to set up stuff but basically well it's a piece of code to generate a ton of for flex to an as jail I would say okay great answer actually um I don't see any more questions in the Q&A portion so I believe that would lead us into the closing thank you so much for being here Michael it seems that you get to have a very basically an extended Q&A portion because we started early yeah thank you yep nice having you
 concludes our introduction and we'll be moving into our first talk which is  bridging the stepping stones using pieces of NixOS without full commitment.  So our speaker for this talk is going to be Michael Raskin and an interesting  thing about Michael is you may have a hard time finding his GitHub account  because it is just completely just like a random like hex dump of just random  hex bytes so I think I believe it's like 7c6f434c and yeah I always have a  really hard time trying to figure out okay what is his like GitHub again  because it's like just random. Okay a little background about Michael is that  he's one of the few people who stopped using mainline NixOS but actually still  uses Nix and Nix packages kernels app like for more than 10 years I believe  and he moved to NixOS from Linux from scratch and he was actually using a  separate union FS slice for each package in that distribution and he's a post  doc in computer science I believe it's theoretical to apply computer science  and he started using NixOS in 2007. Okay I believe Michael cannot take it away.  Hello I'm Michael Raskin and I will tell you about bridging the stepping stones  or how to use pieces of NixOS without full commitment to the full NixOS and  without lips of faith. So what pieces of the ecosystem you are expected to use?  Well first Nix package manager yes you should use it and it leaves well side  by side with anything so the only cost is maybe doubling the space you need for  installing software. Next Nix package collection you most likely want to use  it and what's the cost maybe a few gigabytes of clones lying around and  standard environments build which you would build in a slightly more  space efficient way I guess and then there is NixOS operating system  distribution well maybe you also want to use that after all what's the cost only  changing all of your habits about how to manage an operating system installation.  Of course NixOS has a lot of nice features and many of them are listed on  the home page but basically it boils down to three bullet points. Your system is  Nix package and you happen to have Nix around 2.0. If you boot up to stage 2.0  you are very likely to boot successfully and into a consistent state and  you have a declarative config which is just a single expression which gets  instantiated from zero to complete and can be versioned and whatever. These are  really cool features so now you might be asking yourself can you now finally  safely experiment with all the cool stuff like in-game systems, operating system  kernels and services which you can override to your liking and also can  you finally escape all these complicated interactions between the things  that get installed imperatively. Unfortunately the situation is a bit  more complicated. Well first of all NixOS hardcodes quite a few things. First of  all it is written around a Linux kernel and systemd as the init system and then  the configuration of NixOS, this declarative thing is done by a model  system which is nice for simple things and propagates your preferences across  the configuration correctly. But when you do complicated things you notice that  there are a lot of moving parts and there is a global namespace and they  sometimes touch the same place in the namespace. Of course this gets resolved  automatically but the interaction complexity is back here and also when  you want to play with overriding services it turns out that models are  less overriding than NixPackages packages. And of course NixOS hardcodes how  you describe the core of your system like you have to use this NixOS specific  DSL to describe which file systems to mount on boot. On the one hand it looks  like not a lot of hard coding because what distribution doesn't hard code  such things. On the other hand it turns out that inside of all these are  trapped some less opinionated things like configuration generators for  multiple demons and also start flag knowledge for these demons. And as an  illustration that indeed there is less opinionated code that is kind of  trapped inside, this knowledge is duplicated in NixDarvin even though  for a different service management system and HomeManager which sometimes has  a duplication of functionality with NixOS with slightly different approaches in  a confusing way and of course Nix process management which you will hear  about later today. Of course one can also ask is this code really trapped? I invoke  the law of headlines ending in the question mark to hear the answer. Well  it is complicated. There is a strategy how to use all this code outside the  mainline NixOS installation. You just evaluate NixOS as a function as a Nix  expression with configuration which is be minimal beyond minimal. It is not  enough to build a bootable NixOS or maybe even a container. It only talks  about the service itself and then you grab the parts you care about like most  likely the configuration files in the ATC and also a nice part of NixOS  system-day union generation functionality is possibility to export to the  contents of exact stuff or to a runnable script. Thanks for that it's really useful  in many cases. By the way I use both things I have described here in on my  own system to grab CupsRunning and to get XOR configuration. So currently all  of these functionality I use is online which allows you to grab service  script by service name and NixOS configuration and also ATC file or a  bunch of files in ATC. And all of this is online and has been online for some  time. So what are the implications and limitations of these approaches? I  claim that the main value of NixOS which would take the most time to  replicate even based on Nix is a large database of configuration generator  for many many demos and programs and many services are actually already  reusable which is nice if you know how to do it and you have a use case. Of course  there are some catches you need to pay attention because for example many  services are configured not just under their namespace but all over the place  and also some services are too complicated for a working runner script to  be generated automatically by the generic code. You might be able to grab the  parts of the unit and slap together the correct runner script on a case-by-case  basis and then of course some services have configs that do not go to slash DC  and also do not get an option name for the content. They just get reference  inside the service starter unit and they are annoying to grab. So you see that I  said that I can replace the NixOS boot script so not to run the full NixOS but  still use the services but I said something about the model system and  what would I use instead. I would use something mimicking Nix-Picogas overlays  so make extensible all the things and define a small core system and then the  core system adapts by reading some parameters from self like list of  services which might be empty by default and in overlay lays you overwrite  whatever you want to reconfigure and of course you want to put as much as  possible into this ether set and not into let instances because you want  everything to be inspectable and maybe even modifiable. On the other hand I can  say that what if not the model system is the wrong question. The other question is  can we make it not matter. The question is that services could be like packages  they could be packages with parameters and they could provide rich pass-through  and they could request and inspect other service instances and then its users  choice how to ensure that everything gets passed and that every service gets as  parameter correctly configured instance of dependencies and of course model  system would be the default would be the only thing supported by mainlining  XOS but it would be easier to extract it and replace it and it would be a well  defined top-level layer with clean separation of responsibilities and another  thing that makes us kind of hardcodes is the bootloader handling because of  course Nick says assumes that it's the only thing that could want to configure  bootloader based on system generations managed by Nick and naturally it assumes  that all the layout is the Nick's as layout which leads to some kind of  unfortunate things some unmodularity like that the single  Perl script generating the group entries must know what how Nick's handles Zen  and stuff like that and all of this means that if you have a different system  which creates system generations it won't probably be compatible with Nick's  OS unless I take everything that Nick's OS does exactly and that's why I'm  actually too lazy to do a boot with Nick's OS because I would need to integrate  the two lines of bootloader narration so if I dream what I would dream about I  would like to see Nick's OS services as Nick's like package collection with  services configured by argument of passing and argument overrides and maybe  overlays I would love to see model system then on top of that as one of the ways  to connect things even if the main one and then I would love to see multiple  independence options for core like for boot scripts and stuff arise while  still sharing the service database in a efficient way and then to make it do all  booting such options more convenient between each other maybe Nick's OS  bootloader generator would collect bootloader configuration snippets from  each system generation of course that means that you need to provide snippets  at once for all the bootloaders you theoretically support and probably you  would want to fail unless a special force flag is passed if booted system does  not support the loader you are trying to configure for the new system but still  it could be a step forward and if I'm dreaming anyway I would like Nick's OS  to gain support for atomic slash etc switching it's not completely infeasible  I do use it on my system which is kind of similar to Nick's OS and it's a sim  link into store and so one sim link change and that's it I promise to talk  about a bit about why did I even bother replacing the Nick's OS boot script  well first of all I like that my virtual terminals are not owned by system D  which makes it easier to do my custom tricks around launching XORC and also  it makes it easier to write stuff like power of command is privileged not by  password by buy a check of physical presence sorry no screencast because  it's very annoying to use XORC screencasting software with a lot of  virtual terminal switches you know and then my system is managed by custom  return custom command list them and mainly and for example it integrates  automatic and is jail rotpers so everything I want can be run in jail  because I don't know nobody runs browsers outside containers in 2020  right and of course only handpicked things have sound access which means  that my user doesn't actually have sound access only the things that are run in  jails which are specifically granted sound and then my own boot mounts are a  mess which is for me easier to manage as a straightforward shell script then  inside Nick's DSL and maybe contrary to its assumptions and then just for fun  and ease of debugging I have full versions of everything I have any trim  FS included there so LVM 2 means full LVM 2 yes we full glip see dependency  yes it takes space but it only takes space and in memory until I actually  switch to the main system so who cares and I like to have my services started  via explicitly specified nice traceable scripts so if something goes wrong it's  easier to debug and what you can take out of this if you want commitment neither  to Nick's as approach nor to my approach well you can take rotpers for pieces  extraction out of Nick's as I mentioned and then a roper to create an isolated  debuff session instead a container for something that wants the debuff session  but doesn't deserve access to your main one if you even have the main debuff  session because I don't maybe you might be interested in Firefox empty profile  builder based on launching Firefox once in Xdami and then there is some code for  converting true type fonts to bitmap format for Linux console because  monospace phones often come with very good hinting and font forge is pretty  good at converting fonts between formats thanks for your attention are the questions  okay hello everyone that was Michael Raskin bridging stepping stones using  pieces of nix of us without full commitment and I did get news from the  channel that we did start a bit earlier than expected so I think it was supposed  to start at 715 so I am sorry about that that may make it such that we don't  have a lot of questions or on the Q&A portion let me look into the pad and see  if we have any questions Michael yeah I do not actually see any questions  I will also manually check the channel next con Q&A if anyone has a question you  can ask there yeah I do not see a question yeah I am sorry about that Michael  yeah happens whatever on the other hand maybe it means that the talk was clear  enough yeah that is also a possibility so I would say you could um I would  direct you to like the breakout room which is bridging steps bridging dash  steps and I'm sure that if people watch your talk later or like go back into  the broadcast they can see that and they can ask you questions from there  I think it's Puck says that two questions appear just now I will just read  them from the chat Michael so we have one from SRHB and she said is abstracting  the service generation worth it in NixOS proper maybe  Michael I believe you're muted I'm sorry let me let me work for the word  proper I worth it in NixOS proper maybe well I mean I don't know exactly how much  you want to say that NixOS is the default set of options I believe that you  want well I believe it's valuable to have a service database that we can reuse  and that we can work on collectively the regardless of our specific choices I  don't know maybe what subset of the options should be called NixOS proper  and what subsets of options should be called using the services from the NixOS  database on a different system I think that can be discussed once we have that  call once we have things to compare I'm not sure like well it's okay if NixOS  is an opinionated set of choices in that regard it's just a question of what we  want to allow next to NixOS and then we can decide what to reintegrate  okay I asked that was pretty clear to me actually so you have a question from  Nixer 86 and they say it was not obvious I'm gonna talk or I missed it but do you  still use system D? Well it turns out that I ended up not using system D at all  and I'm not against using Nix system D in some container or something but just  never needed it enough I definitely don't want to have system D own anything on  my top level namespace and yeah well as I don't run it in smaller namespaces  and I don't want to the top level I ended up without using system D at all  well of course I have everything linked against system D libraries but that's  another story okay we have another question from Hyperfect oh I know you  hey how does the usability of the module system and your extensibility  mechanism compare in practice? Well in a sense it's hard to say because for my  needs this extensibility via overlays has better transparency and easier  debugability for my specific use case but that's another story in a sense I  never tried to use model system to have multiple people  make multiple people build vastly different configurations and I cannot  exclude for many people it is a very good choice to use model system because it  propagates your preferences across the system well well at least as long as  you are inside the expectations so it's I don't know I didn't try to see for who  for who I believe that there are situations where for some people and for  some settings different approaches are more usable more debugable more  inspectable than the model system and as I said I am not sure it might be that  it's still the best choice for the mainline most popular version of  NixOS okay we have a question from Ryan TM he says it sounds like you're  suggesting to move modules to packages is maybe the opposite of Elko's proposal  yesterday what do you think haha well I think the following well first of all as  of me not addressing at all any of Elko's points kind of sorry but world of  peace can confirm that my recording has been finished before I had any chance to  see Elko's talk yesterday and on the other hand yeah I think the following I  do believe that well when I looked yesterday for example at the slides of  Elko's talk I was a bit worried then you say okay and you can overwrite the  configs of the module you extend and then you I saw the config overwrite and it was  just global overwrite and so you know it's complicated so I thought that okay  so what are the scoping rules I think that for putting modules inside Nix  things like scoping rules not to stop being you know purely functional  programming language which gives you at least an option to have full referential  transparency not only formally but also really and to avoid global namespaces if  you want to avoid the global namespaces and so on I think Nix being a purely  functional programming language was very valuable from the very beginning and  is still valuable now and so I believe yeah as I said I believe that module  system is a good thing at some levels but I believe there are layers that are  better done as pure functions and fully extend and fully expectable plain data  structures okay I'm looking at the pad again for more questions I do not see  any I think what we did I very cast what's the what's my in it well as for my  in its system well it turns out that if you have a small small enough desktop  system and you actually want to see and control what is there and how it runs it  turns out that if you have just a few things and you immediately observe whether  they are working or not and you restart them for unrelated reasons anyway like I  restart my local bin to when I change the network I'm connected to or just so  that it reconfigures itself a bit I it turns out that in this situation I  don't even need and it's a real in its system in a sense so I do have P1  obviously and P1 is intended as an in it system and it is static a neat as a  neat from suck less I don't like everything here a suck less does in well  in the sense I don't find useful for me all the tools suck less produce but as a  neat I looked at it and it does exactly what I wanted it to do and so yeah my  in it is a sin it and then I have as I said I have this jumper demon which  manages my system in particular it launches the few demons I want to be  to have running on my system so and that's it I'm all this and somehow it  turns out that on a simple enough system with enough RAM and they happen to have  enough RAM you know cups doesn't crash being doesn't crash on its own  XORC doesn't crash on its own and then what is even there for a true system  supervisor to monitor okay great whoop you another question is there something  else apart from NS jail that I could check to learn more about your jail  setup it sounds like you're going to prison well I mean I assume I hope at  some point yeah I will at some point publish the slides I guess and I hope  that is recorded will be there there is well most of my setup because of  course there are some small things which are very local like some of the  specific host names and stuff which might not be fully included but generally  you know I have most of my setup online in my github account as long as and  yeah well you can look at this but basically it's a lot well basically  it's just using NS jail and then there is a lot of things that you need to tell  to an as jail to comfortably rob some application I need to tell an as jail  what directories to provide to the application what of them to provide  read-only or read-write I also run an as jail under a specific sub a specific  user ID which is unique for every application during a single session and  stuff like that and I said the environment variables of course because of  course you need to set up stuff but basically well it's a piece of code to  generate a ton of for flex to an as jail I would say okay great answer actually  um I don't see any more questions in the Q&A portion so I believe that would  lead us into the closing thank you so much for being here Michael it seems that  you get to have a very basically an extended Q&A portion because we started  early yeah thank you yep nice having you
NixCon_2020_Vid_10_0_131.png NixCon_2020_Vid_10_132_383.png NixCon_2020_Vid_10_384_455.png NixCon_2020_Vid_10_456_899.png NixCon_2020_Vid_10_900_1127.png NixCon_2020_Vid_10_1128_1235.png NixCon_2020_Vid_10_1236_1871.png NixCon_2020_Vid_10_1872_15671.png NixCon_2020_Vid_10_15672_17987.png NixCon_2020_Vid_10_17988_22919.png NixCon_2020_Vid_10_22920_23243.png NixCon_2020_Vid_10_23244_23291.png NixCon_2020_Vid_10_23292_23435.png NixCon_2020_Vid_10_23436_23795.png NixCon_2020_Vid_10_23796_23867.png NixCon_2020_Vid_10_23868_24179.png NixCon_2020_Vid_10_24180_24227.png NixCon_2020_Vid_10_24228_24299.png NixCon_2020_Vid_10_24300_24431.png NixCon_2020_Vid_10_24432_24755.png NixCon_2020_Vid_10_24756_27503.png NixCon_2020_Vid_10_27504_27563.png NixCon_2020_Vid_10_27564_28751.png NixCon_2020_Vid_10_28752_28835.png NixCon_2020_Vid_10_28836_28895.png NixCon_2020_Vid_10_28896_30983.png NixCon_2020_Vid_10_30984_31139.png NixCon_2020_Vid_10_31140_31247.png NixCon_2020_Vid_10_31248_34163.png NixCon_2020_Vid_10_34164_37511.png NixCon_2020_Vid_10_37512_39851.png NixCon_2020_Vid_10_39852_40391.png NixCon_2020_Vid_10_40392_40413.png
concludes our introduction and we'll be moving into our first talk which is bridging the stepping stones using pieces of NixOS without full commitment. So our speaker for this talk is going to be Michael Raskin and an interesting thing about Michael is you may have a hard time finding his GitHub account because it is just completely just like a random like hex dump of just random hex bytes so I think I believe it's like 7c6f434c and yeah I always have a really hard time trying to figure out okay what is his like GitHub again because it's like just random. Okay a little background about Michael is that he's one of the few people who stopped using mainline NixOS but actually still uses Nix and Nix packages kernels app like for more than 10 years I believe and he moved to NixOS from Linux from scratch and he was actually using a separate union FS slice for each package in that distribution and he's a post doc in computer science I believe it's theoretical to apply computer science and he started using NixOS in 2007. Okay I believe Michael cannot take it away. Hello I'm Michael Raskin and I will tell you about bridging the stepping stones or how to use pieces of NixOS without full commitment to the full NixOS and without lips of faith. So what pieces of the ecosystem you are expected to use? Well first Nix package manager yes you should use it and it leaves well side by side with anything so the only cost is maybe doubling the space you need for installing software. Next Nix package collection you most likely want to use it and what's the cost maybe a few gigabytes of clones lying around and standard environments build which you would build in a slightly more space efficient way I guess and then there is NixOS operating system distribution well maybe you also want to use that after all what's the cost only changing all of your habits about how to manage an operating system installation. Of course NixOS has a lot of nice features and many of them are listed on the home page but basically it boils down to three bullet points. Your system is Nix package and you happen to have Nix around 2.0. If you boot up to stage 2.0 you are very likely to boot successfully and into a consistent state and you have a declarative config which is just a single expression which gets instantiated from zero to complete and can be versioned and whatever. These are really cool features so now you might be asking yourself can you now finally safely experiment with all the cool stuff like in-game systems, operating system kernels and services which you can override to your liking and also can you finally escape all these complicated interactions between the things that get installed imperatively. Unfortunately the situation is a bit more complicated. Well first of all NixOS hardcodes quite a few things. First of all it is written around a Linux kernel and systemd as the init system and then the configuration of NixOS, this declarative thing is done by a model system which is nice for simple things and propagates your preferences across the configuration correctly. But when you do complicated things you notice that there are a lot of moving parts and there is a global namespace and they sometimes touch the same place in the namespace. Of course this gets resolved automatically but the interaction complexity is back here and also when you want to play with overriding services it turns out that models are less overriding than NixPackages packages. And of course NixOS hardcodes how you describe the core of your system like you have to use this NixOS specific DSL to describe which file systems to mount on boot. On the one hand it looks like not a lot of hard coding because what distribution doesn't hard code such things. On the other hand it turns out that inside of all these are trapped some less opinionated things like configuration generators for multiple demons and also start flag knowledge for these demons. And as an illustration that indeed there is less opinionated code that is kind of trapped inside, this knowledge is duplicated in NixDarvin even though for a different service management system and HomeManager which sometimes has a duplication of functionality with NixOS with slightly different approaches in a confusing way and of course Nix process management which you will hear about later today. Of course one can also ask is this code really trapped? I invoke the law of headlines ending in the question mark to hear the answer. Well it is complicated. There is a strategy how to use all this code outside the mainline NixOS installation. You just evaluate NixOS as a function as a Nix expression with configuration which is be minimal beyond minimal. It is not enough to build a bootable NixOS or maybe even a container. It only talks about the service itself and then you grab the parts you care about like most likely the configuration files in the ATC and also a nice part of NixOS system-day union generation functionality is possibility to export to the contents of exact stuff or to a runnable script. Thanks for that it's really useful in many cases. By the way I use both things I have described here in on my own system to grab CupsRunning and to get XOR configuration. So currently all of these functionality I use is online which allows you to grab service script by service name and NixOS configuration and also ATC file or a bunch of files in ATC. And all of this is online and has been online for some time. So what are the implications and limitations of these approaches? I claim that the main value of NixOS which would take the most time to replicate even based on Nix is a large database of configuration generator for many many demos and programs and many services are actually already reusable which is nice if you know how to do it and you have a use case. Of course there are some catches you need to pay attention because for example many services are configured not just under their namespace but all over the place and also some services are too complicated for a working runner script to be generated automatically by the generic code. You might be able to grab the parts of the unit and slap together the correct runner script on a case-by-case basis and then of course some services have configs that do not go to slash DC and also do not get an option name for the content. They just get reference inside the service starter unit and they are annoying to grab. So you see that I said that I can replace the NixOS boot script so not to run the full NixOS but still use the services but I said something about the model system and what would I use instead. I would use something mimicking Nix-Picogas overlays so make extensible all the things and define a small core system and then the core system adapts by reading some parameters from self like list of services which might be empty by default and in overlay lays you overwrite whatever you want to reconfigure and of course you want to put as much as possible into this ether set and not into let instances because you want everything to be inspectable and maybe even modifiable. On the other hand I can say that what if not the model system is the wrong question. The other question is can we make it not matter. The question is that services could be like packages they could be packages with parameters and they could provide rich pass-through and they could request and inspect other service instances and then its users choice how to ensure that everything gets passed and that every service gets as parameter correctly configured instance of dependencies and of course model system would be the default would be the only thing supported by mainlining XOS but it would be easier to extract it and replace it and it would be a well defined top-level layer with clean separation of responsibilities and another thing that makes us kind of hardcodes is the bootloader handling because of course Nick says assumes that it's the only thing that could want to configure bootloader based on system generations managed by Nick and naturally it assumes that all the layout is the Nick's as layout which leads to some kind of unfortunate things some unmodularity like that the single Perl script generating the group entries must know what how Nick's handles Zen and stuff like that and all of this means that if you have a different system which creates system generations it won't probably be compatible with Nick's OS unless I take everything that Nick's OS does exactly and that's why I'm actually too lazy to do a boot with Nick's OS because I would need to integrate the two lines of bootloader narration so if I dream what I would dream about I would like to see Nick's OS services as Nick's like package collection with services configured by argument of passing and argument overrides and maybe overlays I would love to see model system then on top of that as one of the ways to connect things even if the main one and then I would love to see multiple independence options for core like for boot scripts and stuff arise while still sharing the service database in a efficient way and then to make it do all booting such options more convenient between each other maybe Nick's OS bootloader generator would collect bootloader configuration snippets from each system generation of course that means that you need to provide snippets at once for all the bootloaders you theoretically support and probably you would want to fail unless a special force flag is passed if booted system does not support the loader you are trying to configure for the new system but still it could be a step forward and if I'm dreaming anyway I would like Nick's OS to gain support for atomic slash etc switching it's not completely infeasible I do use it on my system which is kind of similar to Nick's OS and it's a sim link into store and so one sim link change and that's it I promise to talk about a bit about why did I even bother replacing the Nick's OS boot script well first of all I like that my virtual terminals are not owned by system D which makes it easier to do my custom tricks around launching XORC and also it makes it easier to write stuff like power of command is privileged not by password by buy a check of physical presence sorry no screencast because it's very annoying to use XORC screencasting software with a lot of virtual terminal switches you know and then my system is managed by custom return custom command list them and mainly and for example it integrates automatic and is jail rotpers so everything I want can be run in jail because I don't know nobody runs browsers outside containers in 2020 right and of course only handpicked things have sound access which means that my user doesn't actually have sound access only the things that are run in jails which are specifically granted sound and then my own boot mounts are a mess which is for me easier to manage as a straightforward shell script then inside Nick's DSL and maybe contrary to its assumptions and then just for fun and ease of debugging I have full versions of everything I have any trim FS included there so LVM 2 means full LVM 2 yes we full glip see dependency yes it takes space but it only takes space and in memory until I actually switch to the main system so who cares and I like to have my services started via explicitly specified nice traceable scripts so if something goes wrong it's easier to debug and what you can take out of this if you want commitment neither to Nick's as approach nor to my approach well you can take rotpers for pieces extraction out of Nick's as I mentioned and then a roper to create an isolated debuff session instead a container for something that wants the debuff session but doesn't deserve access to your main one if you even have the main debuff session because I don't maybe you might be interested in Firefox empty profile builder based on launching Firefox once in Xdami and then there is some code for converting true type fonts to bitmap format for Linux console because monospace phones often come with very good hinting and font forge is pretty good at converting fonts between formats thanks for your attention are the questions okay hello everyone that was Michael Raskin bridging stepping stones using pieces of nix of us without full commitment and I did get news from the channel that we did start a bit earlier than expected so I think it was supposed to start at 715 so I am sorry about that that may make it such that we don't have a lot of questions or on the Q&A portion let me look into the pad and see if we have any questions Michael yeah I do not actually see any questions I will also manually check the channel next con Q&A if anyone has a question you can ask there yeah I do not see a question yeah I am sorry about that Michael yeah happens whatever on the other hand maybe it means that the talk was clear enough yeah that is also a possibility so I would say you could um I would direct you to like the breakout room which is bridging steps bridging dash steps and I'm sure that if people watch your talk later or like go back into the broadcast they can see that and they can ask you questions from there I think it's Puck says that two questions appear just now I will just read them from the chat Michael so we have one from SRHB and she said is abstracting the service generation worth it in NixOS proper maybe Michael I believe you're muted I'm sorry let me let me work for the word proper I worth it in NixOS proper maybe well I mean I don't know exactly how much you want to say that NixOS is the default set of options I believe that you want well I believe it's valuable to have a service database that we can reuse and that we can work on collectively the regardless of our specific choices I don't know maybe what subset of the options should be called NixOS proper and what subsets of options should be called using the services from the NixOS database on a different system I think that can be discussed once we have that call once we have things to compare I'm not sure like well it's okay if NixOS is an opinionated set of choices in that regard it's just a question of what we want to allow next to NixOS and then we can decide what to reintegrate okay I asked that was pretty clear to me actually so you have a question from Nixer 86 and they say it was not obvious I'm gonna talk or I missed it but do you still use system D? Well it turns out that I ended up not using system D at all and I'm not against using Nix system D in some container or something but just never needed it enough I definitely don't want to have system D own anything on my top level namespace and yeah well as I don't run it in smaller namespaces and I don't want to the top level I ended up without using system D at all well of course I have everything linked against system D libraries but that's another story okay we have another question from Hyperfect oh I know you hey how does the usability of the module system and your extensibility mechanism compare in practice? Well in a sense it's hard to say because for my needs this extensibility via overlays has better transparency and easier debugability for my specific use case but that's another story in a sense I never tried to use model system to have multiple people make multiple people build vastly different configurations and I cannot exclude for many people it is a very good choice to use model system because it propagates your preferences across the system well well at least as long as you are inside the expectations so it's I don't know I didn't try to see for who for who I believe that there are situations where for some people and for some settings different approaches are more usable more debugable more inspectable than the model system and as I said I am not sure it might be that it's still the best choice for the mainline most popular version of NixOS okay we have a question from Ryan TM he says it sounds like you're suggesting to move modules to packages is maybe the opposite of Elko's proposal yesterday what do you think haha well I think the following well first of all as of me not addressing at all any of Elko's points kind of sorry but world of peace can confirm that my recording has been finished before I had any chance to see Elko's talk yesterday and on the other hand yeah I think the following I do believe that well when I looked yesterday for example at the slides of Elko's talk I was a bit worried then you say okay and you can overwrite the configs of the module you extend and then you I saw the config overwrite and it was just global overwrite and so you know it's complicated so I thought that okay so what are the scoping rules I think that for putting modules inside Nix things like scoping rules not to stop being you know purely functional programming language which gives you at least an option to have full referential transparency not only formally but also really and to avoid global namespaces if you want to avoid the global namespaces and so on I think Nix being a purely functional programming language was very valuable from the very beginning and is still valuable now and so I believe yeah as I said I believe that module system is a good thing at some levels but I believe there are layers that are better done as pure functions and fully extend and fully expectable plain data structures okay I'm looking at the pad again for more questions I do not see any I think what we did I very cast what's the what's my in it well as for my in its system well it turns out that if you have a small small enough desktop system and you actually want to see and control what is there and how it runs it turns out that if you have just a few things and you immediately observe whether they are working or not and you restart them for unrelated reasons anyway like I restart my local bin to when I change the network I'm connected to or just so that it reconfigures itself a bit I it turns out that in this situation I don't even need and it's a real in its system in a sense so I do have P1 obviously and P1 is intended as an in it system and it is static a neat as a neat from suck less I don't like everything here a suck less does in well in the sense I don't find useful for me all the tools suck less produce but as a neat I looked at it and it does exactly what I wanted it to do and so yeah my in it is a sin it and then I have as I said I have this jumper demon which manages my system in particular it launches the few demons I want to be to have running on my system so and that's it I'm all this and somehow it turns out that on a simple enough system with enough RAM and they happen to have enough RAM you know cups doesn't crash being doesn't crash on its own XORC doesn't crash on its own and then what is even there for a true system supervisor to monitor okay great whoop you another question is there something else apart from NS jail that I could check to learn more about your jail setup it sounds like you're going to prison well I mean I assume I hope at some point yeah I will at some point publish the slides I guess and I hope that is recorded will be there there is well most of my setup because of course there are some small things which are very local like some of the specific host names and stuff which might not be fully included but generally you know I have most of my setup online in my github account as long as and yeah well you can look at this but basically it's a lot well basically it's just using NS jail and then there is a lot of things that you need to tell to an as jail to comfortably rob some application I need to tell an as jail what directories to provide to the application what of them to provide read-only or read-write I also run an as jail under a specific sub a specific user ID which is unique for every application during a single session and stuff like that and I said the environment variables of course because of course you need to set up stuff but basically well it's a piece of code to generate a ton of for flex to an as jail I would say okay great answer actually um I don't see any more questions in the Q&A portion so I believe that would lead us into the closing thank you so much for being here Michael it seems that you get to have a very basically an extended Q&A portion because we started early yeah thank you yep nice having you
Hello I'm Michael Raskin and I will tell you pieces of NixOS without full commitment to the full NixOS and without lips of faith. So what Well first Nix package manager yes you should use has a lot of nice features and many of them are single expression which gets instantiated from systemd as the init system and then the it turns out that inside of all these are trapped opinionated code that is kind of trapped inside, this knowledge is duplicated in NixDarvin even though for a different service management system and HomeManager which sometimes has a duplication NixOS as a function as a Nix expression with the core system adapts by reading some parameters not matter. The question is that services could be like packages they could be packages with how to ensure that everything gets passed and layout which leads to some kind of unfortunate that and all of this means that if you have a different system which creates system generations so if I dream what I would dream about I would package collection with services configured by system but still it could be a step forward and assumptions and then just for fun and ease of back into the broadcast they can see that and does the usability of the module system and your example at the slides of Elko's talk I was a bit