Nix from the dark ages (without Root)
Goswami, Rohit
“Nix from the dark ages (without Root)” Rohit Goswami · Lightning Talk (5 minutes) Short comments from the trenches of High Performance Clusters on working with Nix on kernel locked-in systems without proot support.
 Okay, everyone go off for theophane or I'll be really disappointed. Okay, next up is nix from the Dark Ages without root This is by Rohit Goswami in this talk is Centralized on comments on the trenches of high-performance clusters on working with nix on kernel locked in systems without period support. So go on Good morning, I'm Rohit Goswami and I'm gonna be talking about nix in the Dark Ages without root It's a short talk. So I'm just gonna run through this quickly. So first and foremost, hello nix con This is my first nix con and I'm super excited to be here. This is me. We can you can check that out later So what's the big picture? What is going on? Well on the right? You'll see an academic cluster picture from GE research and what's the problem really? Well, there's no docker if you're very lucky or in some places you might have singularity You won't have any user space support, which means you can play around with P route and It probably runs into s or something as you can see in the lower right If you check this out later, then you probably also be able to see that these are typically quite powerful That's the whole point of using them. There's a lot of power in these clusters and there's also network file system typically luster fs or cluster fs or even in some cases nxs and There's always a resource queue Either through slum or previous torque or something like that Again, if you're lucky you're managing very you might find support for lmod, which is a path helper as some of you might know and Why is this like why does it run an old centOS? Why you know, what's going on? Well, it's a necessary evil users can't be trusted There are a lot of resources which are being consumed and you have to track these because people pay for them But you know users do need new software developers, especially a lot of my work involves a lot of high performance coding and I need to know reproducibly my stuff works so next is the solution clearly and It's been adopted by some in the scientific community It's been around for a long time now. It's 16 years. Yeah, but does no one use it? That's not exactly true There are classes that use it. They have good support like Rick had even has an ECM paper on it and Flatter Institute and compute Canada also support next but what if no one cared? What if nobody cared that you have problems or you like next well, then you end up here in this presentation So where are we exactly? Well, there's a whole lot of user installed junk to the post which accompanies this and not going to get into all of that But the basic concept is do something to get mixed up in anything unholy things to path You know prune bits of the source code just get it to install somehow and then that makes reinstall itself, right? so I'm not the first to think of this there are these two excellent repositories which I liberally looked into when I was doing this and Unfortunately, you know none of them are up to date hence So are we done is it over? Well, no even for the short presentation is still a lot of time. So not quite So what went wrong? Well, if you follow along the methodology which I covered in the post then well you basically because it's you can't utilize the cache and You're basically running on the login node and you're wasting a lot of resources. Yes, these resources can be tracked down to you You can be censored for it. So, you know in that sense, you're still responsible, but you know, this is not great Right? So what do we need really? We need better permission handling. There was this incredible moment when the permissions were So poorly said that you actually have to run a little watch command, which is terrible and of course, you know, nobody should be 777 in their permissions, but still Okay, so there's another issue which is that the lock is not necessarily released at the same time especially when you're building a large Package and one of the quote-unquote fixes is basically moving the package moving the entire directory and then we're building it Which is clearly not ideal, right? So where are we really? Well Bills and queues we need to know who's building what and we also need to be able to run on the entire cluster Not just the login node, right? So future directions. There's the union mount proposal Which essentially is meant for efficient private stores, but it's a step in the right direction It would reduce compilation. It doesn't actually replace the global store. There's a discussion here on Reddit And my own personal goals like what I want to do well, I was thinking of looking into hashing relative to a prefix and a cleaner setup definitely it is not I mean, it's a long blog post and it is not pretty right now to do this and hopefully I'll be back next year with more complete Project really I mean with more information and with a better approach to this I have a couple of years on my PhD and I plan to use NICs exclusively. So yeah, I'm definitely going to be looking into this March, so That's the end. Well, not exactly. There's the biography and thank you. Thank you for your attention.
 Okay, everyone go off for theophane or I'll be really disappointed. Okay, next up is nix from the Dark Ages without root  This is by Rohit Goswami in this talk is  Centralized on comments on the trenches of high-performance clusters on working with nix on kernel locked in systems without period support. So go on  Good morning, I'm Rohit Goswami and I'm gonna be talking about nix in the Dark Ages without root  It's a short talk. So I'm just gonna run through this quickly. So first and foremost, hello nix con  This is my first nix con and I'm super excited to be here. This is me. We can you can check that out later  So what's the big picture? What is going on? Well on the right?  You'll see an academic cluster picture from GE research and what's the problem really?  Well, there's no docker if you're very lucky or in some places you might have singularity  You won't have any user space support, which means you can play around with P route and  It probably runs into s or something as you can see in the lower right  If you check this out later, then you probably also be able to see that these are typically quite powerful  That's the whole point of using them. There's a lot of power in these clusters and there's also network file system  typically luster fs or cluster fs or even in some cases nxs and  There's always a resource queue  Either through slum or previous torque or something like that  Again, if you're lucky you're managing very you might find support for lmod, which is a path helper as some of you might know  and  Why is this like why does it run an old centOS? Why you know, what's going on?  Well, it's a necessary evil users can't be trusted  There are a lot of resources which are being consumed and you have to track these because people pay for them  But you know users do need new software developers, especially a lot of my work involves a lot of high performance coding and I need to know  reproducibly my stuff works  so next is the solution clearly and  It's been adopted by some in the scientific community  It's been around for a long time now. It's 16 years. Yeah, but does no one use it? That's not exactly true  There are classes that use it. They have good support like Rick had even has an ECM paper on it and  Flatter Institute and compute Canada also support next but what if no one cared?  What if nobody cared that you have problems or you like next well, then you end up here in this presentation  So where are we exactly?  Well, there's a whole lot of user installed junk to the post which accompanies this and not going to get into all of that  But the basic concept is do something to get mixed up in anything unholy things to path  You know prune bits of the source code just get it to install somehow and then that makes reinstall itself, right?  so I'm not the first to think of this there are these two excellent repositories which I  liberally looked into when I was doing this and  Unfortunately, you know none of them are up to date hence  So are we done is it over? Well, no even for the short presentation is still a lot of time. So not quite  So what went wrong? Well, if you follow along the methodology which I covered in the post then well you basically  because it's you can't utilize the cache and  You're basically running on the login node and you're wasting a lot of resources. Yes, these resources can be tracked down to you  You can be censored for it. So, you know in that sense, you're still responsible, but you know, this is not great  Right?  So what do we need really?  We need better permission handling. There was this incredible moment when  the permissions were  So poorly said that you actually have to run a little watch command, which is terrible and of course, you know, nobody should be  777 in their permissions, but still  Okay, so there's another issue which is that the lock is not necessarily released at the same time  especially when you're building a large  Package and one of the quote-unquote fixes is basically moving the package moving the entire directory and then we're building it  Which is clearly not ideal, right?  So where are we really? Well  Bills and queues we need to know who's building what and we also need to be able to run on the entire cluster  Not just the login node, right? So future directions. There's the union mount proposal  Which essentially is meant for efficient private stores, but it's a step in the right direction  It would reduce compilation. It doesn't actually replace the global store. There's a discussion here on Reddit  And my own personal goals like what I want to do well, I was thinking of looking into  hashing relative to a prefix and a cleaner setup definitely it is not  I mean, it's a long blog post and it is not pretty right now to do this and hopefully I'll be back next year with  more complete  Project really I mean with more information and with a better approach to this  I have a couple of years on my PhD and I plan to use NICs exclusively. So yeah, I'm definitely  going to be looking into this March, so  That's the end. Well, not exactly. There's the biography and thank you. Thank you for your attention.
NixCon_2020_Vid_15_0_551.png NixCon_2020_Vid_15_552_2387.png NixCon_2020_Vid_15_2388_3767.png NixCon_2020_Vid_15_3768_4595.png NixCon_2020_Vid_15_4596_4727.png NixCon_2020_Vid_15_4728_6659.png NixCon_2020_Vid_15_6660_7883.png NixCon_2020_Vid_15_7884_7967.png NixCon_2020_Vid_15_7968_8027.png NixCon_2020_Vid_15_8028_8039.png
Okay, everyone go off for theophane or I'll be really disappointed. Okay, next up is nix from the Dark Ages without root This is by Rohit Goswami in this talk is Centralized on comments on the trenches of high-performance clusters on working with nix on kernel locked in systems without period support. So go on Good morning, I'm Rohit Goswami and I'm gonna be talking about nix in the Dark Ages without root It's a short talk. So I'm just gonna run through this quickly. So first and foremost, hello nix con This is my first nix con and I'm super excited to be here. This is me. We can you can check that out later So what's the big picture? What is going on? Well on the right? You'll see an academic cluster picture from GE research and what's the problem really? Well, there's no docker if you're very lucky or in some places you might have singularity You won't have any user space support, which means you can play around with P route and It probably runs into s or something as you can see in the lower right If you check this out later, then you probably also be able to see that these are typically quite powerful That's the whole point of using them. There's a lot of power in these clusters and there's also network file system typically luster fs or cluster fs or even in some cases nxs and There's always a resource queue Either through slum or previous torque or something like that Again, if you're lucky you're managing very you might find support for lmod, which is a path helper as some of you might know and Why is this like why does it run an old centOS? Why you know, what's going on? Well, it's a necessary evil users can't be trusted There are a lot of resources which are being consumed and you have to track these because people pay for them But you know users do need new software developers, especially a lot of my work involves a lot of high performance coding and I need to know reproducibly my stuff works so next is the solution clearly and It's been adopted by some in the scientific community It's been around for a long time now. It's 16 years. Yeah, but does no one use it? That's not exactly true There are classes that use it. They have good support like Rick had even has an ECM paper on it and Flatter Institute and compute Canada also support next but what if no one cared? What if nobody cared that you have problems or you like next well, then you end up here in this presentation So where are we exactly? Well, there's a whole lot of user installed junk to the post which accompanies this and not going to get into all of that But the basic concept is do something to get mixed up in anything unholy things to path You know prune bits of the source code just get it to install somehow and then that makes reinstall itself, right? so I'm not the first to think of this there are these two excellent repositories which I liberally looked into when I was doing this and Unfortunately, you know none of them are up to date hence So are we done is it over? Well, no even for the short presentation is still a lot of time. So not quite So what went wrong? Well, if you follow along the methodology which I covered in the post then well you basically because it's you can't utilize the cache and You're basically running on the login node and you're wasting a lot of resources. Yes, these resources can be tracked down to you You can be censored for it. So, you know in that sense, you're still responsible, but you know, this is not great Right? So what do we need really? We need better permission handling. There was this incredible moment when the permissions were So poorly said that you actually have to run a little watch command, which is terrible and of course, you know, nobody should be 777 in their permissions, but still Okay, so there's another issue which is that the lock is not necessarily released at the same time especially when you're building a large Package and one of the quote-unquote fixes is basically moving the package moving the entire directory and then we're building it Which is clearly not ideal, right? So where are we really? Well Bills and queues we need to know who's building what and we also need to be able to run on the entire cluster Not just the login node, right? So future directions. There's the union mount proposal Which essentially is meant for efficient private stores, but it's a step in the right direction It would reduce compilation. It doesn't actually replace the global store. There's a discussion here on Reddit And my own personal goals like what I want to do well, I was thinking of looking into hashing relative to a prefix and a cleaner setup definitely it is not I mean, it's a long blog post and it is not pretty right now to do this and hopefully I'll be back next year with more complete Project really I mean with more information and with a better approach to this I have a couple of years on my PhD and I plan to use NICs exclusively. So yeah, I'm definitely going to be looking into this March, so That's the end. Well, not exactly. There's the biography and thank you. Thank you for your attention.
on the trenches of high-performance clusters on working with nix on kernel locked in systems I'm Rohit Goswami and I'm gonna be talking about