Using SELinux with container runtimes
Vrabec, Lukas
Lukas Vrabec built a new standalone tool, udica, for generating SELinux policy profiles for containers based on automatic inspecting these containers. We will focus on why udica is needed in the container world and how it can make SELinux and containers work better together. We will show real examples where SELinux separation for containers had to be turned off because the generic SELinux type container_t was too tight. With a tool like “udica”, users and developers can easily customize the policy with limited SELnux policy writing skills. Come to see how easy also you can create custom SELinux policy for your containers! This talk will explain how SELinux works with containers. We will show how to enable/disable SElinux using multiple different container runtimes and define the default types. One issue with these types is that they are tough to customize. The two default types for running containers are containert which is a fully confined domain, which eliminates any use of the host files unless they are relabeled. Or spct, which is the type containers run with when SELinux is disabled for container separation, -privileged mode. As an example, If you had a container that you wanted to be able to gather the logs from /var/log on the host and send them to a centralized server, you have to disable SELinux separation. Writing custom policy for each container that needed additional access would be very difficult and require a container policy writer.
 Hello everyone. Let's welcome our next speaker in security of the room on FOSDEM. It's Lukash Vrabets and he's talk about Selenium's container and time. Let's welcome Lukash. So can you hear me fine? Okay. Before we start with the presentation, I have one question for you. How many of you know Selenium's? So many people. Okay. That's great. So let's start with the introduction. My name is Lukash Vrabets. I'm Selenium's evangelist at Red Hat. Some people already call me that. I'm a member of security technologies team at Red Hat and I'm a rail and Fedora contributor to following packages which is Selenium's policy, XGEST, Udica and Netlabel tools. Here is my blog post. We are interested in some CVEs and how Selenium's mitigate them. You can find some interesting blog posts and GitHub link and also Twitter. So why we are here? I'll use a nice example of one of my colleagues from last week and in one OpenShift cluster, the Selenium's is enforced by default. On this cluster, there are almost 200 pods running more than 600 containers and 134 containers run as privilege. In this case, I mean there is no Selenium's isolation for them and 134 is so many containers. So privilege containers are really scary. Today, I would like to teach you how you can generate Selenium's policy for every container. Quick introduction to Selenium's. I make you Selenium's experts in less than five minutes, hopefully. So Selenium's, you probably know it, is implementation of mandatory access control. But we can say it's a process isolation technology for mitigate processes, for to mitigate attacks via privilege escalation. One of the most important thing you need to know are labels. In this case, there are two examples, container underscore T and container file underscore T. These labels are assigned to processes or to system resources. System resource could be directory, file, Simlink, socket, and many more. These labels are assigned to processes and system resources by Selenium's security policy. So how labels look like in reality, they are stored, related to system resources, they are stored in extended attributes of the file system, for example, X2, X3, and X4. There are two ways how you can see the Selenium's label of some specific file. In this case, it's slash hc slash password. And as you can see, the whole Selenium's label is what is written in red. But in this case, we are interested only about the type, which is the third part of the label, which is password underscore file underscore T. So for processes, we can use parameter capital Z for PS command. And if you want to grab some label of container, you can see that in the example, there are three containers, and each container has the same Selenium's type, which is container underscore T, which is here. Selenium's security policy describes the interaction between the processes and system resources, right? And we are using for the interaction allow rules, and this is the example of the allow rule. The first is keyword allow, that's quite clear. The second thing is saying that any process labeled as container underscore T can read, open, and get attributes of files labeled as container underscore file underscore T. And what is important to say also is that everything is denied by default, so you need to have explicitly allow rule inside the Selenium's policy, which is loaded to kernel, to allow some to allow some Cisco. And basically, that's it. You are now Selenium's experts, and that's all you need to know for generating Selenium's policies for containers, of course. Okay. So now we will talk about generic Selenium's policy. For each container without some custom Selenium's policy, we use the default one, which is container underscore T. This policy protects the host system from the container processes. And processes or containers with these Selenium's policies can only read or execute files in slash user, and these container processes can only read to, sorry, only write to container files. As I show you, process type, that's quite simple. That's container with container underscore T, Selenium's label, and file is every file and directory on the host file system labeled as container underscore file underscore T. So this is a really, really nice slide. I really like it. These are the least, just few of them, few of CVEs of container runtimes when there was a possibility that the process inside the container could escape from container to the host file system, and in several cases gets rude access. Selenium's blocked all of them. The latest one is from February 2019, when Selenium's completely blocked this type of attack. Yep, I say that. Okay. So what about containers attacking each other? For this, we have also solution, and it's a multi-category security, which is part of multi-level security. So this is the way how it works with MCS. We have two containers. Both of them has same Selenium's label, which is container underscore T, but the categories are different. First one has categories C1 and C2. Second one, category C2 and C3. In the middle, there are three files with same Selenium's type, which is container underscore file underscore T. And as we can see, following container can access this file because C1 and C2 is subset of C1 and C2. What is interesting that a file in the middle doesn't have any category, and empty set is also subset of C1 and C2, and same situation for C2 and C3. So this is how we are isolating containers between each other. This is the same slide, but arrow showing what is not allowed. So C1 and C2 is not subset of C2 and C3. So you can ask how you can label some directory as container underscore file underscore T. Container runtimes has really nice feature. It's for sure supported in Podman, and I'm pretty sure that it's also in Docker. If you're starting some specific container, you can buy mount, for example, in the example slash var slash lib slash MariaDB, and you put parameter capital Z. So container and runtime engine will be label on the host system, this directory to container underscore file underscore T, also with the same categories like running container has. That's because of the capital Z here. If you want to have two containers and you want to share the same directory, you can put small Z, which means the engine just reliable, this directory is container underscore file underscore T with no specific categories. So, of course, this solution has several problems. The first one is that one generic default S Linux policy for containers is too strict for certain cases. For example, Fedora civil boot project needs container to read and write user home directories. Second one, project flu and D needs to able to read logs stored in slash log slash var on the host system. There is another case when default S Linux policy is too loose in certain cases. There is no S Linux network controls, which means that container can bind on any port. There is no S Linux control for capabilities, which means that running container has all Linux capabilities. So we are able also to solve this. So what's the current situation? What can you do with MCS and generic S Linux policy? Let's say we want to start container with flu and D project and we want to buy mouth slash var slash log from the host system and we use capital Z as a parameter. It's here. Okay. And this is not good. Actually, it's pretty bad because all other confined services on the host system will not be able to write or read slash var slash log because the label will be container underscore file underscore T instead of var underscore log underscore T, which is the default label for this path. The worst situation, what you can do, you can use this agli parameter, which is here and basically you disable S Linux separation for that container. So this is the worst what you can do. So solutions. You can write completely new S Linux policy for your custom container, but this is the best solution, but it's really too difficult, right? For system administrator because you need to have, let's be honest, quite good S Linux expertise. The second thing what you can do is to add additional S Linux rule for the container underscore T type. This is Tony that ideal because you need to know how to write a S Linux policy and also this additional rules will apply for all containers on your system, not to just your custom container. So to solve all these issues, there is a Uditsa project. Actually, Uditsa mean a fishing route in Slovak and Uditsa is a tool for generating S Linux security profiles or policies for containers. So the best way how to describe Uditsa, let's use for example. Let's start the container mounting slash home with permissions read and write. Let's mount slash var slash spool with read only permissions and let's expose port 2021. So as I told you, generic S Linux policy cannot read and write slash home. Generic S Linux policy cannot read slash var slash spool for containers, but on the other hand can expose all ports. So I'm brave enough to show you live demo, but let's finish the slides and then I show you the demo. But what will be there is we start the container with the by mounts and ports as I described. Then we only run inspection of the latest started container, which is here. And Uditsa will generate the S Linux policy based on the inspection of container with name my underscore container, which is such as the recommends to load following S Linux policies, which is this load command, which has restart, which has restart the container with the following parameter where we are basically saying to not to use generic S Linux policy container underscore T, but our which is my underscore container dot process. And that's it. Then just just to prove it when we run PS, we will see that our container is running with the with our new S Linux label. So how it looks like what's what's under the hood. So the whole concept here is based on the block inheritance feature in a seal language where Uditsa creates or combine the rules from a predefined seal blocks, or we can say templates. So we did science packs inspects container and looking for mounts, ports and capabilities and combines these predefined blocks with the with the with the base one, which is base underscore container underscore. Sorry dot seal. So this is the way we will use the base block, which is needed for reading and executing files in slash user and reading configuration files in slash at C. Then we will use net block, which we need for binding on for binding on TCP port 21, which is labeled as FTP underscore port. And next the block is home, which we need for reading and writing to user home directories. Uditsa basically inherits allow rules inside these blocks and generate new S Linux policy with the name my container. We also mount slash var slash pool right inside the container. But for this there is no predefined block. So what we did second do is to look to directory slash var slash pool, find all possible S Linux labels, which could be inside this directory, generate generate block. In this case, let's call it spool and again inherit all these allow rules to our S Linux policy. And that's it. This kind of generated S Linux policy you can use with for following runtimes, which is podman, Docker, build up and also Kubernetes. So these are the these are the slides slides. So let's let's try it with demo. Okay, so I installed can you see it? Good. Perfect. So I installed all necessary packages, which is Uditsa, podman and settles. I tell you what is settles in the moment. So let's check what's the S Linux status on this system. Hopefully it's enforcing. Okay, cool. So we have a certain in enforcing. Let's start. Let's start the container from the example. So we are mounting slash home slash vars pool and exposing for 21. Okay, container is started. And we see that S Linux label of the of the container is container underscore T. So sorry, it's here. And here are the randomly generated randomly generated containers. Now you see command as a search. Basically, I'm querying the allow rules inside the S Linux policy loaded in into kernel if there is if there is a present allow rule. The first one is related to slash home slash home has a S Linux label home underscore root underscore T. And if there is no, there is no output, which means there is no allow rule. The same situation with the vars pool. That's pretty clear. The pad has a S Linux labels var underscore spool underscore T. T there is no, there is no allow rule. But on the other hand, for the port, we can see that container, which is part of these attributes can basically bind on NETCP ports. Another, let's continue. Now we run Udica. So we need the inspection file of the latest container, put it to standard input of Udica and the parameter is the name of the of the new S Linux policy. So let's start it. Policy is created and it's a recommends to load following following templates and final S Linux policy for our container using command a C module. So let's do that. And we stop the latest container. Perfect. And we start container again. But with following following parameter, which is saying to start container with the, with the following S Linux label. And as we can see, it's not running as container underscore T, but it's our custom S Linux type. And we are running again to SC search. And as we can see, the allow rules are now there, right? So this, this container can, can read and write a director is under slash home can access to slash bar slash pool. And again, on the other hand, can bind only on a port label FTP underscore, poor underscore T, which is, which is a S Linux label for TCP port 21. And basically that's it. So now you are able to generate a S Linux policy really quickly. So here are the links for Fujitsu for podman. If you are interested how it looks like the inheritance in, in, in Fujitsu, you can look on, on Fujitsu POC and you have the links. So that's it. Thank you. And if you have any questions, I'm ready. Thanks for your talk. Now as a new, as a Linux expert, I was wondering why this you did that part. Could this be in principle also part of potman? I mean, you're basically already passing all the information to potman, the port, the read, read write and read only stuff. Yes. It's, it's good point. Actually, I discussed it with the podman, podman maintenance to include it there. But somehow, unfortunately, for me, we agreed that it will be separate too. But we discussed that. I have a question about labeling. So you said that podman is really labeling all the files in the container on container start? Not really. S Linux is not enabled inside the container, right? So the podman is enabling the staff on the host level. So we do mean in, in, in the host, right? So there is no S Linux. There is no S Linux inside the container. There is a lot of file operation. Yes. Okay. So for if containers like large has a lot of files, it's, it would be kind of slow. Yeah. Yes. Yes. We know about this issue. We discussed it recently inside the S Linux team. And most probably we will work on this for future to kind of to make the relabeling faster. Maybe there are, there are ways how to do that. Some other questions? I didn't see there was someone in the back. Do you have some plan to integrate it into Kubernetes? Because right now you're writing podman, then write Odisha and then run podman again. If I'm writing a Kubernetes or any other orchestration manifest, yes, I'm doing it once. So how well it works with Kubernetes, MISOs and whatever else there is on the market. Very good question. Thank you for that. In near future there will be on, on GitHub there are already, there should be already, there should be container with Odisha inside there. And basically you can almost fully automatically generate S Linux policy inside the Kubernetes. We put, we put annotation generate S Linux policy to Docker file, right? So there is, there is operator. It recognized that some container was started with that annotation. The S Linux policies, S Linux policies generated. However, there are still technical issues. For example, how to, how to maintain S Linux policy between the cluster and stuff like that. So it's not easy, but we are working on that. And some proof of concept you can already find on, on Odisha web page. So maybe follow up to the other question. So is there, is it, I mean it takes a while for Odisha to, to generate the policy, but if the policy is really large as they also performance it on every container start to, as a Linux having to, to do that, or is that really fast? And, or can you maybe show what the, the CICIL was, what that Odisha generated earlier? Yeah, yeah, I can, I can do that. Actually, this is, this is pretty fast. This, this is the S Linux policy. What you can see the first one is that the blog is called my underscore container. You see that you can see the second and third line saying that inherit all allow rules from the, these three, these blogs, which is container restricted underscore net container. The next line is related to capabilities. So this is, this is quite quickly. And all the rest of the allow rules are related only to slash bar slash pool because there is no predefined, there is no predefined template. Actually, this is, this is not, this is not problem. We can, we can generate a S Linux policy really quickly because we just, just parsing the inspection file, which is in the JSON format and looking if there is predefined, if there is a predefined blog, then just set the list of capabilities from inspection file. And if there, if there is a need generate also allow rules, but the main issue is related to, to relabeling. Yeah. Okay. Thank you for your talk. We, we've got last half a minute, so we will probably end and this talk. Thank you, Rukai, for presentation. Thank you. Great. Thank you again.
 Hello everyone. Let's welcome our next speaker in security  of the room on FOSDEM. It's Lukash Vrabets and he's talk about Selenium's container  and time. Let's welcome Lukash.  So can you hear me fine? Okay. Before we start with the presentation,  I have one question for you. How many of you know Selenium's?  So many people. Okay. That's great.  So let's start with the introduction. My name is Lukash Vrabets.  I'm Selenium's evangelist at Red Hat. Some people already call me that.  I'm a member of security technologies team at Red Hat and I'm a rail and Fedora contributor  to following packages which is Selenium's policy, XGEST, Udica and Netlabel tools.  Here is my blog post. We are interested in some CVEs and how Selenium's  mitigate them. You can find some interesting blog posts  and GitHub link and also Twitter. So why we are here?  I'll use a nice example of one of my colleagues from last week and in one OpenShift cluster,  the Selenium's is enforced by default. On this cluster, there are almost 200 pods  running more than 600 containers and 134 containers run as privilege.  In this case, I mean there is no Selenium's isolation for them and 134 is so many containers.  So privilege containers are really scary. Today, I would like to teach you how you can  generate Selenium's policy for every container.  Quick introduction to Selenium's. I make you Selenium's experts in less than five minutes,  hopefully. So Selenium's, you probably know it, is implementation  of mandatory access control. But we can say it's a process isolation technology  for mitigate processes, for to mitigate attacks via privilege escalation.  One of the most important thing you need to know are labels.  In this case, there are two examples, container underscore T and container file underscore  T. These labels are assigned to processes or to system resources.  System resource could be directory, file, Simlink, socket, and many more.  These labels are assigned to processes and system resources by Selenium's security policy.  So how labels look like in reality, they are stored, related to system resources, they  are stored in extended attributes of the file system, for example, X2, X3, and X4.  There are two ways how you can see the Selenium's label of some specific file.  In this case, it's slash hc slash password. And as you can see, the whole Selenium's  label is what is written in red. But in this case, we are interested only about the type,  which is the third part of the label, which is password underscore file underscore T.  So for processes, we can use parameter capital Z for PS command.  And if you want to grab some label of container, you can see that in the example, there are  three containers, and each container has the same Selenium's type, which is container underscore  T, which is here. Selenium's security policy describes the interaction  between the processes and system resources, right?  And we are using for the interaction allow rules, and this is the example of the allow  rule. The first is keyword allow, that's quite clear.  The second thing is saying that any process labeled as container underscore T can read,  open, and get attributes of files labeled as container underscore file underscore T.  And what is important to say also is that everything is denied by default, so you need  to have explicitly allow rule inside the Selenium's policy, which is loaded to kernel, to allow  some to allow some Cisco. And basically, that's it.  You are now Selenium's experts, and that's all you need to know for generating Selenium's  policies for containers, of course. Okay.  So now we will talk about generic Selenium's policy.  For each container without some custom Selenium's policy, we use the default one, which is container  underscore T. This policy protects the host system from  the container processes. And processes or containers with these Selenium's  policies can only read or execute files in slash user, and these container processes  can only read to, sorry, only write to container files.  As I show you, process type, that's quite simple. That's container with container underscore  T, Selenium's label, and file is every file and directory on the host file system labeled  as container underscore file underscore T. So this is a really, really nice slide.  I really like it. These are the least, just few of them, few  of CVEs of container runtimes when there was a possibility that the process inside the  container could escape from container to the host file system, and in several cases gets  rude access. Selenium's blocked all of them.  The latest one is from February 2019, when Selenium's completely blocked this type of  attack. Yep, I say that. Okay.  So what about containers attacking each other? For this, we have also solution, and it's  a multi-category security, which is part of multi-level security.  So this is the way how it works with MCS. We have two containers. Both of them has same  Selenium's label, which is container underscore T, but the categories are different. First  one has categories C1 and C2. Second one, category C2 and C3.  In the middle, there are three files with same Selenium's type, which is container underscore  file underscore T. And as we can see, following container can access this file because C1  and C2 is subset of C1 and C2. What is interesting that a file in the middle doesn't have any  category, and empty set is also subset of C1 and C2, and same situation for C2 and C3.  So this is how we are isolating containers between each other. This is the same slide,  but arrow showing what is not allowed. So C1 and C2 is not subset of C2 and C3.  So you can ask how you can label some directory as container underscore file underscore T.  Container runtimes has really nice feature. It's for sure supported in Podman, and I'm  pretty sure that it's also in Docker. If you're starting some specific container, you can  buy mount, for example, in the example slash var slash lib slash MariaDB, and you put parameter  capital Z. So container and runtime engine will be label on the host system, this directory  to container underscore file underscore T, also with the same categories like running  container has. That's because of the capital Z here.  If you want to have two containers and you want to share the same directory, you can  put small Z, which means the engine just reliable, this directory is container underscore file  underscore T with no specific categories. So, of course, this solution has several  problems. The first one is that one generic default  S Linux policy for containers is too strict for certain cases. For example, Fedora civil  boot project needs container to read and write user home directories. Second one, project  flu and D needs to able to read logs stored in slash log slash var on the host system.  There is another case when default S Linux policy is too loose in certain cases. There  is no S Linux network controls, which means that container can bind on any port. There  is no S Linux control for capabilities, which means that running container has all Linux  capabilities. So we are able also to solve this.  So what's the current situation? What can you do with MCS and generic S Linux policy?  Let's say we want to start container with flu and D project and we want to buy mouth  slash var slash log from the host system and we use capital Z as a parameter. It's here.  Okay. And this is not good. Actually, it's pretty  bad because all other confined services on the host system will not be able to write  or read slash var slash log because the label will be container underscore file underscore  T instead of var underscore log underscore T, which is the default label for this path.  The worst situation, what you can do, you can use this agli parameter, which is here  and basically you disable S Linux separation for that container. So this is the worst what  you can do. So solutions. You can write completely new S Linux policy for your custom container,  but this is the best solution, but it's really too difficult, right? For system administrator  because you need to have, let's be honest, quite good S Linux expertise. The second thing  what you can do is to add additional S Linux rule for the container underscore T type. This  is Tony that ideal because you need to know how to write a S Linux policy and also this  additional rules will apply for all containers on your system, not to just your custom container.  So to solve all these issues, there is a Uditsa project. Actually, Uditsa mean a fishing  route in Slovak and Uditsa is a tool for generating S Linux security profiles or policies for  containers. So the best way how to describe Uditsa, let's use for example. Let's start  the container mounting slash home with permissions read and write. Let's mount slash var slash  spool with read only permissions and let's expose port 2021. So as I told you, generic  S Linux policy cannot read and write slash home. Generic S Linux policy cannot read slash  var slash spool for containers, but on the other hand can expose all ports. So I'm brave  enough to show you live demo, but let's finish the slides and then I show you the demo. But  what will be there is we start the container with the by mounts and ports as I described.  Then we only run inspection of the latest started container, which is here. And Uditsa  will generate the S Linux policy based on the inspection of container with name my underscore  container, which is such as the recommends to load following S Linux policies, which  is this load command, which has restart, which has restart the container with the following  parameter where we are basically saying to not to use generic S Linux policy container  underscore T, but our which is my underscore container dot process. And that's it.  Then just just to prove it when we run PS, we will see that our container is running  with the with our new S Linux label. So how it looks like what's what's under the hood.  So the whole concept here is based on the block inheritance feature in a seal language where  Uditsa creates or combine the rules from a predefined seal blocks, or we can say templates.  So we did science packs inspects container and looking for mounts, ports and capabilities  and combines these predefined blocks with the with the with the base one, which is base  underscore container underscore. Sorry dot seal. So this is the way we will use the base  block, which is needed for reading and executing files in slash user and reading configuration  files in slash at C. Then we will use net block, which we need for binding on for binding  on TCP port 21, which is labeled as FTP underscore port. And next the block is home, which we  need for reading and writing to user home directories. Uditsa basically inherits allow  rules inside these blocks and generate new S Linux policy with the name my container.  We also mount slash var slash pool right inside the container. But for this there is no predefined  block. So what we did second do is to look to directory slash var slash pool, find all  possible S Linux labels, which could be inside this directory, generate generate block. In  this case, let's call it spool and again inherit all these allow rules to our S Linux policy.  And that's it. This kind of generated S Linux policy you can use with for following runtimes,  which is podman, Docker, build up and also Kubernetes. So these are the these are the  slides slides. So let's let's try it with demo. Okay, so I installed can you see it?  Good. Perfect. So I installed all necessary packages, which is Uditsa, podman and settles.  I tell you what is settles in the moment. So let's check what's the S Linux status on  this system. Hopefully it's enforcing. Okay, cool. So we have a certain in enforcing. Let's  start. Let's start the container from the example. So we are mounting slash home slash  vars pool and exposing for 21. Okay, container is started. And we see that S Linux label  of the of the container is container underscore T. So sorry, it's here. And here are the randomly  generated randomly generated containers. Now you see command as a search. Basically, I'm  querying the allow rules inside the S Linux policy loaded in into kernel if there is if  there is a present allow rule. The first one is related to slash home slash home has a  S Linux label home underscore root underscore T. And if there is no, there is no output,  which means there is no allow rule. The same situation with the vars pool. That's pretty  clear. The pad has a S Linux labels var underscore spool underscore T. T there is no, there is  no allow rule. But on the other hand, for the port, we can see that container, which is  part of these attributes can basically bind on NETCP ports. Another, let's continue. Now  we run Udica. So we need the inspection file of the latest container, put it to standard  input of Udica and the parameter is the name of the of the new S Linux policy. So let's  start it. Policy is created and it's a recommends to load following following templates and final  S Linux policy for our container using command a C module. So let's do that. And we stop the  latest container. Perfect. And we start container again. But with following following parameter,  which is saying to start container with the, with the following S Linux label. And as we  can see, it's not running as container underscore T, but it's our custom S Linux type. And we  are running again to SC search. And as we can see, the allow rules are now there, right?  So this, this container can, can read and write a director is under slash home can access  to slash bar slash pool. And again, on the other hand, can bind only on a port label  FTP underscore, poor underscore T, which is, which is a S Linux label for TCP port 21. And  basically that's it. So now you are able to generate a S Linux policy really quickly.  So here are the links for Fujitsu for podman. If you are interested how it looks like the  inheritance in, in, in Fujitsu, you can look on, on Fujitsu POC and you have the links.  So that's it. Thank you. And if you have any questions, I'm ready.  Thanks for your talk. Now as a new, as a Linux expert, I was wondering why this you did that  part. Could this be in principle also part of potman? I mean, you're basically already  passing all the information to potman, the port, the read, read write and read only stuff.  Yes. It's, it's good point. Actually, I discussed it with the podman, podman maintenance to  include it there. But somehow, unfortunately, for me, we agreed that it will be separate  too. But we discussed that.  I have a question about labeling. So you said that podman is really labeling all the files  in the container on container start? Not really. S Linux is not enabled inside the  container, right? So the podman is enabling the staff on the host level. So we do mean  in, in, in the host, right? So there is no S Linux. There is no S Linux inside the container.  There is a lot of file operation. Yes. Okay. So for if containers like large has a lot  of files, it's, it would be kind of slow. Yeah. Yes. Yes. We know about this issue. We  discussed it recently inside the S Linux team. And most probably we will work on this for  future to kind of to make the relabeling faster. Maybe there are, there are ways how to do  that.  Some other questions? I didn't see there was someone in the back.  Do you have some plan to integrate it into Kubernetes? Because right now you're writing  podman, then write Odisha and then run podman again. If I'm writing a Kubernetes or any  other orchestration manifest, yes, I'm doing it once. So how well it works with Kubernetes,  MISOs and whatever else there is on the market.  Very good question. Thank you for that. In near future there will be on, on GitHub there  are already, there should be already, there should be container with Odisha inside there.  And basically you can almost fully automatically generate S Linux policy inside the Kubernetes.  We put, we put annotation generate S Linux policy to Docker file, right? So there is,  there is operator. It recognized that some container was started with that annotation.  The S Linux policies, S Linux policies generated. However, there are still technical issues.  For example, how to, how to maintain S Linux policy between the cluster and stuff like  that. So it's not easy, but we are working on that. And some proof of concept you can  already find on, on Odisha web page.  So maybe follow up to the other question. So is there, is it, I mean it takes a while  for Odisha to, to generate the policy, but if the policy is really large as they also  performance it on every container start to, as a Linux having to, to do that, or is that  really fast? And, or can you maybe show what the, the CICIL was, what that Odisha generated  earlier?  Yeah, yeah, I can, I can do that. Actually, this is, this is pretty fast. This, this is  the S Linux policy. What you can see the first one is that the blog is called my underscore  container. You see that you can see the second and third line saying that inherit all allow  rules from the, these three, these blogs, which is container restricted underscore net container.  The next line is related to capabilities. So this is, this is quite quickly. And all  the rest of the allow rules are related only to slash bar slash pool because there is no  predefined, there is no predefined template. Actually, this is, this is not, this is not  problem. We can, we can generate a S Linux policy really quickly because we just, just  parsing the inspection file, which is in the JSON format and looking if there is predefined,  if there is a predefined blog, then just set the list of capabilities from inspection  file. And if there, if there is a need generate also allow rules, but the main issue is related  to, to relabeling. Yeah.  Okay. Thank you for your talk. We, we've got last half a minute, so we will probably end  and this talk. Thank you, Rukai, for presentation.  Thank you.  Great. Thank you again.
security_using_selinux_with_container_runtimes_0_167.png security_using_selinux_with_container_runtimes_168_875.png security_using_selinux_with_container_runtimes_876_1775.png security_using_selinux_with_container_runtimes_1776_2723.png security_using_selinux_with_container_runtimes_2724_3203.png security_using_selinux_with_container_runtimes_3204_3947.png security_using_selinux_with_container_runtimes_3948_4475.png security_using_selinux_with_container_runtimes_4476_7703.png security_using_selinux_with_container_runtimes_7704_8999.png security_using_selinux_with_container_runtimes_9000_11615.png security_using_selinux_with_container_runtimes_11616_11951.png security_using_selinux_with_container_runtimes_11952_13643.png security_using_selinux_with_container_runtimes_13644_17627.png security_using_selinux_with_container_runtimes_17628_17747.png security_using_selinux_with_container_runtimes_17748_18923.png security_using_selinux_with_container_runtimes_18924_21503.png security_using_selinux_with_container_runtimes_21504_22331.png security_using_selinux_with_container_runtimes_22332_23615.png security_using_selinux_with_container_runtimes_23616_23747.png security_using_selinux_with_container_runtimes_23748_25115.png security_using_selinux_with_container_runtimes_25116_25271.png security_using_selinux_with_container_runtimes_25272_28115.png security_using_selinux_with_container_runtimes_28116_28259.png security_using_selinux_with_container_runtimes_28260_29303.png security_using_selinux_with_container_runtimes_29304_33251.png security_using_selinux_with_container_runtimes_33252_33311.png security_using_selinux_with_container_runtimes_33312_33575.png security_using_selinux_with_container_runtimes_33576_35411.png security_using_selinux_with_container_runtimes_35412_35555.png security_using_selinux_with_container_runtimes_35556_37487.png security_using_selinux_with_container_runtimes_37488_37607.png
Hello everyone. Let's welcome our next speaker in security of the room on FOSDEM. It's Lukash Vrabets and he's talk about Selenium's container and time. Let's welcome Lukash. So can you hear me fine? Okay. Before we start with the presentation, I have one question for you. How many of you know Selenium's? So many people. Okay. That's great. So let's start with the introduction. My name is Lukash Vrabets. I'm Selenium's evangelist at Red Hat. Some people already call me that. I'm a member of security technologies team at Red Hat and I'm a rail and Fedora contributor to following packages which is Selenium's policy, XGEST, Udica and Netlabel tools. Here is my blog post. We are interested in some CVEs and how Selenium's mitigate them. You can find some interesting blog posts and GitHub link and also Twitter. So why we are here? I'll use a nice example of one of my colleagues from last week and in one OpenShift cluster, the Selenium's is enforced by default. On this cluster, there are almost 200 pods running more than 600 containers and 134 containers run as privilege. In this case, I mean there is no Selenium's isolation for them and 134 is so many containers. So privilege containers are really scary. Today, I would like to teach you how you can generate Selenium's policy for every container. Quick introduction to Selenium's. I make you Selenium's experts in less than five minutes, hopefully. So Selenium's, you probably know it, is implementation of mandatory access control. But we can say it's a process isolation technology for mitigate processes, for to mitigate attacks via privilege escalation. One of the most important thing you need to know are labels. In this case, there are two examples, container underscore T and container file underscore T. These labels are assigned to processes or to system resources. System resource could be directory, file, Simlink, socket, and many more. These labels are assigned to processes and system resources by Selenium's security policy. So how labels look like in reality, they are stored, related to system resources, they are stored in extended attributes of the file system, for example, X2, X3, and X4. There are two ways how you can see the Selenium's label of some specific file. In this case, it's slash hc slash password. And as you can see, the whole Selenium's label is what is written in red. But in this case, we are interested only about the type, which is the third part of the label, which is password underscore file underscore T. So for processes, we can use parameter capital Z for PS command. And if you want to grab some label of container, you can see that in the example, there are three containers, and each container has the same Selenium's type, which is container underscore T, which is here. Selenium's security policy describes the interaction between the processes and system resources, right? And we are using for the interaction allow rules, and this is the example of the allow rule. The first is keyword allow, that's quite clear. The second thing is saying that any process labeled as container underscore T can read, open, and get attributes of files labeled as container underscore file underscore T. And what is important to say also is that everything is denied by default, so you need to have explicitly allow rule inside the Selenium's policy, which is loaded to kernel, to allow some to allow some Cisco. And basically, that's it. You are now Selenium's experts, and that's all you need to know for generating Selenium's policies for containers, of course. Okay. So now we will talk about generic Selenium's policy. For each container without some custom Selenium's policy, we use the default one, which is container underscore T. This policy protects the host system from the container processes. And processes or containers with these Selenium's policies can only read or execute files in slash user, and these container processes can only read to, sorry, only write to container files. As I show you, process type, that's quite simple. That's container with container underscore T, Selenium's label, and file is every file and directory on the host file system labeled as container underscore file underscore T. So this is a really, really nice slide. I really like it. These are the least, just few of them, few of CVEs of container runtimes when there was a possibility that the process inside the container could escape from container to the host file system, and in several cases gets rude access. Selenium's blocked all of them. The latest one is from February 2019, when Selenium's completely blocked this type of attack. Yep, I say that. Okay. So what about containers attacking each other? For this, we have also solution, and it's a multi-category security, which is part of multi-level security. So this is the way how it works with MCS. We have two containers. Both of them has same Selenium's label, which is container underscore T, but the categories are different. First one has categories C1 and C2. Second one, category C2 and C3. In the middle, there are three files with same Selenium's type, which is container underscore file underscore T. And as we can see, following container can access this file because C1 and C2 is subset of C1 and C2. What is interesting that a file in the middle doesn't have any category, and empty set is also subset of C1 and C2, and same situation for C2 and C3. So this is how we are isolating containers between each other. This is the same slide, but arrow showing what is not allowed. So C1 and C2 is not subset of C2 and C3. So you can ask how you can label some directory as container underscore file underscore T. Container runtimes has really nice feature. It's for sure supported in Podman, and I'm pretty sure that it's also in Docker. If you're starting some specific container, you can buy mount, for example, in the example slash var slash lib slash MariaDB, and you put parameter capital Z. So container and runtime engine will be label on the host system, this directory to container underscore file underscore T, also with the same categories like running container has. That's because of the capital Z here. If you want to have two containers and you want to share the same directory, you can put small Z, which means the engine just reliable, this directory is container underscore file underscore T with no specific categories. So, of course, this solution has several problems. The first one is that one generic default S Linux policy for containers is too strict for certain cases. For example, Fedora civil boot project needs container to read and write user home directories. Second one, project flu and D needs to able to read logs stored in slash log slash var on the host system. There is another case when default S Linux policy is too loose in certain cases. There is no S Linux network controls, which means that container can bind on any port. There is no S Linux control for capabilities, which means that running container has all Linux capabilities. So we are able also to solve this. So what's the current situation? What can you do with MCS and generic S Linux policy? Let's say we want to start container with flu and D project and we want to buy mouth slash var slash log from the host system and we use capital Z as a parameter. It's here. Okay. And this is not good. Actually, it's pretty bad because all other confined services on the host system will not be able to write or read slash var slash log because the label will be container underscore file underscore T instead of var underscore log underscore T, which is the default label for this path. The worst situation, what you can do, you can use this agli parameter, which is here and basically you disable S Linux separation for that container. So this is the worst what you can do. So solutions. You can write completely new S Linux policy for your custom container, but this is the best solution, but it's really too difficult, right? For system administrator because you need to have, let's be honest, quite good S Linux expertise. The second thing what you can do is to add additional S Linux rule for the container underscore T type. This is Tony that ideal because you need to know how to write a S Linux policy and also this additional rules will apply for all containers on your system, not to just your custom container. So to solve all these issues, there is a Uditsa project. Actually, Uditsa mean a fishing route in Slovak and Uditsa is a tool for generating S Linux security profiles or policies for containers. So the best way how to describe Uditsa, let's use for example. Let's start the container mounting slash home with permissions read and write. Let's mount slash var slash spool with read only permissions and let's expose port 2021. So as I told you, generic S Linux policy cannot read and write slash home. Generic S Linux policy cannot read slash var slash spool for containers, but on the other hand can expose all ports. So I'm brave enough to show you live demo, but let's finish the slides and then I show you the demo. But what will be there is we start the container with the by mounts and ports as I described. Then we only run inspection of the latest started container, which is here. And Uditsa will generate the S Linux policy based on the inspection of container with name my underscore container, which is such as the recommends to load following S Linux policies, which is this load command, which has restart, which has restart the container with the following parameter where we are basically saying to not to use generic S Linux policy container underscore T, but our which is my underscore container dot process. And that's it. Then just just to prove it when we run PS, we will see that our container is running with the with our new S Linux label. So how it looks like what's what's under the hood. So the whole concept here is based on the block inheritance feature in a seal language where Uditsa creates or combine the rules from a predefined seal blocks, or we can say templates. So we did science packs inspects container and looking for mounts, ports and capabilities and combines these predefined blocks with the with the with the base one, which is base underscore container underscore. Sorry dot seal. So this is the way we will use the base block, which is needed for reading and executing files in slash user and reading configuration files in slash at C. Then we will use net block, which we need for binding on for binding on TCP port 21, which is labeled as FTP underscore port. And next the block is home, which we need for reading and writing to user home directories. Uditsa basically inherits allow rules inside these blocks and generate new S Linux policy with the name my container. We also mount slash var slash pool right inside the container. But for this there is no predefined block. So what we did second do is to look to directory slash var slash pool, find all possible S Linux labels, which could be inside this directory, generate generate block. In this case, let's call it spool and again inherit all these allow rules to our S Linux policy. And that's it. This kind of generated S Linux policy you can use with for following runtimes, which is podman, Docker, build up and also Kubernetes. So these are the these are the slides slides. So let's let's try it with demo. Okay, so I installed can you see it? Good. Perfect. So I installed all necessary packages, which is Uditsa, podman and settles. I tell you what is settles in the moment. So let's check what's the S Linux status on this system. Hopefully it's enforcing. Okay, cool. So we have a certain in enforcing. Let's start. Let's start the container from the example. So we are mounting slash home slash vars pool and exposing for 21. Okay, container is started. And we see that S Linux label of the of the container is container underscore T. So sorry, it's here. And here are the randomly generated randomly generated containers. Now you see command as a search. Basically, I'm querying the allow rules inside the S Linux policy loaded in into kernel if there is if there is a present allow rule. The first one is related to slash home slash home has a S Linux label home underscore root underscore T. And if there is no, there is no output, which means there is no allow rule. The same situation with the vars pool. That's pretty clear. The pad has a S Linux labels var underscore spool underscore T. T there is no, there is no allow rule. But on the other hand, for the port, we can see that container, which is part of these attributes can basically bind on NETCP ports. Another, let's continue. Now we run Udica. So we need the inspection file of the latest container, put it to standard input of Udica and the parameter is the name of the of the new S Linux policy. So let's start it. Policy is created and it's a recommends to load following following templates and final S Linux policy for our container using command a C module. So let's do that. And we stop the latest container. Perfect. And we start container again. But with following following parameter, which is saying to start container with the, with the following S Linux label. And as we can see, it's not running as container underscore T, but it's our custom S Linux type. And we are running again to SC search. And as we can see, the allow rules are now there, right? So this, this container can, can read and write a director is under slash home can access to slash bar slash pool. And again, on the other hand, can bind only on a port label FTP underscore, poor underscore T, which is, which is a S Linux label for TCP port 21. And basically that's it. So now you are able to generate a S Linux policy really quickly. So here are the links for Fujitsu for podman. If you are interested how it looks like the inheritance in, in, in Fujitsu, you can look on, on Fujitsu POC and you have the links. So that's it. Thank you. And if you have any questions, I'm ready. Thanks for your talk. Now as a new, as a Linux expert, I was wondering why this you did that part. Could this be in principle also part of potman? I mean, you're basically already passing all the information to potman, the port, the read, read write and read only stuff. Yes. It's, it's good point. Actually, I discussed it with the podman, podman maintenance to include it there. But somehow, unfortunately, for me, we agreed that it will be separate too. But we discussed that. I have a question about labeling. So you said that podman is really labeling all the files in the container on container start? Not really. S Linux is not enabled inside the container, right? So the podman is enabling the staff on the host level. So we do mean in, in, in the host, right? So there is no S Linux. There is no S Linux inside the container. There is a lot of file operation. Yes. Okay. So for if containers like large has a lot of files, it's, it would be kind of slow. Yeah. Yes. Yes. We know about this issue. We discussed it recently inside the S Linux team. And most probably we will work on this for future to kind of to make the relabeling faster. Maybe there are, there are ways how to do that. Some other questions? I didn't see there was someone in the back. Do you have some plan to integrate it into Kubernetes? Because right now you're writing podman, then write Odisha and then run podman again. If I'm writing a Kubernetes or any other orchestration manifest, yes, I'm doing it once. So how well it works with Kubernetes, MISOs and whatever else there is on the market. Very good question. Thank you for that. In near future there will be on, on GitHub there are already, there should be already, there should be container with Odisha inside there. And basically you can almost fully automatically generate S Linux policy inside the Kubernetes. We put, we put annotation generate S Linux policy to Docker file, right? So there is, there is operator. It recognized that some container was started with that annotation. The S Linux policies, S Linux policies generated. However, there are still technical issues. For example, how to, how to maintain S Linux policy between the cluster and stuff like that. So it's not easy, but we are working on that. And some proof of concept you can already find on, on Odisha web page. So maybe follow up to the other question. So is there, is it, I mean it takes a while for Odisha to, to generate the policy, but if the policy is really large as they also performance it on every container start to, as a Linux having to, to do that, or is that really fast? And, or can you maybe show what the, the CICIL was, what that Odisha generated earlier? Yeah, yeah, I can, I can do that. Actually, this is, this is pretty fast. This, this is the S Linux policy. What you can see the first one is that the blog is called my underscore container. You see that you can see the second and third line saying that inherit all allow rules from the, these three, these blogs, which is container restricted underscore net container. The next line is related to capabilities. So this is, this is quite quickly. And all the rest of the allow rules are related only to slash bar slash pool because there is no predefined, there is no predefined template. Actually, this is, this is not, this is not problem. We can, we can generate a S Linux policy really quickly because we just, just parsing the inspection file, which is in the JSON format and looking if there is predefined, if there is a predefined blog, then just set the list of capabilities from inspection file. And if there, if there is a need generate also allow rules, but the main issue is related to, to relabeling. Yeah. Okay. Thank you for your talk. We, we've got last half a minute, so we will probably end and this talk. Thank you, Rukai, for presentation. Thank you. Great. Thank you again.
are stored, related to system resources, they are container, you can see that in the example, there are three containers, and each container has the we use the default one, which is container processes or containers with these Selenium's CVEs of container runtimes when there was a works with MCS. We have two containers. Both of them has same Selenium's label, which is there are three files with same Selenium's type, containers between each other. This is the same running container has. That's because of the several problems. The first one is that one generic default S Linux policy for containers is another case when default S Linux policy is too because all other confined services on the host system will not be able to write or read slash separation for that container. So this is the how to write a S Linux policy and also this your system, not to just your custom container. Slovak and Uditsa is a tool for generating S Linux security profiles or policies for Uditsa will generate the S Linux policy based on Linux label. So how it looks like what's what's block, which is needed for reading and executing these blocks and generate new S Linux policy with the container from the example. So we are for your talk. Now as a new, as a Linux expert, I run podman again. If I'm writing a Kubernetes or you can almost fully automatically generate S