SaBRe: Load-time selective binary rewriting
Arras, Paul-Antoine
Andronidis, Anastasios
Pina, Luís
Mituzas, Karolis
Shu, Qianyi
Grumberg, Daniel
Cada, Cristian
Abstract Binary rewriting is a technique that consists in disassembling a program to modify its instructions, with many applications, e.g. monitoring, debugging, reverse engineering and reliability. However, existing solutions suffer from well-known shortcomings in terms of soundness, performance and usability. We present SaBRe, a novel load-time framework for selective binary rewriting. SaBRe rewrites specific constructs of interest — mainly system calls and function prologues — when the program is loaded into memory. This enables users to intercept those constructs at runtime via a modular architecture allowing custom plugins to be linked with SaBRe using a simple and flexible API. We also discuss the theoretical underpinnings of disassembling and rewriting, including conditions for coverage, accuracy, and correctness; and how they affect SaBRe. We developed two backends for SaBRe — one for x86_64 and one for RISC-V — which were in turn used to implement two open-source plugins: a fast system call tracer and a fault injector. Our evaluation shows that SaBRe imposes little performance overhead, between 0.2% and 4.3% on average. In addition to explaining the architecture of SaBRe and demonstrating its performance, we also show on a concrete example how easy creating a new plugin for SaBRe is. SaBRe is a free open-source software released under the GPLv3 license and originally developed as part of the Software Reliabilty Group at Imperial College London. Introduction The goal of binary rewriting is to add, delete and replace instructions in binary code. There are two main types of binary rewriting techniques: static and dynamic. In static binary rewriting, the binary file is statically rewritten on disk, while in dynamic binary rewriting it is rewritten in memory, as the program executes. Static binary rewriting has the advantage that the rewriting process does not incur any overhead during execution, as it is performed before the program starts running. However, static binary rewriting is hard to get right: creating a valid modified executable on disk is challenging, and correctly identifying all the code in the program is error-prone in the presence of variable-length instructions and indirect jumps. By contrast, dynamic binary rewriting modifies the code in memory, during program execution. This is typically accomplished by translating one basic block at a time and caching the results, with branch instructions modified to point to already translated code. Since translation is done at runtime, when the instructions are issued and the targets of indirect branches are already resolved, dynamic binary rewriting does not encounter the challenges discussed above for static binary rewriting. However, the translation is heavyweight and incurs a large runtime overhead. In this presentation, we introduce SaBRe, a system that implements a novel design point for binary rewriting. Unlike prior techniques, SaBRe operates at load-time, after the program is loaded into memory, but before it starts execution. Like static binary rewriting techniques, SaBRe rewrites the code in-place, but the translation is done in memory, as for dynamic binary rewriting. To achieve a high level of both performance and reliability, SaBRe relies by default on trampolines, which are extremely efficient and can be used more than 99.99% of the time, and only falls back on illegal instructions triggering a signal handler for pathological cases. The main limitation of SaBRe is that it is designed to rewrite only certain types of constructs, namely system calls (including vDSO), function prologues and some architecture- specific instructions (e.g. RDTSC in x86). However, as we illustrate later on, this is enough to support a variety of tasks, with much lower overhead than with dynamic binary rewriting and without incurring the precision limitations of static binary rewriting. We implemented two binary rewriters based on this design: one for x86 64 and one for RISC-V code. Both rewriters feature a flexible API, which we used to implement three different plugins: a fast system call tracer, a multi-version execution system (not open-sourced yet) and a fault injector. In summary, our main contributions are: 1. A new design point for selective binary rewriting which translates code in memory in-place at load time, before the program starts execution. 2. An implementation of this approach for two architectures, one for x86 64 and the other for RISC-V. 3. A comprehensive evaluation using two open-source plugins: a fast strace-like system call tracer and a fault injector. 4. An extremely simple API that can be leveraged by users to implement and integrate their own plugins.
 So, hello everybody. We're going to start the next talk which was presented by Paul Antoine Arras and which is a low-level engineer and he will talk about SAP. Thank you. Thank you for the introduction. So, yes, I'm going to tell you about software that I developed with the time selected binary writing. Let's start with some quite simple use case just to see why we need to do such low-level first step. So, for instance, let's say that you are a developer. You've developed a brand new software and you want to test to see how resilient your software is. Well, basically, you want to see if you can find some tricky bugs and fix them easily with some tools. So, basically, if you want to set the full tolerance of your software, so for a particular example, well, you can have this rule of memory RAM exhausted or whatever kind of problem you can run into. And you want to check that your software is able to cope appropriately with this kind of issue. So, the problem is it's hard to reproduce on a real system. Well, if you want to simulate on a real system full disk or lack of RAM, well, that's quite painful. So, usually, what you want to do is to simulate this. So, the question is how could you achieve this? So, it's basically possible. You have really several options. One of the options is to tinker with the kernel, the Linux kernel, for instance. Well, for kernel hackers, that's great. But, well, for most people, it's quite quite painful and dangerous. So, you don't want to do that. You can also try to modify, to adapt or to intercept the libraries that support your software. But, once again, it's quite painful and not really reliable. So, actually, there is an intermediate solution that is trying to see what's between the libraries and the operating system. So, let's have a look at a simple, very simple Python program, for instance. Well, here, the example is with Python, but you can do that with whatever language you want. And, well, with a basic hello world, what you want to do is just print the string hello world onto the string, onto the terminal. So, basically, what happens is that, so, you have your user code, so, basically, this line of code that we call into some libraries. So, in this case, it will be the Python runtime. And at some point, the Python runtime will call into the operating system to, well, actually print the string onto the string. And what we're interested in here is the interface between the library and the operating system. So, this interface is called the system called interface. And this is the single interface that's available to communicate between the user space, your program, and the operating system. There is no other way. So, you are absolutely sure that any code that you are making through the operating system will go through this interface. So, that's very important for us. What's next? So, just let's take a look, a quick look at what is exactly this system called interface. How does it work? What is it for? So, actually, it's a set of low-level operations, like, well, a kind of primitive, really relatively low-level, like if you want to write characters on the screen, as I said previously, or, for instance, to send packets to the networks, or communicate with some random peripheral, well, whatever you want to do, especially in terms of input or output, you have to go through this and use one of the functions that are available. So, basically, system calls are really like regular function. That is, you have a set of arguments that you pass to the system call, and you get one result in return, depending on the number of arguments that vary, usually between zero and six, depending on what you want to do, you will end up using one of these functions. So, for instance, if we go back to our Python hello world, the print hello world will translate into the write system call, which is the way that you use to write some character on the terminal, for instance, or also to write some data into a file. So, in this case, this write call takes three arguments. The first one is the file handle. So, well, as most people know, under Linux, the standard output is associated with the file descriptor number one. That's the reason why we have one at the first argument, then the string that you actually want to print hello world, and finally, the side of the string, which is here, 13. So far, so good. Now, as I said, what we want to achieve is to simulate a fault, a problem in terms of, for instance, we have this rule, what exactly happened. So, we want to take a look at the return value of the system call, because this is what we tell us if the system call has succeeded or if there was an error. This return value can be either positive if it's successful or negative. In this case, the exact value will indicate the type of error. So, in the case of the write system call, if it is successful, you get the side that was actually written. And if it's not, well, you get some error code that will describe the error that you got. For instance, permission denied. The code is E perm or this rule, in which case, you get E no space. So, for instance, let's say that we have this Python program slightly more complex than the previous one. This time, we want to write into a file. So, we use the open function, and we want to write into the file whose path is slash tmp slash hello in write mode. And then, yes, we write the string hello into this file. So, how does it translate in terms of system call? So, we have two system calls. The first one is, well, as you can guess, the open system call that takes, well, basically the same arguments at the real function. And once the file has been opened, if it succeeds, it returns the number of the file descriptor, which is 8 here. And then, we just have to get this number, the file descriptor, and pass it to the write system call in turn. So, that's what we do. And we are calling the write system call with 8 as the first argument, the string and the say of the string. So, here we have two possible outcomes. Either it's successful, in this case, it returns the number of characters returned, 6, or if it's not, well, it will return some negative value, which is there are code that describes the problem. So, for instance, you could get an E-perm code. We tell you that you don't have the necessary permission to write into this file. Right. So, let's go back to our original example of fault injection. We want to simulate some faults. For instance, simulate the fact that the disk is full, so that you can't write to a file. So, the question is, in terms of system call, how does it translate? So, what we want to do, actually, is to take the error code that is really returned by the system call and swap it with an error code. So, that even though the operation has been successful, your application will get an error code. And we think that there was the error decoded and encoded by the error code. And at that point, you'll be able to assess exactly how your application is responding to it. So, yes, let's do this. So, for instance, let's take this write call and replace the return value, the real return value, which is fixed, with an E-perm code. So, how to achieve that? Well, that's where binary writing comes into play. So, what exactly is binary writing? So, in simple terms, it's modifying a program, not at the source code level, but at the machine code level. That is the code, the binary code that is actually read and executed by the CPU. So, the good point is you don't need the source code. Well, sometimes you don't have access to source code at all, but well, most of the time, even if you have the source code, you don't necessarily want to recompile it every time, especially if you want to deal with a false detection. So, well, one good thing with binary writing is that you don't have to modify the source code. You don't need to recompile anything. You just have to get your binary and work directly on it. So, the only requirement for binary writing is disassembling. So, for now, let's keep disassembling apart and let's just say that what we need is the representation of the programs in terms of assembly instructions. So, it basically means that you have your binary code. It's just a sequence of zero and one. So, that's really what the CPUs are seeing and what they are able to execute. But well, we need to be able to have a slightly more human readable version of this. So, this is where this assembly comes to play. It's basically selecting what we have on the left, the sequence of this into something that is more or less, I say more or less, readable by human, which is just a sequence of instruction. So, that's really a translation, a perfect translation. There is no loss of information and you can then reconvert the assembly version into the original bit sequence without any loss of data. So, once we have this disassembly, we are able to do interesting stuff. So, let's take this representation. So, so far, so I've been showing you just a small snippet of assembly and the rest of this talk. Anytime I will show you some assembly, it will look like this. There will be three columns. The first one is your set from the very beginning of your program. So, here, let's say that just an excerpt, so the very beginning of your program, just the first couple of instructions. So, we start with set zero and then we increment. Second column is the size. So, I'll explain later why I should decide because that's really important when we do binary writing. And finally, the human readable version, which is just what the instruction does in a way more readable by human. Okay. So, let's take this and then let's try to see what operation we can do on instruction. So, we can do three, exactly three operations. We can remove instructions, we can replace instructions, or we can add instructions. That's the three types of instructions that we can do in binary writing. And I'm going to show you how we can do each of them. So, let's start with the very first, the simplest thing that we can do is removing instructions. So, here, what we do is that, let's say that we have this program on the left-hand side. So, here we want to remove the instruction in the red, okay? So, the load something. So, what do we do? Well, we just remove it, but we don't just remove it because, well, it's not like in a classic text editor, you can just remove the part that you're not interested in and have everything that will fit between. Here you have to do an additional operation. So, the additional operation is to pad the gap that you've just created by adding some instruction that do basically nothing. So, the instruction that do nothing here is no, no operation. So, since each of these instructions is just one byte long, you put two and then you end up having sealed the space which is two bytes long with the load here, right? Well, so you're going to ask, but why do we need to do that? Why not just remove the load operation and then shift everything? So, that's, well, we end up with no, no load space. Well, the problem is if you do that, you'll change the offset of the instruction. For instance, here you can see that the load instruction has offset one and everything after offset three, eight, et cetera. Here, if you want to remove the load instruction without padding the space, we'll end up having the cool instruction at offset one, the old instruction at offset six, well, everything will be shifted. And the thing is in assembly, in machine code, you need to have your set right to be able to jump from one point of the code to another. So, basically, in a high level language when you have a nif, a while, a loop, or any conditional structure, it's basically saying in terms of assembly, well, we are jumping from one point in the program to another. And the jump in assembly code is achieved by just recording the offset where we want to jump. So, well, that's the reason why we don't do that. We don't just remove the instruction that we're not interested in. We need to keep the offset the same, right? So, that's what's to remove. Now, let's see how we can replace some instruction, slightly more complex, but let's start with a simple case. So, let's say that we want to replace a call. So, call, you read the way that you are calling a routine, subroutine, or a function. So, let's say, for instance, that we have this snippet of assembly and we have this call instruction. We want just to replace it with a jump to somewhere else, okay? So, here is how we translate. So, we just replace the call with a jump and we end up with this assembly sequence. But now, now that we have this, we have to ask ourselves the question, what exactly is the size of this jump instruction? Because as I just said, you have to ensure that you keep the offset of the surrounding code the same. So, the question is, what is the size of the jump? Because if the size is five, well, you will end up with the same offset for the next instruction. It will be eight, but if it's different, well, you will shift everything. So, you have to be careful with the size of the new instruction. So, basically, you need to compare the related sizes of the original instruction and the instruction with which you want it to be replaced. So, yes, that's what I just said. So, let's introduce a bit of notation. So, I note S of O, the size of the original instruction, S of R, the size of the rewritten, the new instruction, and then we compare them. So, in this case, we compare the size of the red instruction, the one that we remove, with the size of this new jump instruction, which is in green. And in this case, okay, let's say, let's say that it's quite simple, we have the same size, five for the call, five for the jump. That's the same thing. In this case, well, quite simple. We just, sorry, quite simple. We just replace and we do nothing else. Nothing to pad, nothing to shift. That just fits. That's perfect. But, well, as you can guess, most of the time, it's not that simple. So, for instance, let's imagine that the jump is shorter. It's three instead of five. So, in this case, well, you can guess what we do. We pad, just like we did when we remove. That's more or less equivalent. But then, obviously, we can get into cases where the jump, so the new instruction is larger than the original one. And in this case, well, what do we do? So, that's quite a problem. They usually, well, quite a basic problem in terms of binary writing. So, there are several ways to deal with that. I'm going to show you the way that we use in Saber, which is called D2R. So, D2R, or D2Ring, is just taking the instruction that do not fit anymore in the original code space. So, where we added an instruction, or we replaced the original instruction with a larger one. So, there are some instructions that can't fit anymore. So, we have to take them out, move them somewhere else. So, the somewhere else is what I will call an out of line scratch space. So, out of line because it's not in line anymore. It's not part of the straight instruction stream. It's somewhere else in the memory. Okay? So, the way we do that is, well, basically, we are allocating memory somewhere else. We are moving the instruction that do not fit anymore in the original main line stream to this out of line scratch space. And then, obviously, we need to connect this out of line space to the original main line. And we do that by inserting some jumps. So, let's take it on a simple example. Let's say that we want to add an instruction with such a D2R. In this case, let's say that we want insert a jump. No, sorry. Let's say that we want to insert some instruction. It can be one instruction. It can be a sequence of instruction. No matter the length of the sequence you want to insert, the principle is the same. So, let's say that we want to insert at the point where you see the green arrow. Okay? So, right after the load and before the call. So, how can we do that? So, first, we are locating the out of line scratch space, which is at the bottom right corner. And there, we move the call instruction because it can't fit anymore. Once we have rewritten this part of the call. So, we move it. The original part, we move it from the main line to the out of line scratch. Then, we insert the add instruction. So, here, I didn't represent them, but you can think of it as just one instruction of a snippet of how many instructions you want. It doesn't matter. And then, last part that I mentioned, you have to connect this out of line scratch space with the rest of the code. And to do that, you use jump. So, first, we insert a jump so that the first green instruction that you can see at offset three. And you jump to some label that is representing the beginning of the outline scratch space. So, the label here is D0. So, when you're here, which should be the place when you have the call instruction in the original code, instead of directly executing the call instruction, you will jump at the beginning of the outline scratch space. And then, at that point, you can execute the original instruction, the call, as if it were still at the same place. The semantics doesn't change. You have this guarantee. And then, you can just execute whatever arbitrary instruction you want. And finally, let's not forget to jump back to the main line with this jump. So, this time, we've inserted another label called L0, which is just right after the jump. To that, when we have executed the instruction, we just come back and then continue, well, I was about to say, as if nothing has happened. Well, something has happened. But the rest of the code, the surrounding code, remains unchanged. Right. So, this is very important. This is really the basics of how everything is working. So, in this case, you can notice that the size of the original instruction, the call here in orange, is the same as the jump. Five. Both are five. Well, that's fine. But as I mentioned before, you have to be careful that when you replace one instruction with the other, you have to compare the relative sizes and see if that fits or if you have to adapt. So, in this case, we do exactly what we've done before. We have to compare this time not the real instruction, but the jump that we insert with the original instruction. So, in this case, let's say that, well, we have basically two cases. Either the jump is less or equal to the size of the original instruction. In this case, well, you just insert it. If need be, you pad with knobs and everybody is happy. Or if it's not the case, if the jump itself is larger than the original instruction, well, in this case, you have to relocate the number of instructions. So, for instance, let's go back to our previous example. We want to replace this load instruction and we want to insert a jump. So, in this case, let's say that the original instruction, the load that we want to replace is only two, two bytes, which is quite short. In comparison, the jump is five. So, well, in this case, the jump is quite larger than the original instruction. So, you need to, well, you need to make space even to insert the jump. So, well, what you do is just, as I mentioned before, you take the call instruction that is just next to the instruction that you want to replace. You move it to the outer line of the space. The rest is the same. You still need to locate the space. So, I don't go back to that. But once you have your outer line's crash space, you just move the surrounding instruction at the right place. So, here, after the instruction that you want to substitute, since it is, since you are relocating the instruction that is after the instruction that you, that you replace, and you end up with this result, so still the jump to the outer line space and then jump back to the main line. So, well, I've, I've basically covered all the possible outcomes, depending on the side, the relative sizes of the original instruction and the return instruction. And same for the jump instruction. In case you have to, to have this outer line's crash space. So, yeah, basically the most complex case that you can face is that you have a couple of instructions around the jump that you insert that you need to relocate. So, now the question is, can instruction always be rewritten, relocated, sorry. Can you always rewritten arbitrarily selecting instruction around the spot where we want to replace or other instruction? Well, you get that if I'm asking the re, the answer is no. So, I'm going to show you the, well, some typical examples of cases where you can't or you have to be very careful when you relocate instructions. So, for instance, PC relative addressing. So, what is PC relative addressing? So, usually when you want to, I give you the example of a jump, when you want to jump to some part of the code, for instance, if you have a loop, a loop is basically having a jump backwards into the instruction stream, so that you will execute this sequence several times. And usually the most efficient way to do that is instead of encoding the address, the full address of the place where we want to end up, the target of the jump, which would be, well, 32, 64 bits, which is quite long. Usually, you just use the displacement, which is the distance, the offset between the source of the jump where we're jumping from to the target where you're jumping to. The difference between the two. Usually, it's, well, a couple of bytes because, well, it's quite rare that you have to jump, especially when you have a typical control structure, that you have to jump to a place very, very far away in the code. So, usually, you just have to encode the displacement, the distance between source and target, and that's much more efficient. So, in this case, for instance, let's say that we want to load some data that is located 48 bytes from the current value of the PC, which is basically where we are, the current instruction that is executed. So, 48 bytes from the place where we are currently executing, right? And we want to exert an instruction where we have the green arrow, right before the load instruction that has the displacement addressing. So, what do we do? Well, we just applied the D2, the D2 mechanism that I just presented earlier. So, we end up with the same kind of stuff with our jump. Well, we need to insert some note because we are relocating the load instruction because, well, we need to make the jump fit. And so, we end up here in the out of line scratch space, we end up with this displacement, 48 from PC. Well, the problem is, since we moved the instruction to somewhere else, you can see here the offset is quite different, it's FFEA, and the original set was two. So, we are very, very far away from the original space. So, this offset is not correct anymore because we didn't move the data that it put into. So, we end up actually with quite a significant mismatch between the right displacement, which is 49 in the original instruction. And after relocating the instruction, you get this displacement, which is 10032, well, that's completely wrong. So, you have to be careful with that. And you have to, well, basically just to compute the actual displacement after having relocated the instruction. So, well, you just compute the difference between the original displacement and the new offset. So, here, 49 minus FFE6, and you end up with the new displacement once you have relocated the instruction, which is minus FFE90. Okay? So, well, that's feasible, that's feasible. But usually, we try to, when we do binary writing, we try to avoid it because we can be quite painful. So, second problem that can arise when you want to relocate some instruction. So, I told you about jumps, I told you about source and targets. So, well, basically, here, imagine that we have a jump backward, like in a loop. And we want to insert an instruction right before the target of the loop, which is here represented by the L0 label, right? So, what do we do? Well, once again, we do the same kind of thing with the detour. We move the original surrounding instruction, which in this case is this load. We add the instruction, et cetera, as usual. No problem. When you end up trying to execute the jump L0 instruction after writing what happened? Well, you've lost the L0 because you rewrote the sequence of instructions, which is load then call. And you can see that the target, the L0 label, is pointing right in the middle of this. And since you've completely overwritten this, if you still want to jump at this address, well, you end up jumping right in the middle of another instruction. So, this is completely wrong. And in this case, well, there is no simple way to solve it. So, what we do is that we scan the code before even starting to rewrite. We record all the branch targets. And we say, okay, this instruction, we don't do anything with them. So, we just keep them at place. And we won't even try to relocate them. We need to do that otherwise. We end up with some trying to decode some non-existent illegal instruction. And that can be quite problematic. So, I just show you two examples of typical problem that you will end up when you want to relocate instruction. There is a third one that is side effects. I won't go into the details of the side effects, but just keep in mind that you have three types of problems or challenges. And you have to, you have just to keep them in mind when you rewrite instruction. So now, what if we end up, for instance, with a target for a jump right before the instruction that we want to insert, and also some instruction with side effect right after, which means that you just, you just write in between two instructions that you can't relocate. So, what happens in this case? Because you may, for very valid reasons, be willing to rewrite some instruction at this precise position. And well, if you end up having surrounding instruction that you can't relocate, well, you're stuck. Well, not completely. You can still end up doing something which is not strictly speaking binary writing, but something that can allow you to do what you want. So, actually, what you can do is just replace the instruction that you are interested in with an illegal one. So, an instruction that is known to not encode an illegal operation. So, you replace it with this instruction. And what you do is that you set up a signal handler that will catch the signal that will be raised by trying to decode an illegal instruction. So, once you've set up this, you just have to catch the signal. Obviously, you need to record the source address of the instruction that triggered the signal. But then, you can do whatever you want inside the signal handler. So, well, as you can imagine, that's quite a significant overhead doing this, having to go through all the mechanisms of the signal handler. But since, well, in our experience, that really, really happened rarely, well, the overhead is, is structural. So, well, that's really the last resource, the last resource solution that, at least, we have a solution when that happens. We mean that we can rewrite virtually all instruction, any arbitrary instruction in the program. So, I promise you that I'll go back to the disassembly problem. Well, at the time, I didn't tell you that there was a problem, but actually, there is a problem. So, let's, let's go back. Let's, let's make one step backward and look at what we exactly have when you, when we actually look at some machine code, some program code in terms of binary. So, we have, well, on the left, this binary sequence, some arbitrary one, and we want to turn it into something that is, well, understandable by a byte human. And especially, we want to, to know exactly which bit, or which bytes correspond exactly to which instruction, because the instruction is the granularity that we are interested in. So, you can see here that actually what it is really meant for is being able to, to match a sequence of bytes with some specific instructions. So, for instance, here you have the yellow sequence of bytes that will be representing a push instruction, the orange one will represent the load, et cetera. So, that's, that's exactly what, what we want to achieve. The problem is, it's not always easy. You don't always get it for free. So, you have to, you have to have some, some, some algorithm or some, some technique that will, that will guarantee that you have the right, the right DDSMB. So, there are, there are different, different techniques that allow you to get, to get the DDSMB. The dynamic one, the static one. Well, both have advantages, but for, for the purpose of binary writing, in Saber, we will use the static one. So, there are basically in terms of starting the assembly, there are many two ways of doing this. Either what you have on the left, which is linear sweep. So, linear sweep is just, well, we basically start at the very beginning of the program and we scan it instruction by instruction. We know, we know where the program starts because, well, otherwise the, the OPC system is not even able to load it. And so, we can, we know, actually that the point where we start is correct, but then we can end up at some point having some garbage instruction. For instance, if you have some, some padding bytes that are, that are just there to, to ensure alignment in the code, linear sweep is not able to, to cut that. On the other hand, you can have what is called recursive traversal, which is in this case something more clever. It's really trying to do what the CPU will do that is following the, the flow, the control flow and when you have the jump, jumping, which has enabled to, to skip the, the garbage bytes, for instance. But at the same time, you can skip some instruction because you, you don't necessarily know where, when, in which condition a block should be taken. So, well, just to say, in Saber, we are using the, the static version because it's really, really efficient in terms of, of cost and previous research has shown that it works very well with, at least with code that is produced by, well, most compilers, GCC, LLVM and for most targets. So, X86, RISC-5 for instance, it's known to not work very well for ARM, but, well, for, at least for, for our purpose, it's, it works quite, quite well. So, okay, let's, let's put it together and try to see what we can achieve with all of this. So, just as a reminder, we want to do this, we want to inject the code, replacing the real error code that is returned by a system called, by some error. So, what we do is that we intercept those system calls. We, well, we just rewrite the instruction that actually executes the system calls. So, for instance, on X86, you have one C-scall instruction. Well, so, what Saber does is just rewriting all the C-scall instructions at once, at the very beginning of the execution. So, what happened is that when the operating system loads the binary into memory, the program, Saber will catch it and you will rewrite everything, well, all the system calls, even before the execution actually starts. So, it does, it does that at the very beginning. So, that you pay relatively small price at low time, but then at runtime, you don't have to do any disassembly, you don't have to do any actual rewriting. The only code that you pay is just what I showed you, just taking the detour, having a, for each system call, a couple of instructions added. So, in terms of overhead, as I show you in a minute, it's really, really good. So, yes, that's basically what we do. So, in terms of how to use it in concrete terms, so Saber itself, what I call the backbone of Saber, it's just a loader that will load your application, rewrite the system calls, and then execute the program as normal. If you want to actually do something with those system calls, you have to write a small plug-in using an API that is really, really simple. So, just an example of how that works. Let's say that we want, well, this is our thing before, just inject false. So, we just have to implement, I say we, the user, the user just has to implement this function, whatever the name or what it does is that, it's like a hook that is called by Saber, every time it reaches a system call, okay? So, at runtime, it goes through the out of line squash space, then it jumps, like with the trampoline, it jumps into this function, and this function, well, it's just a usual C code, so you can do whatever you like. You can change the arguments of the system call, you can change the return value, which is what we want to do. You can even end up not calling the system call at all. And, well, basically, that allows you to do what you want. So, well, let's see what it would look like if we want to achieve this basic fault injection problem. So, we have this function, let's call it a handleSystemCode, which takes the number of arguments, so the first argument is the number of the system call. So, here we want to catch one specific system call, which is the right system call. Well, all the other arguments are just the real argument to the system call. So, we just pass them directly. But in the case, in the very specific case, when we get the right system call, what do we do? In this case, well, we can call it or even decide not to call the system call at all. But in any case, what we want to do is return some arbitrary error code. So, here, for instance, we return the E-perm error code, which is a permission error. And then, your application is going to see this instead of the expected successful return code. It will see this, and at that time, it will have to decide what to do. And you end up seeing what happens. It depends. So, maybe it will be very nicely or you get some error message or a complete crash or whatever. So, that's how you end up seeing what can happen when you have some random bug or some random problem with the operation you're doing with the operating system, right? So, just a few words to tell you that. So, currently, there are three, yes, three plugins that are open source and available for you. So, well, the first one is basically just registering the system call without changing anything. It's basically for testing purpose. You have a full team director that is a bit more sophisticated than the basic one that I showed you. And you have a tracer that is, well, if you know S-trace, it's basically doing the same, just a recording, printing on the screen all the system calls that are called with their arguments and return values. But it's much more efficient. I'm going to show you in a couple of minutes. So, for now, we support two targets. I mentioned before, so x86, 64 and RISC-5. And we get, oh, yes, so the current implementation targets exclusively Linux as an operating system, I guess it would be not too difficult to extend it to other unices, but, well, so far it's not been done. It's written mainly in C with some assembly snippets. Very important, it doesn't use any third-party library, not even for disassembly. So if you look at all the software that exists that do binary writing, usually they rely on a bunch of third-party libraries, especially for disassembly. We don't do that. And that's why we end up with something really, really tiny. So 47 kilobyte for the executable for x86, 40 kilobyte for RISC-5. So you can even use that for small embedded types, embedded targets that will work very well. Just to show you, because I mentioned earlier that Cable is able to be very efficient, so just to show you the kind of overhead that we get. So we measure that with some well-known server application like MAMCHD, Redis, et cetera. So we end up with an average load time overhead that is, not average, sorry, but maximum measure load time overhead that is 60 milliseconds, that is, well, lower than the perceived response time for the user, which is 100 milliseconds. The runtime overhead is very, very low, as I mentioned before. Usually below 3%, in most cases, around 1%. So we compared our system called Tracer with the original S-Trace trying to mimic the output. And we got, well, we got very, very good results. As I can show you here, yes. So, well, native is just the original execution without tracing, without instrumentation or nothing. So we compared the original S-Trace with our implementation of the plugin SBR. And also with some PIN tool doing the equivalent. And, well, as you can see, we are, well, obviously we are, there is an overhead compared to native implementation, but compared to S-Trace, it's really, really efficient. Yeah, the first one, sorry, the first one was on the D-U GNU tool, basically reading and writing in a loop. And we also did some experiments on D-U trying to estimate the usage of a big directory. And we end up with more or less the same results as we did. And finally, yes, the fault injector. So we got, we found some bugs in the core utility tools. Core utility is basically, well, all of the basic tools that you have and that you use all the time in your terminal, CUT, LS, etc. So, yes, we found some bugs, different kind of bugs, thanks to the random probability tool that we have in our fault injector. So to sum up, so Saber, what is he doing? It's selective, selective, well, I didn't define selective before, but as you have understood, selective because it's targets mainly as stem calls. By the way, it's also able to rewrite function for logs if you want to, well, to intercept some specific function. It's also able to rewrite the RDSC instruction in X86. So RDSC is just reading the timestamp counter. So it works at load time, just ahead of time, as we call it, very low overhead. So it's suitable for embedded devices. The API is quite simple. So you can build your plug-in data by basically writing one function in a couple lines of code. We have embedded available plugins, open source on GitHub and, well, everything is GPLv3. So, yes, just feel free to play with it. If you have any questions, I'll be happy to take them. Thank you. Okay, so we have around four minutes for questions. Thank you for this talk, Paul. And who has a question? Here. Thank you for your presentation, most interesting technology. You mentioned a case where you did parts of the rewriting by modifying an error handler. Yeah, sorry, I can't hear very well. Yes. Okay, you mentioned a case where you did part of the rewriting using an error handler. Yes. So now I'm curious, how do you make sure that this same error handler is not invoked from an entirely different place which would disturb the meaning of your system? So if I understand your question correctly, you are asking when we relocate instruction into an error handler which might also be called from a different place. Okay, so when we don't have space to insert the jump, we use a signal handler to catch the signal. So your question is when we do that, how do we ensure that we are actually handling an instruction that we generally have rewritten rather than some real signal, some real illegal instruction that we weren't aware of? Is that your question? Yeah. Yeah, so actually what you do is that you have to, when you're inside your signal handler, you have to check the address of the instruction that just threw the signal handler. So you look at this address, you see what exactly it is, the operation. In this case, you are most interested in system call. So if it's a system call, well, that's fine. If it's something else, when you just fall through and you let it be handled by the operating system. Does that answer your question? Okay. Well, if you have more questions, you can take it offline if you want. Is that a question? Okay. Have you, with your approach, have you encountered any counter mechanism which is provided in, I mean, this way you don't really need the source code, right? You can do this on arbitrary binaries. We don't need the source code yet. Yes. So I assume you have experimented with third party software as well doing your approach here? Well, we've experimented with, well, not all possible, not all possible options that we have made with, for instance, I showed the pin tool to do the same. My question is, if you have any, well, algorithm or any procedure in place, if the binary that you are injecting into has precautions against like injections, if you have stumbled across something like this? I'm not sure I understand what you're asking. What do you want to check again, the code that is injected? Yeah, I'm not sure. This is my question actually. So if you have stumbled across any situations where this approach does not work. Oh, the approach with the detour. Well, the only problem that can, well, I can think of two limitations, two problems mainly. When we can't relocate anything surrounding and well, then you have to use a sigil. It mostly works, but, well, you can end up with problems and with that. The other issue is that I mentioned that you can't relocate instructions that are target of jumps. The problem is for direct jumps, I mean, jumps with the target encoded as an offset as I showed you that works. But if you have an offset, a target of a jump that is dynamically computed, for instance, as a result of a computation, well, that depends on the data. And I want to use this static. You can't, you can't really do anything about that. So yes, that would be the main problem, the problem with indirect jumps. But if I wanted to make my binary resilient against this approach, that I would just require, I would just add a custom sigil handler that I need to invoke. And if you then override it, I know, okay, someone is fiddling, I better quit, I better die. So this, actually, this, as I mentioned, this approach works well with, well, you know, mainstream compiler, GCC, LLVM that do, well, sane and sound stuff. But if you end up with some code that has obfuscation mechanism, because, well, some, that's, that's happened. Yes, with obfuscation, that wouldn't work because, for example, you know, you can have overlapping instructions, you can have embedded instruction, you can even have a self-fuelling code, self-modifying code. Well, in this case, we can't do anything about that. But I would say for 99.999% of the code, that, that works quite well.
 So, hello everybody. We're going to start the next talk which was presented by Paul Antoine  Arras and which is a low-level engineer and he will talk about SAP. Thank you.  Thank you for the introduction. So, yes, I'm going to tell you about software that I developed  with the  time selected binary writing. Let's start with some quite simple use case just to see  why we need to do such low-level first step. So, for instance, let's say that you are a  developer. You've developed a brand new software and you want to test to see how resilient  your software is. Well, basically, you want to see if you can find some tricky bugs and  fix them easily with some tools. So, basically, if you want to set the full tolerance of your  software, so for a particular example, well, you can have this rule of memory RAM exhausted  or whatever kind of problem you can run into. And you want to check that your software is  able to cope appropriately with this kind of issue. So, the problem is it's hard to  reproduce on a real system. Well, if you want to simulate on a real system full disk or lack  of RAM, well, that's quite painful. So, usually, what you want to do is to simulate this. So,  the question is how could you achieve this? So, it's basically possible. You have really  several options. One of the options is to tinker with the kernel, the Linux kernel,  for instance. Well, for kernel hackers, that's great. But, well, for most people, it's quite  quite painful and dangerous. So, you don't want to do that. You can also try to modify,  to adapt or to intercept the libraries that support your software. But, once again, it's  quite painful and not really reliable. So, actually, there is an intermediate solution  that is trying to see what's between the libraries and the operating system. So, let's  have a look at a simple, very simple Python program, for instance. Well, here, the example  is with Python, but you can do that with whatever language you want. And, well, with a basic  hello world, what you want to do is just print the string hello world onto the string, onto  the terminal. So, basically, what happens is that, so, you have your user code, so, basically,  this line of code that we call into some libraries. So, in this case, it will be the Python runtime.  And at some point, the Python runtime will call into the operating system to, well, actually  print the string onto the string. And what we're interested in here is the interface  between the library and the operating system. So, this interface is called the system called  interface. And this is the single interface that's available to communicate between the  user space, your program, and the operating system. There is no other way. So, you are  absolutely sure that any code that you are making through the operating system will go  through this interface. So, that's very important for us. What's next? So, just let's take a  look, a quick look at what is exactly this system called interface. How does it work?  What is it for? So, actually, it's a set of low-level operations, like, well, a kind of  primitive, really relatively low-level, like if you want to write characters on the screen,  as I said previously, or, for instance, to send packets to the networks, or communicate  with some random peripheral, well, whatever you want to do, especially in terms of input  or output, you have to go through this and use one of the functions that are available.  So, basically, system calls are really like regular function. That is, you have a set  of arguments that you pass to the system call, and you get one result in return, depending  on the number of arguments that vary, usually between zero and six, depending on what you  want to do, you will end up using one of these functions. So, for instance, if we go back  to our Python hello world, the print hello world will translate into the write system  call, which is the way that you use to write some character on the terminal, for instance,  or also to write some data into a file. So, in this case, this write call takes three  arguments. The first one is the file handle. So, well, as most people know, under Linux,  the standard output is associated with the file descriptor number one. That's the reason  why we have one at the first argument, then the string that you actually want to print  hello world, and finally, the side of the string, which is here, 13. So far, so good.  Now, as I said, what we want to achieve is to simulate a fault, a problem in terms of,  for instance, we have this rule, what exactly happened. So, we want to take a look at the  return value of the system call, because this is what we tell us if the system call has  succeeded or if there was an error. This return value can be either positive if it's successful  or negative. In this case, the exact value will indicate the type of error. So, in the  case of the write system call, if it is successful, you get the side that was actually written.  And if it's not, well, you get some error code that will describe the error that you  got. For instance, permission denied. The code is E perm or this rule, in which case,  you get E no space. So, for instance, let's say that we have this Python program slightly  more complex than the previous one. This time, we want to write into a file. So, we  use the open function, and we want to write into the file whose path is slash tmp slash  hello in write mode. And then, yes, we write the string hello into this file. So, how does  it translate in terms of system call? So, we have two system calls. The first one is,  well, as you can guess, the open system call that takes, well, basically the same arguments  at the real function. And once the file has been opened, if it succeeds, it returns the  number of the file descriptor, which is 8 here. And then, we just have to get this number,  the file descriptor, and pass it to the write system call in turn. So, that's what we do.  And we are calling the write system call with 8 as the first argument, the string and the  say of the string. So, here we have two possible outcomes. Either it's successful, in this  case, it returns the number of characters returned, 6, or if it's not, well, it will  return some negative value, which is there are code that describes the problem. So, for  instance, you could get an E-perm code. We tell you that you don't have the necessary  permission to write into this file. Right. So, let's go back to our original example  of fault injection. We want to simulate some faults. For instance, simulate the fact that  the disk is full, so that you can't write to a file. So, the question is, in terms of  system call, how does it translate? So, what we want to do, actually, is to take the error  code that is really returned by the system call and swap it with an error code. So, that  even though the operation has been successful, your application will get an error code. And  we think that there was the error decoded and encoded by the error code. And at that  point, you'll be able to assess exactly how your application is responding to it. So,  yes, let's do this. So, for instance, let's take this write call and replace the return  value, the real return value, which is fixed, with an E-perm code. So, how to achieve that?  Well, that's where binary writing comes into play. So, what exactly is binary writing?  So, in simple terms, it's modifying a program, not at the source code level, but at the machine  code level. That is the code, the binary code that is actually read and executed by the  CPU. So, the good point is you don't need the source code. Well, sometimes you don't  have access to source code at all, but well, most of the time, even if you have the source  code, you don't necessarily want to recompile it every time, especially if you want to deal  with a false detection. So, well, one good thing with binary writing is that you don't  have to modify the source code. You don't need to recompile anything. You just have  to get your binary and work directly on it. So, the only requirement for binary writing  is disassembling. So, for now, let's keep disassembling apart and let's just say that  what we need is the representation of the programs in terms of assembly instructions.  So, it basically means that you have your binary code. It's just a sequence of zero  and one. So, that's really what the CPUs are seeing and what they are able to execute.  But well, we need to be able to have a slightly more human readable version of this. So, this  is where this assembly comes to play. It's basically selecting what we have on the left,  the sequence of this into something that is more or less, I say more or less, readable  by human, which is just a sequence of instruction. So, that's really a translation, a perfect  translation. There is no loss of information and you can then reconvert the assembly version  into the original bit sequence without any loss of data. So, once we have this disassembly,  we are able to do interesting stuff. So, let's take this representation. So, so far,  so I've been showing you just a small snippet of assembly and the rest of this talk. Anytime  I will show you some assembly, it will look like this. There will be three columns. The  first one is your set from the very beginning of your program. So, here, let's say that  just an excerpt, so the very beginning of your program, just the first couple of instructions.  So, we start with set zero and then we increment. Second column is the size. So, I'll explain  later why I should decide because that's really important when we do binary writing. And finally,  the human readable version, which is just what the instruction does in a way more readable  by human. Okay. So, let's take this and then let's try to see what operation we can do  on instruction. So, we can do three, exactly three operations. We can remove instructions,  we can replace instructions, or we can add instructions. That's the three types of instructions  that we can do in binary writing. And I'm going to show you how we can do each of them.  So, let's start with the very first, the simplest thing that we can do is removing instructions.  So, here, what we do is that, let's say that we have this program on the left-hand side.  So, here we want to remove the instruction in the red, okay? So, the load something.  So, what do we do? Well, we just remove it, but we don't just remove it because, well,  it's not like in a classic text editor, you can just remove the part that you're not interested  in and have everything that will fit between. Here you have to do an additional operation.  So, the additional operation is to pad the gap that you've just created by adding some  instruction that do basically nothing. So, the instruction that do nothing here is no,  no operation. So, since each of these instructions is just one byte long, you put two and then  you end up having sealed the space which is two bytes long with the load here, right?  Well, so you're going to ask, but why do we need to do that? Why not just remove the load  operation and then shift everything? So, that's, well, we end up with no, no load space. Well,  the problem is if you do that, you'll change the offset of the instruction. For instance,  here you can see that the load instruction has offset one and everything after offset  three, eight, et cetera. Here, if you want to remove the load instruction without padding  the space, we'll end up having the cool instruction at offset one, the old instruction at offset  six, well, everything will be shifted. And the thing is in assembly, in machine code,  you need to have your set right to be able to jump from one point of the code to another.  So, basically, in a high level language when you have a nif, a while, a loop, or any conditional  structure, it's basically saying in terms of assembly, well, we are jumping from one  point in the program to another. And the jump in assembly code is achieved by just recording  the offset where we want to jump. So, well, that's the reason why we don't do that. We  don't just remove the instruction that we're not interested in. We need to keep the offset  the same, right? So, that's what's to remove. Now, let's see how we can replace some instruction,  slightly more complex, but let's start with a simple case. So, let's say that we want  to replace a call. So, call, you read the way that you are calling a routine, subroutine,  or a function. So, let's say, for instance, that we have this snippet of assembly and we  have this call instruction. We want just to replace it with a jump to somewhere else,  okay? So, here is how we translate. So, we just replace the call with a jump and we end  up with this assembly sequence. But now, now that we have this, we have to ask ourselves  the question, what exactly is the size of this jump instruction? Because as I just said,  you have to ensure that you keep the offset of the surrounding code the same. So, the  question is, what is the size of the jump? Because if the size is five, well, you will  end up with the same offset for the next instruction. It will be eight, but if it's different, well,  you will shift everything. So, you have to be careful with the size of the new instruction.  So, basically, you need to compare the related sizes of the original instruction and the  instruction with which you want it to be replaced. So, yes, that's what I just said. So, let's  introduce a bit of notation. So, I note S of O, the size of the original instruction,  S of R, the size of the rewritten, the new instruction, and then we compare them. So,  in this case, we compare the size of the red instruction, the one that we remove, with  the size of this new jump instruction, which is in green. And in this case, okay, let's  say, let's say that it's quite simple, we have the same size, five for the call, five  for the jump. That's the same thing. In this case, well, quite simple. We just, sorry,  quite simple. We just replace and we do nothing else. Nothing to pad, nothing to shift. That  just fits. That's perfect. But, well, as you can guess, most of the time, it's not that  simple. So, for instance, let's imagine that the jump is shorter. It's three instead of  five. So, in this case, well, you can guess what we do. We pad, just like we did when  we remove. That's more or less equivalent. But then, obviously, we can get into cases  where the jump, so the new instruction is larger than the original one. And in this  case, well, what do we do? So, that's quite a problem. They usually, well, quite a basic  problem in terms of binary writing. So, there are several ways to deal with that. I'm going  to show you the way that we use in Saber, which is called D2R. So, D2R, or D2Ring, is  just taking the instruction that do not fit anymore in the original code space. So, where  we added an instruction, or we replaced the original instruction with a larger one. So,  there are some instructions that can't fit anymore. So, we have to take them out, move  them somewhere else. So, the somewhere else is what I will call an out of line scratch  space. So, out of line because it's not in line anymore. It's not part of the straight  instruction stream. It's somewhere else in the memory. Okay? So, the way we do that is,  well, basically, we are allocating memory somewhere else. We are moving the instruction  that do not fit anymore in the original main line stream to this out of line scratch space.  And then, obviously, we need to connect this out of line space to the original main line.  And we do that by inserting some jumps. So, let's take it on a simple example. Let's say  that we want to add an instruction with such a D2R. In this case, let's say that we want  insert a jump. No, sorry. Let's say that we want to insert some instruction. It can be  one instruction. It can be a sequence of instruction. No matter the length of the sequence you want  to insert, the principle is the same. So, let's say that we want to insert at the point  where you see the green arrow. Okay? So, right after the load and before the call. So, how  can we do that? So, first, we are locating the out of line scratch space, which is at  the bottom right corner. And there, we move the call instruction because it can't fit  anymore. Once we have rewritten this part of the call. So, we move it. The original part,  we move it from the main line to the out of line scratch. Then, we insert the add instruction.  So, here, I didn't represent them, but you can think of it as just one instruction of  a snippet of how many instructions you want. It doesn't matter. And then, last part that  I mentioned, you have to connect this out of line scratch space with the rest of the  code. And to do that, you use jump. So, first, we insert a jump so that the first green instruction  that you can see at offset three. And you jump to some label that is representing the  beginning of the outline scratch space. So, the label here is D0. So, when you're here,  which should be the place when you have the call instruction in the original code, instead  of directly executing the call instruction, you will jump at the beginning of the outline  scratch space. And then, at that point, you can execute the original instruction, the  call, as if it were still at the same place. The semantics doesn't change. You have this  guarantee. And then, you can just execute whatever arbitrary instruction you want. And  finally, let's not forget to jump back to the main line with this jump. So, this time,  we've inserted another label called L0, which is just right after the jump. To that, when  we have executed the instruction, we just come back and then continue, well, I was about  to say, as if nothing has happened. Well, something has happened. But the rest of the  code, the surrounding code, remains unchanged. Right. So, this is very important. This is  really the basics of how everything is working. So, in this case, you can notice that the  size of the original instruction, the call here in orange, is the same as the jump. Five.  Both are five. Well, that's fine. But as I mentioned before, you have to be careful  that when you replace one instruction with the other, you have to compare the relative  sizes and see if that fits or if you have to adapt. So, in this case, we do exactly what  we've done before. We have to compare this time not the real instruction, but the jump  that we insert with the original instruction. So, in this case, let's say that, well, we  have basically two cases. Either the jump is less or equal to the size of the original  instruction. In this case, well, you just insert it. If need be, you pad with knobs  and everybody is happy. Or if it's not the case, if the jump itself is larger than the  original instruction, well, in this case, you have to relocate the number of instructions.  So, for instance, let's go back to our previous example. We want to replace this load instruction  and we want to insert a jump. So, in this case, let's say that the original instruction,  the load that we want to replace is only two, two bytes, which is quite short. In comparison,  the jump is five. So, well, in this case, the jump is quite larger than the original  instruction. So, you need to, well, you need to make space even to insert the jump. So,  well, what you do is just, as I mentioned before, you take the call instruction that  is just next to the instruction that you want to replace. You move it to the outer line  of the space. The rest is the same. You still need to locate the space. So, I don't go back  to that. But once you have your outer line's crash space, you just move the surrounding  instruction at the right place. So, here, after the instruction that you want to substitute,  since it is, since you are relocating the instruction that is after the instruction  that you, that you replace, and you end up with this result, so still the jump to the  outer line space and then jump back to the main line. So, well, I've, I've basically  covered all the possible outcomes, depending on the side, the relative sizes of the original  instruction and the return instruction. And same for the jump instruction. In case you  have to, to have this outer line's crash space. So, yeah, basically the most complex case  that you can face is that you have a couple of instructions around the jump that you insert  that you need to relocate. So, now the question is, can instruction always be rewritten, relocated,  sorry. Can you always rewritten arbitrarily selecting instruction around the spot where  we want to replace or other instruction? Well, you get that if I'm asking the re, the answer  is no. So, I'm going to show you the, well, some typical examples of cases where you can't  or you have to be very careful when you relocate instructions. So, for instance, PC relative  addressing. So, what is PC relative addressing? So, usually when you want to, I give you the  example of a jump, when you want to jump to some part of the code, for instance, if you  have a loop, a loop is basically having a jump backwards into the instruction stream,  so that you will execute this sequence several times. And usually the most efficient way  to do that is instead of encoding the address, the full address of the place where we want  to end up, the target of the jump, which would be, well, 32, 64 bits, which is quite long.  Usually, you just use the displacement, which is the distance, the offset between the source  of the jump where we're jumping from to the target where you're jumping to. The difference  between the two. Usually, it's, well, a couple of bytes because, well, it's quite rare that  you have to jump, especially when you have a typical control structure, that you have  to jump to a place very, very far away in the code. So, usually, you just have to encode  the displacement, the distance between source and target, and that's much more efficient.  So, in this case, for instance, let's say that we want to load some data that is located  48 bytes from the current value of the PC, which is basically where we are, the current  instruction that is executed. So, 48 bytes from the place where we are currently executing,  right? And we want to exert an instruction where we have the green arrow, right before  the load instruction that has the displacement addressing. So, what do we do? Well, we just  applied the D2, the D2 mechanism that I just presented earlier. So, we end up with the  same kind of stuff with our jump. Well, we need to insert some note because we are relocating  the load instruction because, well, we need to make the jump fit. And so, we end up here  in the out of line scratch space, we end up with this displacement, 48 from PC. Well,  the problem is, since we moved the instruction to somewhere else, you can see here the offset  is quite different, it's FFEA, and the original set was two. So, we are very, very far away  from the original space. So, this offset is not correct anymore because we didn't move  the data that it put into. So, we end up actually with quite a significant mismatch between  the right displacement, which is 49 in the original instruction. And after relocating  the instruction, you get this displacement, which is 10032, well, that's completely wrong.  So, you have to be careful with that. And you have to, well, basically just to compute  the actual displacement after having relocated the instruction. So, well, you just compute  the difference between the original displacement and the new offset. So, here, 49 minus FFE6,  and you end up with the new displacement once you have relocated the instruction, which  is minus FFE90. Okay? So, well, that's feasible, that's feasible. But usually, we try to, when  we do binary writing, we try to avoid it because we can be quite painful.  So, second problem that can arise when you want to relocate some instruction. So, I told  you about jumps, I told you about source and targets. So, well, basically, here, imagine  that we have a jump backward, like in a loop. And we want to insert an instruction right  before the target of the loop, which is here represented by the L0 label, right? So, what  do we do? Well, once again, we do the same kind of thing with the detour. We move the  original surrounding instruction, which in this case is this load. We add the instruction,  et cetera, as usual. No problem. When you end up trying to execute the jump L0 instruction  after writing what happened? Well, you've lost the L0 because you rewrote the sequence  of instructions, which is load then call. And you can see that the target, the L0 label,  is pointing right in the middle of this. And since you've completely overwritten this,  if you still want to jump at this address, well, you end up jumping right in the middle  of another instruction. So, this is completely wrong. And in this case, well, there is no  simple way to solve it. So, what we do is that we scan the code before even starting  to rewrite. We record all the branch targets. And we say, okay, this instruction, we don't  do anything with them. So, we just keep them at place. And we won't even try to relocate  them. We need to do that otherwise. We end up with some trying to decode some non-existent  illegal instruction. And that can be quite problematic. So, I just show you two examples  of typical problem that you will end up when you want to relocate instruction. There is  a third one that is side effects. I won't go into the details of the side effects, but  just keep in mind that you have three types of problems or challenges. And you have to,  you have just to keep them in mind when you rewrite instruction.  So now, what if we end up, for instance, with a target for a jump right before the instruction  that we want to insert, and also some instruction with side effect right after, which means that  you just, you just write in between two instructions that you can't relocate. So, what happens  in this case? Because you may, for very valid reasons, be willing to rewrite some instruction  at this precise position. And well, if you end up having surrounding instruction that  you can't relocate, well, you're stuck. Well, not completely. You can still end up doing  something which is not strictly speaking binary writing, but something that can allow you  to do what you want. So, actually, what you can do is just replace the instruction that  you are interested in with an illegal one. So, an instruction that is known to not encode  an illegal operation. So, you replace it with this instruction. And what you do is that  you set up a signal handler that will catch the signal that will be raised by trying to  decode an illegal instruction. So, once you've set up this, you just have to catch the signal.  Obviously, you need to record the source address of the instruction that triggered the signal.  But then, you can do whatever you want inside the signal handler. So, well, as you can imagine,  that's quite a significant overhead doing this, having to go through all the mechanisms  of the signal handler. But since, well, in our experience, that really, really happened  rarely, well, the overhead is, is structural. So, well, that's really the last resource,  the last resource solution that, at least, we have a solution when that happens. We mean  that we can rewrite virtually all instruction, any arbitrary instruction in the program.  So, I promise you that I'll go back to the disassembly problem. Well, at the time, I  didn't tell you that there was a problem, but actually, there is a problem. So, let's,  let's go back. Let's, let's make one step backward and look at what we exactly have when you,  when we actually look at some machine code, some program code in terms of binary. So,  we have, well, on the left, this binary sequence, some arbitrary one, and we want to turn it  into something that is, well, understandable by a byte human. And especially, we want to,  to know exactly which bit, or which bytes correspond exactly to which instruction, because the  instruction is the granularity that we are interested in. So, you can see here that actually  what it is really meant for is being able to, to match a sequence of bytes with some specific  instructions. So, for instance, here you have the yellow sequence of bytes that will be  representing a push instruction, the orange one will represent the load, et cetera. So,  that's, that's exactly what, what we want to achieve. The problem is, it's not always  easy. You don't always get it for free. So, you have to, you have to have some, some,  some algorithm or some, some technique that will, that will guarantee that you have the  right, the right DDSMB. So, there are, there are different, different techniques that allow  you to get, to get the DDSMB. The dynamic one, the static one. Well, both have advantages,  but for, for the purpose of binary writing, in Saber, we will use the static one. So,  there are basically in terms of starting the assembly, there are many two ways of doing  this. Either what you have on the left, which is linear sweep. So, linear sweep is just,  well, we basically start at the very beginning of the program and we scan it instruction  by instruction. We know, we know where the program starts because, well, otherwise the,  the OPC system is not even able to load it. And so, we can, we know, actually that the  point where we start is correct, but then we can end up at some point having some garbage  instruction. For instance, if you have some, some padding bytes that are, that are just  there to, to ensure alignment in the code, linear sweep is not able to, to cut that.  On the other hand, you can have what is called recursive traversal, which is in this case  something more clever. It's really trying to do what the CPU will do that is following  the, the flow, the control flow and when you have the jump, jumping, which has enabled  to, to skip the, the garbage bytes, for instance. But at the same time, you can skip some instruction  because you, you don't necessarily know where, when, in which condition a block should be  taken. So, well, just to say, in Saber, we are using the, the static version because  it's really, really efficient in terms of, of cost and previous research has shown that  it works very well with, at least with code that is produced by, well, most compilers,  GCC, LLVM and for most targets. So, X86, RISC-5 for instance, it's known to not work  very well for ARM, but, well, for, at least for, for our purpose, it's, it works quite,  quite well. So, okay, let's, let's put it together and try to see what we can achieve  with all of this. So, just as a reminder, we want to do this, we want to inject the  code, replacing the real error code that is returned by a system called, by some error.  So, what we do is that we intercept those system calls. We, well, we just rewrite the  instruction that actually executes the system calls. So, for instance, on X86, you have  one C-scall instruction. Well, so, what Saber does is just rewriting all the C-scall instructions  at once, at the very beginning of the execution. So, what happened is that when the operating  system loads the binary into memory, the program, Saber will catch it and you will rewrite everything,  well, all the system calls, even before the execution actually starts. So, it does, it  does that at the very beginning. So, that you pay relatively small price at low time,  but then at runtime, you don't have to do any disassembly, you don't have to do any  actual rewriting. The only code that you pay is just what I showed you, just taking the  detour, having a, for each system call, a couple of instructions added. So, in terms  of overhead, as I show you in a minute, it's really, really good. So, yes, that's basically  what we do. So, in terms of how to use it in concrete terms, so Saber itself, what I  call the backbone of Saber, it's just a loader that will load your application, rewrite the  system calls, and then execute the program as normal. If you want to actually do something  with those system calls, you have to write a small plug-in using an API that is really,  really simple. So, just an example of how that works. Let's say that we want, well,  this is our thing before, just inject false. So, we just have to implement, I say we, the  user, the user just has to implement this function, whatever the name or what it does  is that, it's like a hook that is called by Saber, every time it reaches a system call,  okay? So, at runtime, it goes through the out of line squash space, then it jumps, like  with the trampoline, it jumps into this function, and this function, well, it's just a usual  C code, so you can do whatever you like. You can change the arguments of the system call,  you can change the return value, which is what we want to do. You can even end up not  calling the system call at all. And, well, basically, that allows you to do what you  want. So, well, let's see what it would look like if we want to achieve this basic fault  injection problem. So, we have this function, let's call it a handleSystemCode, which takes  the number of arguments, so the first argument is the number of the system call. So, here  we want to catch one specific system call, which is the right system call. Well, all  the other arguments are just the real argument to the system call. So, we just pass them  directly. But in the case, in the very specific case, when we get the right system call, what  do we do? In this case, well, we can call it or even decide not to call the system call  at all. But in any case, what we want to do is return some arbitrary error code. So, here,  for instance, we return the E-perm error code, which is a permission error. And then, your  application is going to see this instead of the expected successful return code. It will  see this, and at that time, it will have to decide what to do. And you end up seeing what  happens. It depends. So, maybe it will be very nicely or you get some error message or a  complete crash or whatever. So, that's how you end up seeing what can happen when you  have some random bug or some random problem with the operation you're doing with the operating  system, right? So, just a few words to tell you that. So, currently, there are three,  yes, three plugins that are open source and available for you. So, well, the first one  is basically just registering the system call without changing anything. It's basically  for testing purpose. You have a full team director that is a bit more sophisticated than  the basic one that I showed you. And you have a tracer that is, well, if you know S-trace,  it's basically doing the same, just a recording, printing on the screen all the system calls  that are called with their arguments and return values. But it's much more efficient. I'm  going to show you in a couple of minutes. So, for now, we support two targets. I mentioned  before, so x86, 64 and RISC-5. And we get, oh, yes, so the current implementation targets  exclusively Linux as an operating system, I guess it would be not too difficult to extend  it to other unices, but, well, so far it's not been done. It's written mainly in C with  some assembly snippets. Very important, it doesn't use any third-party library, not even  for disassembly. So if you look at all the software that exists that do binary writing,  usually they rely on a bunch of third-party libraries, especially for disassembly. We  don't do that. And that's why we end up with something really, really tiny. So 47 kilobyte  for the executable for x86, 40 kilobyte for RISC-5. So you can even use that for small  embedded types, embedded targets that will work very well. Just to show you, because I  mentioned earlier that Cable is able to be very efficient, so just to show you the kind  of overhead that we get. So we measure that with some well-known server application like  MAMCHD, Redis, et cetera. So we end up with an average load time overhead that is, not  average, sorry, but maximum measure load time overhead that is 60 milliseconds, that is,  well, lower than the perceived response time for the user, which is 100 milliseconds. The  runtime overhead is very, very low, as I mentioned before. Usually below 3%, in most cases, around  1%. So we compared our system called Tracer with the original S-Trace trying to mimic  the output. And we got, well, we got very, very good results. As I can show you here,  yes. So, well, native is just the original execution without tracing, without instrumentation  or nothing. So we compared the original S-Trace with our implementation of the plugin SBR.  And also with some PIN tool doing the equivalent. And, well, as you can see, we are, well,  obviously we are, there is an overhead compared to native implementation, but compared to  S-Trace, it's really, really efficient. Yeah, the first one, sorry, the first one was on  the D-U GNU tool, basically reading and writing in a loop. And we also did some experiments  on D-U trying to estimate the usage of a big directory. And we end up with more or less  the same results as we did. And finally, yes, the fault injector. So we got, we found  some bugs in the core utility tools. Core utility is basically, well, all of the basic tools  that you have and that you use all the time in your terminal, CUT, LS, etc. So, yes, we  found some bugs, different kind of bugs, thanks to the random probability tool that we have  in our fault injector. So to sum up, so Saber, what is he doing? It's selective, selective,  well, I didn't define selective before, but as you have understood, selective because  it's targets mainly as stem calls. By the way, it's also able to rewrite function  for logs if you want to, well, to intercept some specific function. It's also able to  rewrite the RDSC instruction in X86. So RDSC is just reading the timestamp counter. So  it works at load time, just ahead of time, as we call it, very low overhead. So it's  suitable for embedded devices. The API is quite simple. So you can build your plug-in  data by basically writing one function in a couple lines of code. We have embedded available  plugins, open source on GitHub and, well, everything is GPLv3. So, yes, just feel free  to play with it. If you have any questions, I'll be happy to take them. Thank you.  Okay, so we have around four minutes for questions. Thank you for this talk, Paul. And who has  a question? Here.  Thank you for your presentation, most interesting technology. You mentioned a case where you  did parts of the rewriting by modifying an error handler.  Yeah, sorry, I can't hear very well. Yes.  Okay, you mentioned a case where you did part of the rewriting using an error handler.  Yes.  So now I'm curious, how do you make sure that this same error handler is not invoked from  an entirely different place which would disturb the meaning of your system?  So if I understand your question correctly, you are asking when we relocate instruction  into an error handler which might also be called from a different place.  Okay, so when we don't have space to insert the jump, we use a signal handler to catch  the signal. So your question is when we do that, how do we ensure that we are actually  handling an instruction that we generally have rewritten rather than some real signal,  some real illegal instruction that we weren't aware of? Is that your question?  Yeah.  Yeah, so actually what you do is that you have to, when you're inside your signal handler,  you have to check the address of the instruction that just threw the signal handler. So you  look at this address, you see what exactly it is, the operation. In this case, you are  most interested in system call. So if it's a system call, well, that's fine. If it's  something else, when you just fall through and you let it be handled by the operating  system.  Does that answer your question?  Okay. Well, if you have more questions, you can take it offline if you want.  Is that a question?  Okay.  Have you, with your approach, have you encountered any counter mechanism which is provided in,  I mean, this way you don't really need the source code, right? You can do this on arbitrary  binaries.  We don't need the source code yet.  Yes. So I assume you have experimented with third party software as well doing your approach  here?  Well, we've experimented with, well, not all possible, not all possible options that we  have made with, for instance, I showed the pin tool to do the same.  My question is, if you have any, well, algorithm or any procedure in place, if the binary that  you are injecting into has precautions against like injections, if you have stumbled across  something like this?  I'm not sure I understand what you're asking. What do you want to check again, the code that  is injected?  Yeah, I'm not sure. This is my question actually. So if you have stumbled across any situations  where this approach does not work.  Oh, the approach with the detour. Well, the only problem that can, well, I can think of  two limitations, two problems mainly. When we can't relocate anything surrounding and  well, then you have to use a sigil. It mostly works, but, well, you can end up with problems  and with that.  The other issue is that I mentioned that you can't relocate instructions that are target  of jumps. The problem is for direct jumps, I mean, jumps with the target encoded as an  offset as I showed you that works.  But if you have an offset, a target of a jump that is dynamically computed, for instance,  as a result of a computation, well, that depends on the data. And I want to use this static.  You can't, you can't really do anything about that. So yes, that would be the main problem,  the problem with indirect jumps.  But if I wanted to make my binary resilient against this approach, that I would just require,  I would just add a custom sigil handler that I need to invoke.  And if you then override it, I know, okay, someone is fiddling, I better quit, I better  die.  So this, actually, this, as I mentioned, this approach works well with, well, you know,  mainstream compiler, GCC, LLVM that do, well, sane and sound stuff. But if you end up with  some code that has obfuscation mechanism, because, well, some, that's, that's happened.  Yes, with obfuscation, that wouldn't work because, for example, you know, you can have  overlapping instructions, you can have embedded instruction, you can even have a self-fuelling  code, self-modifying code. Well, in this case, we can't do anything about that. But I would  say for 99.999% of the code, that, that works quite well.
sabre_0_4343.png sabre_4344_10295.png sabre_10296_11939.png sabre_11940_13811.png sabre_13812_15287.png sabre_15288_17135.png sabre_17136_22223.png sabre_22224_22355.png sabre_22356_22463.png sabre_22464_24287.png sabre_24288_25463.png sabre_25464_28259.png sabre_28260_35987.png sabre_35988_37283.png sabre_37284_39287.png sabre_39288_40703.png sabre_40704_44003.png sabre_44004_45059.png sabre_45060_58031.png sabre_58032_60215.png sabre_60216_60815.png sabre_60816_61283.png sabre_61284_61907.png sabre_61908_63623.png sabre_63624_63803.png sabre_63804_67079.png sabre_67080_70331.png sabre_70332_74003.png sabre_74004_75616.png
So, hello everybody. We're going to start the next talk which was presented by Paul Antoine Arras and which is a low-level engineer and he will talk about SAP. Thank you. Thank you for the introduction. So, yes, I'm going to tell you about software that I developed with the time selected binary writing. Let's start with some quite simple use case just to see why we need to do such low-level first step. So, for instance, let's say that you are a developer. You've developed a brand new software and you want to test to see how resilient your software is. Well, basically, you want to see if you can find some tricky bugs and fix them easily with some tools. So, basically, if you want to set the full tolerance of your software, so for a particular example, well, you can have this rule of memory RAM exhausted or whatever kind of problem you can run into. And you want to check that your software is able to cope appropriately with this kind of issue. So, the problem is it's hard to reproduce on a real system. Well, if you want to simulate on a real system full disk or lack of RAM, well, that's quite painful. So, usually, what you want to do is to simulate this. So, the question is how could you achieve this? So, it's basically possible. You have really several options. One of the options is to tinker with the kernel, the Linux kernel, for instance. Well, for kernel hackers, that's great. But, well, for most people, it's quite quite painful and dangerous. So, you don't want to do that. You can also try to modify, to adapt or to intercept the libraries that support your software. But, once again, it's quite painful and not really reliable. So, actually, there is an intermediate solution that is trying to see what's between the libraries and the operating system. So, let's have a look at a simple, very simple Python program, for instance. Well, here, the example is with Python, but you can do that with whatever language you want. And, well, with a basic hello world, what you want to do is just print the string hello world onto the string, onto the terminal. So, basically, what happens is that, so, you have your user code, so, basically, this line of code that we call into some libraries. So, in this case, it will be the Python runtime. And at some point, the Python runtime will call into the operating system to, well, actually print the string onto the string. And what we're interested in here is the interface between the library and the operating system. So, this interface is called the system called interface. And this is the single interface that's available to communicate between the user space, your program, and the operating system. There is no other way. So, you are absolutely sure that any code that you are making through the operating system will go through this interface. So, that's very important for us. What's next? So, just let's take a look, a quick look at what is exactly this system called interface. How does it work? What is it for? So, actually, it's a set of low-level operations, like, well, a kind of primitive, really relatively low-level, like if you want to write characters on the screen, as I said previously, or, for instance, to send packets to the networks, or communicate with some random peripheral, well, whatever you want to do, especially in terms of input or output, you have to go through this and use one of the functions that are available. So, basically, system calls are really like regular function. That is, you have a set of arguments that you pass to the system call, and you get one result in return, depending on the number of arguments that vary, usually between zero and six, depending on what you want to do, you will end up using one of these functions. So, for instance, if we go back to our Python hello world, the print hello world will translate into the write system call, which is the way that you use to write some character on the terminal, for instance, or also to write some data into a file. So, in this case, this write call takes three arguments. The first one is the file handle. So, well, as most people know, under Linux, the standard output is associated with the file descriptor number one. That's the reason why we have one at the first argument, then the string that you actually want to print hello world, and finally, the side of the string, which is here, 13. So far, so good. Now, as I said, what we want to achieve is to simulate a fault, a problem in terms of, for instance, we have this rule, what exactly happened. So, we want to take a look at the return value of the system call, because this is what we tell us if the system call has succeeded or if there was an error. This return value can be either positive if it's successful or negative. In this case, the exact value will indicate the type of error. So, in the case of the write system call, if it is successful, you get the side that was actually written. And if it's not, well, you get some error code that will describe the error that you got. For instance, permission denied. The code is E perm or this rule, in which case, you get E no space. So, for instance, let's say that we have this Python program slightly more complex than the previous one. This time, we want to write into a file. So, we use the open function, and we want to write into the file whose path is slash tmp slash hello in write mode. And then, yes, we write the string hello into this file. So, how does it translate in terms of system call? So, we have two system calls. The first one is, well, as you can guess, the open system call that takes, well, basically the same arguments at the real function. And once the file has been opened, if it succeeds, it returns the number of the file descriptor, which is 8 here. And then, we just have to get this number, the file descriptor, and pass it to the write system call in turn. So, that's what we do. And we are calling the write system call with 8 as the first argument, the string and the say of the string. So, here we have two possible outcomes. Either it's successful, in this case, it returns the number of characters returned, 6, or if it's not, well, it will return some negative value, which is there are code that describes the problem. So, for instance, you could get an E-perm code. We tell you that you don't have the necessary permission to write into this file. Right. So, let's go back to our original example of fault injection. We want to simulate some faults. For instance, simulate the fact that the disk is full, so that you can't write to a file. So, the question is, in terms of system call, how does it translate? So, what we want to do, actually, is to take the error code that is really returned by the system call and swap it with an error code. So, that even though the operation has been successful, your application will get an error code. And we think that there was the error decoded and encoded by the error code. And at that point, you'll be able to assess exactly how your application is responding to it. So, yes, let's do this. So, for instance, let's take this write call and replace the return value, the real return value, which is fixed, with an E-perm code. So, how to achieve that? Well, that's where binary writing comes into play. So, what exactly is binary writing? So, in simple terms, it's modifying a program, not at the source code level, but at the machine code level. That is the code, the binary code that is actually read and executed by the CPU. So, the good point is you don't need the source code. Well, sometimes you don't have access to source code at all, but well, most of the time, even if you have the source code, you don't necessarily want to recompile it every time, especially if you want to deal with a false detection. So, well, one good thing with binary writing is that you don't have to modify the source code. You don't need to recompile anything. You just have to get your binary and work directly on it. So, the only requirement for binary writing is disassembling. So, for now, let's keep disassembling apart and let's just say that what we need is the representation of the programs in terms of assembly instructions. So, it basically means that you have your binary code. It's just a sequence of zero and one. So, that's really what the CPUs are seeing and what they are able to execute. But well, we need to be able to have a slightly more human readable version of this. So, this is where this assembly comes to play. It's basically selecting what we have on the left, the sequence of this into something that is more or less, I say more or less, readable by human, which is just a sequence of instruction. So, that's really a translation, a perfect translation. There is no loss of information and you can then reconvert the assembly version into the original bit sequence without any loss of data. So, once we have this disassembly, we are able to do interesting stuff. So, let's take this representation. So, so far, so I've been showing you just a small snippet of assembly and the rest of this talk. Anytime I will show you some assembly, it will look like this. There will be three columns. The first one is your set from the very beginning of your program. So, here, let's say that just an excerpt, so the very beginning of your program, just the first couple of instructions. So, we start with set zero and then we increment. Second column is the size. So, I'll explain later why I should decide because that's really important when we do binary writing. And finally, the human readable version, which is just what the instruction does in a way more readable by human. Okay. So, let's take this and then let's try to see what operation we can do on instruction. So, we can do three, exactly three operations. We can remove instructions, we can replace instructions, or we can add instructions. That's the three types of instructions that we can do in binary writing. And I'm going to show you how we can do each of them. So, let's start with the very first, the simplest thing that we can do is removing instructions. So, here, what we do is that, let's say that we have this program on the left-hand side. So, here we want to remove the instruction in the red, okay? So, the load something. So, what do we do? Well, we just remove it, but we don't just remove it because, well, it's not like in a classic text editor, you can just remove the part that you're not interested in and have everything that will fit between. Here you have to do an additional operation. So, the additional operation is to pad the gap that you've just created by adding some instruction that do basically nothing. So, the instruction that do nothing here is no, no operation. So, since each of these instructions is just one byte long, you put two and then you end up having sealed the space which is two bytes long with the load here, right? Well, so you're going to ask, but why do we need to do that? Why not just remove the load operation and then shift everything? So, that's, well, we end up with no, no load space. Well, the problem is if you do that, you'll change the offset of the instruction. For instance, here you can see that the load instruction has offset one and everything after offset three, eight, et cetera. Here, if you want to remove the load instruction without padding the space, we'll end up having the cool instruction at offset one, the old instruction at offset six, well, everything will be shifted. And the thing is in assembly, in machine code, you need to have your set right to be able to jump from one point of the code to another. So, basically, in a high level language when you have a nif, a while, a loop, or any conditional structure, it's basically saying in terms of assembly, well, we are jumping from one point in the program to another. And the jump in assembly code is achieved by just recording the offset where we want to jump. So, well, that's the reason why we don't do that. We don't just remove the instruction that we're not interested in. We need to keep the offset the same, right? So, that's what's to remove. Now, let's see how we can replace some instruction, slightly more complex, but let's start with a simple case. So, let's say that we want to replace a call. So, call, you read the way that you are calling a routine, subroutine, or a function. So, let's say, for instance, that we have this snippet of assembly and we have this call instruction. We want just to replace it with a jump to somewhere else, okay? So, here is how we translate. So, we just replace the call with a jump and we end up with this assembly sequence. But now, now that we have this, we have to ask ourselves the question, what exactly is the size of this jump instruction? Because as I just said, you have to ensure that you keep the offset of the surrounding code the same. So, the question is, what is the size of the jump? Because if the size is five, well, you will end up with the same offset for the next instruction. It will be eight, but if it's different, well, you will shift everything. So, you have to be careful with the size of the new instruction. So, basically, you need to compare the related sizes of the original instruction and the instruction with which you want it to be replaced. So, yes, that's what I just said. So, let's introduce a bit of notation. So, I note S of O, the size of the original instruction, S of R, the size of the rewritten, the new instruction, and then we compare them. So, in this case, we compare the size of the red instruction, the one that we remove, with the size of this new jump instruction, which is in green. And in this case, okay, let's say, let's say that it's quite simple, we have the same size, five for the call, five for the jump. That's the same thing. In this case, well, quite simple. We just, sorry, quite simple. We just replace and we do nothing else. Nothing to pad, nothing to shift. That just fits. That's perfect. But, well, as you can guess, most of the time, it's not that simple. So, for instance, let's imagine that the jump is shorter. It's three instead of five. So, in this case, well, you can guess what we do. We pad, just like we did when we remove. That's more or less equivalent. But then, obviously, we can get into cases where the jump, so the new instruction is larger than the original one. And in this case, well, what do we do? So, that's quite a problem. They usually, well, quite a basic problem in terms of binary writing. So, there are several ways to deal with that. I'm going to show you the way that we use in Saber, which is called D2R. So, D2R, or D2Ring, is just taking the instruction that do not fit anymore in the original code space. So, where we added an instruction, or we replaced the original instruction with a larger one. So, there are some instructions that can't fit anymore. So, we have to take them out, move them somewhere else. So, the somewhere else is what I will call an out of line scratch space. So, out of line because it's not in line anymore. It's not part of the straight instruction stream. It's somewhere else in the memory. Okay? So, the way we do that is, well, basically, we are allocating memory somewhere else. We are moving the instruction that do not fit anymore in the original main line stream to this out of line scratch space. And then, obviously, we need to connect this out of line space to the original main line. And we do that by inserting some jumps. So, let's take it on a simple example. Let's say that we want to add an instruction with such a D2R. In this case, let's say that we want insert a jump. No, sorry. Let's say that we want to insert some instruction. It can be one instruction. It can be a sequence of instruction. No matter the length of the sequence you want to insert, the principle is the same. So, let's say that we want to insert at the point where you see the green arrow. Okay? So, right after the load and before the call. So, how can we do that? So, first, we are locating the out of line scratch space, which is at the bottom right corner. And there, we move the call instruction because it can't fit anymore. Once we have rewritten this part of the call. So, we move it. The original part, we move it from the main line to the out of line scratch. Then, we insert the add instruction. So, here, I didn't represent them, but you can think of it as just one instruction of a snippet of how many instructions you want. It doesn't matter. And then, last part that I mentioned, you have to connect this out of line scratch space with the rest of the code. And to do that, you use jump. So, first, we insert a jump so that the first green instruction that you can see at offset three. And you jump to some label that is representing the beginning of the outline scratch space. So, the label here is D0. So, when you're here, which should be the place when you have the call instruction in the original code, instead of directly executing the call instruction, you will jump at the beginning of the outline scratch space. And then, at that point, you can execute the original instruction, the call, as if it were still at the same place. The semantics doesn't change. You have this guarantee. And then, you can just execute whatever arbitrary instruction you want. And finally, let's not forget to jump back to the main line with this jump. So, this time, we've inserted another label called L0, which is just right after the jump. To that, when we have executed the instruction, we just come back and then continue, well, I was about to say, as if nothing has happened. Well, something has happened. But the rest of the code, the surrounding code, remains unchanged. Right. So, this is very important. This is really the basics of how everything is working. So, in this case, you can notice that the size of the original instruction, the call here in orange, is the same as the jump. Five. Both are five. Well, that's fine. But as I mentioned before, you have to be careful that when you replace one instruction with the other, you have to compare the relative sizes and see if that fits or if you have to adapt. So, in this case, we do exactly what we've done before. We have to compare this time not the real instruction, but the jump that we insert with the original instruction. So, in this case, let's say that, well, we have basically two cases. Either the jump is less or equal to the size of the original instruction. In this case, well, you just insert it. If need be, you pad with knobs and everybody is happy. Or if it's not the case, if the jump itself is larger than the original instruction, well, in this case, you have to relocate the number of instructions. So, for instance, let's go back to our previous example. We want to replace this load instruction and we want to insert a jump. So, in this case, let's say that the original instruction, the load that we want to replace is only two, two bytes, which is quite short. In comparison, the jump is five. So, well, in this case, the jump is quite larger than the original instruction. So, you need to, well, you need to make space even to insert the jump. So, well, what you do is just, as I mentioned before, you take the call instruction that is just next to the instruction that you want to replace. You move it to the outer line of the space. The rest is the same. You still need to locate the space. So, I don't go back to that. But once you have your outer line's crash space, you just move the surrounding instruction at the right place. So, here, after the instruction that you want to substitute, since it is, since you are relocating the instruction that is after the instruction that you, that you replace, and you end up with this result, so still the jump to the outer line space and then jump back to the main line. So, well, I've, I've basically covered all the possible outcomes, depending on the side, the relative sizes of the original instruction and the return instruction. And same for the jump instruction. In case you have to, to have this outer line's crash space. So, yeah, basically the most complex case that you can face is that you have a couple of instructions around the jump that you insert that you need to relocate. So, now the question is, can instruction always be rewritten, relocated, sorry. Can you always rewritten arbitrarily selecting instruction around the spot where we want to replace or other instruction? Well, you get that if I'm asking the re, the answer is no. So, I'm going to show you the, well, some typical examples of cases where you can't or you have to be very careful when you relocate instructions. So, for instance, PC relative addressing. So, what is PC relative addressing? So, usually when you want to, I give you the example of a jump, when you want to jump to some part of the code, for instance, if you have a loop, a loop is basically having a jump backwards into the instruction stream, so that you will execute this sequence several times. And usually the most efficient way to do that is instead of encoding the address, the full address of the place where we want to end up, the target of the jump, which would be, well, 32, 64 bits, which is quite long. Usually, you just use the displacement, which is the distance, the offset between the source of the jump where we're jumping from to the target where you're jumping to. The difference between the two. Usually, it's, well, a couple of bytes because, well, it's quite rare that you have to jump, especially when you have a typical control structure, that you have to jump to a place very, very far away in the code. So, usually, you just have to encode the displacement, the distance between source and target, and that's much more efficient. So, in this case, for instance, let's say that we want to load some data that is located 48 bytes from the current value of the PC, which is basically where we are, the current instruction that is executed. So, 48 bytes from the place where we are currently executing, right? And we want to exert an instruction where we have the green arrow, right before the load instruction that has the displacement addressing. So, what do we do? Well, we just applied the D2, the D2 mechanism that I just presented earlier. So, we end up with the same kind of stuff with our jump. Well, we need to insert some note because we are relocating the load instruction because, well, we need to make the jump fit. And so, we end up here in the out of line scratch space, we end up with this displacement, 48 from PC. Well, the problem is, since we moved the instruction to somewhere else, you can see here the offset is quite different, it's FFEA, and the original set was two. So, we are very, very far away from the original space. So, this offset is not correct anymore because we didn't move the data that it put into. So, we end up actually with quite a significant mismatch between the right displacement, which is 49 in the original instruction. And after relocating the instruction, you get this displacement, which is 10032, well, that's completely wrong. So, you have to be careful with that. And you have to, well, basically just to compute the actual displacement after having relocated the instruction. So, well, you just compute the difference between the original displacement and the new offset. So, here, 49 minus FFE6, and you end up with the new displacement once you have relocated the instruction, which is minus FFE90. Okay? So, well, that's feasible, that's feasible. But usually, we try to, when we do binary writing, we try to avoid it because we can be quite painful. So, second problem that can arise when you want to relocate some instruction. So, I told you about jumps, I told you about source and targets. So, well, basically, here, imagine that we have a jump backward, like in a loop. And we want to insert an instruction right before the target of the loop, which is here represented by the L0 label, right? So, what do we do? Well, once again, we do the same kind of thing with the detour. We move the original surrounding instruction, which in this case is this load. We add the instruction, et cetera, as usual. No problem. When you end up trying to execute the jump L0 instruction after writing what happened? Well, you've lost the L0 because you rewrote the sequence of instructions, which is load then call. And you can see that the target, the L0 label, is pointing right in the middle of this. And since you've completely overwritten this, if you still want to jump at this address, well, you end up jumping right in the middle of another instruction. So, this is completely wrong. And in this case, well, there is no simple way to solve it. So, what we do is that we scan the code before even starting to rewrite. We record all the branch targets. And we say, okay, this instruction, we don't do anything with them. So, we just keep them at place. And we won't even try to relocate them. We need to do that otherwise. We end up with some trying to decode some non-existent illegal instruction. And that can be quite problematic. So, I just show you two examples of typical problem that you will end up when you want to relocate instruction. There is a third one that is side effects. I won't go into the details of the side effects, but just keep in mind that you have three types of problems or challenges. And you have to, you have just to keep them in mind when you rewrite instruction. So now, what if we end up, for instance, with a target for a jump right before the instruction that we want to insert, and also some instruction with side effect right after, which means that you just, you just write in between two instructions that you can't relocate. So, what happens in this case? Because you may, for very valid reasons, be willing to rewrite some instruction at this precise position. And well, if you end up having surrounding instruction that you can't relocate, well, you're stuck. Well, not completely. You can still end up doing something which is not strictly speaking binary writing, but something that can allow you to do what you want. So, actually, what you can do is just replace the instruction that you are interested in with an illegal one. So, an instruction that is known to not encode an illegal operation. So, you replace it with this instruction. And what you do is that you set up a signal handler that will catch the signal that will be raised by trying to decode an illegal instruction. So, once you've set up this, you just have to catch the signal. Obviously, you need to record the source address of the instruction that triggered the signal. But then, you can do whatever you want inside the signal handler. So, well, as you can imagine, that's quite a significant overhead doing this, having to go through all the mechanisms of the signal handler. But since, well, in our experience, that really, really happened rarely, well, the overhead is, is structural. So, well, that's really the last resource, the last resource solution that, at least, we have a solution when that happens. We mean that we can rewrite virtually all instruction, any arbitrary instruction in the program. So, I promise you that I'll go back to the disassembly problem. Well, at the time, I didn't tell you that there was a problem, but actually, there is a problem. So, let's, let's go back. Let's, let's make one step backward and look at what we exactly have when you, when we actually look at some machine code, some program code in terms of binary. So, we have, well, on the left, this binary sequence, some arbitrary one, and we want to turn it into something that is, well, understandable by a byte human. And especially, we want to, to know exactly which bit, or which bytes correspond exactly to which instruction, because the instruction is the granularity that we are interested in. So, you can see here that actually what it is really meant for is being able to, to match a sequence of bytes with some specific instructions. So, for instance, here you have the yellow sequence of bytes that will be representing a push instruction, the orange one will represent the load, et cetera. So, that's, that's exactly what, what we want to achieve. The problem is, it's not always easy. You don't always get it for free. So, you have to, you have to have some, some, some algorithm or some, some technique that will, that will guarantee that you have the right, the right DDSMB. So, there are, there are different, different techniques that allow you to get, to get the DDSMB. The dynamic one, the static one. Well, both have advantages, but for, for the purpose of binary writing, in Saber, we will use the static one. So, there are basically in terms of starting the assembly, there are many two ways of doing this. Either what you have on the left, which is linear sweep. So, linear sweep is just, well, we basically start at the very beginning of the program and we scan it instruction by instruction. We know, we know where the program starts because, well, otherwise the, the OPC system is not even able to load it. And so, we can, we know, actually that the point where we start is correct, but then we can end up at some point having some garbage instruction. For instance, if you have some, some padding bytes that are, that are just there to, to ensure alignment in the code, linear sweep is not able to, to cut that. On the other hand, you can have what is called recursive traversal, which is in this case something more clever. It's really trying to do what the CPU will do that is following the, the flow, the control flow and when you have the jump, jumping, which has enabled to, to skip the, the garbage bytes, for instance. But at the same time, you can skip some instruction because you, you don't necessarily know where, when, in which condition a block should be taken. So, well, just to say, in Saber, we are using the, the static version because it's really, really efficient in terms of, of cost and previous research has shown that it works very well with, at least with code that is produced by, well, most compilers, GCC, LLVM and for most targets. So, X86, RISC-5 for instance, it's known to not work very well for ARM, but, well, for, at least for, for our purpose, it's, it works quite, quite well. So, okay, let's, let's put it together and try to see what we can achieve with all of this. So, just as a reminder, we want to do this, we want to inject the code, replacing the real error code that is returned by a system called, by some error. So, what we do is that we intercept those system calls. We, well, we just rewrite the instruction that actually executes the system calls. So, for instance, on X86, you have one C-scall instruction. Well, so, what Saber does is just rewriting all the C-scall instructions at once, at the very beginning of the execution. So, what happened is that when the operating system loads the binary into memory, the program, Saber will catch it and you will rewrite everything, well, all the system calls, even before the execution actually starts. So, it does, it does that at the very beginning. So, that you pay relatively small price at low time, but then at runtime, you don't have to do any disassembly, you don't have to do any actual rewriting. The only code that you pay is just what I showed you, just taking the detour, having a, for each system call, a couple of instructions added. So, in terms of overhead, as I show you in a minute, it's really, really good. So, yes, that's basically what we do. So, in terms of how to use it in concrete terms, so Saber itself, what I call the backbone of Saber, it's just a loader that will load your application, rewrite the system calls, and then execute the program as normal. If you want to actually do something with those system calls, you have to write a small plug-in using an API that is really, really simple. So, just an example of how that works. Let's say that we want, well, this is our thing before, just inject false. So, we just have to implement, I say we, the user, the user just has to implement this function, whatever the name or what it does is that, it's like a hook that is called by Saber, every time it reaches a system call, okay? So, at runtime, it goes through the out of line squash space, then it jumps, like with the trampoline, it jumps into this function, and this function, well, it's just a usual C code, so you can do whatever you like. You can change the arguments of the system call, you can change the return value, which is what we want to do. You can even end up not calling the system call at all. And, well, basically, that allows you to do what you want. So, well, let's see what it would look like if we want to achieve this basic fault injection problem. So, we have this function, let's call it a handleSystemCode, which takes the number of arguments, so the first argument is the number of the system call. So, here we want to catch one specific system call, which is the right system call. Well, all the other arguments are just the real argument to the system call. So, we just pass them directly. But in the case, in the very specific case, when we get the right system call, what do we do? In this case, well, we can call it or even decide not to call the system call at all. But in any case, what we want to do is return some arbitrary error code. So, here, for instance, we return the E-perm error code, which is a permission error. And then, your application is going to see this instead of the expected successful return code. It will see this, and at that time, it will have to decide what to do. And you end up seeing what happens. It depends. So, maybe it will be very nicely or you get some error message or a complete crash or whatever. So, that's how you end up seeing what can happen when you have some random bug or some random problem with the operation you're doing with the operating system, right? So, just a few words to tell you that. So, currently, there are three, yes, three plugins that are open source and available for you. So, well, the first one is basically just registering the system call without changing anything. It's basically for testing purpose. You have a full team director that is a bit more sophisticated than the basic one that I showed you. And you have a tracer that is, well, if you know S-trace, it's basically doing the same, just a recording, printing on the screen all the system calls that are called with their arguments and return values. But it's much more efficient. I'm going to show you in a couple of minutes. So, for now, we support two targets. I mentioned before, so x86, 64 and RISC-5. And we get, oh, yes, so the current implementation targets exclusively Linux as an operating system, I guess it would be not too difficult to extend it to other unices, but, well, so far it's not been done. It's written mainly in C with some assembly snippets. Very important, it doesn't use any third-party library, not even for disassembly. So if you look at all the software that exists that do binary writing, usually they rely on a bunch of third-party libraries, especially for disassembly. We don't do that. And that's why we end up with something really, really tiny. So 47 kilobyte for the executable for x86, 40 kilobyte for RISC-5. So you can even use that for small embedded types, embedded targets that will work very well. Just to show you, because I mentioned earlier that Cable is able to be very efficient, so just to show you the kind of overhead that we get. So we measure that with some well-known server application like MAMCHD, Redis, et cetera. So we end up with an average load time overhead that is, not average, sorry, but maximum measure load time overhead that is 60 milliseconds, that is, well, lower than the perceived response time for the user, which is 100 milliseconds. The runtime overhead is very, very low, as I mentioned before. Usually below 3%, in most cases, around 1%. So we compared our system called Tracer with the original S-Trace trying to mimic the output. And we got, well, we got very, very good results. As I can show you here, yes. So, well, native is just the original execution without tracing, without instrumentation or nothing. So we compared the original S-Trace with our implementation of the plugin SBR. And also with some PIN tool doing the equivalent. And, well, as you can see, we are, well, obviously we are, there is an overhead compared to native implementation, but compared to S-Trace, it's really, really efficient. Yeah, the first one, sorry, the first one was on the D-U GNU tool, basically reading and writing in a loop. And we also did some experiments on D-U trying to estimate the usage of a big directory. And we end up with more or less the same results as we did. And finally, yes, the fault injector. So we got, we found some bugs in the core utility tools. Core utility is basically, well, all of the basic tools that you have and that you use all the time in your terminal, CUT, LS, etc. So, yes, we found some bugs, different kind of bugs, thanks to the random probability tool that we have in our fault injector. So to sum up, so Saber, what is he doing? It's selective, selective, well, I didn't define selective before, but as you have understood, selective because it's targets mainly as stem calls. By the way, it's also able to rewrite function for logs if you want to, well, to intercept some specific function. It's also able to rewrite the RDSC instruction in X86. So RDSC is just reading the timestamp counter. So it works at load time, just ahead of time, as we call it, very low overhead. So it's suitable for embedded devices. The API is quite simple. So you can build your plug-in data by basically writing one function in a couple lines of code. We have embedded available plugins, open source on GitHub and, well, everything is GPLv3. So, yes, just feel free to play with it. If you have any questions, I'll be happy to take them. Thank you. Okay, so we have around four minutes for questions. Thank you for this talk, Paul. And who has a question? Here. Thank you for your presentation, most interesting technology. You mentioned a case where you did parts of the rewriting by modifying an error handler. Yeah, sorry, I can't hear very well. Yes. Okay, you mentioned a case where you did part of the rewriting using an error handler. Yes. So now I'm curious, how do you make sure that this same error handler is not invoked from an entirely different place which would disturb the meaning of your system? So if I understand your question correctly, you are asking when we relocate instruction into an error handler which might also be called from a different place. Okay, so when we don't have space to insert the jump, we use a signal handler to catch the signal. So your question is when we do that, how do we ensure that we are actually handling an instruction that we generally have rewritten rather than some real signal, some real illegal instruction that we weren't aware of? Is that your question? Yeah. Yeah, so actually what you do is that you have to, when you're inside your signal handler, you have to check the address of the instruction that just threw the signal handler. So you look at this address, you see what exactly it is, the operation. In this case, you are most interested in system call. So if it's a system call, well, that's fine. If it's something else, when you just fall through and you let it be handled by the operating system. Does that answer your question? Okay. Well, if you have more questions, you can take it offline if you want. Is that a question? Okay. Have you, with your approach, have you encountered any counter mechanism which is provided in, I mean, this way you don't really need the source code, right? You can do this on arbitrary binaries. We don't need the source code yet. Yes. So I assume you have experimented with third party software as well doing your approach here? Well, we've experimented with, well, not all possible, not all possible options that we have made with, for instance, I showed the pin tool to do the same. My question is, if you have any, well, algorithm or any procedure in place, if the binary that you are injecting into has precautions against like injections, if you have stumbled across something like this? I'm not sure I understand what you're asking. What do you want to check again, the code that is injected? Yeah, I'm not sure. This is my question actually. So if you have stumbled across any situations where this approach does not work. Oh, the approach with the detour. Well, the only problem that can, well, I can think of two limitations, two problems mainly. When we can't relocate anything surrounding and well, then you have to use a sigil. It mostly works, but, well, you can end up with problems and with that. The other issue is that I mentioned that you can't relocate instructions that are target of jumps. The problem is for direct jumps, I mean, jumps with the target encoded as an offset as I showed you that works. But if you have an offset, a target of a jump that is dynamically computed, for instance, as a result of a computation, well, that depends on the data. And I want to use this static. You can't, you can't really do anything about that. So yes, that would be the main problem, the problem with indirect jumps. But if I wanted to make my binary resilient against this approach, that I would just require, I would just add a custom sigil handler that I need to invoke. And if you then override it, I know, okay, someone is fiddling, I better quit, I better die. So this, actually, this, as I mentioned, this approach works well with, well, you know, mainstream compiler, GCC, LLVM that do, well, sane and sound stuff. But if you end up with some code that has obfuscation mechanism, because, well, some, that's, that's happened. Yes, with obfuscation, that wouldn't work because, for example, you know, you can have overlapping instructions, you can have embedded instruction, you can even have a self-fuelling code, self-modifying code. Well, in this case, we can't do anything about that. But I would say for 99.999% of the code, that, that works quite well.
developed a brand new software and you want to kind of issue. So, the problem is it's hard to Now, as I said, what we want to achieve is to simulate a fault, a problem in terms of, for the write system call, if it is successful, you one. This time, we want to write into a file. So, write system call in turn. So, that's what we do. fact that the disk is full, so that you can't So, in simple terms, it's modifying a program, code level. That is the code, the binary code writing is that you don't have to modify the binary writing is disassembling. So, for now, they are able to execute. But well, we need to be version of this. So, this is where this assembly of instruction. So, that's really a translation, remove instructions, we can replace instructions, types of instructions that we can do in binary another. So, basically, in a high level language when you have a nif, a while, a loop, or any point in the program to another. And the jump in original instruction and the instruction with of the time, it's not that simple. So, for on a simple example. Let's say that we want to add an instruction with such a D2R. In this case, So, right after the load and before the call. So, have rewritten this part of the call. So, we move it. The original part, we move it from the main instruction of a snippet of how many instructions right after the jump. To that, when we have as the jump. Five. Both are five. Well, that's with the other, you have to compare the relative done before. We have to compare this time not the that we want to replace is only two, two bytes, and then jump back to the main line. So, well, between the source of the jump where we're target, and that's much more efficient. So, in the branch targets. And we say, okay, this non-existent illegal instruction. And that can be when you want to relocate instruction. There is a challenges. And you have to, you have just to valid reasons, be willing to rewrite some you do is that you set up a signal handler that code in terms of binary. So, we have, well, on of bytes with some specific instructions. So, for always easy. You don't always get it for free. some algorithm or some, some technique that will, different techniques that allow you to get, to get the DDSMB. The dynamic one, the static one. Well, both have advantages, but for, for the know where, when, in which condition a block intercept those system calls. We, well, we just rewrite the instruction that actually executes the execution. So, what happened is that when the operating system loads the binary into memory, even before the execution actually starts. So, it but then at runtime, you don't have to do any the system calls, and then execute the program as plug-in using an API that is really, really implement, I say we, the user, the user just has with the trampoline, it jumps into this function, which is the right system call. Well, all the currently, there are three, yes, three plugins than the basic one that I showed you. And you have a tracer that is, well, if you know S-trace, we support two targets. I mentioned before, so x86, 64 and RISC-5. And we get, oh, yes, so the kilobyte for the executable for x86, 40 kilobyte so just to show you the kind of overhead that we get. So we measure that with some well-known So we end up with an average load time overhead is, well, lower than the perceived response time cases, around 1%. So we compared our system without tracing, without instrumentation or our implementation of the plugin SBR. And also And finally, yes, the fault injector. So we got, understood, selective because it's targets mainly to intercept some specific function. It's also works at load time, just ahead of time, as we and, well, everything is GPLv3. So, yes, just rewriting by modifying an error handler. Yeah, correctly, you are asking when we relocate in system call. So if it's a system call, well, this approach does not work. Oh, the approach well, I can think of two limitations, two problem, the problem with indirect jumps. But if add a custom sigil handler that I need to invoke.