Finding Xori: Malware Analysis Triage with Automated Disassembly
Rousseau, Amanda
Seymour, Rich
In a world of high volume malware and limited researchers we need a dramatic improvement in our ability to process and analyze new and old malware at scale. Unfortunately what is currently available to the community is incredibly cost prohibitive or does not rise to the challenge. As malware authors and distributors share code and prepackaged tool kits, the corporate sponsored research community is dominated by solutions aimed at profit as opposed to augmenting capabilities available to the broader community. With that in mind, we are introducing our library for malware disassembly called Xori as an open source project. Xori is focused on helping reverse engineers analyze binaries, optimizing for time and effort spent per sample. Xori is an automation-ready disassembly and static analysis library that consumes shellcode or PE binaries and provides triage analysis data. This Rust library emulates the stack, register states, and reference tables to identify suspicious functionality for manual analysis. Xori extracts structured data from binaries to use in machine learning and data science pipelines. We will go over the pain-points of conventional open source disassemblers that Xori solves, examples of identifying suspicious functionality, and some of the interesting things we've done with the library. We invite everyone in the community to use it, help contribute and make it an increasingly valuable tool for researchers alike.
 Please join me in welcoming Amanda Russo and Rich Seymour. Hello, everyone. I'm here to talk to you about finding Zori. It's a malware analysis triage with automated disassembly. A little bit about ourselves. My name is Amanda Russo, also known as Mauer Unicorn on Twitter. I'm currently a malware researcher at Endgame. My day-to-day activities is looking at malware every day, writing detections for them, teaching my interns something, and then Rich. Hey, what's up? My name is Rich Seymour, also known by my hacker handle RCMor on Twitter. I'm a data scientist at Endgame, and I got a chance to work with Amanda on this project. I'm really psyched about it. We started it in January, and we're about six months later. It's about 36,000 lines of code, and we're going to tell you about it. Thanks. A little bit before we begin, I want to go over the quick overview of current assemblers of today. I'll talk about what they do for PE right now, the functionality and features, and I wanted to go over a usage and demo. First, I actually want to show you the tool before we get into it so you know what the hell I'm talking about. It's awesome. That loaded up pretty quick. If you don't know what a disassembler is already, it's kind of taking the binary code and turning it into assembly for you so you can analyze this, the malware, if you're looking at malware. I'm just going to go ahead and drop this WannaCry sample in here really quickly. It will disassemble it really quickly and put it to the screen, which is browser-based, so it's cross-platform for you to run on any Linux, OSX, or Windows distribution. You can play around with it. You can see here's the short circuit thing that MauerTech registered. As well as linear disassembly. Let me make this a little bit bigger for you. It's all there for you to play with in the UI. No crazy buttons. It's really easy to mess with. It has a click and drag. We made this in two weeks for you. We know we use it for automated disassembly, but we wanted you to have a tool that you could play with in the browser on any platform and get used to looking at disassembly. I'm going to switch back to my slides and continue presentation here. Now that you've seen what the hell it is, now we can talk about what it does and how it compares to other disassemblers. What is the problem? My boss told me look at 1,000 samples in three days. I'm like, there's no way I can look at 1,000 samples quick enough and triage it and sit there and copy one sample into a VM, open it up in IDA and analyze it. What did we do? We had to think of a better way to scale our automation. We wanted to basically be able to take our pipeline right now which handles training our models and also add to it disassembly, which is something we've wanted to do at Endgame since before I was at Endgame. It allows us now to basically take our sample set and turn it into disassembly features that we can read because it throws out right now JSON, but you could basically serialize it to any format you can integrate into your pipeline. We thought about doing automation. We wanted to try some disassemblers that were already out there, open source, if it's a library and try to compare them across what we needed. We were thinking about how big the size of the code was, how big it was in memory, if it was stable, how much the price was, it was a cross platform, was it easy to use, is output accurate and can we integrate it into our services already? I have this little chart that we have good, okay, and not so good comparing Capstone or Darii 2, Ida Pro, Hopper and Binary Ninja, which are pretty much the common disassembly libraries out there. So requirements. We needed fast development. We needed stability and resilience because if we're running through a bunch of services, we'll need it to spin up again. It had to be cross platform because we're not just using it on Windows or OS X and we're also running it on a server. Output can be easily integrated into our services, which JSON is really useful for that. It had to be really easy to use. It had to have a core feature set and output had to be accurate. All right. So the first step we did was diving into the code of all of these disassemblers and trying to pick and choose what are the pros and cons of each and kind of build it into something that we needed in our own internal framework. So we adopted a different aspects of what Capstone does since it's based on the LLVM and GDB repositories. Also a little bit of emulation that, you know, Kimu does as well as understand how to put that logic using Rust and then fix some bugs along the way. So evaluating example. Say you have an X86 32-bit instruction here. So typically the 66 represents the op size of the instruction, which tells you is the operand bigger or larger than it should be. And then you have the op code, which tells you what instruction actually is. So how many of you are familiar with op code 90? Yeah. So it's not right. Not not not. So really this is actually exchange AXX in object dump and LLVM because of that op size is there. So I did it correctly. Capstone doesn't do it correctly and Distarmed doesn't do it correctly. So some of these like minor changes between different disassemblers is going to give you different output when they disassemble the instruction. Exchange AXX actually is a knob, but it technically is not based off of the CPU. So here's where I ask who here knows what Rust is? Okay. And who here is programmed in Rust? Okay. That looks like 12 people maybe. That sounds great. I'm glad everyone's going to be able to contribute to this project. I think the thing that we want to emphasize besides the bullet points on the screen is that I didn't know Rust really. I mean I'd been tracking its existence, but that's a big difference from like diving into writing a disassembler in a few months. So Rust gives you all these nice things. Like it protects you from stomping on the OS from your stock. It handles memory much more cleanly than any programming language I've used. It's really fast. And the oddest thing about it is that it's actually it was really fast for us to develop in because all of the really dumb bugs that bog you down for a week and then it's another week and then it's the next month and you're like, why can't I get this to work? Basically get caught either by the compiler not letting you do it or well yeah, by the compiler basically. So it's really helpful and I suggest you learn it. All right. So current features. Yes, it's open source. I was really like held back on my heat open source. You can contribute now. We already had like how many contributions last night? We've got 180 some stars. We've got two contributions, a lot of issues, requests coming in. And it's only been live for two days. So it's really bumping and grinding. Yeah. Currently it supports x86 and 64 bit architecture. I only had time for that since my job requires PE executables. It displays strings from reference memory. If you're familiar with the flare on plugins they have or if you're pushing strings to the stack and it emulates it and produces a string for you, that's already built in. It manages memory. It outputs JSON for your service consumption. It has two modes, light emulation, which means it does handle register stack and some instructions as well as full emulation, which I don't recommend because it's really slow right now, but you can do it if you want to get through a packer or some type of encryption routine. I also was able to simulate the thread environment block as well as the process environment block so that you can do dynamic API call loading as well as evaluate functions from DLL exports so you can populate those in the code itself. All right. So the design. So we have a PE loader similar to what Windows does for loading a PE. It's very basic, loads the tab as well as DLLs from the PE image header. Also a memory manager that manages all those memory spaces so that in case the malware is trying to access an address outside of those memory spaces it won't allow you to, as well as the analysis struct which contains all the functions, the disassembly and all the imports. And then finally when you're analyzing through all of the different sections of code it has a state which tracks the CPU registers and all the flags, a new version of the stack as well as tracking loops. I wish I would get into later. So if you're wondering how Chump like me contributed to this project, I decided to play not invented here syndrome on a PE parser. I use this great library called NAMM which I think originated in France. It's a parser combination framework and it allowed us to parse the header, PE headers rather safely. If you've ever messed, who here has ever looked at a PE header? You know, MZ is a real person. And the thing about headers, like if you've looked at Corkami's excellent work on like describing what this stuff is to people, there's a lot that's useful to the machine to run like getting the imports correct and the tab and the entry point and all that. But there's also a lot of stuff that's just kind of like retro garbage from the 8-bit DOS days, 16-bit Windows days, 32-bit Windows days and the 64-bit Windows days. So the cool thing about this parser is it basically takes everything so we can data science the heck out of it later. If we want to figure out that corrupted DOS stub means that it's some threat actor we've never heard of, then we can do that now. So I'm pretty excited about it. And yeah, that's my little thing. So the information that we take from the PE header is used to build the PE image into memory. So that means building all the sections, making sure they're read, write, executable and tracking that as well. Building the stack, building the DLLs into memory in the right locations. You can actually configure that. We have a little JSON config file that you can set whatever address offset you want for the DLLs or the Teb and PEP as well as building the Teb and PEP in memory. So what I meant by loading the DLL exports, we have this giant JSON of all the DLLs exported from like Windows 7. And it pulls all of the API calls out of there. So you don't have to keep using the DLLs. You can just use this giant JSON. And it will load it up and rebuild all of the addresses as it would in the PE image. And fill all of those addresses for you. So if it's doing any dynamic API calling and it moves an address into a register or an address into a new memory location, that address will actually be there simulated. All right. So like I said, dealing with dynamic API calls, you need the Teb and PEP. So we kind of recreate that for the Windows 7 environment. It handles memory. So anytime it accesses one of those, you can choose to allow it or not allow it. As well as managing all of those addresses on the stack, which is nice. And let me explain what I mean by this. So we have this shell code doing some interesting stuff. In the header imports, the only API calls it's actually using is exit process, get last error, get local time, and get module handle A, which we don't actually see load library A, virtual protect, or shell execute A, right? So here, you can see it's taking the string called load library A and pushing it onto the stack. From there, this function right below here, it's going to call this function and get that address off of the PEP. So that means it traverses all of the DLLs as well as the names from the PE headers in order to collect that address offset. Once it's done, it's going to put that address into EAX and then move into 401004, that pointer. And then down here, it's actually going to call that newly moved address to load shell32.dll, so it can also load shell execute A afterwards. So in reality, the dynamic imports are load library A, virtual protect, and shell execute A. So this is actually the output from Zory, and you'll be able to see all of that happen in there with the light emulation. All right. So the teb and peb, I didn't do anything really crazy. There's so many resources out there to look at the whole struct for the teb and peb. Some versions of Windows are going to be different, but I just used, you know, whatever was available out there. You don't need the whole, like, hidden features of the struct at all. Here's just, like, the top of it, for example. So I recreate, the nice thing about Rust is that you can take a struct and serialize that struct into a binary array or a binary vector, and you can have that vector being accessed by the memory manager as if the assembly instructions were to access those addresses as if it was actually on the OS, which is really nice. And so I had to figure out how to do a link list in bytes, serialized bytes, and put that all together into a structure so that the shell code could use it. All right. And so another thing to help with the analysis is that we have this thing called analysis queue where it will take the branch instruction, like, if else, and there's a call. You'll have two different directions in order to take for the call. So the right-hand side of the branch will go to the top of the queue while the other part of the branch will go to the bottom of the queue, which will help the code analysis follow the actual control flow for you. So it follows both directions. And I also introduced looping for the emulation. So say you run into an infinite loop, and you know it's an infinite loop, but you want to short circuit that loop case. There is a configuration there for you to short circuit all of the looping in there. Say you only want to do 4,000 times, you can set that information. I don't recommend using the emulation mode, but if you want to, you can play with it. I really need to improve it a little bit when you read a couple of research papers. And then, me? So signature analysis is a way of adding annotations to your code so that you don't have to be able to read hex like Amanda. You can just get a nice signature to tell you what the function was. I spent the last week and a half of my life trying to reverse a SIG parser from IDEP format so we could use it. I failed catastrophically in that those formats are always changing. It's very hard to keep up with it. That said, we do have signature matching in Zori right now, but we're always looking for help to make it better. And luckily some folks are already stepping up. So. Yeah. Oh, mine is me too. Oh my goodness. We're prepared. So this is the cool part for me as a data scientist, not a reverser. We took 100 Ember samples, which is a data set we, our company released of hashes. We took the actual binaries, ran them on this old Dell box that was under my desk. The only new thing new about it was SSD and it could get through all these samples in 20 minutes. So that was 1000 samples, 20 minutes, about 1.25 seconds of sample running in parallel, you know, maxing out all the CPUs. And it just dumps a huge amount of data and sometimes too much data if the binary is really big. And then you can do whatever you want. You can grep through it. You can find function names. You can find imports. You can do graph analysis over in the AI village. There's a bunch of people who think they can do a lot of cool things. So I'm really psyched about this. All right. Examples. So it's really easy to build Zory. All you need to do is make sure you install Rust, Cargo build release, target release Zory dash F, WannaCry.exe and it'll produce some results. So if you don't want to use analysis part of Zory, you can just pop in the disassembler part of it and throw it like this nice chunk of bytes and it'll disassemble this Hello World program. But it doesn't really show you any strings. It just gives you the disassembly output. So if you want to use the analysis part, it will actually populate all those strings when it's like popped off the stack and show you when a function ends and annotates all of those nice lovely comments for you. And comparing Zory with IDA Pro, we have the WannaCry ransomware file. You can see we don't have all of the nice argument enumerations yet, but you know, six months is what we can do. We can also see the short circuit string like I showed you in the beginning. This is just the normal output from the terminal. But you know, we have the UI for you to play with so you don't really have to use this part of it. But for development, maybe if you want to use it from the terminal, you can. Here's Radari 2 comparing the two different functions here. It's kind of weird because Radari does this thing where if it identifies API call within like the first couple instructions of a function, that's what it names the function. In this case, we don't try to do that. It's just a regular function call. But it's got parity between the two. And then I showed you the demo earlier, but I can show it to you again with a different sample. I'll just do a normal bin file. So it does PEN binary. So this is the same Hello World program. It's really fast. You just want to see the linear disassembly. You can see it populates Hello World. And you can see the function in the graph view as well. Or if you want to do something more fun with a bunch of jumps, oh, actually, wrong one. You can see how fast it is. This one was Rumbertick. It's actually part of my RE102 course. The longest loading time is actually the libraries there. But you can see this was a fun function, and it populates everything there for you. I don't have any string analysis since this was built for automation, but hopefully soon it will have all that in there for you. So I made it look really cool and spent the whole weekend on that, so hope you guys like it. Thank you. If you want to go download the code, it's already on github.com and gaming slash sorry. People are already playing with it. It's there for you to play with. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you.
 Please join me in welcoming Amanda Russo and Rich Seymour.  Hello, everyone.  I'm here to talk to you about finding Zori.  It's a malware analysis triage with automated disassembly.  A little bit about ourselves.  My name is Amanda Russo, also known as Mauer Unicorn on Twitter.  I'm currently a malware researcher at Endgame.  My day-to-day activities is looking at malware every day, writing detections for them, teaching  my interns something, and then Rich.  Hey, what's up?  My name is Rich Seymour, also known by my hacker handle RCMor on Twitter.  I'm a data scientist at Endgame, and I got a chance to work with Amanda on this project.  I'm really psyched about it.  We started it in January, and we're about six months later.  It's about 36,000 lines of code, and we're going to tell you about it.  Thanks.  A little bit before we begin, I want to go over the quick overview of current assemblers  of today.  I'll talk about what they do for PE right now, the functionality and features, and I  wanted to go over a usage and demo.  First, I actually want to show you the tool before we get into it so you know what the  hell I'm talking about.  It's awesome.  That loaded up pretty quick.  If you don't know what a disassembler is already, it's kind of taking the binary code  and turning it into assembly for you so you can analyze this, the malware, if you're looking  at malware.  I'm just going to go ahead and drop this WannaCry sample in here really quickly.  It will disassemble it really quickly and put it to the screen, which is browser-based,  so it's cross-platform for you to run on any Linux, OSX, or Windows distribution.  You can play around with it.  You can see here's the short circuit thing that MauerTech registered.  As well as linear disassembly.  Let me make this a little bit bigger for you.  It's all there for you to play with in the UI.  No crazy buttons.  It's really easy to mess with.  It has a click and drag.  We made this in two weeks for you.  We know we use it for automated disassembly, but we wanted you to have a tool that you  could play with in the browser on any platform and get used to looking at disassembly.  I'm going to switch back to my slides and continue presentation here.  Now that you've seen what the hell it is, now we can talk about what it does and how  it compares to other disassemblers.  What is the problem?  My boss told me look at 1,000 samples in three days.  I'm like, there's no way I can look at 1,000 samples quick enough and triage it and sit  there and copy one sample into a VM, open it up in IDA and analyze it.  What did we do?  We had to think of a better way to scale our automation.  We wanted to basically be able to take our pipeline right now which handles training  our models and also add to it disassembly, which is something we've wanted to do at Endgame  since before I was at Endgame.  It allows us now to basically take our sample set and turn it into disassembly features  that we can read because it throws out right now JSON, but you could basically serialize  it to any format you can integrate into your pipeline.  We thought about doing automation.  We wanted to try some disassemblers that were already out there, open source, if it's  a library and try to compare them across what we needed.  We were thinking about how big the size of the code was, how big it was in memory, if  it was stable, how much the price was, it was a cross platform, was it easy to use, is  output accurate and can we integrate it into our services already?  I have this little chart that we have good, okay, and not so good comparing Capstone or  Darii 2, Ida Pro, Hopper and Binary Ninja, which are pretty much the common disassembly  libraries out there.  So requirements.  We needed fast development.  We needed stability and resilience because if we're running through a bunch of services,  we'll need it to spin up again.  It had to be cross platform because we're not just using it on Windows or OS X and we're  also running it on a server.  Output can be easily integrated into our services, which JSON is really useful for that.  It had to be really easy to use.  It had to have a core feature set and output had to be accurate.  All right.  So the first step we did was diving into the code of all of these disassemblers and trying  to pick and choose what are the pros and cons of each and kind of build it into something  that we needed in our own internal framework.  So we adopted a different aspects of what Capstone does since it's based on the LLVM  and GDB repositories.  Also a little bit of emulation that, you know, Kimu does as well as understand how to put  that logic using Rust and then fix some bugs along the way.  So evaluating example.  Say you have an X86 32-bit instruction here.  So typically the 66 represents the op size of the instruction, which tells you is the  operand bigger or larger than it should be.  And then you have the op code, which tells you what instruction actually is.  So how many of you are familiar with op code 90?  Yeah.  So it's not right.  Not not not.  So really this is actually exchange AXX in object dump and LLVM because of that op size  is there.  So I did it correctly.  Capstone doesn't do it correctly and Distarmed doesn't do it correctly.  So some of these like minor changes between different disassemblers is going to give you  different output when they disassemble the instruction.  Exchange AXX actually is a knob, but it technically is not based off of the CPU.  So here's where I ask who here knows what Rust is?  Okay.  And who here is programmed in Rust?  Okay.  That looks like 12 people maybe.  That sounds great.  I'm glad everyone's going to be able to contribute to this project.  I think the thing that we want to emphasize besides the bullet points on the screen is  that I didn't know Rust really.  I mean I'd been tracking its existence, but that's a big difference from like diving  into writing a disassembler in a few months.  So Rust gives you all these nice things.  Like it protects you from stomping on the OS from your stock.  It handles memory much more cleanly than any programming language I've used.  It's really fast.  And the oddest thing about it is that it's actually it was really fast for us to develop  in because all of the really dumb bugs that bog you down for a week and then it's another  week and then it's the next month and you're like, why can't I get this to work?  Basically get caught either by the compiler not letting you do it or well yeah, by the  compiler basically.  So it's really helpful and I suggest you learn it.  All right.  So current features.  Yes, it's open source.  I was really like held back on my heat open source.  You can contribute now.  We already had like how many contributions last night?  We've got 180 some stars.  We've got two contributions, a lot of issues, requests coming in.  And it's only been live for two days.  So it's really bumping and grinding.  Yeah.  Currently it supports x86 and 64 bit architecture.  I only had time for that since my job requires PE executables.  It displays strings from reference memory.  If you're familiar with the flare on plugins they have or if you're pushing strings to  the stack and it emulates it and produces a string for you, that's already built in.  It manages memory.  It outputs JSON for your service consumption.  It has two modes, light emulation, which means it does handle register stack and some instructions  as well as full emulation, which I don't recommend because it's really slow right now, but you  can do it if you want to get through a packer or some type of encryption routine.  I also was able to simulate the thread environment block as well as the process environment block  so that you can do dynamic API call loading as well as evaluate functions from DLL exports  so you can populate those in the code itself.  All right.  So the design.  So we have a PE loader similar to what Windows does for loading a PE.  It's very basic, loads the tab as well as DLLs from the PE image header.  Also a memory manager that manages all those memory spaces so that in case the malware  is trying to access an address outside of those memory spaces it won't allow you to,  as well as the analysis struct which contains all the functions, the disassembly and all  the imports.  And then finally when you're analyzing through all of the different sections of code it has  a state which tracks the CPU registers and all the flags, a new version of the stack  as well as tracking loops.  I wish I would get into later.  So if you're wondering how Chump like me contributed to this project, I decided to play not invented  here syndrome on a PE parser.  I use this great library called NAMM which I think originated in France.  It's a parser combination framework and it allowed us to parse the header, PE headers  rather safely.  If you've ever messed, who here has ever looked at a PE header?  You know, MZ is a real person.  And the thing about headers, like if you've looked at Corkami's excellent work on like  describing what this stuff is to people, there's a lot that's useful to the machine to run  like getting the imports correct and the tab and the entry point and all that.  But there's also a lot of stuff that's just kind of like retro garbage from the 8-bit  DOS days, 16-bit Windows days, 32-bit Windows days and the 64-bit Windows days.  So the cool thing about this parser is it basically takes everything so we can data  science the heck out of it later.  If we want to figure out that corrupted DOS stub means that it's some threat actor we've  never heard of, then we can do that now.  So I'm pretty excited about it.  And yeah, that's my little thing.  So the information that we take from the PE header is used to build the PE image into  memory.  So that means building all the sections, making sure they're read, write, executable and tracking  that as well.  Building the stack, building the DLLs into memory in the right locations.  You can actually configure that.  We have a little JSON config file that you can set whatever address offset you want for  the DLLs or the Teb and PEP as well as building the Teb and PEP in memory.  So what I meant by loading the DLL exports, we have this giant JSON of all the DLLs exported  from like Windows 7.  And it pulls all of the API calls out of there.  So you don't have to keep using the DLLs.  You can just use this giant JSON.  And it will load it up and rebuild all of the addresses as it would in the PE image.  And fill all of those addresses for you.  So if it's doing any dynamic API calling and it moves an address into a register or an  address into a new memory location, that address will actually be there simulated.  All right.  So like I said, dealing with dynamic API calls, you need the Teb and PEP.  So we kind of recreate that for the Windows 7 environment.  It handles memory.  So anytime it accesses one of those, you can choose to allow it or not allow it.  As well as managing all of those addresses on the stack, which is nice.  And let me explain what I mean by this.  So we have this shell code doing some interesting stuff.  In the header imports, the only API calls it's actually using is exit process, get last error,  get local time, and get module handle A, which we don't actually see load library A, virtual  protect, or shell execute A, right?  So here, you can see it's taking the string called load library A and pushing it onto  the stack.  From there, this function right below here, it's going to call this function and get that  address off of the PEP.  So that means it traverses all of the DLLs as well as the names from the PE headers in  order to collect that address offset.  Once it's done, it's going to put that address into EAX and then move into 401004, that pointer.  And then down here, it's actually going to call that newly moved address to load shell32.dll,  so it can also load shell execute A afterwards.  So in reality, the dynamic imports are load library A, virtual protect, and shell execute  A. So this is actually the output from Zory, and you'll be able to see all of that happen  in there with the light emulation.  All right.  So the teb and peb, I didn't do anything really crazy.  There's so many resources out there to look at the whole struct for the teb and peb.  Some versions of Windows are going to be different, but I just used, you know, whatever was available  out there.  You don't need the whole, like, hidden features of the struct at all.  Here's just, like, the top of it, for example.  So I recreate, the nice thing about Rust is that you can take a struct and serialize that  struct into a binary array or a binary vector, and you can have that vector being accessed  by the memory manager as if the assembly instructions were to access those addresses  as if it was actually on the OS, which is really nice.  And so I had to figure out how to do a link list in bytes, serialized bytes, and put that  all together into a structure so that the shell code could use it.  All right.  And so another thing to help with the analysis is that we have this thing called analysis  queue where it will take the branch instruction, like, if else, and there's a call.  You'll have two different directions in order to take for the call.  So the right-hand side of the branch will go to the top of the queue while the other  part of the branch will go to the bottom of the queue, which will help the code analysis  follow the actual control flow for you.  So it follows both directions.  And I also introduced looping for the emulation.  So say you run into an infinite loop, and you know it's an infinite loop, but you want  to short circuit that loop case.  There is a configuration there for you to short circuit all of the looping in there.  Say you only want to do 4,000 times, you can set that information.  I don't recommend using the emulation mode, but if you want to, you can play with it.  I really need to improve it a little bit when you read a couple of research papers.  And then, me?  So signature analysis is a way of adding annotations to your code so that you don't have to be  able to read hex like Amanda.  You can just get a nice signature to tell you what the function was.  I spent the last week and a half of my life trying to reverse a SIG parser from IDEP format  so we could use it.  I failed catastrophically in that those formats are always changing.  It's very hard to keep up with it.  That said, we do have signature matching in Zori right now, but we're always looking for  help to make it better.  And luckily some folks are already stepping up.  So.  Yeah.  Oh, mine is me too.  Oh my goodness.  We're prepared.  So this is the cool part for me as a data scientist, not a reverser.  We took 100 Ember samples, which is a data set we, our company released of hashes.  We took the actual binaries, ran them on this old Dell box that was under my desk.  The only new thing new about it was SSD and it could get through all these samples in  20 minutes.  So that was 1000 samples, 20 minutes, about 1.25 seconds of sample running in parallel,  you know, maxing out all the CPUs.  And it just dumps a huge amount of data and sometimes too much data if the binary is really  big.  And then you can do whatever you want.  You can grep through it.  You can find function names.  You can find imports.  You can do graph analysis over in the AI village.  There's a bunch of people who think they can do a lot of cool things.  So I'm really psyched about this.  All right.  Examples.  So it's really easy to build Zory.  All you need to do is make sure you install Rust, Cargo build release, target release  Zory dash F, WannaCry.exe and it'll produce some results.  So if you don't want to use analysis part of Zory, you can just pop in the disassembler  part of it and throw it like this nice chunk of bytes and it'll disassemble this Hello World  program.  But it doesn't really show you any strings.  It just gives you the disassembly output.  So if you want to use the analysis part, it will actually populate all those strings  when it's like popped off the stack and show you when a function ends and annotates all  of those nice lovely comments for you.  And comparing Zory with IDA Pro, we have the WannaCry ransomware file.  You can see we don't have all of the nice argument enumerations yet, but you know, six  months is what we can do.  We can also see the short circuit string like I showed you in the beginning.  This is just the normal output from the terminal.  But you know, we have the UI for you to play with so you don't really have to use this  part of it.  But for development, maybe if you want to use it from the terminal, you can.  Here's Radari 2 comparing the two different functions here.  It's kind of weird because Radari does this thing where if it identifies API call within  like the first couple instructions of a function, that's what it names the function.  In this case, we don't try to do that.  It's just a regular function call.  But it's got parity between the two.  And then I showed you the demo earlier, but I can show it to you again with a different  sample.  I'll just do a normal bin file.  So it does PEN binary.  So this is the same Hello World program.  It's really fast.  You just want to see the linear disassembly.  You can see it populates Hello World.  And you can see the function in the graph view as well.  Or if you want to do something more fun with a bunch of jumps, oh, actually, wrong one.  You can see how fast it is.  This one was Rumbertick.  It's actually part of my RE102 course.  The longest loading time is actually the libraries there.  But you can see this was a fun function, and it populates everything there for you.  I don't have any string analysis since this was built for automation, but hopefully soon  it will have all that in there for you.  So I made it look really cool and spent the whole weekend on that, so hope you guys like  it.  Thank you.  If you want to go download the code, it's already on github.com and gaming slash sorry.  People are already playing with it.  It's there for you to play with.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.
DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_0_2819.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_2820_2867.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_2868_2915.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_2916_3047.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_3048_3755.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_3756_5747.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_5748_9851.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_9852_11663.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_11664_17447.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_17448_18359.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_18360_20855.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_20856_24071.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_24072_26447.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_26448_28427.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_28428_28691.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_28692_28835.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_28836_29111.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_29112_29171.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_29172_29639.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_29640_29687.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_29688_29987.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_29988_30059.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_30060_30215.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_30216_30383.png DEF_CON_26_-_Rousseau_and_Seymour_-_Finding_Xori_Malware_Analysis_Triage_with_Automate_30384_30823.png
Please join me in welcoming Amanda Russo and Rich Seymour. Hello, everyone. I'm here to talk to you about finding Zori. It's a malware analysis triage with automated disassembly. A little bit about ourselves. My name is Amanda Russo, also known as Mauer Unicorn on Twitter. I'm currently a malware researcher at Endgame. My day-to-day activities is looking at malware every day, writing detections for them, teaching my interns something, and then Rich. Hey, what's up? My name is Rich Seymour, also known by my hacker handle RCMor on Twitter. I'm a data scientist at Endgame, and I got a chance to work with Amanda on this project. I'm really psyched about it. We started it in January, and we're about six months later. It's about 36,000 lines of code, and we're going to tell you about it. Thanks. A little bit before we begin, I want to go over the quick overview of current assemblers of today. I'll talk about what they do for PE right now, the functionality and features, and I wanted to go over a usage and demo. First, I actually want to show you the tool before we get into it so you know what the hell I'm talking about. It's awesome. That loaded up pretty quick. If you don't know what a disassembler is already, it's kind of taking the binary code and turning it into assembly for you so you can analyze this, the malware, if you're looking at malware. I'm just going to go ahead and drop this WannaCry sample in here really quickly. It will disassemble it really quickly and put it to the screen, which is browser-based, so it's cross-platform for you to run on any Linux, OSX, or Windows distribution. You can play around with it. You can see here's the short circuit thing that MauerTech registered. As well as linear disassembly. Let me make this a little bit bigger for you. It's all there for you to play with in the UI. No crazy buttons. It's really easy to mess with. It has a click and drag. We made this in two weeks for you. We know we use it for automated disassembly, but we wanted you to have a tool that you could play with in the browser on any platform and get used to looking at disassembly. I'm going to switch back to my slides and continue presentation here. Now that you've seen what the hell it is, now we can talk about what it does and how it compares to other disassemblers. What is the problem? My boss told me look at 1,000 samples in three days. I'm like, there's no way I can look at 1,000 samples quick enough and triage it and sit there and copy one sample into a VM, open it up in IDA and analyze it. What did we do? We had to think of a better way to scale our automation. We wanted to basically be able to take our pipeline right now which handles training our models and also add to it disassembly, which is something we've wanted to do at Endgame since before I was at Endgame. It allows us now to basically take our sample set and turn it into disassembly features that we can read because it throws out right now JSON, but you could basically serialize it to any format you can integrate into your pipeline. We thought about doing automation. We wanted to try some disassemblers that were already out there, open source, if it's a library and try to compare them across what we needed. We were thinking about how big the size of the code was, how big it was in memory, if it was stable, how much the price was, it was a cross platform, was it easy to use, is output accurate and can we integrate it into our services already? I have this little chart that we have good, okay, and not so good comparing Capstone or Darii 2, Ida Pro, Hopper and Binary Ninja, which are pretty much the common disassembly libraries out there. So requirements. We needed fast development. We needed stability and resilience because if we're running through a bunch of services, we'll need it to spin up again. It had to be cross platform because we're not just using it on Windows or OS X and we're also running it on a server. Output can be easily integrated into our services, which JSON is really useful for that. It had to be really easy to use. It had to have a core feature set and output had to be accurate. All right. So the first step we did was diving into the code of all of these disassemblers and trying to pick and choose what are the pros and cons of each and kind of build it into something that we needed in our own internal framework. So we adopted a different aspects of what Capstone does since it's based on the LLVM and GDB repositories. Also a little bit of emulation that, you know, Kimu does as well as understand how to put that logic using Rust and then fix some bugs along the way. So evaluating example. Say you have an X86 32-bit instruction here. So typically the 66 represents the op size of the instruction, which tells you is the operand bigger or larger than it should be. And then you have the op code, which tells you what instruction actually is. So how many of you are familiar with op code 90? Yeah. So it's not right. Not not not. So really this is actually exchange AXX in object dump and LLVM because of that op size is there. So I did it correctly. Capstone doesn't do it correctly and Distarmed doesn't do it correctly. So some of these like minor changes between different disassemblers is going to give you different output when they disassemble the instruction. Exchange AXX actually is a knob, but it technically is not based off of the CPU. So here's where I ask who here knows what Rust is? Okay. And who here is programmed in Rust? Okay. That looks like 12 people maybe. That sounds great. I'm glad everyone's going to be able to contribute to this project. I think the thing that we want to emphasize besides the bullet points on the screen is that I didn't know Rust really. I mean I'd been tracking its existence, but that's a big difference from like diving into writing a disassembler in a few months. So Rust gives you all these nice things. Like it protects you from stomping on the OS from your stock. It handles memory much more cleanly than any programming language I've used. It's really fast. And the oddest thing about it is that it's actually it was really fast for us to develop in because all of the really dumb bugs that bog you down for a week and then it's another week and then it's the next month and you're like, why can't I get this to work? Basically get caught either by the compiler not letting you do it or well yeah, by the compiler basically. So it's really helpful and I suggest you learn it. All right. So current features. Yes, it's open source. I was really like held back on my heat open source. You can contribute now. We already had like how many contributions last night? We've got 180 some stars. We've got two contributions, a lot of issues, requests coming in. And it's only been live for two days. So it's really bumping and grinding. Yeah. Currently it supports x86 and 64 bit architecture. I only had time for that since my job requires PE executables. It displays strings from reference memory. If you're familiar with the flare on plugins they have or if you're pushing strings to the stack and it emulates it and produces a string for you, that's already built in. It manages memory. It outputs JSON for your service consumption. It has two modes, light emulation, which means it does handle register stack and some instructions as well as full emulation, which I don't recommend because it's really slow right now, but you can do it if you want to get through a packer or some type of encryption routine. I also was able to simulate the thread environment block as well as the process environment block so that you can do dynamic API call loading as well as evaluate functions from DLL exports so you can populate those in the code itself. All right. So the design. So we have a PE loader similar to what Windows does for loading a PE. It's very basic, loads the tab as well as DLLs from the PE image header. Also a memory manager that manages all those memory spaces so that in case the malware is trying to access an address outside of those memory spaces it won't allow you to, as well as the analysis struct which contains all the functions, the disassembly and all the imports. And then finally when you're analyzing through all of the different sections of code it has a state which tracks the CPU registers and all the flags, a new version of the stack as well as tracking loops. I wish I would get into later. So if you're wondering how Chump like me contributed to this project, I decided to play not invented here syndrome on a PE parser. I use this great library called NAMM which I think originated in France. It's a parser combination framework and it allowed us to parse the header, PE headers rather safely. If you've ever messed, who here has ever looked at a PE header? You know, MZ is a real person. And the thing about headers, like if you've looked at Corkami's excellent work on like describing what this stuff is to people, there's a lot that's useful to the machine to run like getting the imports correct and the tab and the entry point and all that. But there's also a lot of stuff that's just kind of like retro garbage from the 8-bit DOS days, 16-bit Windows days, 32-bit Windows days and the 64-bit Windows days. So the cool thing about this parser is it basically takes everything so we can data science the heck out of it later. If we want to figure out that corrupted DOS stub means that it's some threat actor we've never heard of, then we can do that now. So I'm pretty excited about it. And yeah, that's my little thing. So the information that we take from the PE header is used to build the PE image into memory. So that means building all the sections, making sure they're read, write, executable and tracking that as well. Building the stack, building the DLLs into memory in the right locations. You can actually configure that. We have a little JSON config file that you can set whatever address offset you want for the DLLs or the Teb and PEP as well as building the Teb and PEP in memory. So what I meant by loading the DLL exports, we have this giant JSON of all the DLLs exported from like Windows 7. And it pulls all of the API calls out of there. So you don't have to keep using the DLLs. You can just use this giant JSON. And it will load it up and rebuild all of the addresses as it would in the PE image. And fill all of those addresses for you. So if it's doing any dynamic API calling and it moves an address into a register or an address into a new memory location, that address will actually be there simulated. All right. So like I said, dealing with dynamic API calls, you need the Teb and PEP. So we kind of recreate that for the Windows 7 environment. It handles memory. So anytime it accesses one of those, you can choose to allow it or not allow it. As well as managing all of those addresses on the stack, which is nice. And let me explain what I mean by this. So we have this shell code doing some interesting stuff. In the header imports, the only API calls it's actually using is exit process, get last error, get local time, and get module handle A, which we don't actually see load library A, virtual protect, or shell execute A, right? So here, you can see it's taking the string called load library A and pushing it onto the stack. From there, this function right below here, it's going to call this function and get that address off of the PEP. So that means it traverses all of the DLLs as well as the names from the PE headers in order to collect that address offset. Once it's done, it's going to put that address into EAX and then move into 401004, that pointer. And then down here, it's actually going to call that newly moved address to load shell32.dll, so it can also load shell execute A afterwards. So in reality, the dynamic imports are load library A, virtual protect, and shell execute A. So this is actually the output from Zory, and you'll be able to see all of that happen in there with the light emulation. All right. So the teb and peb, I didn't do anything really crazy. There's so many resources out there to look at the whole struct for the teb and peb. Some versions of Windows are going to be different, but I just used, you know, whatever was available out there. You don't need the whole, like, hidden features of the struct at all. Here's just, like, the top of it, for example. So I recreate, the nice thing about Rust is that you can take a struct and serialize that struct into a binary array or a binary vector, and you can have that vector being accessed by the memory manager as if the assembly instructions were to access those addresses as if it was actually on the OS, which is really nice. And so I had to figure out how to do a link list in bytes, serialized bytes, and put that all together into a structure so that the shell code could use it. All right. And so another thing to help with the analysis is that we have this thing called analysis queue where it will take the branch instruction, like, if else, and there's a call. You'll have two different directions in order to take for the call. So the right-hand side of the branch will go to the top of the queue while the other part of the branch will go to the bottom of the queue, which will help the code analysis follow the actual control flow for you. So it follows both directions. And I also introduced looping for the emulation. So say you run into an infinite loop, and you know it's an infinite loop, but you want to short circuit that loop case. There is a configuration there for you to short circuit all of the looping in there. Say you only want to do 4,000 times, you can set that information. I don't recommend using the emulation mode, but if you want to, you can play with it. I really need to improve it a little bit when you read a couple of research papers. And then, me? So signature analysis is a way of adding annotations to your code so that you don't have to be able to read hex like Amanda. You can just get a nice signature to tell you what the function was. I spent the last week and a half of my life trying to reverse a SIG parser from IDEP format so we could use it. I failed catastrophically in that those formats are always changing. It's very hard to keep up with it. That said, we do have signature matching in Zori right now, but we're always looking for help to make it better. And luckily some folks are already stepping up. So. Yeah. Oh, mine is me too. Oh my goodness. We're prepared. So this is the cool part for me as a data scientist, not a reverser. We took 100 Ember samples, which is a data set we, our company released of hashes. We took the actual binaries, ran them on this old Dell box that was under my desk. The only new thing new about it was SSD and it could get through all these samples in 20 minutes. So that was 1000 samples, 20 minutes, about 1.25 seconds of sample running in parallel, you know, maxing out all the CPUs. And it just dumps a huge amount of data and sometimes too much data if the binary is really big. And then you can do whatever you want. You can grep through it. You can find function names. You can find imports. You can do graph analysis over in the AI village. There's a bunch of people who think they can do a lot of cool things. So I'm really psyched about this. All right. Examples. So it's really easy to build Zory. All you need to do is make sure you install Rust, Cargo build release, target release Zory dash F, WannaCry.exe and it'll produce some results. So if you don't want to use analysis part of Zory, you can just pop in the disassembler part of it and throw it like this nice chunk of bytes and it'll disassemble this Hello World program. But it doesn't really show you any strings. It just gives you the disassembly output. So if you want to use the analysis part, it will actually populate all those strings when it's like popped off the stack and show you when a function ends and annotates all of those nice lovely comments for you. And comparing Zory with IDA Pro, we have the WannaCry ransomware file. You can see we don't have all of the nice argument enumerations yet, but you know, six months is what we can do. We can also see the short circuit string like I showed you in the beginning. This is just the normal output from the terminal. But you know, we have the UI for you to play with so you don't really have to use this part of it. But for development, maybe if you want to use it from the terminal, you can. Here's Radari 2 comparing the two different functions here. It's kind of weird because Radari does this thing where if it identifies API call within like the first couple instructions of a function, that's what it names the function. In this case, we don't try to do that. It's just a regular function call. But it's got parity between the two. And then I showed you the demo earlier, but I can show it to you again with a different sample. I'll just do a normal bin file. So it does PEN binary. So this is the same Hello World program. It's really fast. You just want to see the linear disassembly. You can see it populates Hello World. And you can see the function in the graph view as well. Or if you want to do something more fun with a bunch of jumps, oh, actually, wrong one. You can see how fast it is. This one was Rumbertick. It's actually part of my RE102 course. The longest loading time is actually the libraries there. But you can see this was a fun function, and it populates everything there for you. I don't have any string analysis since this was built for automation, but hopefully soon it will have all that in there for you. So I made it look really cool and spent the whole weekend on that, so hope you guys like it. Thank you. If you want to go download the code, it's already on github.com and gaming slash sorry. People are already playing with it. It's there for you to play with. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you.
before we begin, I want to go over the quick functionality and features, and I wanted to go the malware, if you're looking at malware. I'm a tool that you could play with in the browser on sample into a VM, open it up in IDA and analyze way to scale our automation. We wanted to it disassembly, which is something we've wanted disassemblers that were already out there, open open source. You can contribute now. We already displays strings from reference memory. If you're if you're pushing strings to the stack and it emulates it and produces a string for you, that's handle register stack and some instructions as the process environment block so that you can do Windows does for loading a PE. It's very basic, to this project, I decided to play not invented basically takes everything so we can data science a register or an address into a new memory explain what I mean by this. So we have this shell code doing some interesting stuff. In the time, and get module handle A, which we don't taking the string called load library A and whatever was available out there. You don't need help with the analysis is that we have this thing side of the branch will go to the top of the actual binaries, ran them on this old Dell box you need to do is make sure you install Rust, development, maybe if you want to use it from the