BLUE TEAM VILLAGE - SOC of 2020
IrishMASMS
The security operations aspect of your Information Security risk management program is where the “rubber meets the road” — the tools and people you have to implement the process and procedures you put together to find the badness and put out the fires. How has the concept of security operations evolved, and where are we headed? There is plenty of buzzword bingo: UBA, UEBA, machine learning and artificial intelligence, network abnormality detection, the marketing conversations of evolving to that SOC of 2020 — what do all these really mean to you and your operations and which can be useful in your efforts to find the badness?
 Hello, ladies and gentlemen, boys and girls. How are we doing today? Wow, we're still hung over, still, certainly not still drunk. Not at all. Let's start out with the disclaimer. This was not reviewed by my current, past or future employers. They have no insight into what I have been talking about this, nor that I was intending to. So don't hold them accountable, please. Basically, I speak no one. Nobody speaks for me. Various other legalese in there. So, who am I? Yeah, I'll drop a couple F bombs here and there. I apologize. But it's nice to be able to give this talk because so many years ago I actually started out as a SOC analyst, Inner Security Operations Center here in Las Vegas itself, worked there for many years, transferred to work being the technical lead for a incident response team at a large research laboratory and been downhill ever since. So this is going to talk about why is this important? Because in my current day job, I get to talk with vars and vendors and they're talking about evolving to the SOC of 2020 and this tool will solve all your problems and there's a hiring shortage, air quotes, et cetera, et cetera. And I just keep throwing up the bullshit flags. So, let's talk about this. Let's talk about how we've gotten into this situation. What can we really do to improve and what things should we be working on? And when those vendors come in going, hey, this is a great tool. You should look at it. You should buy it. You know how to see through that. So what is security operations? Right? What are we looking at here? What is the real meaning of that first? Let's set the stage for what we should be doing and then how we should be evolving to. I look at security operations as being that where the rubber hits the road for your organization, where your policies, procedures, what you're looking at is all those tool sets that you have bought, those operating systems that you're running, those applications, databases, et cetera, what they're feeding into, what are you looking for to find the badness? What are you looking for to find when Tim's been sending those harassing text messages to your secretary or surfing for porn or down a landing mount, et cetera? Not that I've ever done that. This is where it all feeds together, right? If you're not getting the right logs, if you're not getting the right information, how do you get that visibility? Normally we don't, right? That feeds into when we find something how we're going to do incident response with it, right? Oh, this is involving certain person. Now we need to do a forensic investigation in regards to what they've been up to. That should also feed into our security awareness program with our employees. And here's the things that we're seeing. Here's that phishing attack that our CEO got yesterday about, hey, I need to change my W2 deposit. From, yes, from some likehost.com account because I'm locked out of my domain account, right? Security operations and computer network defense. We can call it a SOC, we can call it a CSER, we can call it an S-SER, throw whatever three or four letter acronyms you want in there. Fundamentally is this is the group of folks of individuals or one individual in some places, right? That's looking for the badness and trying to put the fire out and then figuring out what we can do to make this better so it isn't as painful the next time. Though we sometimes forget there's three key pieces of this. People process technology. What do we tend to focus on because we're tech hacker geek nerds and we don't like talking to people and we hate writing down process. As we have, some of us have evolved and moved up the food chain as we're into sucky management positions that we don't get to do all the cool technical stuff, we need to focus on all three. How do we leverage each of the pieces? There is no silver bullet, right? No matter how often our CISO is looking for that, right? Or no, we don't have a CISO, let's talk about the CTL. There are certain necessary capabilities that each organization should have. Two varying degrees of how big they are, what the resources they have, what their threat profile look like, what critical infrastructure sector they're in, et cetera. Looking at the events and what they're happening, trying to correlate those to get some quantifying idea of what those events are looking like, right? Leveraging that threat intel, leveraging that indicators, putting those together to get an understanding of what your threat profile looks like and when the shit hits the fan, incident response. Put the fire out, clean it up, what can we do to make it better and try to prevent it from happening the next time, right? If it's more in depth, do we need to do digital forensics on it, right? Is this some commodity ransomware we've dealt with a million times or is this something new targeting us? Is this something that we can do some analysis on and share with the rest of the threat intel and incident response communities? And then just supporting the tool sets, the things that we have. Hey, Tim, how come we haven't gotten a report from the DLP system for two weeks? Oh, yeah, because we're not doing a performance monitoring yet and the drive filled up. DERP. Those audit and compliance controls, why should we sit in a weekly meeting going through spreadsheets? Why can't we leverage the technology and tools to do these cross checks for us, right? The intent of audit and compliance was, okay, kids, this is, you have to be this tall to ride this ride because companies didn't want to do security. So let's set something up as bare minimum to do. Well, with the tool sets we have, what things can we do to have this do it for us? And then certainly some insider threat detection, how many people are stealing source code, how many sales people are stealing the entire customer database for the next company they go to? Not that that ever happens, right? Key piece, vulnerability management. What phones do you have trying to correlate that with which ones the bad guys are trying to target and what things do we need to get the network server, desktop folks to fix priority, right? And then outreach. As all we're collecting all this stuff, as we have this ability and understanding, how do we communicate that in a way that the business folks understand it, the pointy head managers understand it, the C levels and even sometimes the board and for some of us, even to our customers, right? Because all those capabilities, because Matt did a great job at putting another pyramid together, how do we build from that? How do we need those different pieces in getting that understanding, right? And if you're looking at the SANS top 20, if you're looking at the NIST and all those other standards out there, this is pretty along the same lines, it just gives you better, pretty pictures and another good way to display it to the pointy head manager that you have to try to explain it to. But let's try to group those together and Matt did a great way of doing this, right? So when we're building and trying to mature our security operations programs, this gives us some guidelines and some tool sets to help evangelize this to the folks that we need to get those resources, whether it be money for tools, cooperation with other teams, head count, et cetera. And Wendy did this great one. I really like this. She just tweeted us out about two weeks ago. I'm like, oh yeah, this is going into talk. Because we all need some kind of pyramid, right? But yeah, fundamentally knowing what you have and what is it doing. How many times I sat in that sock not too many miles away from here and I'm dealing with first and second tier sock analysts are like, oh yeah, that's normal. And I started digging into it anyway and ended up being one of the biggest incidents for that quarter. Controlling those changes, understanding those threats and who's tacking them and using the products effectively and also monitoring their performance. Because there's that capability chasm and here John Lambert did a really good way of identifying those different areas that a lot of security operations centers or security operations capabilities fall into problem-wise. Not having executive support, falling into various traps. You don't have the right data or you have too much of it. Basically how do we as teams stay poor? How do we keep running around trying to put out fires instead of taking a couple hours every Friday afternoon and doing something more strategic to help us in the long run? So there's different types of security operations, right? First one, bare bones beginning, ad hoc when it hits the fan, when there's a certain individual or two from a certain three-lighter agency that's packing a weapon and knocks on your door going, we need to have a conversation, right? Not that I've ever experienced that. Yeah, the organizations that have evolved to a point of hey, we might have one or two security people within IT that do this. We actually have a full-fledged C-Cert whether it's dedicated or assigned people to full-fledged security operations centers. In this table here, I'll outline the organizational model, kind of the size of the organization and we'll touch on how to quantify that in a moment. And then geographical scope. And this is directly from the MITRE 10 strategies of World Class Cyber Security Operations Center. I will reference this book a couple different times even though I don't agree with everything it says, it has some really good information in it. So let's talk about that evolution. How have we started? How did we evolve? How are we still hunched over on that crappy S desk? Well, we started as cavemen, right? We might not be talking about Wesley Dale quite yet, but we're certainly looking in that direction, right? This is that ad hoc when it hits a fan. This is that IT guy that goes, oh, I don't need blogs. This is not knowing, but we're starting, right? We're trying to figure out the wheel. We're trying to figure out how to get fire. And then we kick out the person from FireEye. Because, you know, the IT guys, this is so easy a caveman can do it, right? But, you know, Irish, there's no evil bit on a packet, so we just let all of it through, right? This is the organization that doesn't know what they don't know, right? Meanwhile, all their data is being ex-filled. And some foreign countries making product based on their IP that, you know, they didn't have to do any R&D for other than figure out how to compromise your network. So let's go to the Middle Ages, right? Let's go to the somewhat organized ad hoc as needed when we need to go conquer Jerusalem or need to go conquer a local nation state. We need to go put out the fire and then we disband when we don't need it anymore, right? So when it does hit the fan, we might have some process and procedures, we might have some things going on, but it's all old from the last time we did it. That distributed co-managed. I look at this as being the explorers here in the United States, right? Let's go explore what's out there. We have an organized team doing it, but we still need some local help. Do we need an MSSP? Do we need somebody on retainer to do instant response, do forensics? What did, you know, thinking of Lewis and Clark and what things they needed to do to make their progress? Explorers are so meaningful in documenting all the birds they found, all of the nature that they found, right? How do we do that within our security operations center where we kind of hit this level, right? Have that ticketing system and document every little piece. And then leveraging that as a resource that we're searching for the next time something comes up. Oh, hey, we've had this ticket, four or five tickets with this same source IP because it's been beaten on our network. Leveraging the tools and the data that we're pulling together so that our exploration, our voyage is that much more meaningful. Then we have some sort of C-cert capability, right? We're in that organized way and trained across the desert. Hopefully we don't get stuck in truckie for the winter, right? But it's a group of individuals working together in some sort of organized way, right? For a common goal. Hopefully you don't kill too many of the natives while doing it. But you also have those security operations centers that are like the gold rush, right? They keep running to every place they hear that there's gold and they dig and dig and what do they find? And they completely run out all the land that's there that we completely devastated looking for that one nugget. But then we can do security operations at scale. Let's get a tier one, tier two, tier three, let's get a bunch of tier one folks doing the same process manually over and over and over again, just like sewing. Just like the seamstresses that died in that fire. Because they're going to burn out, they're going to say F this I'm going somewhere else. So let's try to put some automation into it, right? What's those things that we can do that we do every day? How can we automatically create a ticket and pull in a threat intel from our feeds and from internal and connect all the tickets that have that same IP address in it? Let's try to make it easier for ourselves, easier to make those determinations on how risky is it and how much do we have to take care of this, right? So I mentioned the MITRE book and the process, certainly determining optimal manning levels I worked with multiple clients in regards to how do we figure this out? Well, certainly there's no firm answer, there's no real equation of figure this out, but I have put this together based on MITRE, based on my own experiences as an analyst and working with multiple clients and working in multiple places for many years and how can we get at least into the ballpark of how many people we need and how do we leverage them? This also you need to look at how you as a company are growing so you can project how you need to grow your team, right? So first thing, just like the pyramids, just like the sans critical controls, you have to know number of what you have in your organization and what you're trying to monitor, at least the ballpark, right? Including the number of sensors you think you're going to need to use, how many cloud providers, domains, et cetera, internal, external, because you're going to be getting the logs from that. Then what's your threat? Who's targeting you and how? What compliance and regulatory obligations you have? Which are geographical soap, et cetera. Add all that up. If you have less than a thousand, then you probably don't need a 24-7, but depending on your threats and your risk plan, you're probably going to need something, all right? And if you have many thousands, then you definitely need something. If you don't, I really like to know how you're dealing right now. And I'll buy you some alcohol to deal with it, right? So specifically for analysts, specifically folks that's going through all the data, good ballpark number is one FTE per 50 to 75 devices, IPs, domains, et cetera. This certainly depends on the number of logs you're getting from those. And the intent is to handle all those critical or literature getting after turning your tools as well as process and procedures. So, optimal situation, right? Number for ratio from tier one to tier two for basically every two tier one analysts between one and four tier two. And if you're doing FTE for a 24 by 7 security operation center, it's more like five to one or three to one in a sock because you need shift operations, you need scheduling, you need floaters for when people are out sick or say, F this, I'm leaving the company, I got a better job, et cetera. Or, hey, the half the sock is going to DEF CON. Not that that ever happens, right? So for engineers, you know, as the folks that we have that are doing the sensor tuning or soon the sensor maintenance, basically the stuff, all that tool sets, platforms that we're getting the data from, about half FTE, depending on the deployment and also the tool sets you're using, right? If you're using certain SIMs that start with an A, you're probably going to need a few more, right? Other ones, potentially less just because of what's involved in keeping them running. If you have your system administrators broken out from your engineers, then your typical ratio of one to one or one or two to sysadmin engineer types. Depends on your organization, depends on how mature you are, depending on your separation of duties. And to your fellow sysadmin out there, a belated sysadmin day. So placement in your organization for your security program, because this is going to directly relate, right? Do you have the wherewithal? Do you have the resources? Does the individual that you report to know the balance of the confidentiality, risk, et cetera? Right? Or is it going to be like, no, don't turn that off. We need that for such a business thing. We don't care. So you have the conflict of interest. Yeah, we might have a CISO, but who does that CISO report to? Does he report or she report directly to the CTO? You still have that conflict of interest, right? Do you actually mature enough or as big enough of an organization that you have a chief risk officer that looks all of this? And some organizations, hey, they might have security in the sock underneath legal, which is awesome, but are they tech savvy enough to understand what you're talking about when you come and tell them that your website has a cross-site scripting attack going on? So common conversation is, do you have something internal or do you need to contract it all out? Give it to an MSSP. Certainly, it could be useful to augment your internal capabilities for your smaller, less mature organizations, but, you know, MSSP services are commoditized. There's those common issues you deal with when you talk with certain organizations of, how come you don't do this? Oh, well, you didn't buy that package. You didn't look through the terms and conditions, TNC, so we don't do that. So certainly, if you're in the market for getting an MSSP, understand what they're going to be providing you. There are the big ones. There's the small ones and then there's the one MSSP's in the middle. I kind of, I try to, I call those the Poteek MSSP's. Those tend to fit most clients, most folks I work with a little bit better in regards to their capabilities and being able to pay attention to you and your needs versus just shooting alerts via email or to your ticketing system. Key point is that last bullet there. You, I need to manage that relationship and nurture that relationship. If you're not talking with them on a regular basis, they're not going to care. If you don't hold them accountable, they're certainly not going to care because they're just collecting your money. Oh, well, I don't need to buy an MSSP. I don't need to sign it up. I'll just do it myself. How well has that gone? Certainly, if you're the Facebook, Google's of the world and you have a shit ton of developers and that's all they love to do, go ahead. I've seen some of their projects that they use internally that they started 10 years ago and then the guy left and nobody's maintained it since. So what's your software development site to look like? What happens when the folks that maintain that get hit by a bus or leave the company or retire? I got my stock. I'm out. Fitts. What is it about those open source tools that are already out there? What is it about Greylog that you don't like? Why that you can't use it in your environment that you're going to need to create your known for brand new? Why can't we just fix what you don't like and improve the open source project that's out there? Tied to that is also how do we staff our stock? So that MITRE book heavily encouraged that we only needed developers in the stock. I call bullshit. Sorry. It's that diversity piece of getting different individuals that have experiences in different areas within information technology and information security that in the middle of an incident go, hey, I was a sysadmin on a sun system and this is what it does. This is how we can go and find it. This is how we can fight the badness, right? No slam against developers but most developers don't run their own systems, right? They don't have that expertise. It's having different individuals and different perspectives in to deal with that fire fight sooner and faster and also come up with better ideas on how to prevent this from happening in the future based on the background experience that we have. We need real and valuable metrics. This is how we use to justify that security operations center we have. This is the baseline that we use to figure out how are we going to evolve to the stock of 2020. If we don't know how we're doing now, if we're not monitoring the performance on that DOP system to realize that the drive filled up, how are we supposed to know how we need to evolve to? So I did a Google search. This is the first image that came up. In evolving to the... Thank you marketing. All right. Because that's a good portion of it, right? There's a whole lot of companies out there trying to pitch this stuff. What's really going to make a difference? So the first thing I've seen is having some robust monitoring detection, right? But we have span ports. All right. Wait a minute. Span ports were never intended to be a permanent fixture. They were only supposed to be a diagnostic tool, right? They're only a sample of the traffic and the first to die when that device gets overloaded. And when I'm in the middle, too many times I've been in the middle incident and I'm trying to figure out why our devices are not triggering on the badness because that span port is fucking dying on me. Basically we're missing activity. If we can't see it, we can't fight it, we can't put the fire out. When does this normally become a problem? When the organization runs out of span ports. A little too late, right? But, all right. So the packet brokers, there's a couple other terms out there. Basically it's a middleman for your network traffic. So instead of putting multiple devices in your network in line, you put the packet broker there and then make a copy of all that traffic and then filter it however you want to your monitoring detection tools. So it makes, it's a force multiplier for the tool sets that's monitoring your network, not just for security but also for performance monitoring. So this is a tag team effort with your network folks. Some of these also give you IP fix DNS, other data out of it. This also lets you use less tech, less platforms. So they're in a prior life, particular organization got a quote for, I think it was 35 boxes this company wanted to install in this environment. You can imagine the price tag on that. Brought in a packet broker, we were able to cut that in half. That quote went down significantly. That vendor was not happy-ish, but at least they were happy to get the opportunity to deal. And we were able to help that organization out by getting better coverage for their environment and into the tool set they wanted to deploy. Not all packet brokers are created equal. There's certainly a ton of marketing, a ton of foot out there. The UX UI experience with some of these are horrible. Some abilities you can only access through the command line, not through the UI. Also, check the security posture of these devices themselves. Have they updated their firmware lately? What's their software development life cycle? Certainly do a third party risk assessment before you buy these platforms. Now internally, who's going to maintain these? Is this your networking team, is this a security team? Those are some questions that come up? And certainly the cost of them. Quick picture on how those are normally deployed. Let's talk about data integration, right? Getting that data into your monitoring detection platforms. Hey, I already have X platform. It has the data feeds coming into it. Great. How are you using those feeds in your SIM, in your other platforms? You able to leverage those as a pipeline, as it stove pipes into each of these tools and you're not sharing them back and forth? Is it feeding your ticketing system for when hits happen and a ticket gets open? Need that situation awareness? Provides better mitigation decisions? Basing it on real data, not that gut feel that we tend to do with insecurity. These tools are not intended to be myopic. Why do we keep trying to deploy them as such? And then from that, let's automate and orchestrate using the real data. Address that low hanging fruits or you're less of a target. Hey. I just call it automated orchestration. Gartner likes to make big acronym words or whatever. This is no different than the Python, other scripts I did 10, 15 years ago. Right? But now we have a platform. Okay. Awesome. Because it's a relatively easy deployment. I can have a tier one, tier two analysts go in and do some clicks and create or update a playbook. And the next place we go to or the new person that comes into the team is already familiar with it. We can't say that about that Perlin Python script that I wrote on a Sunday night after a couple of Bourbon Bear laid doubts, right? When you're looking at these, be cautious. There's been plenty of acquisition. There's a ton of buzzword bingo out there. Do a POC, do a POV, look at the acquisition space and who's looking at it and what's going on. Oh, yeah. How many times did we see this on the black hat floor yesterday, day before? Right? The premise is to take all that information, your logs, and come up with that user account or that entity account and try to identify that this is risky or there's a major threat. All right? Yes. It is useful in ways of if you're mature enough and you have a threat hunting or you have a wolf team that go digging, this gives them some indicators to go chase down. This also will help some of your first tier analysts in if you're not mature enough to create some of those pretty advanced rule sets in your SIM, this will help simmer some of those things up to the top and help correlate some of those log entries that, you know, don't really quite make sense. Good case in point is, was working with a client. They had an alert come up that one of their employees, one of their phones was in South America trying to log into all their accounts. Okay. The analyst tried saying, oh, false positive, this product sucks, get it out of our environment. Wait a minute. This is a question of risk, all right? Is that employee actually in South America? Well, call HR. Yes, she's on vacation. Okay, good. Second question, why did she take her work phone with her and why is she trying to log in? And does she still have her phone with her because in that particular country, it could have been stolen and quite likely could have been. So is that question of risk when you're looking at these tool sets, it might be less risky for your organization but potentially not for others. And that's where we need to look at for these tool sets of it just helping giving you indicators of rabbit holes that go down into to make that assessment. It can't get judgment call on risk for you. So with that UBA, UBA, let's add the MLAI. And this is getting thrown on a lot as well, right? Is it really machine learning? Is it really artificial intelligence? Or is it that startup that a friend of mine worked at that they called MLAI by just counting how many times an end user would log in? They didn't do really any of the potential math after that. So definitely dig into when you're talking with these vendors to cut through the bullshit, cut through the marketing buzz. Similar with that potential perception of false positives, similar to UBA, UBA, what's that question of risk in those alerts that it's providing you? Because when you have those things alert, it's not like an old IDS alert anymore where this is bad, this packet was bad. These tool sets are trying to correlate all these different things together to go this is kind of fuzzy. This might be bad. So it's a question of how we're going to assess risk. So let's take that MLAI and look at all the network activity from your span port or your packet broker, right? And then look at that endpoint and look at that traffic to figure out how bad it is. Be very cautious with some of the vendors in this space. Some of you folks, I'm sure, have already gone through the sales pitch with a certain company that only hires very pretty dude bros right out of college that give you the hard sell that have completely drank the Kool-Aid and if they get any inkling that they are not making a sale, they completely flip, rip the gear out of your data center and they will never talk to you again. Even if you see them on a conference show floor, they will turn the other way because you didn't drink their Kool-Aid. What does that say about their product, their sale cycle and what they can really do with that is their marketing approach and sales approach. The thing about, I have been able to use in the non-detection space is very similar of looking at this traffic over a long term to try to figure out what's really bad. What is those protocols that we're not expecting? Why are these two DNS servers talking to each other over open VPN on our network? Are the tool sets we're using, our IDS, our IPS, the ML AI, all these other tools, are they working as they're intended and supposed to? How do we take the scientific method to prove or disprove that that IDS is feeding into our SIM and we're getting the right alerts and we're taking the right automation tools, measures to stop the badness, right? So there's some companies out there that are doing this threat simulation platforms is what I'm calling it, right? Basically, you establish some fake endpoints and some systems internally, there's a similar system outside your network and they run attacks in between those pockets and does your tool sets adequately detect through the kill chain as they're simulating them what do you need to see to find the badness when it's the real bad? Like anything else? There's some thud, be cautious, right? How are they integrated? Is it really a separate platform or is it a function of some existing tools you already have internally just haven't leveraged yet? And then deception, which again, just like some of the automation things we have done in the 10, 15 some odd years ago and there's some stuff we've done on our own in regards to deception and sending up honeypots and fake documents and fake email addresses and that sort of thing. Here now you have a vendor that's providing a platform to do that for you and monitor for the badness and activity. Again, like everything else we've been talking about, there's a whole lot of thud out there. What would be useful to you and how mature you are as an organization? Can you do this yourself? Can you deploy it out yourself or can you leverage this vendor to push it out and maintain it in an easier way with less resources? So how are we doing? How do we measure ourselves? How do we say that we need to go in this direction or that direction, right? There's those optimal characteristics and capabilities like we mentioned earlier. You can group it in different areas. There's that organizational model and how we're structured. Do we have the authority to do what we need to do? Do we have the capabilities, et cetera? Do we have the right process procedures to implement all this? How do we get there? How do we have that capability? Internally, we can run with ENSA, the C-Cert self-assessment. The first GCS also has a maturity survey. Certainly leveraging the SANS critical controls. We have the NIST ISOEEC. So as part of our security program, as part of these process procedures that we're putting together, we should also put in cross checks for us to check if we're doing the right things. Are we feeding the right stuff? And on a periodic basis, depending on your organization and your threat profile, finding somebody to bring from an external agency to make that assessment and make recommendations on how you should improve. Hopefully with an organization you already have a report with that already know you. Taking that measurements and the model of, hey, here's where you at and from our conversations and your threat profile and where you should be as an organization, here's where you should be and you can put together a roadmap of how do you get there. And then you take that roadmap to your director of security and your CISO and they go and fight for the budget and headcount, et cetera. And here's our plan on how we're going to mature as an organization. So where do we go from that? We have our log processors, we have our dirty sock, we have the pen tester come in every so often. What's next? Through the ages, we've evolved but we certainly have not matured as an entity, as an organization, as a profession. So as security professionals, as hackers, what can we do to help with that? We need to ensure the basics are complete. Back to those triangles, do we have that foundation to build from or have we jumped ahead? Be willing to hire, be willing to invest in those hires? Be wise to the hiring in your local area to your needs and capabilities and what's available in your local market because if you're trying to go, oh, we're going to deploy this tool but there's nobody available in your local market at an affordable level for your organization to run it and maintain it, does that make economic sense for your organization? Does that make proper sense in regards to risk for your organization? And is it good enough or are we trying for that perfect solution? So many times we're having conversations, not just what last week, week before, oh, Reddit got pooned because they were using SMS two-factor and SMS two-factor sucks and nobody should be using it and blah, blah, blah, blah, blah. Well, for them, they made a risk call and they said, hey, it's good enough. Certainly they didn't probably have, potentially have the right monitoring pieces in there to go, are these admins getting targeted? If it's good enough, have that monitor there to be able to detect and see, is it not good enough anymore and we need to add another mitigation, other compensating controls? So some takeaways from me rambling for the last few minutes. Be careful. Look past that buzzword bingo and the marketing. Use that real data to try to determine what's useful for your environment, your risk posture. What's that size and scope for your organization? When do you need to step aside and go, hey, I'm in over my head, when do I need to bring somebody from external help us with this? Certainly with that, hire to your realistic needs and be willing to invest in them. If you can't find somebody to hire for this, why should you have that position open for six months? Hire somebody now, you can train them up in that six months, right? A couple of different references, I've already mentioned a couple of them. Quite actually all of them it looks like. Thank you for joining me, Matt, John, and so first, the other organizations, thank to all of us that have come before us, that have worked in SOCs and have helped us evolve to where we are today. And thank you all that are going to take that next step and help us evolve to the SOC 2020. Any questions? Ah, yes, so if you have any questions, let's meet outside in the hallway next to the bar. Thank you very much all. Have a good day. Enjoy DEF CON.
 Hello, ladies and gentlemen, boys and girls. How are we doing today? Wow, we're still hung  over, still, certainly not still drunk. Not at all. Let's start out with the disclaimer.  This was not reviewed by my current, past or future employers. They have no insight into  what I have been talking about this, nor that I was intending to. So don't hold them accountable,  please. Basically, I speak no one. Nobody speaks for me. Various other legalese in there. So,  who am I? Yeah, I'll drop a couple F bombs here and there. I apologize. But it's nice to be able  to give this talk because so many years ago I actually started out as a SOC analyst,  Inner Security Operations Center here in Las Vegas itself, worked there for many years,  transferred to work being the technical lead for a incident response team at a large research  laboratory and been downhill ever since. So this is going to talk about why is this important?  Because in my current day job, I get to talk with vars and vendors and they're talking about  evolving to the SOC of 2020 and this tool will solve all your problems and there's a hiring  shortage, air quotes, et cetera, et cetera. And I just keep throwing up the bullshit flags. So,  let's talk about this. Let's talk about how we've gotten into this situation. What can we  really do to improve and what things should we be working on? And when those vendors come in  going, hey, this is a great tool. You should look at it. You should buy it. You know how to see  through that. So what is security operations? Right? What are we looking at here? What is the  real meaning of that first? Let's set the stage for what we should be doing and then how we  should be evolving to. I look at security operations as being that where the rubber hits the  road for your organization, where your policies, procedures, what you're looking at is all those  tool sets that you have bought, those operating systems that you're running, those applications,  databases, et cetera, what they're feeding into, what are you looking for to find the  badness? What are you looking for to find when Tim's been sending those harassing text  messages to your secretary or surfing for porn or down a landing mount, et cetera? Not that  I've ever done that. This is where it all feeds together, right? If you're not getting the  right logs, if you're not getting the right information, how do you get that visibility?  Normally we don't, right? That feeds into when we find something how we're going to do incident  response with it, right? Oh, this is involving certain person. Now we need to do a forensic  investigation in regards to what they've been up to. That should also feed into our security  awareness program with our employees. And here's the things that we're seeing. Here's that  phishing attack that our CEO got yesterday about, hey, I need to change my W2 deposit.  From, yes, from some likehost.com account because I'm locked out of my domain account,  right? Security operations and computer network defense. We can call it a SOC, we can call  it a CSER, we can call it an S-SER, throw whatever three or four letter acronyms you  want in there. Fundamentally is this is the group of folks of individuals or one individual  in some places, right? That's looking for the badness and trying to put the fire out  and then figuring out what we can do to make this better so it isn't as painful the next  time. Though we sometimes forget there's three key pieces of this. People process technology.  What do we tend to focus on because we're tech hacker geek nerds and we don't like talking  to people and we hate writing down process. As we have, some of us have evolved and moved  up the food chain as we're into sucky management positions that we don't get to do all the  cool technical stuff, we need to focus on all three. How do we leverage each of the pieces?  There is no silver bullet, right? No matter how often our CISO is looking for that, right?  Or no, we don't have a CISO, let's talk about the CTL. There are certain necessary capabilities  that each organization should have. Two varying degrees of how big they are, what the resources  they have, what their threat profile look like, what critical infrastructure sector they're  in, et cetera. Looking at the events and what they're happening, trying to correlate those  to get some quantifying idea of what those events are looking like, right? Leveraging  that threat intel, leveraging that indicators, putting those together to get an understanding  of what your threat profile looks like and when the shit hits the fan, incident response.  Put the fire out, clean it up, what can we do to make it better and try to prevent it  from happening the next time, right? If it's more in depth, do we need to do digital forensics  on it, right? Is this some commodity ransomware we've dealt with a million times or is this  something new targeting us? Is this something that we can do some analysis on and share  with the rest of the threat intel and incident response communities? And then just supporting  the tool sets, the things that we have. Hey, Tim, how come we haven't gotten a report from  the DLP system for two weeks? Oh, yeah, because we're not doing a performance monitoring yet  and the drive filled up. DERP. Those audit and compliance controls, why should we sit  in a weekly meeting going through spreadsheets? Why can't we leverage the technology and tools  to do these cross checks for us, right? The intent of audit and compliance was, okay,  kids, this is, you have to be this tall to ride this ride because companies didn't want  to do security. So let's set something up as bare minimum to do. Well, with the tool  sets we have, what things can we do to have this do it for us? And then certainly some  insider threat detection, how many people are stealing source code, how many sales people  are stealing the entire customer database for the next company they go to? Not that  that ever happens, right? Key piece, vulnerability management. What phones do you have trying  to correlate that with which ones the bad guys are trying to target and what things do  we need to get the network server, desktop folks to fix priority, right? And then outreach.  As all we're collecting all this stuff, as we have this ability and understanding, how  do we communicate that in a way that the business folks understand it, the pointy head managers  understand it, the C levels and even sometimes the board and for some of us, even to our  customers, right? Because all those capabilities, because Matt did a great job at putting another  pyramid together, how do we build from that? How do we need those different pieces in getting  that understanding, right? And if you're looking at the SANS top 20, if you're looking at the  NIST and all those other standards out there, this is pretty along the same lines, it just  gives you better, pretty pictures and another good way to display it to the pointy head  manager that you have to try to explain it to. But let's try to group those together  and Matt did a great way of doing this, right? So when we're building and trying to mature  our security operations programs, this gives us some guidelines and some tool sets to help  evangelize this to the folks that we need to get those resources, whether it be money  for tools, cooperation with other teams, head count, et cetera. And Wendy did this great  one. I really like this. She just tweeted us out about two weeks ago. I'm like, oh yeah,  this is going into talk. Because we all need some kind of pyramid, right? But yeah, fundamentally  knowing what you have and what is it doing. How many times I sat in that sock not too many  miles away from here and I'm dealing with first and second tier sock analysts are like,  oh yeah, that's normal. And I started digging into it anyway and ended up being one of the  biggest incidents for that quarter. Controlling those changes, understanding those threats  and who's tacking them and using the products effectively and also monitoring their performance.  Because there's that capability chasm and here John Lambert did a really good way of  identifying those different areas that a lot of security operations centers or security  operations capabilities fall into problem-wise. Not having executive support, falling into  various traps. You don't have the right data or you have too much of it. Basically how do  we as teams stay poor? How do we keep running around trying to put out fires instead of taking  a couple hours every Friday afternoon and doing something more strategic to help us in the  long run? So there's different types of security operations, right? First one, bare bones beginning,  ad hoc when it hits the fan, when there's a certain individual or two from a certain  three-lighter agency that's packing a weapon and knocks on your door going, we need to  have a conversation, right? Not that I've ever experienced that. Yeah, the organizations that have  evolved to a point of hey, we might have one or two security people within IT that do this.  We actually have a full-fledged C-Cert whether it's dedicated or assigned people to full-fledged  security operations centers. In this table here, I'll outline the organizational model, kind of  the size of the organization and we'll touch on how to quantify that in a moment. And then  geographical scope. And this is directly from the MITRE 10 strategies of World Class Cyber  Security Operations Center. I will reference this book a couple different times even though I  don't agree with everything it says, it has some really good information in it. So let's talk  about that evolution. How have we started? How did we evolve? How are we still hunched over on  that crappy S desk? Well, we started as cavemen, right? We might not be talking about  Wesley Dale quite yet, but we're certainly looking in that direction, right? This is that ad hoc  when it hits a fan. This is that IT guy that goes, oh, I don't need blogs. This is not knowing,  but we're starting, right? We're trying to figure out the wheel. We're trying to figure out how to  get fire. And then we kick out the person from FireEye. Because, you know, the IT guys, this is  so easy a caveman can do it, right? But, you know, Irish, there's no evil bit on a packet, so we  just let all of it through, right? This is the organization that doesn't know what they don't  know, right? Meanwhile, all their data is being ex-filled. And some foreign countries making  product based on their IP that, you know, they didn't have to do any R&D for other than figure  out how to compromise your network. So let's go to the Middle Ages, right? Let's go to the  somewhat organized ad hoc as needed when we need to go conquer Jerusalem or need to go conquer a  local nation state. We need to go put out the fire and then we disband when we don't need it  anymore, right? So when it does hit the fan, we might have some process and procedures, we might  have some things going on, but it's all old from the last time we did it. That distributed  co-managed. I look at this as being the explorers here in the United States, right? Let's go  explore what's out there. We have an organized team doing it, but we still need some local help. Do  we need an MSSP? Do we need somebody on retainer to do instant response, do forensics? What did,  you know, thinking of Lewis and Clark and what things they needed to do to make their  progress? Explorers are so meaningful in documenting all the birds they found, all of the nature  that they found, right? How do we do that within our security operations center where we kind of hit  this level, right? Have that ticketing system and document every little piece. And then leveraging  that as a resource that we're searching for the next time something comes up. Oh, hey, we've had this  ticket, four or five tickets with this same source IP because it's been beaten on our network. Leveraging  the tools and the data that we're pulling together so that our exploration, our voyage is that much  more meaningful. Then we have some sort of C-cert capability, right? We're in that organized  way and trained across the desert. Hopefully we don't get stuck in truckie for the winter, right? But  it's a group of individuals working together in some sort of organized way, right? For a common  goal. Hopefully you don't kill too many of the natives while doing it. But you also have those  security operations centers that are like the gold rush, right? They keep running to every place they  hear that there's gold and they dig and dig and what do they find? And they completely run out all  the land that's there that we completely devastated looking for that one nugget. But then we can do  security operations at scale. Let's get a tier one, tier two, tier three, let's get a bunch of tier  one folks doing the same process manually over and over and over again, just like sewing. Just like  the seamstresses that died in that fire. Because they're going to burn out, they're going to say F this  I'm going somewhere else. So let's try to put some automation into it, right? What's those things  that we can do that we do every day? How can we automatically create a ticket and pull in a  threat intel from our feeds and from internal and connect all the tickets that have that same IP  address in it? Let's try to make it easier for ourselves, easier to make those determinations on  how risky is it and how much do we have to take care of this, right? So I mentioned the MITRE  book and the process, certainly determining optimal manning levels I worked with multiple  clients in regards to how do we figure this out? Well, certainly there's no firm answer, there's  no real equation of figure this out, but I have put this together based on MITRE, based on my  own experiences as an analyst and working with multiple clients and working in multiple places  for many years and how can we get at least into the ballpark of how many people we need and how  do we leverage them? This also you need to look at how you as a company are growing so you can  project how you need to grow your team, right? So first thing, just like the pyramids, just like  the sans critical controls, you have to know number of what you have in your organization and  what you're trying to monitor, at least the ballpark, right? Including the number of sensors  you think you're going to need to use, how many cloud providers, domains, et cetera, internal,  external, because you're going to be getting the logs from that. Then what's your threat? Who's  targeting you and how? What compliance and regulatory obligations you have? Which are geographical  soap, et cetera. Add all that up. If you have less than a thousand, then you probably don't need a  24-7, but depending on your threats and your risk plan, you're probably going to need something,  all right? And if you have many thousands, then you definitely need something. If you don't, I  really like to know how you're dealing right now. And I'll buy you some alcohol to deal with it,  right? So specifically for analysts, specifically folks that's going through all the data, good  ballpark number is one FTE per 50 to 75 devices, IPs, domains, et cetera. This certainly depends on  the number of logs you're getting from those. And the intent is to handle all those critical  or literature getting after turning your tools as well as process and procedures. So, optimal  situation, right? Number for ratio from tier one to tier two for basically every two tier one  analysts between one and four tier two. And if you're doing FTE for a 24 by 7 security  operation center, it's more like five to one or three to one in a sock because you need shift  operations, you need scheduling, you need floaters for when people are out sick or say,  F this, I'm leaving the company, I got a better job, et cetera. Or, hey, the half the sock is going  to DEF CON. Not that that ever happens, right? So for engineers, you know, as the folks that we  have that are doing the sensor tuning or soon the sensor maintenance, basically the stuff, all  that tool sets, platforms that we're getting the data from, about half FTE, depending on the  deployment and also the tool sets you're using, right? If you're using certain SIMs that start  with an A, you're probably going to need a few more, right? Other ones, potentially less just  because of what's involved in keeping them running. If you have your system administrators  broken out from your engineers, then your typical ratio of one to one or one or two to  sysadmin engineer types. Depends on your organization, depends on how mature you are,  depending on your separation of duties. And to your fellow sysadmin out there, a belated  sysadmin day. So placement in your organization for your security program, because this is  going to directly relate, right? Do you have the wherewithal? Do you have the resources?  Does the individual that you report to know the balance of the confidentiality, risk,  et cetera? Right? Or is it going to be like, no, don't turn that off. We need that for such a  business thing. We don't care. So you have the conflict of interest. Yeah, we might have a  CISO, but who does that CISO report to? Does he report or she report directly to the CTO?  You still have that conflict of interest, right? Do you actually mature enough or as big  enough of an organization that you have a chief risk officer that looks all of this? And  some organizations, hey, they might have security in the sock underneath legal, which is awesome,  but are they tech savvy enough to understand what you're talking about when you come and tell  them that your website has a cross-site scripting attack going on? So common conversation is, do  you have something internal or do you need to contract it all out? Give it to an MSSP.  Certainly, it could be useful to augment your internal capabilities for your smaller, less  mature organizations, but, you know, MSSP services are commoditized. There's those common issues  you deal with when you talk with certain organizations of, how come you don't do this? Oh, well,  you didn't buy that package. You didn't look through the terms and conditions, TNC, so we  don't do that. So certainly, if you're in the market for getting an MSSP, understand what  they're going to be providing you. There are the big ones. There's the small ones and then  there's the one MSSP's in the middle. I kind of, I try to, I call those the Poteek MSSP's.  Those tend to fit most clients, most folks I work with a little bit better in regards to their  capabilities and being able to pay attention to you and your needs versus just shooting  alerts via email or to your ticketing system. Key point is that last bullet there. You, I  need to manage that relationship and nurture that relationship. If you're not talking with  them on a regular basis, they're not going to care. If you don't hold them accountable,  they're certainly not going to care because they're just collecting your money. Oh, well,  I don't need to buy an MSSP. I don't need to sign it up. I'll just do it myself. How well  has that gone? Certainly, if you're the Facebook, Google's of the world and you have a shit  ton of developers and that's all they love to do, go ahead. I've seen some of their projects  that they use internally that they started 10 years ago and then the guy left and nobody's  maintained it since. So what's your software development site to look like? What happens  when the folks that maintain that get hit by a bus or leave the company or retire? I got  my stock. I'm out. Fitts. What is it about those open source tools that are already out  there? What is it about Greylog that you don't like? Why that you can't use it in your environment  that you're going to need to create your known for brand new? Why can't we just fix what  you don't like and improve the open source project that's out there? Tied to that is  also how do we staff our stock? So that MITRE book heavily encouraged that we only needed  developers in the stock. I call bullshit. Sorry. It's that diversity piece of getting  different individuals that have experiences in different areas within information technology  and information security that in the middle of an incident go, hey, I was a sysadmin on  a sun system and this is what it does. This is how we can go and find it. This is how  we can fight the badness, right? No slam against developers but most developers don't run  their own systems, right? They don't have that expertise. It's having different individuals  and different perspectives in to deal with that fire fight sooner and faster and also  come up with better ideas on how to prevent this from happening in the future based on  the background experience that we have. We need real and valuable metrics. This is how  we use to justify that security operations center we have. This is the baseline that we  use to figure out how are we going to evolve to the stock of 2020. If we don't know how  we're doing now, if we're not monitoring the performance on that DOP system to realize  that the drive filled up, how are we supposed to know how we need to evolve to? So I did  a Google search. This is the first image that came up. In evolving to the... Thank you  marketing. All right. Because that's a good portion of it, right? There's a whole lot of  companies out there trying to pitch this stuff. What's really going to make a difference?  So the first thing I've seen is having some robust monitoring detection, right? But we  have span ports. All right. Wait a minute. Span ports were never intended to be a permanent  fixture. They were only supposed to be a diagnostic tool, right? They're only a sample of the  traffic and the first to die when that device gets overloaded. And when I'm in the middle,  too many times I've been in the middle incident and I'm trying to figure out why our devices  are not triggering on the badness because that span port is fucking dying on me. Basically  we're missing activity. If we can't see it, we can't fight it, we can't put the fire out.  When does this normally become a problem? When the organization runs out of span ports.  A little too late, right? But, all right. So the packet brokers, there's a couple other  terms out there. Basically it's a middleman for your network traffic. So instead of putting  multiple devices in your network in line, you put the packet broker there and then make  a copy of all that traffic and then filter it however you want to your monitoring detection  tools. So it makes, it's a force multiplier for the tool sets that's monitoring your network,  not just for security but also for performance monitoring. So this is a tag team effort with  your network folks. Some of these also give you IP fix DNS, other data out of it. This  also lets you use less tech, less platforms. So they're in a prior life, particular organization  got a quote for, I think it was 35 boxes this company wanted to install in this environment.  You can imagine the price tag on that. Brought in a packet broker, we were able to cut that  in half. That quote went down significantly. That vendor was not happy-ish, but at least  they were happy to get the opportunity to deal. And we were able to help that organization  out by getting better coverage for their environment and into the tool set they wanted to deploy.  Not all packet brokers are created equal. There's certainly a ton of marketing, a ton  of foot out there. The UX UI experience with some of these are horrible. Some abilities  you can only access through the command line, not through the UI. Also, check the security  posture of these devices themselves. Have they updated their firmware lately? What's their  software development life cycle? Certainly do a third party risk assessment before you  buy these platforms. Now internally, who's going to maintain these? Is this your networking  team, is this a security team? Those are some questions that come up? And certainly the cost  of them. Quick picture on how those are normally deployed. Let's talk about data integration,  right? Getting that data into your monitoring detection platforms. Hey, I already have X  platform. It has the data feeds coming into it. Great. How are you using those feeds in  your SIM, in your other platforms? You able to leverage those as a pipeline, as it stove  pipes into each of these tools and you're not sharing them back and forth? Is it feeding  your ticketing system for when hits happen and a ticket gets open? Need that situation  awareness? Provides better mitigation decisions? Basing it on real data, not that gut feel  that we tend to do with insecurity. These tools are not intended to be myopic. Why do we keep  trying to deploy them as such? And then from that, let's automate and orchestrate using the  real data. Address that low hanging fruits or you're less of a target. Hey. I just call  it automated orchestration. Gartner likes to make big acronym words or whatever. This  is no different than the Python, other scripts I did 10, 15 years ago. Right? But now we  have a platform. Okay. Awesome. Because it's a relatively easy deployment. I can have a  tier one, tier two analysts go in and do some clicks and create or update a playbook. And  the next place we go to or the new person that comes into the team is already familiar  with it. We can't say that about that Perlin Python script that I wrote on a Sunday night  after a couple of Bourbon Bear laid doubts, right? When you're looking at these, be cautious.  There's been plenty of acquisition. There's a ton of buzzword bingo out there. Do a POC,  do a POV, look at the acquisition space and who's looking at it and what's going on. Oh,  yeah. How many times did we see this on the black hat floor yesterday, day before? Right?  The premise is to take all that information, your logs, and come up with that user account  or that entity account and try to identify that this is risky or there's a major threat.  All right? Yes. It is useful in ways of if you're mature enough and you have a threat  hunting or you have a wolf team that go digging, this gives them some indicators to go chase  down. This also will help some of your first tier analysts in if you're not mature enough to  create some of those pretty advanced rule sets in your SIM, this will help simmer some of those  things up to the top and help correlate some of those log entries that, you know, don't really  quite make sense. Good case in point is, was working with a client. They had an alert come  up that one of their employees, one of their phones was in South America trying to log into  all their accounts. Okay. The analyst tried saying, oh, false positive, this product sucks,  get it out of our environment. Wait a minute. This is a question of risk, all right? Is  that employee actually in South America? Well, call HR. Yes, she's on vacation. Okay, good.  Second question, why did she take her work phone with her and why is she trying to log in?  And does she still have her phone with her because in that particular country, it could  have been stolen and quite likely could have been. So is that question of risk when you're  looking at these tool sets, it might be less risky for your organization but potentially  not for others. And that's where we need to look at for these tool sets of it just helping  giving you indicators of rabbit holes that go down into to make that assessment. It can't  get judgment call on risk for you. So with that UBA, UBA, let's add the MLAI. And this  is getting thrown on a lot as well, right? Is it really machine learning? Is it really  artificial intelligence? Or is it that startup that a friend of mine worked at that they  called MLAI by just counting how many times an end user would log in? They didn't do really  any of the potential math after that. So definitely dig into when you're talking with these vendors  to cut through the bullshit, cut through the marketing buzz. Similar with that potential  perception of false positives, similar to UBA, UBA, what's that question of risk in those  alerts that it's providing you? Because when you have those things alert, it's not like  an old IDS alert anymore where this is bad, this packet was bad. These tool sets are trying  to correlate all these different things together to go this is kind of fuzzy. This might be  bad. So it's a question of how we're going to assess risk. So let's take that MLAI and  look at all the network activity from your span port or your packet broker, right? And  then look at that endpoint and look at that traffic to figure out how bad it is. Be very  cautious with some of the vendors in this space. Some of you folks, I'm sure, have already  gone through the sales pitch with a certain company that only hires very pretty dude bros  right out of college that give you the hard sell that have completely drank the Kool-Aid  and if they get any inkling that they are not making a sale, they completely flip, rip  the gear out of your data center and they will never talk to you again. Even if you see them  on a conference show floor, they will turn the other way because you didn't drink their  Kool-Aid. What does that say about their product, their sale cycle and what they can really do  with that is their marketing approach and sales approach. The thing about, I have been able  to use in the non-detection space is very similar of looking at this traffic over a long  term to try to figure out what's really bad. What is those protocols that we're not expecting?  Why are these two DNS servers talking to each other over open VPN on our network? Are the  tool sets we're using, our IDS, our IPS, the ML AI, all these other tools, are they working  as they're intended and supposed to? How do we take the scientific method to prove or  disprove that that IDS is feeding into our SIM and we're getting the right alerts and we're  taking the right automation tools, measures to stop the badness, right? So there's some  companies out there that are doing this threat simulation platforms is what I'm calling it,  right? Basically, you establish some fake endpoints and some systems internally, there's a similar  system outside your network and they run attacks in between those pockets and does your tool  sets adequately detect through the kill chain as they're simulating them what do you need to see  to find the badness when it's the real bad? Like anything else? There's some thud, be  cautious, right? How are they integrated? Is it really a separate platform or is it a function  of some existing tools you already have internally just haven't leveraged yet? And then  deception, which again, just like some of the automation things we have done in the 10, 15  some odd years ago and there's some stuff we've done on our own in regards to deception and  sending up honeypots and fake documents and fake email addresses and that sort of thing.  Here now you have a vendor that's providing a platform to do that for you and monitor for  the badness and activity. Again, like everything else we've been talking about, there's a whole  lot of thud out there. What would be useful to you and how mature you are as an organization?  Can you do this yourself? Can you deploy it out yourself or can you leverage this vendor to  push it out and maintain it in an easier way with less resources? So how are we doing?  How do we measure ourselves? How do we say that we need to go in this direction or that  direction, right? There's those optimal characteristics and capabilities like we mentioned  earlier. You can group it in different areas. There's that organizational model and how we're  structured. Do we have the authority to do what we need to do? Do we have the capabilities,  et cetera? Do we have the right process procedures to implement all this? How do we get there?  How do we have that capability? Internally, we can run with ENSA, the C-Cert self-assessment.  The first GCS also has a maturity survey. Certainly leveraging the SANS critical controls.  We have the NIST ISOEEC. So as part of our security program, as part of these process procedures  that we're putting together, we should also put in cross checks for us to check if we're  doing the right things. Are we feeding the right stuff? And on a periodic basis, depending  on your organization and your threat profile, finding somebody to bring from an external  agency to make that assessment and make recommendations on how you should improve. Hopefully with  an organization you already have a report with that already know you. Taking that measurements  and the model of, hey, here's where you at and from our conversations and your threat  profile and where you should be as an organization, here's where you should be and you can put  together a roadmap of how do you get there. And then you take that roadmap to your director  of security and your CISO and they go and fight for the budget and headcount, et cetera.  And here's our plan on how we're going to mature as an organization. So where do we go from  that? We have our log processors, we have our dirty sock, we have the pen tester come  in every so often. What's next? Through the ages, we've evolved but we certainly have not  matured as an entity, as an organization, as a profession. So as security professionals,  as hackers, what can we do to help with that? We need to ensure the basics are complete.  Back to those triangles, do we have that foundation to build from or have we jumped ahead? Be  willing to hire, be willing to invest in those hires? Be wise to the hiring in your local area  to your needs and capabilities and what's available in your local market because if you're trying  to go, oh, we're going to deploy this tool but there's nobody available in your local market  at an affordable level for your organization to run it and maintain it, does that make  economic sense for your organization? Does that make proper sense in regards to risk for  your organization? And is it good enough or are we trying for that perfect solution?  So many times we're having conversations, not just what last week, week before, oh,  Reddit got pooned because they were using SMS two-factor and SMS two-factor sucks and  nobody should be using it and blah, blah, blah, blah, blah. Well, for them, they made a risk  call and they said, hey, it's good enough. Certainly they didn't probably have, potentially  have the right monitoring pieces in there to go, are these admins getting targeted? If  it's good enough, have that monitor there to be able to detect and see, is it not good  enough anymore and we need to add another mitigation, other compensating controls?  So some takeaways from me rambling for the last few minutes. Be careful. Look past that  buzzword bingo and the marketing. Use that real data to try to determine what's useful  for your environment, your risk posture. What's that size and scope for your organization?  When do you need to step aside and go, hey, I'm in over my head, when do I need to bring  somebody from external help us with this? Certainly with that, hire to your realistic  needs and be willing to invest in them. If you can't find somebody to hire for this,  why should you have that position open for six months? Hire somebody now, you can train  them up in that six months, right? A couple of different references, I've already mentioned  a couple of them. Quite actually all of them it looks like. Thank you for joining me, Matt,  John, and so first, the other organizations, thank to all of us that have come before us,  that have worked in SOCs and have helped us evolve to where we are today. And thank you  all that are going to take that next step and help us evolve to the SOC 2020. Any questions?  Ah, yes, so if you have any questions, let's meet outside in the hallway next to the bar.  Thank you very much all. Have a good day. Enjoy DEF CON.
DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_0_299.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_300_827.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_828_1631.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_1632_1991.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_1992_2699.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_2700_3011.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_3012_3071.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_3072_3215.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_3216_3347.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_3348_4415.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_4416_4547.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_4548_4811.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_4812_5075.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_5076_5123.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_5124_5915.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_5916_7223.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_7224_9143.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_9144_11123.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_11124_11891.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_11892_12503.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_12504_13691.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_13692_14723.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_14724_16403.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_16404_16655.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_16656_17435.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_17436_18191.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_18192_18971.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_18972_20891.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_20892_21695.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_21696_22319.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_22320_22919.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_22920_23579.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_23580_23927.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_23928_25379.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_25380_25427.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_25428_25715.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_25716_26387.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_26388_27695.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_27696_28163.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_28164_28583.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_28584_29123.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_29124_30563.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_30564_32783.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_32784_34259.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_34260_35783.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_35784_36611.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_36612_37031.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_37032_37091.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_37092_37247.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_37248_38447.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_38448_40415.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_40416_41471.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_41472_41579.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_41580_42911.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_42912_44555.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_44556_47783.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_47784_48995.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_48996_51683.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_51684_53519.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_53520_54671.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_54672_54887.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_54888_55475.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_55476_56555.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_56556_57167.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_57168_57227.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_57228_57503.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_57504_58943.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_58944_60059.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_60060_61199.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_61200_61355.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_61356_61787.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_61788_62327.png DEF_CON_26_BLUE_TEAM_VILLAGE_-_IrishMASMS_-_Evolving_Security_Operations_to_the_Year_2020_m-FcPVJ3x-_62328_62363.png
Hello, ladies and gentlemen, boys and girls. How are we doing today? Wow, we're still hung over, still, certainly not still drunk. Not at all. Let's start out with the disclaimer. This was not reviewed by my current, past or future employers. They have no insight into what I have been talking about this, nor that I was intending to. So don't hold them accountable, please. Basically, I speak no one. Nobody speaks for me. Various other legalese in there. So, who am I? Yeah, I'll drop a couple F bombs here and there. I apologize. But it's nice to be able to give this talk because so many years ago I actually started out as a SOC analyst, Inner Security Operations Center here in Las Vegas itself, worked there for many years, transferred to work being the technical lead for a incident response team at a large research laboratory and been downhill ever since. So this is going to talk about why is this important? Because in my current day job, I get to talk with vars and vendors and they're talking about evolving to the SOC of 2020 and this tool will solve all your problems and there's a hiring shortage, air quotes, et cetera, et cetera. And I just keep throwing up the bullshit flags. So, let's talk about this. Let's talk about how we've gotten into this situation. What can we really do to improve and what things should we be working on? And when those vendors come in going, hey, this is a great tool. You should look at it. You should buy it. You know how to see through that. So what is security operations? Right? What are we looking at here? What is the real meaning of that first? Let's set the stage for what we should be doing and then how we should be evolving to. I look at security operations as being that where the rubber hits the road for your organization, where your policies, procedures, what you're looking at is all those tool sets that you have bought, those operating systems that you're running, those applications, databases, et cetera, what they're feeding into, what are you looking for to find the badness? What are you looking for to find when Tim's been sending those harassing text messages to your secretary or surfing for porn or down a landing mount, et cetera? Not that I've ever done that. This is where it all feeds together, right? If you're not getting the right logs, if you're not getting the right information, how do you get that visibility? Normally we don't, right? That feeds into when we find something how we're going to do incident response with it, right? Oh, this is involving certain person. Now we need to do a forensic investigation in regards to what they've been up to. That should also feed into our security awareness program with our employees. And here's the things that we're seeing. Here's that phishing attack that our CEO got yesterday about, hey, I need to change my W2 deposit. From, yes, from some likehost.com account because I'm locked out of my domain account, right? Security operations and computer network defense. We can call it a SOC, we can call it a CSER, we can call it an S-SER, throw whatever three or four letter acronyms you want in there. Fundamentally is this is the group of folks of individuals or one individual in some places, right? That's looking for the badness and trying to put the fire out and then figuring out what we can do to make this better so it isn't as painful the next time. Though we sometimes forget there's three key pieces of this. People process technology. What do we tend to focus on because we're tech hacker geek nerds and we don't like talking to people and we hate writing down process. As we have, some of us have evolved and moved up the food chain as we're into sucky management positions that we don't get to do all the cool technical stuff, we need to focus on all three. How do we leverage each of the pieces? There is no silver bullet, right? No matter how often our CISO is looking for that, right? Or no, we don't have a CISO, let's talk about the CTL. There are certain necessary capabilities that each organization should have. Two varying degrees of how big they are, what the resources they have, what their threat profile look like, what critical infrastructure sector they're in, et cetera. Looking at the events and what they're happening, trying to correlate those to get some quantifying idea of what those events are looking like, right? Leveraging that threat intel, leveraging that indicators, putting those together to get an understanding of what your threat profile looks like and when the shit hits the fan, incident response. Put the fire out, clean it up, what can we do to make it better and try to prevent it from happening the next time, right? If it's more in depth, do we need to do digital forensics on it, right? Is this some commodity ransomware we've dealt with a million times or is this something new targeting us? Is this something that we can do some analysis on and share with the rest of the threat intel and incident response communities? And then just supporting the tool sets, the things that we have. Hey, Tim, how come we haven't gotten a report from the DLP system for two weeks? Oh, yeah, because we're not doing a performance monitoring yet and the drive filled up. DERP. Those audit and compliance controls, why should we sit in a weekly meeting going through spreadsheets? Why can't we leverage the technology and tools to do these cross checks for us, right? The intent of audit and compliance was, okay, kids, this is, you have to be this tall to ride this ride because companies didn't want to do security. So let's set something up as bare minimum to do. Well, with the tool sets we have, what things can we do to have this do it for us? And then certainly some insider threat detection, how many people are stealing source code, how many sales people are stealing the entire customer database for the next company they go to? Not that that ever happens, right? Key piece, vulnerability management. What phones do you have trying to correlate that with which ones the bad guys are trying to target and what things do we need to get the network server, desktop folks to fix priority, right? And then outreach. As all we're collecting all this stuff, as we have this ability and understanding, how do we communicate that in a way that the business folks understand it, the pointy head managers understand it, the C levels and even sometimes the board and for some of us, even to our customers, right? Because all those capabilities, because Matt did a great job at putting another pyramid together, how do we build from that? How do we need those different pieces in getting that understanding, right? And if you're looking at the SANS top 20, if you're looking at the NIST and all those other standards out there, this is pretty along the same lines, it just gives you better, pretty pictures and another good way to display it to the pointy head manager that you have to try to explain it to. But let's try to group those together and Matt did a great way of doing this, right? So when we're building and trying to mature our security operations programs, this gives us some guidelines and some tool sets to help evangelize this to the folks that we need to get those resources, whether it be money for tools, cooperation with other teams, head count, et cetera. And Wendy did this great one. I really like this. She just tweeted us out about two weeks ago. I'm like, oh yeah, this is going into talk. Because we all need some kind of pyramid, right? But yeah, fundamentally knowing what you have and what is it doing. How many times I sat in that sock not too many miles away from here and I'm dealing with first and second tier sock analysts are like, oh yeah, that's normal. And I started digging into it anyway and ended up being one of the biggest incidents for that quarter. Controlling those changes, understanding those threats and who's tacking them and using the products effectively and also monitoring their performance. Because there's that capability chasm and here John Lambert did a really good way of identifying those different areas that a lot of security operations centers or security operations capabilities fall into problem-wise. Not having executive support, falling into various traps. You don't have the right data or you have too much of it. Basically how do we as teams stay poor? How do we keep running around trying to put out fires instead of taking a couple hours every Friday afternoon and doing something more strategic to help us in the long run? So there's different types of security operations, right? First one, bare bones beginning, ad hoc when it hits the fan, when there's a certain individual or two from a certain three-lighter agency that's packing a weapon and knocks on your door going, we need to have a conversation, right? Not that I've ever experienced that. Yeah, the organizations that have evolved to a point of hey, we might have one or two security people within IT that do this. We actually have a full-fledged C-Cert whether it's dedicated or assigned people to full-fledged security operations centers. In this table here, I'll outline the organizational model, kind of the size of the organization and we'll touch on how to quantify that in a moment. And then geographical scope. And this is directly from the MITRE 10 strategies of World Class Cyber Security Operations Center. I will reference this book a couple different times even though I don't agree with everything it says, it has some really good information in it. So let's talk about that evolution. How have we started? How did we evolve? How are we still hunched over on that crappy S desk? Well, we started as cavemen, right? We might not be talking about Wesley Dale quite yet, but we're certainly looking in that direction, right? This is that ad hoc when it hits a fan. This is that IT guy that goes, oh, I don't need blogs. This is not knowing, but we're starting, right? We're trying to figure out the wheel. We're trying to figure out how to get fire. And then we kick out the person from FireEye. Because, you know, the IT guys, this is so easy a caveman can do it, right? But, you know, Irish, there's no evil bit on a packet, so we just let all of it through, right? This is the organization that doesn't know what they don't know, right? Meanwhile, all their data is being ex-filled. And some foreign countries making product based on their IP that, you know, they didn't have to do any R&D for other than figure out how to compromise your network. So let's go to the Middle Ages, right? Let's go to the somewhat organized ad hoc as needed when we need to go conquer Jerusalem or need to go conquer a local nation state. We need to go put out the fire and then we disband when we don't need it anymore, right? So when it does hit the fan, we might have some process and procedures, we might have some things going on, but it's all old from the last time we did it. That distributed co-managed. I look at this as being the explorers here in the United States, right? Let's go explore what's out there. We have an organized team doing it, but we still need some local help. Do we need an MSSP? Do we need somebody on retainer to do instant response, do forensics? What did, you know, thinking of Lewis and Clark and what things they needed to do to make their progress? Explorers are so meaningful in documenting all the birds they found, all of the nature that they found, right? How do we do that within our security operations center where we kind of hit this level, right? Have that ticketing system and document every little piece. And then leveraging that as a resource that we're searching for the next time something comes up. Oh, hey, we've had this ticket, four or five tickets with this same source IP because it's been beaten on our network. Leveraging the tools and the data that we're pulling together so that our exploration, our voyage is that much more meaningful. Then we have some sort of C-cert capability, right? We're in that organized way and trained across the desert. Hopefully we don't get stuck in truckie for the winter, right? But it's a group of individuals working together in some sort of organized way, right? For a common goal. Hopefully you don't kill too many of the natives while doing it. But you also have those security operations centers that are like the gold rush, right? They keep running to every place they hear that there's gold and they dig and dig and what do they find? And they completely run out all the land that's there that we completely devastated looking for that one nugget. But then we can do security operations at scale. Let's get a tier one, tier two, tier three, let's get a bunch of tier one folks doing the same process manually over and over and over again, just like sewing. Just like the seamstresses that died in that fire. Because they're going to burn out, they're going to say F this I'm going somewhere else. So let's try to put some automation into it, right? What's those things that we can do that we do every day? How can we automatically create a ticket and pull in a threat intel from our feeds and from internal and connect all the tickets that have that same IP address in it? Let's try to make it easier for ourselves, easier to make those determinations on how risky is it and how much do we have to take care of this, right? So I mentioned the MITRE book and the process, certainly determining optimal manning levels I worked with multiple clients in regards to how do we figure this out? Well, certainly there's no firm answer, there's no real equation of figure this out, but I have put this together based on MITRE, based on my own experiences as an analyst and working with multiple clients and working in multiple places for many years and how can we get at least into the ballpark of how many people we need and how do we leverage them? This also you need to look at how you as a company are growing so you can project how you need to grow your team, right? So first thing, just like the pyramids, just like the sans critical controls, you have to know number of what you have in your organization and what you're trying to monitor, at least the ballpark, right? Including the number of sensors you think you're going to need to use, how many cloud providers, domains, et cetera, internal, external, because you're going to be getting the logs from that. Then what's your threat? Who's targeting you and how? What compliance and regulatory obligations you have? Which are geographical soap, et cetera. Add all that up. If you have less than a thousand, then you probably don't need a 24-7, but depending on your threats and your risk plan, you're probably going to need something, all right? And if you have many thousands, then you definitely need something. If you don't, I really like to know how you're dealing right now. And I'll buy you some alcohol to deal with it, right? So specifically for analysts, specifically folks that's going through all the data, good ballpark number is one FTE per 50 to 75 devices, IPs, domains, et cetera. This certainly depends on the number of logs you're getting from those. And the intent is to handle all those critical or literature getting after turning your tools as well as process and procedures. So, optimal situation, right? Number for ratio from tier one to tier two for basically every two tier one analysts between one and four tier two. And if you're doing FTE for a 24 by 7 security operation center, it's more like five to one or three to one in a sock because you need shift operations, you need scheduling, you need floaters for when people are out sick or say, F this, I'm leaving the company, I got a better job, et cetera. Or, hey, the half the sock is going to DEF CON. Not that that ever happens, right? So for engineers, you know, as the folks that we have that are doing the sensor tuning or soon the sensor maintenance, basically the stuff, all that tool sets, platforms that we're getting the data from, about half FTE, depending on the deployment and also the tool sets you're using, right? If you're using certain SIMs that start with an A, you're probably going to need a few more, right? Other ones, potentially less just because of what's involved in keeping them running. If you have your system administrators broken out from your engineers, then your typical ratio of one to one or one or two to sysadmin engineer types. Depends on your organization, depends on how mature you are, depending on your separation of duties. And to your fellow sysadmin out there, a belated sysadmin day. So placement in your organization for your security program, because this is going to directly relate, right? Do you have the wherewithal? Do you have the resources? Does the individual that you report to know the balance of the confidentiality, risk, et cetera? Right? Or is it going to be like, no, don't turn that off. We need that for such a business thing. We don't care. So you have the conflict of interest. Yeah, we might have a CISO, but who does that CISO report to? Does he report or she report directly to the CTO? You still have that conflict of interest, right? Do you actually mature enough or as big enough of an organization that you have a chief risk officer that looks all of this? And some organizations, hey, they might have security in the sock underneath legal, which is awesome, but are they tech savvy enough to understand what you're talking about when you come and tell them that your website has a cross-site scripting attack going on? So common conversation is, do you have something internal or do you need to contract it all out? Give it to an MSSP. Certainly, it could be useful to augment your internal capabilities for your smaller, less mature organizations, but, you know, MSSP services are commoditized. There's those common issues you deal with when you talk with certain organizations of, how come you don't do this? Oh, well, you didn't buy that package. You didn't look through the terms and conditions, TNC, so we don't do that. So certainly, if you're in the market for getting an MSSP, understand what they're going to be providing you. There are the big ones. There's the small ones and then there's the one MSSP's in the middle. I kind of, I try to, I call those the Poteek MSSP's. Those tend to fit most clients, most folks I work with a little bit better in regards to their capabilities and being able to pay attention to you and your needs versus just shooting alerts via email or to your ticketing system. Key point is that last bullet there. You, I need to manage that relationship and nurture that relationship. If you're not talking with them on a regular basis, they're not going to care. If you don't hold them accountable, they're certainly not going to care because they're just collecting your money. Oh, well, I don't need to buy an MSSP. I don't need to sign it up. I'll just do it myself. How well has that gone? Certainly, if you're the Facebook, Google's of the world and you have a shit ton of developers and that's all they love to do, go ahead. I've seen some of their projects that they use internally that they started 10 years ago and then the guy left and nobody's maintained it since. So what's your software development site to look like? What happens when the folks that maintain that get hit by a bus or leave the company or retire? I got my stock. I'm out. Fitts. What is it about those open source tools that are already out there? What is it about Greylog that you don't like? Why that you can't use it in your environment that you're going to need to create your known for brand new? Why can't we just fix what you don't like and improve the open source project that's out there? Tied to that is also how do we staff our stock? So that MITRE book heavily encouraged that we only needed developers in the stock. I call bullshit. Sorry. It's that diversity piece of getting different individuals that have experiences in different areas within information technology and information security that in the middle of an incident go, hey, I was a sysadmin on a sun system and this is what it does. This is how we can go and find it. This is how we can fight the badness, right? No slam against developers but most developers don't run their own systems, right? They don't have that expertise. It's having different individuals and different perspectives in to deal with that fire fight sooner and faster and also come up with better ideas on how to prevent this from happening in the future based on the background experience that we have. We need real and valuable metrics. This is how we use to justify that security operations center we have. This is the baseline that we use to figure out how are we going to evolve to the stock of 2020. If we don't know how we're doing now, if we're not monitoring the performance on that DOP system to realize that the drive filled up, how are we supposed to know how we need to evolve to? So I did a Google search. This is the first image that came up. In evolving to the... Thank you marketing. All right. Because that's a good portion of it, right? There's a whole lot of companies out there trying to pitch this stuff. What's really going to make a difference? So the first thing I've seen is having some robust monitoring detection, right? But we have span ports. All right. Wait a minute. Span ports were never intended to be a permanent fixture. They were only supposed to be a diagnostic tool, right? They're only a sample of the traffic and the first to die when that device gets overloaded. And when I'm in the middle, too many times I've been in the middle incident and I'm trying to figure out why our devices are not triggering on the badness because that span port is fucking dying on me. Basically we're missing activity. If we can't see it, we can't fight it, we can't put the fire out. When does this normally become a problem? When the organization runs out of span ports. A little too late, right? But, all right. So the packet brokers, there's a couple other terms out there. Basically it's a middleman for your network traffic. So instead of putting multiple devices in your network in line, you put the packet broker there and then make a copy of all that traffic and then filter it however you want to your monitoring detection tools. So it makes, it's a force multiplier for the tool sets that's monitoring your network, not just for security but also for performance monitoring. So this is a tag team effort with your network folks. Some of these also give you IP fix DNS, other data out of it. This also lets you use less tech, less platforms. So they're in a prior life, particular organization got a quote for, I think it was 35 boxes this company wanted to install in this environment. You can imagine the price tag on that. Brought in a packet broker, we were able to cut that in half. That quote went down significantly. That vendor was not happy-ish, but at least they were happy to get the opportunity to deal. And we were able to help that organization out by getting better coverage for their environment and into the tool set they wanted to deploy. Not all packet brokers are created equal. There's certainly a ton of marketing, a ton of foot out there. The UX UI experience with some of these are horrible. Some abilities you can only access through the command line, not through the UI. Also, check the security posture of these devices themselves. Have they updated their firmware lately? What's their software development life cycle? Certainly do a third party risk assessment before you buy these platforms. Now internally, who's going to maintain these? Is this your networking team, is this a security team? Those are some questions that come up? And certainly the cost of them. Quick picture on how those are normally deployed. Let's talk about data integration, right? Getting that data into your monitoring detection platforms. Hey, I already have X platform. It has the data feeds coming into it. Great. How are you using those feeds in your SIM, in your other platforms? You able to leverage those as a pipeline, as it stove pipes into each of these tools and you're not sharing them back and forth? Is it feeding your ticketing system for when hits happen and a ticket gets open? Need that situation awareness? Provides better mitigation decisions? Basing it on real data, not that gut feel that we tend to do with insecurity. These tools are not intended to be myopic. Why do we keep trying to deploy them as such? And then from that, let's automate and orchestrate using the real data. Address that low hanging fruits or you're less of a target. Hey. I just call it automated orchestration. Gartner likes to make big acronym words or whatever. This is no different than the Python, other scripts I did 10, 15 years ago. Right? But now we have a platform. Okay. Awesome. Because it's a relatively easy deployment. I can have a tier one, tier two analysts go in and do some clicks and create or update a playbook. And the next place we go to or the new person that comes into the team is already familiar with it. We can't say that about that Perlin Python script that I wrote on a Sunday night after a couple of Bourbon Bear laid doubts, right? When you're looking at these, be cautious. There's been plenty of acquisition. There's a ton of buzzword bingo out there. Do a POC, do a POV, look at the acquisition space and who's looking at it and what's going on. Oh, yeah. How many times did we see this on the black hat floor yesterday, day before? Right? The premise is to take all that information, your logs, and come up with that user account or that entity account and try to identify that this is risky or there's a major threat. All right? Yes. It is useful in ways of if you're mature enough and you have a threat hunting or you have a wolf team that go digging, this gives them some indicators to go chase down. This also will help some of your first tier analysts in if you're not mature enough to create some of those pretty advanced rule sets in your SIM, this will help simmer some of those things up to the top and help correlate some of those log entries that, you know, don't really quite make sense. Good case in point is, was working with a client. They had an alert come up that one of their employees, one of their phones was in South America trying to log into all their accounts. Okay. The analyst tried saying, oh, false positive, this product sucks, get it out of our environment. Wait a minute. This is a question of risk, all right? Is that employee actually in South America? Well, call HR. Yes, she's on vacation. Okay, good. Second question, why did she take her work phone with her and why is she trying to log in? And does she still have her phone with her because in that particular country, it could have been stolen and quite likely could have been. So is that question of risk when you're looking at these tool sets, it might be less risky for your organization but potentially not for others. And that's where we need to look at for these tool sets of it just helping giving you indicators of rabbit holes that go down into to make that assessment. It can't get judgment call on risk for you. So with that UBA, UBA, let's add the MLAI. And this is getting thrown on a lot as well, right? Is it really machine learning? Is it really artificial intelligence? Or is it that startup that a friend of mine worked at that they called MLAI by just counting how many times an end user would log in? They didn't do really any of the potential math after that. So definitely dig into when you're talking with these vendors to cut through the bullshit, cut through the marketing buzz. Similar with that potential perception of false positives, similar to UBA, UBA, what's that question of risk in those alerts that it's providing you? Because when you have those things alert, it's not like an old IDS alert anymore where this is bad, this packet was bad. These tool sets are trying to correlate all these different things together to go this is kind of fuzzy. This might be bad. So it's a question of how we're going to assess risk. So let's take that MLAI and look at all the network activity from your span port or your packet broker, right? And then look at that endpoint and look at that traffic to figure out how bad it is. Be very cautious with some of the vendors in this space. Some of you folks, I'm sure, have already gone through the sales pitch with a certain company that only hires very pretty dude bros right out of college that give you the hard sell that have completely drank the Kool-Aid and if they get any inkling that they are not making a sale, they completely flip, rip the gear out of your data center and they will never talk to you again. Even if you see them on a conference show floor, they will turn the other way because you didn't drink their Kool-Aid. What does that say about their product, their sale cycle and what they can really do with that is their marketing approach and sales approach. The thing about, I have been able to use in the non-detection space is very similar of looking at this traffic over a long term to try to figure out what's really bad. What is those protocols that we're not expecting? Why are these two DNS servers talking to each other over open VPN on our network? Are the tool sets we're using, our IDS, our IPS, the ML AI, all these other tools, are they working as they're intended and supposed to? How do we take the scientific method to prove or disprove that that IDS is feeding into our SIM and we're getting the right alerts and we're taking the right automation tools, measures to stop the badness, right? So there's some companies out there that are doing this threat simulation platforms is what I'm calling it, right? Basically, you establish some fake endpoints and some systems internally, there's a similar system outside your network and they run attacks in between those pockets and does your tool sets adequately detect through the kill chain as they're simulating them what do you need to see to find the badness when it's the real bad? Like anything else? There's some thud, be cautious, right? How are they integrated? Is it really a separate platform or is it a function of some existing tools you already have internally just haven't leveraged yet? And then deception, which again, just like some of the automation things we have done in the 10, 15 some odd years ago and there's some stuff we've done on our own in regards to deception and sending up honeypots and fake documents and fake email addresses and that sort of thing. Here now you have a vendor that's providing a platform to do that for you and monitor for the badness and activity. Again, like everything else we've been talking about, there's a whole lot of thud out there. What would be useful to you and how mature you are as an organization? Can you do this yourself? Can you deploy it out yourself or can you leverage this vendor to push it out and maintain it in an easier way with less resources? So how are we doing? How do we measure ourselves? How do we say that we need to go in this direction or that direction, right? There's those optimal characteristics and capabilities like we mentioned earlier. You can group it in different areas. There's that organizational model and how we're structured. Do we have the authority to do what we need to do? Do we have the capabilities, et cetera? Do we have the right process procedures to implement all this? How do we get there? How do we have that capability? Internally, we can run with ENSA, the C-Cert self-assessment. The first GCS also has a maturity survey. Certainly leveraging the SANS critical controls. We have the NIST ISOEEC. So as part of our security program, as part of these process procedures that we're putting together, we should also put in cross checks for us to check if we're doing the right things. Are we feeding the right stuff? And on a periodic basis, depending on your organization and your threat profile, finding somebody to bring from an external agency to make that assessment and make recommendations on how you should improve. Hopefully with an organization you already have a report with that already know you. Taking that measurements and the model of, hey, here's where you at and from our conversations and your threat profile and where you should be as an organization, here's where you should be and you can put together a roadmap of how do you get there. And then you take that roadmap to your director of security and your CISO and they go and fight for the budget and headcount, et cetera. And here's our plan on how we're going to mature as an organization. So where do we go from that? We have our log processors, we have our dirty sock, we have the pen tester come in every so often. What's next? Through the ages, we've evolved but we certainly have not matured as an entity, as an organization, as a profession. So as security professionals, as hackers, what can we do to help with that? We need to ensure the basics are complete. Back to those triangles, do we have that foundation to build from or have we jumped ahead? Be willing to hire, be willing to invest in those hires? Be wise to the hiring in your local area to your needs and capabilities and what's available in your local market because if you're trying to go, oh, we're going to deploy this tool but there's nobody available in your local market at an affordable level for your organization to run it and maintain it, does that make economic sense for your organization? Does that make proper sense in regards to risk for your organization? And is it good enough or are we trying for that perfect solution? So many times we're having conversations, not just what last week, week before, oh, Reddit got pooned because they were using SMS two-factor and SMS two-factor sucks and nobody should be using it and blah, blah, blah, blah, blah. Well, for them, they made a risk call and they said, hey, it's good enough. Certainly they didn't probably have, potentially have the right monitoring pieces in there to go, are these admins getting targeted? If it's good enough, have that monitor there to be able to detect and see, is it not good enough anymore and we need to add another mitigation, other compensating controls? So some takeaways from me rambling for the last few minutes. Be careful. Look past that buzzword bingo and the marketing. Use that real data to try to determine what's useful for your environment, your risk posture. What's that size and scope for your organization? When do you need to step aside and go, hey, I'm in over my head, when do I need to bring somebody from external help us with this? Certainly with that, hire to your realistic needs and be willing to invest in them. If you can't find somebody to hire for this, why should you have that position open for six months? Hire somebody now, you can train them up in that six months, right? A couple of different references, I've already mentioned a couple of them. Quite actually all of them it looks like. Thank you for joining me, Matt, John, and so first, the other organizations, thank to all of us that have come before us, that have worked in SOCs and have helped us evolve to where we are today. And thank you all that are going to take that next step and help us evolve to the SOC 2020. Any questions? Ah, yes, so if you have any questions, let's meet outside in the hallway next to the bar. Thank you very much all. Have a good day. Enjoy DEF CON.
SOC of 2020 and this tool will solve all your So what is security operations? Right? What are being that where the rubber hits the road for might have some process and procedures, we might evolving to the... Thank you marketing. All in your network in line, you put the packet your monitoring detection tools. So it makes, these, be cautious. There's been plenty of acquisition. There's a ton of buzzword bingo out really machine learning? Is it really artificial what do you need to see to find the badness when there. What would be useful to you and how mature procedures to implement all this? How do we get can put together a roadmap of how do you get