Strategic refactoring using static analysis
Bjartnes, Bjørn Einar
Decoupling legacy code bases is not only hard, but often we don’t even have a clear idea of the current couplings that exist in our system. Without a clear overview of the current state we can’t make sound decisions on what we should do to improve. This fact leads to refactorings that are subjective, these refactorings might not yield value except for some subjective measure of “less ugly”.Refactoring with a purpose. I would argue reading code is like walking around in the forrest, enjoying the small details - the scale is 1:1. Static analysis can draw maps from a higher perspective, which is critical for strategic thinking. By using static analysis we can make a map of the current couplings in our system, make a plan for refactorings based on that map, and verify that we have obtained the goals we set out to accomplish by re-running the analysis. This can make refactoring easier, cheaper and yield code-bases that can be proven to be more modular. Using examples from NRK TV (tv.nrk.no and psapi.nrk.no) I will demonstrate, using NDepend, how to analyze unwanted couplings and make rules to make sure they never again will appear in your code base.
 Okay, good morning everyone. Well, I've left the slide up for a while so you've probably seen the topic, strategic refactoring using static analysis. So first to MI, I'm Björn Einlad. I work at NRK. That's the National Norwegian Broadcasting in Norway. So, well, the revisions of you know what this is. For the foreign people, it's like the BBC of Norway. So I like to use this hashtag when I talk about NRK TV. You can if you want to. Tweet. So the agenda for today. I want to try to keep it loose. So if you have any questions, like feel free to interrupt me. Just throw questions at me. Just shout out so I can see it. I can't see you very well because of the lights. Just scream loud. But I'll try to go through some simple definition of what I mean by strategy, what I mean when I say static analysis in this context. I'll be showing you the domain we're working at. Then I'll go through End-upend, which is the tool that I'm using when I'm doing static analysis. Then we'll look a little bit of refactoring at the assembly project level. And I'll go a little bit more into detail on End-upend. And we'll go and look at namespaces and some more complex metrics and stuff like that. And everything I'll be presenting today is like Israel Life Code. Everything is just stuff we have in production. And there's no secrets. I'll, you know, happy to show everything we have here. So just ask questions and I'll dig into it. So first, some definitions. Well, refactoring, it's basically rewriting the code without changing any of the functional specifications. That's the way I see it to improve some other metric or some other aspect of the code. When I say strategic in this context, I just mean strategic at the domain level, which I'll go into a little bit, or strategic that we're talking about refactoring at assembly, dependency, namespace level, not necessarily, you know, making one method cleaner or nicer by some other metric. And when I say static code analysis, well, that's a big topic. I'm no expert in static analysis in general at all. I'm not going to claim that. In the.NET space, there's, you know, everything from StyleCop to Resharper to FXCop and the code analysis tools and the end-depend, which I know of that's doing static analysis. So it's everything that's just running and analyzing your code without actually executing the code. So StyleCop is, you know, it's very at the syntactic level at your code, how do you name your methods and so on, you know, lower casing, upper cases, and can't do any of the strategic stuff that we're talking about here. There's FXCop and Resharper is doing a lot of these things as well. I'll show you a little bit. But to me, the role choice of the static analysis tools is end-dependent, and you'll see why when we get into the details. So that's the tool I've chose to use here. And well, you know, we're talking about legacy applications here. For a definition on that, I will go to Karolina's talk in an hour. So, but I guess all the code we have in production, I define it as legacy. I don't know. So I'm a big fan of military strategy and stuff, and, you know, it's sad. It's like when you listen to gangster rap and you feel cool when you refactor stuff. It's very sad. Some of us do this, you know, you feel like a general when you're refactoring and making strategic decisions. I like this metaphor even though I think they're quite sad. But there are some similarities, right, that we're able to work at a higher level of code than sitting in the trenches or sitting in the code base that we're able to lift our perspective to a higher level and make some higher level decisions. So, you know, we want to make some kind of map of our code that are showing assemblies, namespaces, dependencies between them and so on. This is how I think of a lot of the clean code movement that's going around. Not that it's bad to write clean code, but people seem to be like this guy, like, I'm the cleanest fighter. Like, he's in the trenches. His friends there are like rotting corpses, like smelling around them, but this is a claim to be the cleanest. And it's a very, very local optimization to clean yourself when you're in a trench, like, and I feel a lot of the same with clean code. Like, rewriting a method, it's fine, like, do it, but it's not going to achieve any higher level strategic objective for your business. At least that's my claim. So, I think I've narrowed it down a bit to talk about strategic refactoring of our applications and the applications I'll show is NRK TV, NRK radio, and NRK super with NDPen. So, we sort of narrowed it, made it a little bit more concrete. So, I'll just quickly show you the domain we're working on. So, this is NRK TV. So, test programs, you can watch TV programs. There's lots and lots of programs in here. There's a login, so I'm logged in, so I have my content, so I can continue watching stuff that I've seen. There's my programs. There's also, like, this is, we'll write, we have very high focus on privacy. So, like, now, apparently, I've seen frekje fortellingen, which is a Danish series about erotica, so I can, like, I can delete that because I probably don't want people to see that. And there's, like, sex educations. Oh, I probably should delete that as well, right? So, it's personalized. There's some live TV, you know, TV program guides, categories, so that's the TV. Then there's the TV for children, which is aimed at a younger age group. It looks a bit different. I think this is quite interesting. Can any guess why there's, like, Q and X, even though there's no programs in Q and X? Like, why don't we filter it out? And it's like, because these kids are learning the alphabet, so if we take out Q and X, we confuse the kids. So, it's like, it's a very different age group. And it's not only cartoons in here, it's, like, it looks a little bit, like, you know, cartoonish, but there's also some serious stuff about learning, learning alphabets, but also we go into hever topics, that's substance abuse and things like that. So, it's a mix of education and entertainment. And then there's NRK radio, which is a live radio. That's a bit different. You can listen to, you get some notifications of what is currently playing in. These things are a bit different. And then there's the API team that I'm working on, which is the backend. We'll talk about the backend here. There's a, with all the, we're basically all these applications go in fetch their data. This is like a quick overview of the domain. This is our story. Parts of it is in Norwegian. I joined around here, like, September last year, so I can't take credit for any of this. But basically, the idea is that there are lots and lots and lots and lots of clients. The stuff that we looked on now, it's just a web client. So, that's just a web client, but there are smart TVs and there is apps and all kinds of clients that are playing the content from the same API. But we'll be looking mostly at the front and stuff on the web. I just took this picture before I left. This is our team. These are the frontend people here, these four. Aynard is doing a talk later today's here. There's Stola, which is on the backend team. Steinard, which is on the audience is sitting behind here. And, you know, nice little group. There's our operations. There's Ops. He has a cop that says no. And there's the release manager that, like, you know, clicks octopus and decides what goes into production. Again, this is an overview of all the clients. The stuff we'll be looking at now are the desktop, the web stuff and how that relates to the backend. I took this. This is showing how many users we have on the desktop and there's a lot, almost like a million unique users a week. This is just to show that the desktop stuff is really important. And I chose a metric that shows that it's really important. Like, if you look at how many minutes people actually watch content, that's different. And this is just a little bit of background of why we chose to do refactoring of these applications to begin with. And that was during the implementation of this part, which is like direct feed. I just took this yesterday. This is stuff that is live, but it's not necessarily broadcasted on TV. And to implement this feature, basically, we finally took too long. We spent several months getting the MVP of this product out. The reason being that the code base was too complex and too work-with, basically. And we decided, you know, something needs to be cleaned up here. We need to be able to move faster than what we were able to do. And all this, basically, all the stuff that you've seen now with the NRK Super, NRK Radio, NRK TV is the same web application. It's just configured to look differently running at different hosts, which was probably a good idea at some point because you could get very quickly out something that looked quite similar. But then it's grown different over time. Like, you had the login stuff on one page. You had some live view on the radio was coming in and play. And the TV stuff looks completely different. So at some point, stuff that looks very, looks quite similar end up being very different. And another aspect is that you can see here on the children stuff, they watch it mostly on the tablet. They don't use the desktop that much, whereas NRK TV is very heavily used here. So, like, when we want to make changes to our very, very large audience on the desktop side, we always need to think of what happens to this stuff, which isn't as business critical. So we have these strategic decisions as well, where the fact that the these applications intertwine is not too good. Okay, so we need a plan, right? And this was supposed not too long ago. This is Montgomery and the plan for D-Day. And the overall of the plan is just like one very simple page. And I think we need sort of the similar things when we work with complex code bases. We need to have a simple plan when we work with complex stuff. It says like heavy air bombing from a student's light permits until after eight hours. And there's the forward body and there's the main body to follow. And it says down here, the key note of everything to be simplicity. This is a quite heavy, large operation. The other similarity I found is like main lot of the after they found the beaches and landed the main troops, the objectives to cut dependencies as well, like cut railroad, cut tunnels, cut bridges to be able to sustain further operations. After that, I think all metaphors break down, but that's fine. All metaphors should do. So, end the pen. Let's start looking at the tools and see some concrete examples. The thing about end the pen is through what they say here. You get this insight into absolutely didn't have before. There's a breadth and depth of information. But and we can just take a look at how that looks. If we just did, we can build a very small and simple application like a console app. I can, you know, if, if args any will do something, right, will do, oops. I'm not going to write too much code, but I just need, we'll say, you know, high and DC and else, you know, I don't know, something else. Doesn't matter. We're not going to run this code, right? We're just going to analyze it statically. So, we're just going to build it. So, I built the code. We should be a folder here with an application. We're not going to run it. We're just doing static stuff that I can analyze. So, this is end the pen, the full tool and I can, what I can do is I can just say analyze some depend, some assemblies in a folder. I'll take out the visual studio host the exe file and just say analyze this exe file. Do your job, you know, generate your reports and it's true. It's like there's lots of data. It says here that my max and average cyclometric complexity is two, which is, so it makes very sense, like if this do that, else do this. But I already have like critical rules violated. I have actually, I just, I violated six rules already, right? And we can look at what it is. It's not that important, but, but the idea here is that there's a breadth and variety of information and graphs and charts and things we can analyze about the code base. And I just talked about like, what did I say? Like simple. The simplicity is key to it all. Like so, how do we use a complex tool in a simple way? So, the first thing is that we sort of, we make a plan or we say, we get an overview of the assemblies. And the first mission is to, you know, cut the back end from the front and they'll show you in the code after a while and apply some custom metrics when we refactor. I'll go into the, to the details now. But the thing is that you'll be able to come up with a simple plan. So, the way this looks at all the three websites you saw, which is one website today, one code base, has some couplings to the API of historical reasons. So, the first plan is cut those, cut those dependencies. Second plan is cut, take the super part out of the other applications. You can probably as well, they get the radio out and we could do a lot of more things, but that, that's the plan, two things. So, that's a simple plan, right? So, the first thing NDPend can do for us is to generate a map of all the pendent, all the dependencies we have. So, what you see here is that the front end web application, it has some common settings, common logger, common cache. It's fine, right? But there's this arrow here that points to web API models. So, it's using some code from some back end models for some reason. And that has transient dependencies that will pull in the entire back end library. Also, elastic search, which is used in the back end for the indexing. And, you know, so you get this entire thing. So, the first objective is basically cut this line, right? And that's what I mean by strategic is just like, cut that line. I don't care how it's done. Just, you know, and NDPend can also tell me that line, what does that depend on? It still doesn't make a lot of sense to me reading out, like there's a plug profile configure that is using some get serious ID from a plug view model. But I don't really care, like, all I want to do is to give this to someone and say, you know, delete these dependencies. I don't care how you achieve that objective. At that point, we're at the tactical level and I'm not going to go into details on that. Like, you know, you can, that's at a local level. You can refactor it. You can duplicate the code, you know, whatever to get rid of it. I'll show you how we can build this as well, the graphs. And this is just to show you that the same image can be generated from Resharper using the architect overview there. So this is basically the same image with the dependencies between assemblies is just oriented a bit differently. I could just show you how we, how this is done. So right now I'm going to analyze the, this super application which have been split out from the code base. When I analyze it, I can ask and depend, you know, view application assemblies only in a graph and then it will draw this, where did it go? It will show, generate this nice little diagram for me, which is the same we saw. And I can ask it, the way I've generated this graph is just like build a graph of code elements involved in this dependency, for example. So this is a quite, this is quite heavy in this case. So it's a more difficult objective to cut out. But you can get this. So what you see is that even though I'm at this level only caring about cutting these assemblies and you could, there's still like 73 violated rules, 8000 violations, seven critical rules. But the way I use underbund is that I don't, we can't go and start refactoring and cleaning up all that stuff. I'm just using it to achieve some objectives. And I think it's important when we work with a tool like this to not get all hung up in getting, you know, doing everything the tool tells us because then we needed to start a lot earlier than, you know, many, many years into the project. But anyway, what you can see here is that what we've achieved at this point is getting rid of the backend assemblies. In this case, the supersite has been split out and just renamed, duplicated out and, but you can see that there are no backend references. What this is, just that simple thing reduced like the size of the package that we send that we deploy from like 300 something megabytes to 100. Still way too big and we're, I think now we're down at maybe 30. But just this one change, like just have dramatic impacts. Okay, so at this point what we had done is that we had separated the super website out, duplicated all the code, get rid of the backend references, which is all good. But we duplicated 12,000 logical lines of code. We just copy pasted it. We took all the complexity from super and OKTV and we duplicated it, which makes, gives us the opportunity to change one without affecting the other. But there's still lots of dead and legacy code in both of the applications. So the question is how do we get rid of that? The image that I sort of used to explain what we're doing is just basically we had the yarn or string, blue and red string like intertwined into a ball. And how do you separate that out? Do you like try to get the threads out one by one or do you just duplicate them and cut off the blue parts from one end and the red parts from the other? And to sort of, to know that you're actually cutting out thread and not just duplicating it, we're using EnderPand at the build server. I'll just show you how we've set that up. And we basically measure all of that in, hold on. So we set up a nightly build with EnderPand. I can go into details later on how it's all set up. That is giving us these friend charts, which trends, well, this is lines of code, but we have set up with EnderPand like this, which is lines of code in the suprassemblies, lines of code in namespaces, lines of code for view-generated code, lines of code for controllers and lines of code for the managers in this, also for average complexity and max cyclomatic complexity in the different namespaces in the duplicated code. So what you see is that when we duplicated the code, it wasn't that long ago basically. It's like just in time for this talk, first of June, that was merged into master branch and we have like just in the suprassembly 10,000 logical lines of code. That's a lot more in lines of code. This is actually just, you know, your statements and expressions and so on. And at this point, we haven't really achieved any, you know, cleaning up. Then the guys started deleting code. And you see, we have deleted approximately, I would say like, you know, 40% of the code from supr, which was TV and radio stuff, which is not used at all. And we can see by the complexity that we haven't really reduced any complexity. And I think that is because we've just deleted the controllers and the views and the stuff that is really separate, but we haven't started going into refactoring the methods where they could be really simplified inside controllers and so on. So I would say at this point, we should continue into refactoring out more local complexity before we stop. And this is nice because we can go to the managers and say, you know, we need a few more weeks. You know, we've already accomplished this. We've reached, you know, the reduced complexity and the amount of code by half, but we need some more time to clean up some of the other stuff. And they can take this to their managers and tell them what they're doing, you know, to see that we are actually able to prove that we're getting some thing out of that. I think too many times developers go like, oh, the managers don't understand why we need time to refactoring, but it's because we're not able to show any results a lot of the time. We just say that, you know, but it's cleaner now. We've done some major refactoring. It's better now. But I think we should show with some graphs and charts how we've actually improved. So I will show you a little bit of how we go about the setup making this metrics that we're using when we're refactoring. So one of the most powerful things that End up and gives you, you know, in addition to these charts and graphs and trends that you can dig into is like a domain model almost of your code using types, methods, fields, names, business assemblies that you can query with what they call CQL links. We can use link to query your code. So like even I can met a program against the code base to make this metric. So I'll just show you how, like, how do I make a custom metric like that for my code base? We can look at the dashboard again. I should be here. It's a bit like flying a spaceship navigating around here, but it's, if you look, this is one of the graphs I made, right? How do we make these graphs that contain, for example, lines of code for controller use? You know, this is the code required. It's like application namespaces where the name contains controllers, child methods, some lines of code. And this is how you give it a name and this is the unit. So if I want to make another metric, I can just, you know, I just take an existing metric. Normally, I just start with one of End up and metrics, right? I just say, I like to make something like this, but I want to tweak it for my own use. I can make, like, a new metric in here. And I can say, you know, where it contains, I don't know, ASP. And you see, I get the result immediately under here and I can call it lines of code for ASP. This is like the generated code in the ASP namespace. And now I see there's 2,000 lines of code. I can make a new cren chart that I call, you know, NDC. And I can say, I want to have this, I guess that's in, here it just shows up. Stuff, is that name that I gave it here in the comments and the units show up here. And I can add that to my graph and I get a graph. And I can check that in and it will show up in the reports that it builds server. And the level of meta program, it's like super simple. If you know Link and you have any idea of, like, that your application consists of namespace methods and namespaces, basically you can start writing very complex and nice queries. This is how you do cyclomatic complexity. This is how they do IL nesting depth. And you can find all the rules and stuff and you can make your own. For example, you can make, you can look at your dependency graph again. And you can go like, I want to cut this as well, right? This common stuff. And I could go and say, I could, you know, edit a code query, generate a code rule that matches if this dependency exists. And I can make a code rule, check that in, that generates a warning if it exists, check it into the server, get a warning at the server and refactor it out to see that it goes green. So it's almost like test-driven, strategic refactoring. You can do. Okay, but we're still, like, at the assembly level right now, how to cut project references and assembly references. Yeah. Also, I need to give some credit to the guys who actually deleted the code because I haven't touched the code base. I just put up the metrics and so on. I'm on another team and I'm the API team. I'm not touching the front-end code. So these are, to be as a, there are consultants in our team from Foshe and Netlight and they did all the deletions. I should actually show you just how it looks because it looks very simple to delete 2,000 logical lines of code. And then you look at, like, some of the pull requests right here and it's like, remove unused controllers from super. And you have this, oh, 11,000 lines of code. And this is maybe a 1,500 logical lines of code and you're like, you code review this stuff. You know, file change, 200 file change. So this is the stuff where it gets tactical. This is where I move out. Like, I like to be at the strategic level when it's about code reviewing this stuff. It's like, as you see, you know, people are a little bit, aren't you test driven to want to test look after you? And I'm like, yeah, you're deleting your tests as well, right? So you need to really know what you're doing. The almost end, only thing you can do is rely on your integration tests that are testing your UI and your manual testing process to catch any errors that you're doing right here. But I think they had some fun as well. I found I was looking through the pull request yesterday. And some of them had, like, yeah, I think, oh, I got a little bit eager, right? And I think people get a little bit more eager when they get actually these graphs and measurements and they get some feedback from the build server that you're actually doing a good job and we can go to management, you know? So it's a little bit competitive and actually getting rid of all the legacy code and getting rid of complexity. Yeah. So you have this here as well. This is, oh, this is a small one. It's only 2,000 lines of code. But it was like in the beginning it was a bit demotivating because you're like, oh, I deleted 8,000 lines of code and I would see this little dip in the graph because it was like 1,500 of logical lines of code that we had to delete a lot more. But that just end up being real and telling you how things really are and not how you want it to be. So this was the assembly level. And now we can go in and look internally at assemblies. This is when it gets really nasty because unlike F-sharp, which tells you you have to do stuff in order and you can't go and reference stuff cyclically, C-sharp lets you have references between namespaces without really giving you a warning. The only tool that I've seen that really gives these warnings is if you use independent quite actively. So how it looks like if I open this and I look at, okay, where are the namespaces inside here? And all the red arrows are namespaces that are mutually dependent. And if you try this on your own code base, unless you are very strict about it and you use a tool like this, this is often how things look. It can be innocent things sometimes, like just like the interfaces was put in the same assembly as the implementation maybe or it was using some concrete types that are quite easy to clean out and sometimes they're just horrible. So like here we have like, you can see, okay, so my models and my controller has such a mutual dependency, though that means the models reference the controllers and the controllers reference the model. So why is the model using the controllers? So now we're inside the assembly looking at how the namespaces relate, right? So I can see, okay. And now graphs don't work very well anymore because there's just too many namespaces. So we use the matrix view. So I say, keep only the involved namespaces and show me this cycle. This is pretty cool. Well, where did the graph matrix go? Wow. It's the spaceship view again. I just need to find the, where did it depends? I think it hit behind here. Yeah. This is the models namespace and controllers namespaces and I can click them and I can get the view of only the related stuff. This is probably why it's hard to do just by visual inspection and reading code without the proper tool because it is quite complex. So what you see, the green ones here are the ones that go from controllers to models which are fine but like where are the, the blue one is going in the other way. So you can see what is that one? It's the extra material season model and the serious controller that has a real relation. And there's one more blue guy like that and I just, this is the same slide I just showed you in real life. So this is the dependency. So basically there's one method that Endepend tells me about which is in the model class to get some URL to a season reference for a program that uses the MVC framework to get the serious controller and call a method on it. And after we had split it out and cleaned out some to code, this was dead. This is dead code. And just by deleting that, now I could easily delete it. It's only used on TV anyway. And when I run the analysis again, I get this nice little one directional arrow. So at this point basically we can start iteratively just cleaning out and hopefully if you get that, all those dependencies straightened out, you get a nice non-cyclic view. You should really try this on your own code base. I tried it on some open source projects and so on. And the code isn't as clean as the people like to think, I think, because we a lot of times don't run this kind of analysis. Yeah, the last thing I just wanted to show you is how this stuff is actually just part of the build process. So the first thing I showed you was just an analyze manualism folders in an assembly. But what you can do is you can configure these projects like I'm working on now is an ndproject file. So this is basically, I can show you where it is. This is just like in a repo, for example, in the super here, there's the ndproject file. That is basically where generated by the end-up and tool. And you can, I don't care. Where did it go? And I normally modify this a little bit by hand as well. This tells you which assemblies you want to scan, which assemblies are part of the namespace. And the tool helps you building these, but you can easily modify them. Where do you find code analysis reports? What are it? Here are all the definitions of the rules and the series and the custom stuff that you made. So like the charts that I made by right clicking and stuff, it just gets saved like this. So it's quite easy to edit in pure text as well. And this is all the definition for the reports that gets run by underpand. And to get those on the build server, it's basically we just, we just go into the build configuration in TeamCity. There's a build step. So after the build command is run, we're using fake here. So we're just running a build command. And after that, I have an end-depend step. And that just, it's a built-in integration to TeamCity. So I just say, you know, there's an end-depend runner. There's my ndproject file, which is the stuff that contains all the metrics, all the graphs, all the rules, all the warnings, and so on. And I just say, you know, compare it to a less successful build. And then you get this nice graph that everyone can enjoy. There's a lot of more features to the tool that I haven't even looked at, which is like code changes between versions, which is more like a code, the code diff in like in GitHub only tells you like what are the lines of text that has been changed, right? Whereas, and the pentast is a higher level view of the code, so we can tell you more logically what has changed and where has stuff changed in your code between commits. There's a lot more advanced queries and metrics you can do. What I would like to hear about are like other people showing like how they do refactoring with at a strategic level using a tool like this, because I think this gives us sort of a common model to talk about refactoring that are at a higher level than just working with the code itself. And I would really like to hear how other people do it. What I really miss is like more data and what effect these metrics have and what I mean by that is like, okay, so we've done this stuff. We cut, we reduced amount of code by two. We reduced complexity by this and this much. What effect does that have on the speed of the team? I know that's really hard to measure, but that's something I would be very interested in understanding. Yeah, the stuff about the managers, I think that's really important because we need, I think what I found is that when I'm able to give these graphs to my managers and they can show this, this is how the code looked before. It was like a mess of red arrows that got in all directions and this is how it looks now. Now it's clean. They can put that on two slides and say, you know, we had two front-end guys working for two weeks and this is what they accomplished. You know, can they get two more weeks to clean up this stuff? Chances are they will, you know, yeah, cool, continue going on. But what a lot of time I find that developers expect is that they go to the managers and they say, like, we've been refactoring, we're really proud, we made a lot of good progress. Can we continue? And if you can show any, you know, evidence of your improvements, it's very hard to actually get any more time. And yeah, so the main reason I'm doing this talk is because I would like to have other people sharing their experiences at this level. Most of the stuff that I've heard about End-Depend up until now when I talk to people are like, yeah, I used to end-depend. Oh, what did you use it for? And it's always like, oh, this client wanted me to come in and analyze the code base of someone else, you know, to give them a report on the status of the code base. But I haven't seen too many experiences of how it's actually being used in day-to-day maintenance. So does anyone have any questions or comments or things? Yeah, sure. Yes, yes, okay. So the question is, if I understand you correctly, can you, can you when you integrated into continuous integration, stop the build process and stop the continue, if there's something wrong? Yeah, yeah. And yes, okay, I will show you. Because if you, so this is, in this case, I'm not doing it right now. I probably should. But I haven't done it like that. But this is our CI build. And I have this nightly end-depend build. So it's just on the side. And this goes red all the time. Because there's the reason is because the inspections error from end-depend are too high. But what I could do if I go into the CI build now, hopefully the guys won't be too annoyed. And I'll go to the build step. I add end-depend. And I'll say the project file is, hold on, see if I can find it. So you would have to configure all the rules just like we have in this project file. And you say, this is where all the rules are. And you say, now it will run that. And we can say build failure condition is if inspection errors are more than zero. Right now they're going to have a problem. Because the number of inspection errors in this analysis as it's currently configured is something like 8,000. So if I save it now, they need to actually fix 8,000 inspection errors for the build to go green. Yes. So the comment is that you could tell it what is important. What is important. What should stop the build and not. And you can. And that's how, what you would do in here. You would actually go in here and you would say, you know, what is the rules that are violated right now? Methods with too many parameters, for example, it's a critical rule that stops them from releasing the build. And you could open this rule, which is defined in this and the project file. And you could say, you know, sorry about the messy windows here, but it says warn if count is larger than zero. And you could go like, I guess inform me for what the name of the command is, but you could take the level of warnings down. And then you could just say that, you know, these are the definitions that will break the build and these will just be logged as inspection warnings. And you could set it up to break at, you know, inspection warnings is more than zero and inspection errors. Like so you could say, you know, disable failing the build on warnings, fail it on errors. And you would, you could go in and set the errors, everything that are errors, not the warnings and so on. So definitely you can and I would say you should do it or I should do it. I just haven't gotten that far yet. Because, yeah. And definitely when you remove a dependency, you should add a rule and add it into CEI and make it stop the build. The thing is that, yeah, what I'm showing you now is just what we have actually done and we haven't actually done that. But what I showed you now is how to do it and we could and should probably do it like that. And then just tighten the rules, you know, day by day. Yeah. The point I wanted to make in a way is just that it's possible to use and depend without caring about all the fancy features to begin with. You can use it strategically to do a few things and don't even care about it. Because I think what happens sometimes is that people open the tool and they get 10,000 warnings and they're like, this is super cool but there's no way I can relate to 10,000 warnings and they close the tool and they go about doing their own day-to-day work again. But no, I think I should definitely basically go in and put all the errors to warnings and break the build. And the other thing is just that we have four build agents and I have one license so I just run and depend on one of the agents so I'd license this for one agent. If I want to run it on each CEI build, I would probably have to license all the agents which would cost me a little bit more money. And I feel confident to go and ask for that money now but when I started the project, I was just experimenting with Independence to see if it had any value. Now that I see that it has value for us, I would probably ask for money to license it on the other agents and start integrating it into our CEI builds. Thanks. Any other questions? Cool. If you want to see any details of it, just ping me and I'll show you around in the tools. Oh, yeah. Yeah. So the question is that you can use the metrics to convince the metrics something valuable on them. Oh, yeah. So the question is that you can use the metrics to show your managers that you're doing, you know, you're getting progress but how do you make sure that it's, how do you convince them that these are good metrics to focus on? I don't know, actually. I just have, I think managers like graphs, some of these graphs going down, they like, you know, users going up, complexity going down. That's my simple answer to it. That's worked for me. But sure, that can be a problem like, you know, you still have to convince people. I think the one thing, the only thing that I found and there's papers on this, I don't have the reference right now is but that says that when you double the amount of code or complexity of code, you use four times, you know, the exponential time to understand it. And when all your namespaces are referring each other, you have to understand the entire thing and that's maybe ten times larger which takes hundred times longer to understand and if you can understand one namespace as a module at the time. So I think you could argue in that way that there are some science behind this that argues that if you reduce the amount of code and the amount of code that is independent on each other, understanding it, reading it, to fixing it is faster. But I think that's the best I can give you. Yeah, to cry the graphs and if the graphs don't work, then go look for the papers. If that doesn't work, then, you know, move along. Yeah. Oh, that's a good question. When you use dependency inversion, how can you track all the types of stuff? And that's, I didn't mention that. After I used end-depend, I hate assembly scanning like with a passion. Like when you just go scan assemblies and pick up interfaces where the references are just a string to some other assembly, that's like when you have a blood hound chasing you and you just cross a river to get rid of your tracks, right? You just end-depend doesn't have anything to work with because you just told them that all the types and all the stuff that are in this file, you should really go and look for it and ask it to do whatever metaprogramming and reflection on your code that is. So the thing is that I get more and more focused on helping, helping the, helping end-depend now. So instead of going like assembly scanning stuff, you would go like implement this class with this interface, you know, and then it's possible to see what's being used where. But I think being explicit about which interfaces you are using and what concrete types are behind the implementations and so on helps a bit. But sure. So, yeah, well, that's my best answer. You can come and try if you want later and see what it can and can't find. But helping, you know, being explicit about your types and interfaces helps end-depend and helps your analysis. We've got bitten a lot of time by assembly scanning, right? You can't see it. It doesn't break anything. Then you try to run it and things break at runtime. And it can't, you need to do runtime analysis to catch that. And that end-depend can't do so. But if you just help it a little bit statically, it's, you know, everybody's happier, I think. Yeah. Okay. Thanks. Thank you.
 Okay, good morning everyone. Well, I've left the slide up for a while so you've probably  seen the topic, strategic refactoring using static analysis. So first to MI, I'm Björn  Einlad. I work at NRK. That's the National Norwegian Broadcasting in Norway. So, well,  the revisions of you know what this is. For the foreign people, it's like the BBC of Norway.  So I like to use this hashtag when I talk about NRK TV. You can if you want to. Tweet.  So the agenda for today. I want to try to keep it loose. So if you have any questions,  like feel free to interrupt me. Just throw questions at me. Just shout out so I can see  it. I can't see you very well because of the lights. Just scream loud. But I'll try to  go through some simple definition of what I mean by strategy, what I mean when I say  static analysis in this context. I'll be showing you the domain we're working at. Then I'll  go through End-upend, which is the tool that I'm using when I'm doing static analysis.  Then we'll look a little bit of refactoring at the assembly project level. And I'll go  a little bit more into detail on End-upend. And we'll go and look at namespaces and some  more complex metrics and stuff like that. And everything I'll be presenting today is  like Israel Life Code. Everything is just stuff we have in production. And there's no secrets.  I'll, you know, happy to show everything we have here. So just ask questions and I'll  dig into it. So first, some definitions. Well, refactoring, it's basically rewriting the  code without changing any of the functional specifications. That's the way I see it to  improve some other metric or some other aspect of the code. When I say strategic in this  context, I just mean strategic at the domain level, which I'll go into a little bit, or  strategic that we're talking about refactoring at assembly, dependency, namespace level,  not necessarily, you know, making one method cleaner or nicer by some other metric. And  when I say static code analysis, well, that's a big topic. I'm no expert in static analysis  in general at all. I'm not going to claim that. In the.NET space, there's, you know,  everything from StyleCop to Resharper to FXCop and the code analysis tools and the end-depend,  which I know of that's doing static analysis. So it's everything that's just running and  analyzing your code without actually executing the code. So StyleCop is, you know, it's very  at the syntactic level at your code, how do you name your methods and so on, you know,  lower casing, upper cases, and can't do any of the strategic stuff that we're talking  about here. There's FXCop and Resharper is doing a lot of these things as well. I'll  show you a little bit. But to me, the role choice of the static analysis tools is end-dependent,  and you'll see why when we get into the details. So that's the tool I've chose to use here.  And well, you know, we're talking about legacy applications here. For a definition on that,  I will go to Karolina's talk in an hour. So, but I guess all the code we have in production,  I define it as legacy. I don't know. So I'm a big fan of military strategy and stuff,  and, you know, it's sad. It's like when you listen to gangster rap and you feel cool when  you refactor stuff. It's very sad. Some of us do this, you know, you feel like a general  when you're refactoring and making strategic decisions. I like this metaphor even though  I think they're quite sad. But there are some similarities, right, that we're able to work  at a higher level of code than sitting in the trenches or sitting in the code base that  we're able to lift our perspective to a higher level and make some higher level decisions.  So, you know, we want to make some kind of map of our code that are showing assemblies,  namespaces, dependencies between them and so on. This is how I think of a lot of the  clean code movement that's going around. Not that it's bad to write clean code, but people  seem to be like this guy, like, I'm the cleanest fighter. Like, he's in the trenches. His friends  there are like rotting corpses, like smelling around them, but this is a claim to be the  cleanest. And it's a very, very local optimization to clean yourself when you're in a trench,  like, and I feel a lot of the same with clean code. Like, rewriting a method, it's fine,  like, do it, but it's not going to achieve any higher level strategic objective for your  business. At least that's my claim. So, I think I've narrowed it down a bit to talk about  strategic refactoring of our applications and the applications I'll show is NRK TV,  NRK radio, and NRK super with NDPen. So, we sort of narrowed it, made it a little bit  more concrete. So, I'll just quickly show you the domain we're working on. So, this  is NRK TV. So, test programs, you can watch TV programs. There's lots and lots of programs  in here. There's a login, so I'm logged in, so I have my content, so I can continue watching  stuff that I've seen. There's my programs. There's also, like, this is, we'll write,  we have very high focus on privacy. So, like, now, apparently, I've seen frekje fortellingen,  which is a Danish series about erotica, so I can, like, I can delete that because I probably  don't want people to see that. And there's, like, sex educations. Oh, I probably should  delete that as well, right? So, it's personalized. There's some live TV, you know, TV program  guides, categories, so that's the TV. Then there's the TV for children, which is aimed  at a younger age group. It looks a bit different. I think this is quite interesting. Can any  guess why there's, like, Q and X, even though there's no programs in Q and X? Like, why  don't we filter it out? And it's like, because these kids are learning the alphabet, so if  we take out Q and X, we confuse the kids. So, it's like, it's a very different age group.  And it's not only cartoons in here, it's, like, it looks a little bit, like, you know,  cartoonish, but there's also some serious stuff about learning, learning alphabets, but also  we go into hever topics, that's substance abuse and things like that. So, it's a mix  of education and entertainment. And then there's NRK radio, which is a live radio. That's  a bit different. You can listen to, you get some notifications of what is currently playing  in. These things are a bit different. And then there's the API team that I'm working  on, which is the backend. We'll talk about the backend here. There's a, with all the,  we're basically all these applications go in fetch their data. This is like a quick  overview of the domain. This is our story. Parts of it is in Norwegian. I joined around  here, like, September last year, so I can't take credit for any of this. But basically,  the idea is that there are lots and lots and lots and lots of clients. The stuff that we  looked on now, it's just a web client. So, that's just a web client, but there are smart  TVs and there is apps and all kinds of clients that are playing the content from the same  API. But we'll be looking mostly at the front and stuff on the web. I just took this  picture before I left. This is our team. These are the frontend people here, these four.  Aynard is doing a talk later today's here. There's Stola, which is on the backend team.  Steinard, which is on the audience is sitting behind here. And, you know, nice little group.  There's our operations. There's Ops. He has a cop that says no. And there's the release  manager that, like, you know, clicks octopus and decides what goes into production. Again,  this is an overview of all the clients. The stuff we'll be looking at now are the desktop,  the web stuff and how that relates to the backend. I took this. This is showing how  many users we have on the desktop and there's a lot, almost like a million unique users  a week. This is just to show that the desktop stuff is really important. And I chose a metric  that shows that it's really important. Like, if you look at how many minutes people actually  watch content, that's different. And this is just a little bit of background of why we  chose to do refactoring of these applications to begin with. And that was during the implementation  of this part, which is like direct feed. I just took this yesterday. This is stuff that  is live, but it's not necessarily broadcasted on TV. And to implement this feature, basically,  we finally took too long. We spent several months getting the MVP of this product out.  The reason being that the code base was too complex and too work-with, basically. And  we decided, you know, something needs to be cleaned up here. We need to be able to move  faster than what we were able to do. And all this, basically, all the stuff that you've  seen now with the NRK Super, NRK Radio, NRK TV is the same web application. It's just  configured to look differently running at different hosts, which was probably a good  idea at some point because you could get very quickly out something that looked quite similar.  But then it's grown different over time. Like, you had the login stuff on one page. You had  some live view on the radio was coming in and play. And the TV stuff looks completely  different. So at some point, stuff that looks very, looks quite similar end up being very  different. And another aspect is that you can see here on the children stuff, they watch  it mostly on the tablet. They don't use the desktop that much, whereas NRK TV is very  heavily used here. So, like, when we want to make changes to our very, very large audience  on the desktop side, we always need to think of what happens to this stuff, which isn't  as business critical. So we have these strategic decisions as well, where the fact that the  these applications intertwine is not too good. Okay, so we need a plan, right? And this was  supposed not too long ago. This is Montgomery and the plan for D-Day. And the overall of  the plan is just like one very simple page. And I think we need sort of the similar things  when we work with complex code bases. We need to have a simple plan when we work with complex  stuff. It says like heavy air bombing from a student's light permits until after eight  hours. And there's the forward body and there's the main body to follow. And it says down  here, the key note of everything to be simplicity. This is a quite heavy, large operation. The  other similarity I found is like main lot of the after they found the beaches and landed  the main troops, the objectives to cut dependencies as well, like cut railroad, cut tunnels, cut  bridges to be able to sustain further operations. After that, I think all metaphors break down,  but that's fine. All metaphors should do. So, end the pen. Let's start looking at the  tools and see some concrete examples. The thing about end the pen is through what they  say here. You get this insight into absolutely didn't have before. There's a breadth and  depth of information. But and we can just take a look at how that looks. If we just  did, we can build a very small and simple application like a console app. I can, you  know, if, if args any will do something, right, will do, oops. I'm not going to write too  much code, but I just need, we'll say, you know, high and DC and else, you know, I don't  know, something else. Doesn't matter. We're not going to run this code, right? We're just  going to analyze it statically. So, we're just going to build it. So, I built the code.  We should be a folder here with an application. We're not going to run it. We're just doing  static stuff that I can analyze. So, this is end the pen, the full tool and I can, what  I can do is I can just say analyze some depend, some assemblies in a folder. I'll take out  the visual studio host the exe file and just say analyze this exe file. Do your job, you  know, generate your reports and it's true. It's like there's lots of data. It says here  that my max and average cyclometric complexity is two, which is, so it makes very sense,  like if this do that, else do this. But I already have like critical rules violated.  I have actually, I just, I violated six rules already, right? And we can look at what it  is. It's not that important, but, but the idea here is that there's a breadth and variety  of information and graphs and charts and things we can analyze about the code base. And I  just talked about like, what did I say? Like simple. The simplicity is key to it all. Like  so, how do we use a complex tool in a simple way? So, the first thing is that we sort of,  we make a plan or we say, we get an overview of the assemblies. And the first mission is  to, you know, cut the back end from the front and they'll show you in the code after a while  and apply some custom metrics when we refactor. I'll go into the, to the details now. But  the thing is that you'll be able to come up with a simple plan. So, the way this looks  at all the three websites you saw, which is one website today, one code base, has some  couplings to the API of historical reasons. So, the first plan is cut those, cut those  dependencies. Second plan is cut, take the super part out of the other applications.  You can probably as well, they get the radio out and we could do a lot of more things,  but that, that's the plan, two things. So, that's a simple plan, right? So, the first  thing NDPend can do for us is to generate a map of all the pendent, all the dependencies  we have. So, what you see here is that the front end web application, it has some common  settings, common logger, common cache. It's fine, right? But there's this arrow here that  points to web API models. So, it's using some code from some back end models for some reason.  And that has transient dependencies that will pull in the entire back end library. Also,  elastic search, which is used in the back end for the indexing. And, you know, so you  get this entire thing. So, the first objective is basically cut this line, right? And that's  what I mean by strategic is just like, cut that line. I don't care how it's done. Just,  you know, and NDPend can also tell me that line, what does that depend on? It still doesn't  make a lot of sense to me reading out, like there's a plug profile configure that is using  some get serious ID from a plug view model. But I don't really care, like, all I want  to do is to give this to someone and say, you know, delete these dependencies. I don't  care how you achieve that objective. At that point, we're at the tactical level and I'm  not going to go into details on that. Like, you know, you can, that's at a local level.  You can refactor it. You can duplicate the code, you know, whatever to get rid of it.  I'll show you how we can build this as well, the graphs. And this is just to show you that  the same image can be generated from Resharper using the architect overview there. So this  is basically the same image with the dependencies between assemblies is just oriented a bit  differently. I could just show you how we, how this is done. So right now I'm going to  analyze the, this super application which have been split out from the code base. When  I analyze it, I can ask and depend, you know, view application assemblies only in a graph  and then it will draw this, where did it go? It will show, generate this nice little diagram  for me, which is the same we saw. And I can ask it, the way I've generated this graph  is just like build a graph of code elements involved in this dependency, for example.  So this is a quite, this is quite heavy in this case. So it's a more difficult objective  to cut out. But you can get this. So what you see is that even though I'm at this level  only caring about cutting these assemblies and you could, there's still like 73 violated  rules, 8000 violations, seven critical rules. But the way I use underbund is that I don't,  we can't go and start refactoring and cleaning up all that stuff. I'm just using it to achieve  some objectives. And I think it's important when we work with a tool like this to not  get all hung up in getting, you know, doing everything the tool tells us because then  we needed to start a lot earlier than, you know, many, many years into the project. But  anyway, what you can see here is that what we've achieved at this point is getting rid  of the backend assemblies. In this case, the supersite has been split out and just renamed,  duplicated out and, but you can see that there are no backend references. What this is, just  that simple thing reduced like the size of the package that we send that we deploy from  like 300 something megabytes to 100. Still way too big and we're, I think now we're down  at maybe 30. But just this one change, like just have dramatic impacts. Okay, so at this  point what we had done is that we had separated the super website out, duplicated all the  code, get rid of the backend references, which is all good. But we duplicated 12,000 logical  lines of code. We just copy pasted it. We took all the complexity from super and OKTV  and we duplicated it, which makes, gives us the opportunity to change one without affecting  the other. But there's still lots of dead and legacy code in both of the applications.  So the question is how do we get rid of that? The image that I sort of used to explain what  we're doing is just basically we had the yarn or string, blue and red string like intertwined  into a ball. And how do you separate that out? Do you like try to get the threads out  one by one or do you just duplicate them and cut off the blue parts from one end and the  red parts from the other? And to sort of, to know that you're actually cutting out thread  and not just duplicating it, we're using EnderPand at the build server. I'll just show you how  we've set that up. And we basically measure all of that in, hold on. So we set up a nightly  build with EnderPand. I can go into details later on how it's all set up. That is giving  us these friend charts, which trends, well, this is lines of code, but we have set up  with EnderPand like this, which is lines of code in the suprassemblies, lines of code  in namespaces, lines of code for view-generated code, lines of code for controllers and lines  of code for the managers in this, also for average complexity and max cyclomatic complexity  in the different namespaces in the duplicated code. So what you see is that when we duplicated  the code, it wasn't that long ago basically. It's like just in time for this talk, first  of June, that was merged into master branch and we have like just in the suprassembly  10,000 logical lines of code. That's a lot more in lines of code. This is actually just,  you know, your statements and expressions and so on. And at this point, we haven't really  achieved any, you know, cleaning up. Then the guys started deleting code. And you see,  we have deleted approximately, I would say like, you know, 40% of the code from supr,  which was TV and radio stuff, which is not used at all. And we can see by the complexity  that we haven't really reduced any complexity. And I think that is because we've just deleted  the controllers and the views and the stuff that is really separate, but we haven't started  going into refactoring the methods where they could be really simplified inside controllers  and so on. So I would say at this point, we should continue into refactoring out more  local complexity before we stop. And this is nice because we can go to the managers  and say, you know, we need a few more weeks. You know, we've already accomplished this.  We've reached, you know, the reduced complexity and the amount of code by half, but we need  some more time to clean up some of the other stuff. And they can take this to their managers  and tell them what they're doing, you know, to see that we are actually able to prove  that we're getting some thing out of that. I think too many times developers go like,  oh, the managers don't understand why we need time to refactoring, but it's because we're  not able to show any results a lot of the time. We just say that, you know, but it's  cleaner now. We've done some major refactoring. It's better now. But I think we should show  with some graphs and charts how we've actually improved.  So I will show you a little bit of how we go about the setup making this metrics that  we're using when we're refactoring. So one of the most powerful things that End up and  gives you, you know, in addition to these charts and graphs and trends that you can  dig into is like a domain model almost of your code using types, methods, fields, names,  business assemblies that you can query with what they call CQL links. We can use link  to query your code. So like even I can met a program against the code base to make this  metric. So I'll just show you how, like, how do I make a custom metric like that for my  code base? We can look at the dashboard again. I should be here. It's a bit like flying a  spaceship navigating around here, but it's, if you look, this is one of the graphs I made,  right? How do we make these graphs that contain, for example, lines of code for controller  use? You know, this is the code required. It's like application namespaces where the  name contains controllers, child methods, some lines of code. And this is how you give it  a name and this is the unit. So if I want to make another metric, I can just, you know,  I just take an existing metric. Normally, I just start with one of End up and metrics,  right? I just say, I like to make something like this, but I want to tweak it for my own  use. I can make, like, a new metric in here. And I can say, you know, where it contains,  I don't know, ASP. And you see, I get the result immediately under here and I can call  it lines of code for ASP. This is like the generated code in the ASP namespace. And now  I see there's 2,000 lines of code. I can make a new cren chart that I call, you know, NDC.  And I can say, I want to have this, I guess that's in, here it just shows up. Stuff, is  that name that I gave it here in the comments and the units show up here. And I can add  that to my graph and I get a graph. And I can check that in and it will show up in the  reports that it builds server. And the level of meta program, it's like super simple. If  you know Link and you have any idea of, like, that your application consists of namespace  methods and namespaces, basically you can start writing very complex and nice queries.  This is how you do cyclomatic complexity. This is how they do IL nesting depth. And  you can find all the rules and stuff and you can make your own. For example, you can make,  you can look at your dependency graph again. And you can go like, I want to cut this as  well, right? This common stuff. And I could go and say, I could, you know, edit a code  query, generate a code rule that matches if this dependency exists. And I can make a code  rule, check that in, that generates a warning if it exists, check it into the server, get  a warning at the server and refactor it out to see that it goes green. So it's almost  like test-driven, strategic refactoring. You can do. Okay, but we're still, like, at the  assembly level right now, how to cut project references and assembly references. Yeah.  Also, I need to give some credit to the guys who actually deleted the code because I haven't  touched the code base. I just put up the metrics and so on. I'm on another team and  I'm the API team. I'm not touching the front-end code. So these are, to be as a, there are  consultants in our team from Foshe and Netlight and they did all the deletions. I should actually  show you just how it looks because it looks very simple to delete 2,000 logical lines  of code. And then you look at, like, some of the pull requests right here and it's like,  remove unused controllers from super. And you have this, oh, 11,000 lines of code. And  this is maybe a 1,500 logical lines of code and you're like, you code review this stuff.  You know, file change, 200 file change. So this is the stuff where it gets tactical. This  is where I move out. Like, I like to be at the strategic level when it's about code reviewing  this stuff. It's like, as you see, you know, people are a little bit, aren't you test driven  to want to test look after you? And I'm like, yeah, you're deleting your tests as well,  right? So you need to really know what you're doing. The almost end, only thing you can  do is rely on your integration tests that are testing your UI and your manual testing  process to catch any errors that you're doing right here. But I think they had some fun  as well. I found I was looking through the pull request yesterday. And some of them had,  like, yeah, I think, oh, I got a little bit eager, right? And I think people get a little  bit more eager when they get actually these graphs and measurements and they get some  feedback from the build server that you're actually doing a good job and we can go to  management, you know? So it's a little bit competitive and actually getting rid of all  the legacy code and getting rid of complexity. Yeah. So you have this here as well. This  is, oh, this is a small one. It's only 2,000 lines of code. But it was like in the beginning  it was a bit demotivating because you're like, oh, I deleted 8,000 lines of code and I would  see this little dip in the graph because it was like 1,500 of logical lines of code that  we had to delete a lot more. But that just end up being real and telling you how things  really are and not how you want it to be. So this was the assembly level. And now we  can go in and look internally at assemblies. This is when it gets really nasty because  unlike F-sharp, which tells you you have to do stuff in order and you can't go and reference  stuff cyclically, C-sharp lets you have references between namespaces without really giving you  a warning. The only tool that I've seen that really gives these warnings is if you use  independent quite actively. So how it looks like if I open this and I look at, okay, where  are the namespaces inside here? And all the red arrows are namespaces that are mutually  dependent. And if you try this on your own code base, unless you are very strict about  it and you use a tool like this, this is often how things look. It can be innocent things  sometimes, like just like the interfaces was put in the same assembly as the implementation  maybe or it was using some concrete types that are quite easy to clean out and sometimes  they're just horrible. So like here we have like, you can see, okay, so my models and  my controller has such a mutual dependency, though that means the models reference the  controllers and the controllers reference the model. So why is the model using the controllers?  So now we're inside the assembly looking at how the namespaces relate, right? So I can  see, okay. And now graphs don't work very well anymore because there's just too many  namespaces. So we use the matrix view. So I say, keep only the involved namespaces and  show me this cycle. This is pretty cool. Well, where did the graph matrix go? Wow. It's  the spaceship view again. I just need to find the, where did it depends? I think it hit  behind here. Yeah. This is the models namespace and controllers namespaces and I can click  them and I can get the view of only the related stuff. This is probably why it's hard to do  just by visual inspection and reading code without the proper tool because it is quite  complex. So what you see, the green ones here are the ones that go from controllers to models  which are fine but like where are the, the blue one is going in the other way. So you  can see what is that one? It's the extra material season model and the serious controller that  has a real relation. And there's one more blue guy like that and I just, this is the  same slide I just showed you in real life. So this is the dependency. So basically there's  one method that Endepend tells me about which is in the model class to get some URL to a  season reference for a program that uses the MVC framework to get the serious controller  and call a method on it. And after we had split it out and cleaned out some to code,  this was dead. This is dead code. And just by deleting that, now I could easily delete  it. It's only used on TV anyway. And when I run the analysis again, I get this nice  little one directional arrow. So at this point basically we can start iteratively just cleaning  out and hopefully if you get that, all those dependencies straightened out, you get a nice  non-cyclic view. You should really try this on your own code base. I tried it on some  open source projects and so on. And the code isn't as clean as the people like to think,  I think, because we a lot of times don't run this kind of analysis. Yeah, the last thing  I just wanted to show you is how this stuff is actually just part of the build process.  So the first thing I showed you was just an analyze manualism folders in an assembly.  But what you can do is you can configure these projects like I'm working on now is  an ndproject file. So this is basically, I can show you where it is. This is just like  in a repo, for example, in the super here, there's the ndproject file. That is basically  where generated by the end-up and tool. And you can, I don't care. Where did it go? And  I normally modify this a little bit by hand as well. This tells you which assemblies you  want to scan, which assemblies are part of the namespace. And the tool helps you building  these, but you can easily modify them. Where do you find code analysis reports? What are  it? Here are all the definitions of the rules and the series and the custom stuff that you  made. So like the charts that I made by right clicking and stuff, it just gets saved like  this. So it's quite easy to edit in pure text as well. And this is all the definition for  the reports that gets run by underpand. And to get those on the build server, it's basically  we just, we just go into the build configuration in TeamCity. There's a build step. So after  the build command is run, we're using fake here. So we're just running a build command.  And after that, I have an end-depend step. And that just, it's a built-in integration  to TeamCity. So I just say, you know, there's an end-depend runner. There's my ndproject  file, which is the stuff that contains all the metrics, all the graphs, all the rules,  all the warnings, and so on. And I just say, you know, compare it to a less successful  build. And then you get this nice graph that everyone can enjoy. There's a lot of more  features to the tool that I haven't even looked at, which is like code changes between versions,  which is more like a code, the code diff in like in GitHub only tells you like what are  the lines of text that has been changed, right? Whereas, and the pentast is a higher level  view of the code, so we can tell you more logically what has changed and where has stuff  changed in your code between commits. There's a lot more advanced queries and metrics you  can do. What I would like to hear about are like other people showing like how they do  refactoring with at a strategic level using a tool like this, because I think this gives  us sort of a common model to talk about refactoring that are at a higher level than just working  with the code itself. And I would really like to hear how other people do it. What I really  miss is like more data and what effect these metrics have and what I mean by that is like,  okay, so we've done this stuff. We cut, we reduced amount of code by two. We reduced complexity  by this and this much. What effect does that have on the speed of the team? I know that's  really hard to measure, but that's something I would be very interested in understanding.  Yeah, the stuff about the managers, I think that's really important because we need, I  think what I found is that when I'm able to give these graphs to my managers and they  can show this, this is how the code looked before. It was like a mess of red arrows that  got in all directions and this is how it looks now. Now it's clean. They can put that on two  slides and say, you know, we had two front-end guys working for two weeks and this is what  they accomplished. You know, can they get two more weeks to clean up this stuff? Chances  are they will, you know, yeah, cool, continue going on. But what a lot of time I find that  developers expect is that they go to the managers and they say, like, we've been refactoring,  we're really proud, we made a lot of good progress. Can we continue? And if you can show any,  you know, evidence of your improvements, it's very hard to actually get any more time.  And yeah, so the main reason I'm doing this talk is because I would like to have other  people sharing their experiences at this level. Most of the stuff that I've heard about  End-Depend up until now when I talk to people are like, yeah, I used to end-depend. Oh,  what did you use it for? And it's always like, oh, this client wanted me to come in and  analyze the code base of someone else, you know, to give them a report on the status  of the code base. But I haven't seen too many experiences of how it's actually being used  in day-to-day maintenance. So does anyone have any questions or comments or things?  Yeah, sure.  Yes, yes, okay. So the question is, if I understand you correctly, can you, can you  when you integrated into continuous integration, stop the build process and stop the continue,  if there's something wrong? Yeah, yeah. And yes, okay, I will show you. Because if you,  so this is, in this case, I'm not doing it right now. I probably should. But I haven't  done it like that. But this is our CI build. And I have this nightly end-depend build.  So it's just on the side. And this goes red all the time. Because there's the reason is  because the inspections error from end-depend are too high. But what I could do if I go  into the CI build now, hopefully the guys won't be too annoyed. And I'll go to the build  step. I add end-depend. And I'll say the project file is, hold on, see if I can find it. So  you would have to configure all the rules just like we have in this project file. And  you say, this is where all the rules are. And you say, now it will run that. And we can  say build failure condition is if inspection errors are more than zero. Right now they're  going to have a problem. Because the number of inspection errors in this analysis as it's  currently configured is something like 8,000. So if I save it now, they need to actually  fix 8,000 inspection errors for the build to go green.  Yes. So the comment is that you could tell it what is important. What is important. What  should stop the build and not. And you can. And that's how, what you would do in here.  You would actually go in here and you would say, you know, what is the rules that are  violated right now? Methods with too many parameters, for example, it's a critical rule  that stops them from releasing the build. And you could open this rule, which is defined  in this and the project file. And you could say, you know, sorry about the messy windows  here, but it says warn if count is larger than zero. And you could go like, I guess inform  me for what the name of the command is, but you could take the level of warnings down.  And then you could just say that, you know, these are the definitions that will break  the build and these will just be logged as inspection warnings. And you could set it  up to break at, you know, inspection warnings is more than zero and inspection errors. Like  so you could say, you know, disable failing the build on warnings, fail it on errors.  And you would, you could go in and set the errors, everything that are errors, not the  warnings and so on. So definitely you can and I would say you should do it or I should  do it. I just haven't gotten that far yet. Because, yeah. And definitely when you remove  a dependency, you should add a rule and add it into CEI and make it stop the build. The  thing is that, yeah, what I'm showing you now is just what we have actually done and  we haven't actually done that. But what I showed you now is how to do it and we could  and should probably do it like that. And then just tighten the rules, you know, day by day.  Yeah. The point I wanted to make in a way is just that it's possible to use and depend  without caring about all the fancy features to begin with. You can use it strategically  to do a few things and don't even care about it. Because I think what happens sometimes  is that people open the tool and they get 10,000 warnings and they're like, this is super  cool but there's no way I can relate to 10,000 warnings and they close the tool and they  go about doing their own day-to-day work again. But no, I think I should definitely basically  go in and put all the errors to warnings and break the build. And the other thing is just  that we have four build agents and I have one license so I just run and depend on one  of the agents so I'd license this for one agent. If I want to run it on each CEI build,  I would probably have to license all the agents which would cost me a little bit more money.  And I feel confident to go and ask for that money now but when I started the project,  I was just experimenting with Independence to see if it had any value. Now that I see  that it has value for us, I would probably ask for money to license it on the other agents  and start integrating it into our CEI builds. Thanks. Any other questions? Cool. If you want  to see any details of it, just ping me and I'll show you around in the tools. Oh, yeah.  Yeah.  So the question is that you can use the metrics to convince the metrics something valuable  on them.  Oh, yeah. So the question is that you can use the metrics to show your managers that  you're doing, you know, you're getting progress but how do you make sure that it's, how do  you convince them that these are good metrics to focus on? I don't know, actually. I just  have, I think managers like graphs, some of these graphs going down, they like, you know,  users going up, complexity going down. That's my simple answer to it. That's worked for  me. But sure, that can be a problem like, you know, you still have to convince people.  I think the one thing, the only thing that I found and there's papers on this, I don't  have the reference right now is but that says that when you double the amount of code or  complexity of code, you use four times, you know, the exponential time to understand it.  And when all your namespaces are referring each other, you have to understand the entire  thing and that's maybe ten times larger which takes hundred times longer to understand and  if you can understand one namespace as a module at the time. So I think you could argue in  that way that there are some science behind this that argues that if you reduce the amount  of code and the amount of code that is independent on each other, understanding it, reading it,  to fixing it is faster. But I think that's the best I can give you.  Yeah, to cry the graphs and if the graphs don't work, then go look for the papers. If  that doesn't work, then, you know, move along. Yeah.  Oh, that's a good question. When you use dependency inversion, how can you track all the types  of stuff? And that's, I didn't mention that. After I used end-depend, I hate assembly scanning  like with a passion. Like when you just go scan assemblies and pick up interfaces where  the references are just a string to some other assembly, that's like when you have a blood  hound chasing you and you just cross a river to get rid of your tracks, right? You just  end-depend doesn't have anything to work with because you just told them that all the types  and all the stuff that are in this file, you should really go and look for it and ask it  to do whatever metaprogramming and reflection on your code that is. So the thing is that  I get more and more focused on helping, helping the, helping end-depend now. So instead of  going like assembly scanning stuff, you would go like implement this class with this interface,  you know, and then it's possible to see what's being used where. But I think being explicit  about which interfaces you are using and what concrete types are behind the implementations  and so on helps a bit. But sure. So, yeah, well, that's my best answer. You can come  and try if you want later and see what it can and can't find. But helping, you know,  being explicit about your types and interfaces helps end-depend and helps your analysis.  We've got bitten a lot of time by assembly scanning, right? You can't see it. It doesn't  break anything. Then you try to run it and things break at runtime. And it can't, you  need to do runtime analysis to catch that. And that end-depend can't do so. But if you  just help it a little bit statically, it's, you know, everybody's happier, I think. Yeah.  Okay. Thanks.  Thank you.
Strategic_refactoring_using_static_analysis_0_59.png Strategic_refactoring_using_static_analysis_60_8795.png Strategic_refactoring_using_static_analysis_8796_8939.png Strategic_refactoring_using_static_analysis_8940_8987.png Strategic_refactoring_using_static_analysis_8988_9083.png Strategic_refactoring_using_static_analysis_9084_9131.png Strategic_refactoring_using_static_analysis_9132_9179.png Strategic_refactoring_using_static_analysis_9180_9227.png Strategic_refactoring_using_static_analysis_9228_9299.png Strategic_refactoring_using_static_analysis_9300_9419.png Strategic_refactoring_using_static_analysis_9420_10235.png Strategic_refactoring_using_static_analysis_10236_10343.png Strategic_refactoring_using_static_analysis_10344_10451.png Strategic_refactoring_using_static_analysis_10452_10511.png Strategic_refactoring_using_static_analysis_10512_10679.png Strategic_refactoring_using_static_analysis_10680_11207.png Strategic_refactoring_using_static_analysis_11208_11255.png Strategic_refactoring_using_static_analysis_11256_11327.png Strategic_refactoring_using_static_analysis_11328_11411.png Strategic_refactoring_using_static_analysis_11412_11459.png Strategic_refactoring_using_static_analysis_11460_11519.png Strategic_refactoring_using_static_analysis_11520_11567.png Strategic_refactoring_using_static_analysis_11568_11675.png Strategic_refactoring_using_static_analysis_11676_11783.png Strategic_refactoring_using_static_analysis_11784_11915.png Strategic_refactoring_using_static_analysis_11916_12023.png Strategic_refactoring_using_static_analysis_12024_12335.png Strategic_refactoring_using_static_analysis_12336_12395.png Strategic_refactoring_using_static_analysis_12396_13175.png Strategic_refactoring_using_static_analysis_13176_13919.png Strategic_refactoring_using_static_analysis_13920_14219.png Strategic_refactoring_using_static_analysis_14220_14855.png Strategic_refactoring_using_static_analysis_14856_15911.png Strategic_refactoring_using_static_analysis_15912_17639.png Strategic_refactoring_using_static_analysis_17640_19163.png Strategic_refactoring_using_static_analysis_19164_19679.png Strategic_refactoring_using_static_analysis_19680_22751.png Strategic_refactoring_using_static_analysis_22752_22859.png Strategic_refactoring_using_static_analysis_22860_26855.png Strategic_refactoring_using_static_analysis_26856_28235.png Strategic_refactoring_using_static_analysis_28236_29063.png Strategic_refactoring_using_static_analysis_29064_32855.png Strategic_refactoring_using_static_analysis_32856_32951.png Strategic_refactoring_using_static_analysis_32952_32999.png Strategic_refactoring_using_static_analysis_33000_37523.png Strategic_refactoring_using_static_analysis_37524_37571.png Strategic_refactoring_using_static_analysis_37572_41891.png Strategic_refactoring_using_static_analysis_41892_42467.png Strategic_refactoring_using_static_analysis_42468_42959.png Strategic_refactoring_using_static_analysis_42960_43067.png Strategic_refactoring_using_static_analysis_43068_43511.png Strategic_refactoring_using_static_analysis_43512_43595.png Strategic_refactoring_using_static_analysis_43596_43643.png Strategic_refactoring_using_static_analysis_43644_45563.png Strategic_refactoring_using_static_analysis_45564_46787.png Strategic_refactoring_using_static_analysis_46788_46859.png Strategic_refactoring_using_static_analysis_46860_47579.png Strategic_refactoring_using_static_analysis_47580_49415.png Strategic_refactoring_using_static_analysis_49416_49595.png Strategic_refactoring_using_static_analysis_49596_49691.png Strategic_refactoring_using_static_analysis_49692_50039.png Strategic_refactoring_using_static_analysis_50040_50891.png Strategic_refactoring_using_static_analysis_50892_51035.png Strategic_refactoring_using_static_analysis_51036_51947.png Strategic_refactoring_using_static_analysis_51948_52667.png Strategic_refactoring_using_static_analysis_52668_53027.png Strategic_refactoring_using_static_analysis_53028_55619.png Strategic_refactoring_using_static_analysis_55620_56699.png Strategic_refactoring_using_static_analysis_56700_62051.png Strategic_refactoring_using_static_analysis_62052_64499.png Strategic_refactoring_using_static_analysis_64500_65219.png Strategic_refactoring_using_static_analysis_65220_76583.png Strategic_refactoring_using_static_analysis_76584_76631.png Strategic_refactoring_using_static_analysis_76632_77181.png
Okay, good morning everyone. Well, I've left the slide up for a while so you've probably seen the topic, strategic refactoring using static analysis. So first to MI, I'm Björn Einlad. I work at NRK. That's the National Norwegian Broadcasting in Norway. So, well, the revisions of you know what this is. For the foreign people, it's like the BBC of Norway. So I like to use this hashtag when I talk about NRK TV. You can if you want to. Tweet. So the agenda for today. I want to try to keep it loose. So if you have any questions, like feel free to interrupt me. Just throw questions at me. Just shout out so I can see it. I can't see you very well because of the lights. Just scream loud. But I'll try to go through some simple definition of what I mean by strategy, what I mean when I say static analysis in this context. I'll be showing you the domain we're working at. Then I'll go through End-upend, which is the tool that I'm using when I'm doing static analysis. Then we'll look a little bit of refactoring at the assembly project level. And I'll go a little bit more into detail on End-upend. And we'll go and look at namespaces and some more complex metrics and stuff like that. And everything I'll be presenting today is like Israel Life Code. Everything is just stuff we have in production. And there's no secrets. I'll, you know, happy to show everything we have here. So just ask questions and I'll dig into it. So first, some definitions. Well, refactoring, it's basically rewriting the code without changing any of the functional specifications. That's the way I see it to improve some other metric or some other aspect of the code. When I say strategic in this context, I just mean strategic at the domain level, which I'll go into a little bit, or strategic that we're talking about refactoring at assembly, dependency, namespace level, not necessarily, you know, making one method cleaner or nicer by some other metric. And when I say static code analysis, well, that's a big topic. I'm no expert in static analysis in general at all. I'm not going to claim that. In the.NET space, there's, you know, everything from StyleCop to Resharper to FXCop and the code analysis tools and the end-depend, which I know of that's doing static analysis. So it's everything that's just running and analyzing your code without actually executing the code. So StyleCop is, you know, it's very at the syntactic level at your code, how do you name your methods and so on, you know, lower casing, upper cases, and can't do any of the strategic stuff that we're talking about here. There's FXCop and Resharper is doing a lot of these things as well. I'll show you a little bit. But to me, the role choice of the static analysis tools is end-dependent, and you'll see why when we get into the details. So that's the tool I've chose to use here. And well, you know, we're talking about legacy applications here. For a definition on that, I will go to Karolina's talk in an hour. So, but I guess all the code we have in production, I define it as legacy. I don't know. So I'm a big fan of military strategy and stuff, and, you know, it's sad. It's like when you listen to gangster rap and you feel cool when you refactor stuff. It's very sad. Some of us do this, you know, you feel like a general when you're refactoring and making strategic decisions. I like this metaphor even though I think they're quite sad. But there are some similarities, right, that we're able to work at a higher level of code than sitting in the trenches or sitting in the code base that we're able to lift our perspective to a higher level and make some higher level decisions. So, you know, we want to make some kind of map of our code that are showing assemblies, namespaces, dependencies between them and so on. This is how I think of a lot of the clean code movement that's going around. Not that it's bad to write clean code, but people seem to be like this guy, like, I'm the cleanest fighter. Like, he's in the trenches. His friends there are like rotting corpses, like smelling around them, but this is a claim to be the cleanest. And it's a very, very local optimization to clean yourself when you're in a trench, like, and I feel a lot of the same with clean code. Like, rewriting a method, it's fine, like, do it, but it's not going to achieve any higher level strategic objective for your business. At least that's my claim. So, I think I've narrowed it down a bit to talk about strategic refactoring of our applications and the applications I'll show is NRK TV, NRK radio, and NRK super with NDPen. So, we sort of narrowed it, made it a little bit more concrete. So, I'll just quickly show you the domain we're working on. So, this is NRK TV. So, test programs, you can watch TV programs. There's lots and lots of programs in here. There's a login, so I'm logged in, so I have my content, so I can continue watching stuff that I've seen. There's my programs. There's also, like, this is, we'll write, we have very high focus on privacy. So, like, now, apparently, I've seen frekje fortellingen, which is a Danish series about erotica, so I can, like, I can delete that because I probably don't want people to see that. And there's, like, sex educations. Oh, I probably should delete that as well, right? So, it's personalized. There's some live TV, you know, TV program guides, categories, so that's the TV. Then there's the TV for children, which is aimed at a younger age group. It looks a bit different. I think this is quite interesting. Can any guess why there's, like, Q and X, even though there's no programs in Q and X? Like, why don't we filter it out? And it's like, because these kids are learning the alphabet, so if we take out Q and X, we confuse the kids. So, it's like, it's a very different age group. And it's not only cartoons in here, it's, like, it looks a little bit, like, you know, cartoonish, but there's also some serious stuff about learning, learning alphabets, but also we go into hever topics, that's substance abuse and things like that. So, it's a mix of education and entertainment. And then there's NRK radio, which is a live radio. That's a bit different. You can listen to, you get some notifications of what is currently playing in. These things are a bit different. And then there's the API team that I'm working on, which is the backend. We'll talk about the backend here. There's a, with all the, we're basically all these applications go in fetch their data. This is like a quick overview of the domain. This is our story. Parts of it is in Norwegian. I joined around here, like, September last year, so I can't take credit for any of this. But basically, the idea is that there are lots and lots and lots and lots of clients. The stuff that we looked on now, it's just a web client. So, that's just a web client, but there are smart TVs and there is apps and all kinds of clients that are playing the content from the same API. But we'll be looking mostly at the front and stuff on the web. I just took this picture before I left. This is our team. These are the frontend people here, these four. Aynard is doing a talk later today's here. There's Stola, which is on the backend team. Steinard, which is on the audience is sitting behind here. And, you know, nice little group. There's our operations. There's Ops. He has a cop that says no. And there's the release manager that, like, you know, clicks octopus and decides what goes into production. Again, this is an overview of all the clients. The stuff we'll be looking at now are the desktop, the web stuff and how that relates to the backend. I took this. This is showing how many users we have on the desktop and there's a lot, almost like a million unique users a week. This is just to show that the desktop stuff is really important. And I chose a metric that shows that it's really important. Like, if you look at how many minutes people actually watch content, that's different. And this is just a little bit of background of why we chose to do refactoring of these applications to begin with. And that was during the implementation of this part, which is like direct feed. I just took this yesterday. This is stuff that is live, but it's not necessarily broadcasted on TV. And to implement this feature, basically, we finally took too long. We spent several months getting the MVP of this product out. The reason being that the code base was too complex and too work-with, basically. And we decided, you know, something needs to be cleaned up here. We need to be able to move faster than what we were able to do. And all this, basically, all the stuff that you've seen now with the NRK Super, NRK Radio, NRK TV is the same web application. It's just configured to look differently running at different hosts, which was probably a good idea at some point because you could get very quickly out something that looked quite similar. But then it's grown different over time. Like, you had the login stuff on one page. You had some live view on the radio was coming in and play. And the TV stuff looks completely different. So at some point, stuff that looks very, looks quite similar end up being very different. And another aspect is that you can see here on the children stuff, they watch it mostly on the tablet. They don't use the desktop that much, whereas NRK TV is very heavily used here. So, like, when we want to make changes to our very, very large audience on the desktop side, we always need to think of what happens to this stuff, which isn't as business critical. So we have these strategic decisions as well, where the fact that the these applications intertwine is not too good. Okay, so we need a plan, right? And this was supposed not too long ago. This is Montgomery and the plan for D-Day. And the overall of the plan is just like one very simple page. And I think we need sort of the similar things when we work with complex code bases. We need to have a simple plan when we work with complex stuff. It says like heavy air bombing from a student's light permits until after eight hours. And there's the forward body and there's the main body to follow. And it says down here, the key note of everything to be simplicity. This is a quite heavy, large operation. The other similarity I found is like main lot of the after they found the beaches and landed the main troops, the objectives to cut dependencies as well, like cut railroad, cut tunnels, cut bridges to be able to sustain further operations. After that, I think all metaphors break down, but that's fine. All metaphors should do. So, end the pen. Let's start looking at the tools and see some concrete examples. The thing about end the pen is through what they say here. You get this insight into absolutely didn't have before. There's a breadth and depth of information. But and we can just take a look at how that looks. If we just did, we can build a very small and simple application like a console app. I can, you know, if, if args any will do something, right, will do, oops. I'm not going to write too much code, but I just need, we'll say, you know, high and DC and else, you know, I don't know, something else. Doesn't matter. We're not going to run this code, right? We're just going to analyze it statically. So, we're just going to build it. So, I built the code. We should be a folder here with an application. We're not going to run it. We're just doing static stuff that I can analyze. So, this is end the pen, the full tool and I can, what I can do is I can just say analyze some depend, some assemblies in a folder. I'll take out the visual studio host the exe file and just say analyze this exe file. Do your job, you know, generate your reports and it's true. It's like there's lots of data. It says here that my max and average cyclometric complexity is two, which is, so it makes very sense, like if this do that, else do this. But I already have like critical rules violated. I have actually, I just, I violated six rules already, right? And we can look at what it is. It's not that important, but, but the idea here is that there's a breadth and variety of information and graphs and charts and things we can analyze about the code base. And I just talked about like, what did I say? Like simple. The simplicity is key to it all. Like so, how do we use a complex tool in a simple way? So, the first thing is that we sort of, we make a plan or we say, we get an overview of the assemblies. And the first mission is to, you know, cut the back end from the front and they'll show you in the code after a while and apply some custom metrics when we refactor. I'll go into the, to the details now. But the thing is that you'll be able to come up with a simple plan. So, the way this looks at all the three websites you saw, which is one website today, one code base, has some couplings to the API of historical reasons. So, the first plan is cut those, cut those dependencies. Second plan is cut, take the super part out of the other applications. You can probably as well, they get the radio out and we could do a lot of more things, but that, that's the plan, two things. So, that's a simple plan, right? So, the first thing NDPend can do for us is to generate a map of all the pendent, all the dependencies we have. So, what you see here is that the front end web application, it has some common settings, common logger, common cache. It's fine, right? But there's this arrow here that points to web API models. So, it's using some code from some back end models for some reason. And that has transient dependencies that will pull in the entire back end library. Also, elastic search, which is used in the back end for the indexing. And, you know, so you get this entire thing. So, the first objective is basically cut this line, right? And that's what I mean by strategic is just like, cut that line. I don't care how it's done. Just, you know, and NDPend can also tell me that line, what does that depend on? It still doesn't make a lot of sense to me reading out, like there's a plug profile configure that is using some get serious ID from a plug view model. But I don't really care, like, all I want to do is to give this to someone and say, you know, delete these dependencies. I don't care how you achieve that objective. At that point, we're at the tactical level and I'm not going to go into details on that. Like, you know, you can, that's at a local level. You can refactor it. You can duplicate the code, you know, whatever to get rid of it. I'll show you how we can build this as well, the graphs. And this is just to show you that the same image can be generated from Resharper using the architect overview there. So this is basically the same image with the dependencies between assemblies is just oriented a bit differently. I could just show you how we, how this is done. So right now I'm going to analyze the, this super application which have been split out from the code base. When I analyze it, I can ask and depend, you know, view application assemblies only in a graph and then it will draw this, where did it go? It will show, generate this nice little diagram for me, which is the same we saw. And I can ask it, the way I've generated this graph is just like build a graph of code elements involved in this dependency, for example. So this is a quite, this is quite heavy in this case. So it's a more difficult objective to cut out. But you can get this. So what you see is that even though I'm at this level only caring about cutting these assemblies and you could, there's still like 73 violated rules, 8000 violations, seven critical rules. But the way I use underbund is that I don't, we can't go and start refactoring and cleaning up all that stuff. I'm just using it to achieve some objectives. And I think it's important when we work with a tool like this to not get all hung up in getting, you know, doing everything the tool tells us because then we needed to start a lot earlier than, you know, many, many years into the project. But anyway, what you can see here is that what we've achieved at this point is getting rid of the backend assemblies. In this case, the supersite has been split out and just renamed, duplicated out and, but you can see that there are no backend references. What this is, just that simple thing reduced like the size of the package that we send that we deploy from like 300 something megabytes to 100. Still way too big and we're, I think now we're down at maybe 30. But just this one change, like just have dramatic impacts. Okay, so at this point what we had done is that we had separated the super website out, duplicated all the code, get rid of the backend references, which is all good. But we duplicated 12,000 logical lines of code. We just copy pasted it. We took all the complexity from super and OKTV and we duplicated it, which makes, gives us the opportunity to change one without affecting the other. But there's still lots of dead and legacy code in both of the applications. So the question is how do we get rid of that? The image that I sort of used to explain what we're doing is just basically we had the yarn or string, blue and red string like intertwined into a ball. And how do you separate that out? Do you like try to get the threads out one by one or do you just duplicate them and cut off the blue parts from one end and the red parts from the other? And to sort of, to know that you're actually cutting out thread and not just duplicating it, we're using EnderPand at the build server. I'll just show you how we've set that up. And we basically measure all of that in, hold on. So we set up a nightly build with EnderPand. I can go into details later on how it's all set up. That is giving us these friend charts, which trends, well, this is lines of code, but we have set up with EnderPand like this, which is lines of code in the suprassemblies, lines of code in namespaces, lines of code for view-generated code, lines of code for controllers and lines of code for the managers in this, also for average complexity and max cyclomatic complexity in the different namespaces in the duplicated code. So what you see is that when we duplicated the code, it wasn't that long ago basically. It's like just in time for this talk, first of June, that was merged into master branch and we have like just in the suprassembly 10,000 logical lines of code. That's a lot more in lines of code. This is actually just, you know, your statements and expressions and so on. And at this point, we haven't really achieved any, you know, cleaning up. Then the guys started deleting code. And you see, we have deleted approximately, I would say like, you know, 40% of the code from supr, which was TV and radio stuff, which is not used at all. And we can see by the complexity that we haven't really reduced any complexity. And I think that is because we've just deleted the controllers and the views and the stuff that is really separate, but we haven't started going into refactoring the methods where they could be really simplified inside controllers and so on. So I would say at this point, we should continue into refactoring out more local complexity before we stop. And this is nice because we can go to the managers and say, you know, we need a few more weeks. You know, we've already accomplished this. We've reached, you know, the reduced complexity and the amount of code by half, but we need some more time to clean up some of the other stuff. And they can take this to their managers and tell them what they're doing, you know, to see that we are actually able to prove that we're getting some thing out of that. I think too many times developers go like, oh, the managers don't understand why we need time to refactoring, but it's because we're not able to show any results a lot of the time. We just say that, you know, but it's cleaner now. We've done some major refactoring. It's better now. But I think we should show with some graphs and charts how we've actually improved. So I will show you a little bit of how we go about the setup making this metrics that we're using when we're refactoring. So one of the most powerful things that End up and gives you, you know, in addition to these charts and graphs and trends that you can dig into is like a domain model almost of your code using types, methods, fields, names, business assemblies that you can query with what they call CQL links. We can use link to query your code. So like even I can met a program against the code base to make this metric. So I'll just show you how, like, how do I make a custom metric like that for my code base? We can look at the dashboard again. I should be here. It's a bit like flying a spaceship navigating around here, but it's, if you look, this is one of the graphs I made, right? How do we make these graphs that contain, for example, lines of code for controller use? You know, this is the code required. It's like application namespaces where the name contains controllers, child methods, some lines of code. And this is how you give it a name and this is the unit. So if I want to make another metric, I can just, you know, I just take an existing metric. Normally, I just start with one of End up and metrics, right? I just say, I like to make something like this, but I want to tweak it for my own use. I can make, like, a new metric in here. And I can say, you know, where it contains, I don't know, ASP. And you see, I get the result immediately under here and I can call it lines of code for ASP. This is like the generated code in the ASP namespace. And now I see there's 2,000 lines of code. I can make a new cren chart that I call, you know, NDC. And I can say, I want to have this, I guess that's in, here it just shows up. Stuff, is that name that I gave it here in the comments and the units show up here. And I can add that to my graph and I get a graph. And I can check that in and it will show up in the reports that it builds server. And the level of meta program, it's like super simple. If you know Link and you have any idea of, like, that your application consists of namespace methods and namespaces, basically you can start writing very complex and nice queries. This is how you do cyclomatic complexity. This is how they do IL nesting depth. And you can find all the rules and stuff and you can make your own. For example, you can make, you can look at your dependency graph again. And you can go like, I want to cut this as well, right? This common stuff. And I could go and say, I could, you know, edit a code query, generate a code rule that matches if this dependency exists. And I can make a code rule, check that in, that generates a warning if it exists, check it into the server, get a warning at the server and refactor it out to see that it goes green. So it's almost like test-driven, strategic refactoring. You can do. Okay, but we're still, like, at the assembly level right now, how to cut project references and assembly references. Yeah. Also, I need to give some credit to the guys who actually deleted the code because I haven't touched the code base. I just put up the metrics and so on. I'm on another team and I'm the API team. I'm not touching the front-end code. So these are, to be as a, there are consultants in our team from Foshe and Netlight and they did all the deletions. I should actually show you just how it looks because it looks very simple to delete 2,000 logical lines of code. And then you look at, like, some of the pull requests right here and it's like, remove unused controllers from super. And you have this, oh, 11,000 lines of code. And this is maybe a 1,500 logical lines of code and you're like, you code review this stuff. You know, file change, 200 file change. So this is the stuff where it gets tactical. This is where I move out. Like, I like to be at the strategic level when it's about code reviewing this stuff. It's like, as you see, you know, people are a little bit, aren't you test driven to want to test look after you? And I'm like, yeah, you're deleting your tests as well, right? So you need to really know what you're doing. The almost end, only thing you can do is rely on your integration tests that are testing your UI and your manual testing process to catch any errors that you're doing right here. But I think they had some fun as well. I found I was looking through the pull request yesterday. And some of them had, like, yeah, I think, oh, I got a little bit eager, right? And I think people get a little bit more eager when they get actually these graphs and measurements and they get some feedback from the build server that you're actually doing a good job and we can go to management, you know? So it's a little bit competitive and actually getting rid of all the legacy code and getting rid of complexity. Yeah. So you have this here as well. This is, oh, this is a small one. It's only 2,000 lines of code. But it was like in the beginning it was a bit demotivating because you're like, oh, I deleted 8,000 lines of code and I would see this little dip in the graph because it was like 1,500 of logical lines of code that we had to delete a lot more. But that just end up being real and telling you how things really are and not how you want it to be. So this was the assembly level. And now we can go in and look internally at assemblies. This is when it gets really nasty because unlike F-sharp, which tells you you have to do stuff in order and you can't go and reference stuff cyclically, C-sharp lets you have references between namespaces without really giving you a warning. The only tool that I've seen that really gives these warnings is if you use independent quite actively. So how it looks like if I open this and I look at, okay, where are the namespaces inside here? And all the red arrows are namespaces that are mutually dependent. And if you try this on your own code base, unless you are very strict about it and you use a tool like this, this is often how things look. It can be innocent things sometimes, like just like the interfaces was put in the same assembly as the implementation maybe or it was using some concrete types that are quite easy to clean out and sometimes they're just horrible. So like here we have like, you can see, okay, so my models and my controller has such a mutual dependency, though that means the models reference the controllers and the controllers reference the model. So why is the model using the controllers? So now we're inside the assembly looking at how the namespaces relate, right? So I can see, okay. And now graphs don't work very well anymore because there's just too many namespaces. So we use the matrix view. So I say, keep only the involved namespaces and show me this cycle. This is pretty cool. Well, where did the graph matrix go? Wow. It's the spaceship view again. I just need to find the, where did it depends? I think it hit behind here. Yeah. This is the models namespace and controllers namespaces and I can click them and I can get the view of only the related stuff. This is probably why it's hard to do just by visual inspection and reading code without the proper tool because it is quite complex. So what you see, the green ones here are the ones that go from controllers to models which are fine but like where are the, the blue one is going in the other way. So you can see what is that one? It's the extra material season model and the serious controller that has a real relation. And there's one more blue guy like that and I just, this is the same slide I just showed you in real life. So this is the dependency. So basically there's one method that Endepend tells me about which is in the model class to get some URL to a season reference for a program that uses the MVC framework to get the serious controller and call a method on it. And after we had split it out and cleaned out some to code, this was dead. This is dead code. And just by deleting that, now I could easily delete it. It's only used on TV anyway. And when I run the analysis again, I get this nice little one directional arrow. So at this point basically we can start iteratively just cleaning out and hopefully if you get that, all those dependencies straightened out, you get a nice non-cyclic view. You should really try this on your own code base. I tried it on some open source projects and so on. And the code isn't as clean as the people like to think, I think, because we a lot of times don't run this kind of analysis. Yeah, the last thing I just wanted to show you is how this stuff is actually just part of the build process. So the first thing I showed you was just an analyze manualism folders in an assembly. But what you can do is you can configure these projects like I'm working on now is an ndproject file. So this is basically, I can show you where it is. This is just like in a repo, for example, in the super here, there's the ndproject file. That is basically where generated by the end-up and tool. And you can, I don't care. Where did it go? And I normally modify this a little bit by hand as well. This tells you which assemblies you want to scan, which assemblies are part of the namespace. And the tool helps you building these, but you can easily modify them. Where do you find code analysis reports? What are it? Here are all the definitions of the rules and the series and the custom stuff that you made. So like the charts that I made by right clicking and stuff, it just gets saved like this. So it's quite easy to edit in pure text as well. And this is all the definition for the reports that gets run by underpand. And to get those on the build server, it's basically we just, we just go into the build configuration in TeamCity. There's a build step. So after the build command is run, we're using fake here. So we're just running a build command. And after that, I have an end-depend step. And that just, it's a built-in integration to TeamCity. So I just say, you know, there's an end-depend runner. There's my ndproject file, which is the stuff that contains all the metrics, all the graphs, all the rules, all the warnings, and so on. And I just say, you know, compare it to a less successful build. And then you get this nice graph that everyone can enjoy. There's a lot of more features to the tool that I haven't even looked at, which is like code changes between versions, which is more like a code, the code diff in like in GitHub only tells you like what are the lines of text that has been changed, right? Whereas, and the pentast is a higher level view of the code, so we can tell you more logically what has changed and where has stuff changed in your code between commits. There's a lot more advanced queries and metrics you can do. What I would like to hear about are like other people showing like how they do refactoring with at a strategic level using a tool like this, because I think this gives us sort of a common model to talk about refactoring that are at a higher level than just working with the code itself. And I would really like to hear how other people do it. What I really miss is like more data and what effect these metrics have and what I mean by that is like, okay, so we've done this stuff. We cut, we reduced amount of code by two. We reduced complexity by this and this much. What effect does that have on the speed of the team? I know that's really hard to measure, but that's something I would be very interested in understanding. Yeah, the stuff about the managers, I think that's really important because we need, I think what I found is that when I'm able to give these graphs to my managers and they can show this, this is how the code looked before. It was like a mess of red arrows that got in all directions and this is how it looks now. Now it's clean. They can put that on two slides and say, you know, we had two front-end guys working for two weeks and this is what they accomplished. You know, can they get two more weeks to clean up this stuff? Chances are they will, you know, yeah, cool, continue going on. But what a lot of time I find that developers expect is that they go to the managers and they say, like, we've been refactoring, we're really proud, we made a lot of good progress. Can we continue? And if you can show any, you know, evidence of your improvements, it's very hard to actually get any more time. And yeah, so the main reason I'm doing this talk is because I would like to have other people sharing their experiences at this level. Most of the stuff that I've heard about End-Depend up until now when I talk to people are like, yeah, I used to end-depend. Oh, what did you use it for? And it's always like, oh, this client wanted me to come in and analyze the code base of someone else, you know, to give them a report on the status of the code base. But I haven't seen too many experiences of how it's actually being used in day-to-day maintenance. So does anyone have any questions or comments or things? Yeah, sure. Yes, yes, okay. So the question is, if I understand you correctly, can you, can you when you integrated into continuous integration, stop the build process and stop the continue, if there's something wrong? Yeah, yeah. And yes, okay, I will show you. Because if you, so this is, in this case, I'm not doing it right now. I probably should. But I haven't done it like that. But this is our CI build. And I have this nightly end-depend build. So it's just on the side. And this goes red all the time. Because there's the reason is because the inspections error from end-depend are too high. But what I could do if I go into the CI build now, hopefully the guys won't be too annoyed. And I'll go to the build step. I add end-depend. And I'll say the project file is, hold on, see if I can find it. So you would have to configure all the rules just like we have in this project file. And you say, this is where all the rules are. And you say, now it will run that. And we can say build failure condition is if inspection errors are more than zero. Right now they're going to have a problem. Because the number of inspection errors in this analysis as it's currently configured is something like 8,000. So if I save it now, they need to actually fix 8,000 inspection errors for the build to go green. Yes. So the comment is that you could tell it what is important. What is important. What should stop the build and not. And you can. And that's how, what you would do in here. You would actually go in here and you would say, you know, what is the rules that are violated right now? Methods with too many parameters, for example, it's a critical rule that stops them from releasing the build. And you could open this rule, which is defined in this and the project file. And you could say, you know, sorry about the messy windows here, but it says warn if count is larger than zero. And you could go like, I guess inform me for what the name of the command is, but you could take the level of warnings down. And then you could just say that, you know, these are the definitions that will break the build and these will just be logged as inspection warnings. And you could set it up to break at, you know, inspection warnings is more than zero and inspection errors. Like so you could say, you know, disable failing the build on warnings, fail it on errors. And you would, you could go in and set the errors, everything that are errors, not the warnings and so on. So definitely you can and I would say you should do it or I should do it. I just haven't gotten that far yet. Because, yeah. And definitely when you remove a dependency, you should add a rule and add it into CEI and make it stop the build. The thing is that, yeah, what I'm showing you now is just what we have actually done and we haven't actually done that. But what I showed you now is how to do it and we could and should probably do it like that. And then just tighten the rules, you know, day by day. Yeah. The point I wanted to make in a way is just that it's possible to use and depend without caring about all the fancy features to begin with. You can use it strategically to do a few things and don't even care about it. Because I think what happens sometimes is that people open the tool and they get 10,000 warnings and they're like, this is super cool but there's no way I can relate to 10,000 warnings and they close the tool and they go about doing their own day-to-day work again. But no, I think I should definitely basically go in and put all the errors to warnings and break the build. And the other thing is just that we have four build agents and I have one license so I just run and depend on one of the agents so I'd license this for one agent. If I want to run it on each CEI build, I would probably have to license all the agents which would cost me a little bit more money. And I feel confident to go and ask for that money now but when I started the project, I was just experimenting with Independence to see if it had any value. Now that I see that it has value for us, I would probably ask for money to license it on the other agents and start integrating it into our CEI builds. Thanks. Any other questions? Cool. If you want to see any details of it, just ping me and I'll show you around in the tools. Oh, yeah. Yeah. So the question is that you can use the metrics to convince the metrics something valuable on them. Oh, yeah. So the question is that you can use the metrics to show your managers that you're doing, you know, you're getting progress but how do you make sure that it's, how do you convince them that these are good metrics to focus on? I don't know, actually. I just have, I think managers like graphs, some of these graphs going down, they like, you know, users going up, complexity going down. That's my simple answer to it. That's worked for me. But sure, that can be a problem like, you know, you still have to convince people. I think the one thing, the only thing that I found and there's papers on this, I don't have the reference right now is but that says that when you double the amount of code or complexity of code, you use four times, you know, the exponential time to understand it. And when all your namespaces are referring each other, you have to understand the entire thing and that's maybe ten times larger which takes hundred times longer to understand and if you can understand one namespace as a module at the time. So I think you could argue in that way that there are some science behind this that argues that if you reduce the amount of code and the amount of code that is independent on each other, understanding it, reading it, to fixing it is faster. But I think that's the best I can give you. Yeah, to cry the graphs and if the graphs don't work, then go look for the papers. If that doesn't work, then, you know, move along. Yeah. Oh, that's a good question. When you use dependency inversion, how can you track all the types of stuff? And that's, I didn't mention that. After I used end-depend, I hate assembly scanning like with a passion. Like when you just go scan assemblies and pick up interfaces where the references are just a string to some other assembly, that's like when you have a blood hound chasing you and you just cross a river to get rid of your tracks, right? You just end-depend doesn't have anything to work with because you just told them that all the types and all the stuff that are in this file, you should really go and look for it and ask it to do whatever metaprogramming and reflection on your code that is. So the thing is that I get more and more focused on helping, helping the, helping end-depend now. So instead of going like assembly scanning stuff, you would go like implement this class with this interface, you know, and then it's possible to see what's being used where. But I think being explicit about which interfaces you are using and what concrete types are behind the implementations and so on helps a bit. But sure. So, yeah, well, that's my best answer. You can come and try if you want later and see what it can and can't find. But helping, you know, being explicit about your types and interfaces helps end-depend and helps your analysis. We've got bitten a lot of time by assembly scanning, right? You can't see it. It doesn't break anything. Then you try to run it and things break at runtime. And it can't, you need to do runtime analysis to catch that. And that end-depend can't do so. But if you just help it a little bit statically, it's, you know, everybody's happier, I think. Yeah. Okay. Thanks. Thank you.
topic, strategic refactoring using static choice of the static analysis tools is on that, I will go to Karolina's talk in an hour. lift our perspective to a higher level and make want to make some kind of map of our code that applications I'll show is NRK TV, NRK radio, and a very different age group. And it's not only their data. This is like a quick overview of the critical. So we have these strategic decisions as ago. This is Montgomery and the plan for D-Day. things we can analyze about the code base. And I which have been split out from the code base. don't, we can't go and start refactoring and you know Link and you have any idea of, like, do just by visual inspection and reading code versions, which is more like a code, the code changed in your code between commits. There's a people showing like how they do refactoring with know that's really hard to measure, but that's say, you know, what is the rules that are so on. So definitely you can and I would say you don't even care about it. Because I think what again. But no, I think I should definitely ping me and I'll show you around in the tools. sure, that can be a problem like, you know, you