Patterns of large-scale JavaScript applications
Bekkelund, Kim Joar
JavaScript has come a long way in few years. As we no longer need to battle DOM differences we can focus on building our applications instead. Now, however, as more and more application logic move from the server to the client our main problem is that we need to unlearn our earlier DOM-centric approach to JavaScript. Tools such as Backbone and Angular help, but before we are able to use the effectively we have to change some of our neural pathways. In this talk I will look at a couple of patterns that will help you move away from jQuery spaghetti and get you started on a foundation for building large-scale JavaScript applications.
 Yes, should we get started? So today we're going to talk a little bit about patterns of large scale JavaScript applications. There's been a lot of cool stuff going on these last few years. So I want to talk a little bit about some of the things that I've experienced and learned. Hello, my name is Kim. I work at Beck Consulting here in Oslo. And you can find more about me here. So we're going to talk about large JavaScript applications. But people tend to think that I'm talking about really large JavaScript applications when I say this. 20,000 lines of code or more. And it doesn't really have to be that way. All of these things that I'm talking about also the same concept apply for smaller applications. They can be a thousand lines of code. They can be a couple of thousand lines of code. And a lot of the patterns are still important to think about. So the main thing usually is that these are applications that need to live for a long time. And also applications that have unforeseen requirements. So most of this will apply to all of the JavaScript apps we write, at least for work. So the problem with JavaScript is that we tend to start out with something like this. It's a small feature. We send all the HTML from the server to the browser. And then we start creating a couple of small features. Five lines, 10 lines, 20 lines of code. And as the time goes by, we need more and more functionality. And then we suddenly have 5,000 lines of code in one file. And this happens all the time. I've been on several projects where this happened and it happens to everyone. I think this is a good term for it, jQuery soup. We get this brittle, tangled mess. And it's really hard to understand and hard to code in those apps. And maybe the worst thing is that it's hard to reason about the code. So that makes it difficult to create new features. So if you go back a couple of years to around 2006, we had an obtrusive JavaScript. We sent HTML to the browser and we used JavaScript to obtrusively add functionality. So it worked without JavaScript and we added functionality with JavaScript. And this year we also got jQuery. So the browsers back then was IE6. IE7 came out in 2006. We had Firefox 2. And the state of the web was different back then. So JavaScript was an amazing library. We really needed jQuery. The thing is that it's an amazing library for DOM manipulation. But we tend to follow a lot of anti-patterns when our apps grow. So we have things like coupling ourselves too much to the DOM. So when you change the HTML, you actually end up breaking your JavaScript. You put everything inside document ready. So whenever the code loads, it starts the entire thing and you have no way of testing it. And it's also hard to reuse. And also everything tends to end up in a jQuery plugin. And also all the things that don't have anything to do with jQuery. And maybe the worst thing is that too often the business logic is just hidden inside this brittle, tangled mess of a code base. So if we just look at a reasonably simple example, we have a form here. And when you submit the form, we do an IEX request. And on success on that request, we actually receive some... We get the data and we put it into the DOM. But the thing is that right here there's a lot of stuff going on. We have page events and user events and network IEO. We have network events. We do templating, parsing of the response and a lot of stuff. And we also get callback L. We got these structures that just go in and in and in. So there's callbacks inside, callbacks inside, callbacks. And what happens then when we have this considerable increase in the amount of JavaScript? We have a lot of more interactive pages than before. And there's a lot of complexity just in the user interfaces. We could have something like the new Google Maps. So here we require JavaScript for anything to function at all. So we've come a long way. And what has happened is that we're pushing more and more stuff from the back end to the front end. So a lot of the stuff that we before did on the back end, we now do dynamically with JavaScript on the front end. And also we've got a lot of new capabilities in the browsers. There's still a browser war. There's still really pushing for new stuff into the browser. So you have geolocation, local storage, canvas, and a lot of other stuff. So this means that there's more and more complexity in your front end application. So the problem is no longer the browser itself and the capabilities you have there. It's the size and complexity of our applications. I saw a great talk about a year ago from Brandon Keepers who works at GitHub. He said that the litmus test for your application is that if your site breaks, when your JavaScript fails, you should treat it like a real language. And this is mainly what we're going to focus on, what it means to treat JavaScript as a real language. Because what we want is testable, scalable, and maintainable JavaScript. This is where we want to be. The problem, though, is that this is really difficult. It takes a lot of time to learn. I've been doing this full time for a couple of years, and we've had a really hard time to get there. We're starting to get there. So the problem is that we need to unlearn a lot of stuff, because we've coded JavaScript for 10 years, a lot of us, and we're used to doing it the jQuery way. And it's so easy to create brittle systems when we're using jQuery. But the first thing is that we need to unlearn our DOM-centric approach to creating websites. So the problem with the DOM is that it's this big blob of state. So you have this stateful and global object you're working with all the time. So this makes it really hard to maintain your codebase when your application grows, because you're working on this one blob of state. And it's also really hard to test. And debugging some of these applications can also be really hard, because there's usually a lot of stuff going on, and it depends on the state you're currently in. We also need to unlearn how we write JavaScript. We need to learn from how they do it on the back end, and we need to get away from 5,000 lines of code in a single file. We also need to watch out for globals, not just the DOM itself, but also globals in general. So basically, learn more from the back end. And we need to unlearn how we use jQuery. This is a great quote from growing object-oriented software guided by tests, which is an amazing book. They say that when your code is difficult to test, it's most likely because you need to improve your design. Justin Meyer, who created a JavaScript MVC library, said this, is that to build large applications, you need to never build large applications. You need to build small pieces and assemble those into your application. So we need to get away from these big files. And these last couple of years, we've started to get some help here. We've got a lot of new libraries, so we have things like knockouts and backbone, which are reasonably small. We also have things like Ember and AngularJS, which is more full-fledged frameworks. So they're trying to help solve the problem, and they have quite different philosophies, so we're going to try to understand some of the things that they do. Because right now, it's easy to feel overwhelmed by choice. It's a daunting task just to be able to choose one of them. I would have struggled with that if I was starting a new project right now. So the thing is that we need a foundation to discuss the features they have and how they do things, and also to understand the value they add and how you can solve it yourself without using a library or framework at all. So the most important thing is to get the state away from the DOM. This is the same for all of those libraries. They all want you to get the state away from the DOM. And one way of doing this is NBC. This is what most of those libraries are inspired by. So if we just look at these last few years, we have from 2005 here, and this is a Google search for JavaScript NBC, or the Google Trends. And we see that this is really growing. So the main thing here about NBC, I guess a lot of you are back-end developers, so you know what NBC is. But the main point here is that you want this clean separation between your data and the DOM. So the controller here is the thing that mediates input and manipulates the model. However, you won't see that cleanly applied in a lot of front-end JavaScript frameworks and libraries. There's a lot of variation there, so you often see this instead, ModelViewStar. And this is just because a lot of them has a different take on what that last part is. So the thing is you want to split your data and the DOM, and how you do it, I think you can call it that, is quite different from library to library. So you can have something like a ModelView presenter where all the presentation logic is pushed into the presenter. So now the view doesn't know about the model at all. It's the presenter that gives the view the data. You can have something like ModelViewViewModel, where the view model is the one that exposes data to the view. So this is a quote from one of the core developers in AngularJS. He says that AngularJS is a ModelViewWhatever framework, where whatever stands for whatever works for you. They started out being mostly MVC and has gradually become more NVVM. But the thing is they also say this very clearly that it is ModelViewWhatever. So the most important thing here is separating the state and the DOM. So this is the first step. We've already gotten somewhere. So just by thinking about this, it should be easier to work with JavaScript. MVStar is not enough, though. Just having this pattern to follow still doesn't say a lot about how your code should look like and how it should work. And a lot of things that you need to think about. For example, here, Joel Hooks wrote a great blog post about a month ago, where he talked about his experiences with AngularJS. And what he says is that when he started on this project, there was a lot of monolithic single files that contained too much code. So just following MVStar itself is not enough. So he talked a lot about the importance of code organization and how important that was for actually getting an application that was easy to work with and where you can add features rapidly. And this is also a great quote by Jim Levin, who says that, just how you split your code has an impact on the maintainability of the application. So really what we want to do is have far more files and far less in the file size. So we want to get away from these 5,000 lines of code in a single file. It's better to have more files that are smaller, easier to work with. So you want more focused files. There's not enough just to split up. You need to think about how you do it. So there's two ways, for example, there are two ways we can think about this. We could package our code by feature or we could package it by layer. So if we look at this, is it large enough? So for example, if you package your code by feature, you could have a module like this, a module folder like this, where you have, you can see the high level scope of the application. So here we're working with BankID, we have some customer information and some other stuff. Or you could have package it by layer, so you have a modules, models folder, a collections folder, a views folder, a routers folder. So this is from a backbone project. So I prefer packaging by feature and I usually tend to call them modules. That was also the module name on the folder they were in. So a module is something like this. So here we see just the search bar on Facebook. That can be a module. So it's a bite-sized piece of your application. It could also be something like this, where we input the status. So it's still a small feature on Facebook itself. Or it could be something like this on the right side. So modules are the things that contain business logic. So this is mainly where you want to have all your business logic and all of the stuff that is special about your application. All the other stuff is usually mostly set up and reusable functions. So the modules should be small. They should try to solve one problem and should really try to relentlessly split into modules. Of course you shouldn't go for like 10 lines and everything, but you should really focus on what feature are we working on here. And try to separate them from the rest of the site. So this should be focused and they should really do one thing really well. I also think that modules should be decoupled from other modules. So you don't need absolute separation, but they really should be separated from each other. So you can place it in a new context. You can move it around on the site and it doesn't break anything. That's not easy in a lot of jQuery apps. If you move it around, you change the DOM structure so your application breaks. At least it's more difficult to do this in a jQuery application. And also it should, just doing this should make your app easier to reason about. It's easier to understand what's happening where. And I think that they should preferably be reusable. This is of course not a requirement, but it's a good thing to have in mind because it puts you in another mindset. So you start thinking about the things you're creating. So I tend to think a lot about this when I work with what I call modules. And I also think that they should be easily testable and they will be more easily testable just by creating these smaller, more focused modules instead of having your big application. So just by following the steps themselves, it should be easier to test your modules and your application in general. So there's still a lot of things we need to think about to get testable JavaScript because testable JavaScript is really hard. There's a lot of books written about it recently. There's a lot of presentations about it. There's a lot of people talking about it, but still it's quite hard to get there. We've been working quite hard with it for a couple of years and we're still struggling with really testing our JavaScript in a good way. And one of the core problems here is that we need to work with a DOM that is actually stateful and that is the big blob. So testing JavaScript is hard. Again, from growing object oriented software, they say that your system is easier to change if its objects are context independent. So that's precisely what we want to do with our modules. So the object shouldn't have any built-in knowledge about the system in which it executes. So now moving them around is far easier. If they don't depend on the DOM in the same way, it's easier to reuse them in another context. So you can move that search bar to a new page. It's now easier. So we can set down some ground rules for our modules. So just a couple of minor rules. And the first one is that they should never access the DOM outside the module. As soon as you do this, it's far easier to reason about your code because this module works on this part of the site. It doesn't suddenly work on a totally other place on the site. So this might seem like a natural thing to do, but it's so easy to start working on other places on the site. They also shouldn't create global variables. As soon as it creates global variables, someone else could work on those global variables and could put your system in the wrong state. So generally, try to keep away from global variables as much as possible. This is perhaps one of the most important things to think about when working with larger applications. Because the global variables are really hard to track what state they're in. And also don't access global variables, except, of course, the native built-in ones in JavaScript. So keeping these things in mind should really get you a long way. There's just a couple of simple steps, a couple of simple things to think about. So you have mvstar, just split, separate the DOM from your state or your data. And then you have modules. It's just a way to think about how you separate those things. So you create these small, bite-sized pieces of your application instead of having these large JavaScript files. So next up then is how we handle dependencies. This is also quite hard in JavaScript. We have a lot of different types of dependencies. We might depend on other code. So that's when the application runs. We actually need to have access to some objects. We need to have access to a place in the DOM where we should place our module. And there's a lot of dependencies you have in your code. You also tend to depend on other files. As soon as we start thinking about modules, you usually need to depend on something else. So you need to get that into your application in some way. And we also depend on third-party libraries. So things like jQuery, Backbone, Angular, and those things. So if we're just going to look at one of them at a time, so the first one is that we depend on other code. And the problem here is that we tend to depend far too much on global stuff because JavaScript makes this really easy. So here's an example of that. We have this small bit of jQuery. We find an element in the DOM. We click that element. We wait for a click on that element. And then we do an IAX request. And on success, we update the DOM. So just a minor piece of code. The problem here is that we have a bit of code that can't be reused. We have this anonymous function that is called directly. So this is really hard to reuse in another context. What if you need to fetch the same data? It's also difficult to test because you need to actually have the DOM available and you need to bind this click event and then click on the button. And that's really difficult. You don't need that at all. You shouldn't need that at all. And you also have a couple of globals. So here we're using jQuery. So jQuery.get. We do an IAX request. And we also update the DOM directly here. So by doing some changes here, we can really make this far better. So the first step could just be to name that function and take it out of the click handler. So now we have a method here that we can call directly. So now you don't have to create the DOM with the click handler, the click event. So now you can call the function directly, but you still have a couple of problems. You have this IAX request that is difficult to test. So now when you call this bit of code, it will do an IAX request. So you need to either muck out jQuery or you need to muck out all IAX requests. You shouldn't need to do that. You also still have the globals. You rely on having jQuery available in the global namespace. And you also depend on having the DOM. So we kind of got away from the DOM problem, but we also still have a DOM problem. So here we're just changing it up a little bit. Instead of relying on jQuery being available in the global namespace, we send it into the function. So we could have called in something else here, like just jQuery, when we sent it in. But here I called it IAX, so we can see IAX.get. So now if we want to test this, this is far easier. Now we can muck this out. We can send in something that has a get method that does just what we want to do. We have also sent in the HTML element where results should live. So now we can create an empty element with jQuery and send that in, for example. So you don't need to set up the entire DOM anymore. So now this is far easier to test, and this is actually far easier to reuse in other contexts. If you need to do the same IAX request and update another place in the DOM, that is really easy now. So the thing is that now we can control the dependencies. So we're still at a reasonably low level of abstraction here. So we could have had higher level abstractions for IAX and everything. We're still just working with jQuery itself. It's just that we're using jQuery a little bit differently than we usually do. So I wrote a blog post about this. I think it's about a year ago where I talked a little bit about a view abstraction based on backbone. So just using pure JavaScript instead of relying on backbone, which even though it's small, it's a thousand lines of code. So you could create, say that you have a view with just some subsection of your site, for example, the search bar that we saw on Facebook. So you could have a user view where you say that you want to receive, do I have the mouse here? Maybe? So you say here that you want to have an HTML element that you receive, and you also want to receive a user. So with user view now, we can, this is on the bottom here, we can say that we want to create an instance of this. So with JavaScript you can create an instance of your function, and then this will be for that instance. So we put the HTML element that we send in, which is here something we found in the DOM, and we also send in user and put that on the instance. So now we can create methods on this view. So we can create a showImageInstant method that just appends an image to the DOM. And now we can call that method. So the thing is now that we have another way to think about how we use jQuery. Now we send it in, now we have wrapped it in a reasonably simple abstraction. So this is no library at all, this is just a way to use pure JavaScript. And this is basically what, for example, Backbone. It's just a couple of helpers in addition to this. So what I would say is always inject your dependencies. This really helps with structure, it helps with testing, and it helps with creating larger applications. And an example of someone who has really embraced this is AngularJS. Here's an example of some AngularJS code. You have a controller, which is the thing that puts stuff, for example, on or makes stuff available for your DOM, for your HTML. So the thing that you put on scope here, which is something you get into your controller, will be available for your template. So here you can see that. So let's see if we have the mouse here. What you put on scope here will be available. So now we can do an IX request. And then on success, we update the scope, which then makes it available for the template. The thing is that it doesn't matter which way you order your arguments here. If you just change that around, Angular will still handle this with no problem. So they have a built-in service locator. So they see the function, they can see the name of the parameters, and then they go looking for that. So here it's built in functionality in AngularJS, the HTTP and the scope. But you can also register your own bits of code here. So to take an example from AngularJS, you could do something like this. You say that you want to create a module. Here it's named NDC, and it has no dependencies. That's the empty array. Then you can say that you want to create a function, which is named awesome. And that function should return awesome conference. And then you can create an injector. And in your injector, you say that you depend on NDC, which is your module, and you also depend on NG, which is a built-in Angular module. So this happens behind the scenes in AngularJS itself, but it's quite easy to do with the built-in stuff. So here we see that the NDC is just the NDC module. And now we can invoke a function on that injector. So here we see that we have a function, which takes in awesome, and then it console logs. So this will actually console log awesome conference. So Angular finds the name here, and it finds it in the things you have registered on the NDC module. Another cool thing that is going on in a lot of new libraries and frameworks is two-way binding. So this is a really cool way to work with the DOM. It's done in, amongst others, knockout, Angular, and Ember. So if you look at an example from EmberJS, we have a template at the top here, where it says if it's expanded, then it should show some data. And it should also show a button where you can press on to contract the input. And then if it is expanded, it should show a button where it says show more. So if you look at the post controller, which is the thing that gives the data, give the template its data, you can see that it's expanded defaults to being false. So the first time you render this template, it will go to showing the button that says show more. And on the show more button, you have this action where it say expand. So when you click on expand, click on this button, it calls the expand action, which is on the post controller. So here we see that we update is expanded to true. So now it will automatically update the template. So the thing is here, we have no jQuery at all, but we're still working with the DOM. So this is just another way to work with the DOM that is really, it's used quite a bit in the three libraries I mentioned. So you get rid of event handlers, you get rid of manual DOM updates, and you can ensure that your template is in sync with your data without having all of this jQuery hacking. So that was the first thing. Now we have another way to work on, work with other code. So now if we go on to look on other files, the problem now is that we depend on the order of script elements in our index HTML. And when you get hundreds of files, this is really a problem. It becomes unmaintainable, and it also becomes really brittle. And it's hard to know what needs to be there for something to function, especially when you get, say that you have 100 files, this is really hard. So we want to get away from this. We need something better. Because the thing is that we introduced lots of globals here. So we pull in jQuery, then we have jQuery as a global. We pull in Backbone, then we have Backbone on the global namespace. We have our own code that is on the global namespace. So you have a lot of weakly stated dependencies. It's hard to see what needs to be there for something else to function. So we need a better way to refer to other files. Two ways of doing this is commonJS and AMD. CommonJS is the way that Node.js solves this. It looks something like this. We say that we require jQuery, and we take the results from that require, and we put it on the dollar. And then we can also say that we export something from our current file. So here we export my function, so that when other require our file, that file, it has my function available. AMD is another way of doing this. Here you see that when you define this bit of functionality, you depend on jQuery. And it's also put in here into the dollar. And what you return from this function will be available when someone else say that they depend on this file. So now we can avoid globals. Now we can get away from having jQuery on the global namespace. And we don't need to use backbone from the global namespace. So now it's really easy to see dependencies. When you open a file, you can see it at the top of the file, what this bit of code depend on. And that makes your code, I would say, easier to understand. You can see right then and there what you depend on. And you can also see that you depend on too much stuff. When you have 50 things you depend on, that's usually a sign that you need to do something. So this is also a way to help you. And I'll split your code into new modules to split that file into something less than the 5,000 lines of code. Right now AMD works better in current browsers. You have some ways of running common JS, such as a library called Browserify. But right now AMD works a little bit better because it's asynchronous. So what happens here is that usually when you say that you depend on jQuery, the library can see have you loaded jQuery already? No, you haven't. Okay, then I go dynamically look for jQuery on the server. And you can also minify that into one file for production. For example, with something like RequireJS, there's a lot of implementations of AMD right now. I've used RequireJS quite a bit. And it works quite good. Of course there's a bit of complexity when you pull in something like this because this is a central part of your site, how you handle modules. So RequireJS has worked quite good for me. It also has a minifier. So you can have 200 files in development and you only want one file in production. There's a minifier there that you can run to get this one file. So now we have handled how we depend on other files. So now we can see that there isn't really a problem depending on 200 other files when you have something that can solve it for you. There's also stuff going on here for a new versions of JavaScript. So there will be some built-in modules stuff coming in a new version of JavaScript. So the last thing for dependencies is that we depend on third-party libraries. And we need to keep this up to date. Now you usually need to go to jQuery.com, see that, okay, jQuery is updated, and then you need to download it and everything. And it's also really hard to see, is it updated, without going to all of these pages to see. So you can use a package manager like Component or Bower. This is still, I don't know, quite... This is new stuff. This is happening right now. Twitter created Bower or released Bower. It's not a long time ago. So this is quite new and it might not be the perfect solution yet. And there's a lot of difficult problems to solve here. But with Bower, for example, you can do something like this. You say that in your Bower.json file, you can say that you depend on AngularJS, you depend on AngularResource, and you can see that we depend here on version 107. We can also say that we depend on jQuery. And now, when you type Bower install, it will go looking for those on GitHub. And the nice thing now is that if you type BowerList, you can see that you have downloaded Angular, AngularResource, which depend on Angular, and you can also see that jQuery is not up to date. So there are new versions available. And then you have 20, 30, 40 dependencies. This makes your code really easy. It makes it far easier to update dependencies regularly. So, for example, here, if we have written BowerUpdate, we could update these dependencies. And to save a new dependency into our Bower.json file, we can say Bower install, dash dash save bootstrap, for example, to download Twitter Bootstrap. So this is somewhat similar to NPM for those that I've played with Node.js. So, we've been through quite a lot of stuff. There's a lot of new stuff going on in JavaScript right now, and there has been for a couple of years, and there will happen a lot of stuff the next couple of years. But your question might be, seriously, do you need to do all this stuff? And I think it depends on the type of application you're making. So usually, we have applications that need to live for a long time. You have a lot of unforeseen requirements. You need to adapt to what's going on. So there's this big push for having even more interactive stuff on the front end. So we have more and more complex applications. And when we have more and more complex applications, we really need to think about how we develop our applications. So I absolutely think it's necessary to start thinking about these things and learn more about them. And not all of them are ready for production use if you're not ready to debug it and stuff. I've been debugging Bower somewhat lately. It doesn't run that good on Windows. So I think we need it, but we're still not quite there yet. For some of the things we are, so you should absolutely go check out Angular and Ember and Knockout and Backbone at least. There's a lot of cool stuff going on. So if you want to learn more about this, me and a couple of friends created this resource for finding good articles and videos on this stuff because we thought it was really difficult to find good stuff on JavaScript. So if you have any good ideas for articles, videos, presentations, whatever, just send us an email because we really want to have a really good resource for people to find good stuff for creating large JavaScript applications. So that's all. I've put the presentation online so you can find it at Speaker Deck. So any questions? And if you don't have any questions right now, you can also find me afterwards. I will also be here on both tomorrow and on Friday, so you can just come by and talk about JavaScript. So if there are no questions, I think we're done. Thank you.
 Yes, should we get started?  So today we're going to talk a little bit about patterns of large scale JavaScript applications.  There's been a lot of cool stuff going on these last few years.  So I want to talk a little bit about some of the things that I've experienced and learned.  Hello, my name is Kim. I work at Beck Consulting here in Oslo.  And you can find more about me here.  So we're going to talk about large JavaScript applications.  But people tend to think that I'm talking about really large JavaScript applications when I say this.  20,000 lines of code or more.  And it doesn't really have to be that way.  All of these things that I'm talking about also the same concept apply for smaller applications.  They can be a thousand lines of code. They can be a couple of thousand lines of code.  And a lot of the patterns are still important to think about.  So the main thing usually is that these are applications that need to live for a long time.  And also applications that have unforeseen requirements.  So most of this will apply to all of the JavaScript apps we write, at least for work.  So the problem with JavaScript is that we tend to start out with something like this.  It's a small feature. We send all the HTML from the server to the browser.  And then we start creating a couple of small features.  Five lines, 10 lines, 20 lines of code.  And as the time goes by, we need more and more functionality.  And then we suddenly have 5,000 lines of code in one file.  And this happens all the time.  I've been on several projects where this happened and it happens to everyone.  I think this is a good term for it, jQuery soup.  We get this brittle, tangled mess.  And it's really hard to understand and hard to code in those apps.  And maybe the worst thing is that it's hard to reason about the code.  So that makes it difficult to create new features.  So if you go back a couple of years to around 2006, we had an obtrusive JavaScript.  We sent HTML to the browser and we used JavaScript to obtrusively add functionality.  So it worked without JavaScript and we added functionality with JavaScript.  And this year we also got jQuery.  So the browsers back then was IE6.  IE7 came out in 2006.  We had Firefox 2.  And the state of the web was different back then.  So JavaScript was an amazing library.  We really needed jQuery.  The thing is that it's an amazing library for DOM manipulation.  But we tend to follow a lot of anti-patterns when our apps grow.  So we have things like coupling ourselves too much to the DOM.  So when you change the HTML, you actually end up breaking your JavaScript.  You put everything inside document ready.  So whenever the code loads, it starts the entire thing and you have no way of testing it.  And it's also hard to reuse.  And also everything tends to end up in a jQuery plugin.  And also all the things that don't have anything to do with jQuery.  And maybe the worst thing is that too often the business logic is just hidden inside this brittle, tangled mess of a code base.  So if we just look at a reasonably simple example, we have a form here.  And when you submit the form, we do an IEX request.  And on success on that request, we actually receive some...  We get the data and we put it into the DOM.  But the thing is that right here there's a lot of stuff going on.  We have page events and user events and network IEO.  We have network events.  We do templating, parsing of the response and a lot of stuff.  And we also get callback L.  We got these structures that just go in and in and in.  So there's callbacks inside, callbacks inside, callbacks.  And what happens then when we have this considerable increase in the amount of JavaScript?  We have a lot of more interactive pages than before.  And there's a lot of complexity just in the user interfaces.  We could have something like the new Google Maps.  So here we require JavaScript for anything to function at all.  So we've come a long way.  And what has happened is that we're pushing more and more stuff from the back end to the front end.  So a lot of the stuff that we before did on the back end,  we now do dynamically with JavaScript on the front end.  And also we've got a lot of new capabilities in the browsers.  There's still a browser war.  There's still really pushing for new stuff into the browser.  So you have geolocation, local storage, canvas, and a lot of other stuff.  So this means that there's more and more complexity in your front end application.  So the problem is no longer the browser itself and the capabilities you have there.  It's the size and complexity of our applications.  I saw a great talk about a year ago from Brandon Keepers who works at GitHub.  He said that the litmus test for your application is that if your site breaks,  when your JavaScript fails, you should treat it like a real language.  And this is mainly what we're going to focus on,  what it means to treat JavaScript as a real language.  Because what we want is testable, scalable, and maintainable JavaScript.  This is where we want to be.  The problem, though, is that this is really difficult.  It takes a lot of time to learn.  I've been doing this full time for a couple of years,  and we've had a really hard time to get there.  We're starting to get there.  So the problem is that we need to unlearn a lot of stuff,  because we've coded JavaScript for 10 years, a lot of us,  and we're used to doing it the jQuery way.  And it's so easy to create brittle systems when we're using jQuery.  But the first thing is that we need to unlearn our DOM-centric approach to creating websites.  So the problem with the DOM is that it's this big blob of state.  So you have this stateful and global object you're working with all the time.  So this makes it really hard to maintain your codebase  when your application grows, because you're working on this one blob of state.  And it's also really hard to test.  And debugging some of these applications can also be really hard,  because there's usually a lot of stuff going on,  and it depends on the state you're currently in.  We also need to unlearn how we write JavaScript.  We need to learn from how they do it on the back end,  and we need to get away from 5,000 lines of code in a single file.  We also need to watch out for globals,  not just the DOM itself, but also globals in general.  So basically, learn more from the back end.  And we need to unlearn how we use jQuery.  This is a great quote from growing object-oriented software guided by tests,  which is an amazing book.  They say that when your code is difficult to test,  it's most likely because you need to improve your design.  Justin Meyer, who created a JavaScript MVC library, said this,  is that to build large applications, you need to never build large applications.  You need to build small pieces and assemble those into your application.  So we need to get away from these big files.  And these last couple of years, we've started to get some help here.  We've got a lot of new libraries, so we have things like knockouts and backbone,  which are reasonably small.  We also have things like Ember and AngularJS,  which is more full-fledged frameworks.  So they're trying to help solve the problem,  and they have quite different philosophies,  so we're going to try to understand some of the things that they do.  Because right now, it's easy to feel overwhelmed by choice.  It's a daunting task just to be able to choose one of them.  I would have struggled with that if I was starting a new project right now.  So the thing is that we need a foundation to discuss the features they have  and how they do things, and also to understand the value they add  and how you can solve it yourself without using a library or framework at all.  So the most important thing is to get the state away from the DOM.  This is the same for all of those libraries.  They all want you to get the state away from the DOM.  And one way of doing this is NBC.  This is what most of those libraries are inspired by.  So if we just look at these last few years, we have from 2005 here,  and this is a Google search for JavaScript NBC, or the Google Trends.  And we see that this is really growing.  So the main thing here about NBC, I guess a lot of you are back-end developers,  so you know what NBC is.  But the main point here is that you want this clean separation between your data and the DOM.  So the controller here is the thing that mediates input and manipulates the model.  However, you won't see that cleanly applied in a lot of front-end JavaScript frameworks and libraries.  There's a lot of variation there, so you often see this instead, ModelViewStar.  And this is just because a lot of them has a different take on what that last part is.  So the thing is you want to split your data and the DOM, and how you do it,  I think you can call it that, is quite different from library to library.  So you can have something like a ModelView presenter where all the presentation logic is pushed into the presenter.  So now the view doesn't know about the model at all.  It's the presenter that gives the view the data.  You can have something like ModelViewViewModel, where the view model is the one that exposes data to the view.  So this is a quote from one of the core developers in AngularJS.  He says that AngularJS is a ModelViewWhatever framework, where whatever stands for whatever works for you.  They started out being mostly MVC and has gradually become more NVVM.  But the thing is they also say this very clearly that it is ModelViewWhatever.  So the most important thing here is separating the state and the DOM.  So this is the first step.  We've already gotten somewhere.  So just by thinking about this, it should be easier to work with JavaScript.  MVStar is not enough, though.  Just having this pattern to follow still doesn't say a lot about how your code should look like and how it should work.  And a lot of things that you need to think about.  For example, here, Joel Hooks wrote a great blog post about a month ago, where he talked about his experiences with AngularJS.  And what he says is that when he started on this project, there was a lot of monolithic single files that contained too much code.  So just following MVStar itself is not enough.  So he talked a lot about the importance of code organization and how important that was for actually getting an application that was easy to work with  and where you can add features rapidly.  And this is also a great quote by Jim Levin, who says that,  just how you split your code has an impact on the maintainability of the application.  So really what we want to do is have far more files and far less in the file size.  So we want to get away from these 5,000 lines of code in a single file.  It's better to have more files that are smaller, easier to work with.  So you want more focused files.  There's not enough just to split up.  You need to think about how you do it.  So there's two ways, for example, there are two ways we can think about this.  We could package our code by feature or we could package it by layer.  So if we look at this, is it large enough?  So for example, if you package your code by feature, you could have a module like this, a module folder like this, where you have,  you can see the high level scope of the application.  So here we're working with BankID, we have some customer information and some other stuff.  Or you could have package it by layer, so you have a modules, models folder, a collections folder, a views folder, a routers folder.  So this is from a backbone project.  So I prefer packaging by feature and I usually tend to call them modules.  That was also the module name on the folder they were in.  So a module is something like this.  So here we see just the search bar on Facebook.  That can be a module.  So it's a bite-sized piece of your application.  It could also be something like this, where we input the status.  So it's still a small feature on Facebook itself.  Or it could be something like this on the right side.  So modules are the things that contain business logic.  So this is mainly where you want to have all your business logic and all of the stuff that is special about your application.  All the other stuff is usually mostly set up and reusable functions.  So the modules should be small.  They should try to solve one problem and should really try to relentlessly split into modules.  Of course you shouldn't go for like 10 lines and everything, but you should really focus on what feature are we working on here.  And try to separate them from the rest of the site.  So this should be focused and they should really do one thing really well.  I also think that modules should be decoupled from other modules.  So you don't need absolute separation, but they really should be separated from each other.  So you can place it in a new context.  You can move it around on the site and it doesn't break anything.  That's not easy in a lot of jQuery apps.  If you move it around, you change the DOM structure so your application breaks.  At least it's more difficult to do this in a jQuery application.  And also it should, just doing this should make your app easier to reason about.  It's easier to understand what's happening where.  And I think that they should preferably be reusable.  This is of course not a requirement, but it's a good thing to have in mind because it puts you in another mindset.  So you start thinking about the things you're creating.  So I tend to think a lot about this when I work with what I call modules.  And I also think that they should be easily testable and they will be more easily testable just by creating these smaller, more focused modules instead of having your big application.  So just by following the steps themselves, it should be easier to test your modules and your application in general.  So there's still a lot of things we need to think about to get testable JavaScript because testable JavaScript is really hard.  There's a lot of books written about it recently.  There's a lot of presentations about it.  There's a lot of people talking about it, but still it's quite hard to get there.  We've been working quite hard with it for a couple of years and we're still struggling with really testing our JavaScript in a good way.  And one of the core problems here is that we need to work with a DOM that is actually stateful and that is the big blob.  So testing JavaScript is hard.  Again, from growing object oriented software, they say that your system is easier to change if its objects are context independent.  So that's precisely what we want to do with our modules.  So the object shouldn't have any built-in knowledge about the system in which it executes.  So now moving them around is far easier.  If they don't depend on the DOM in the same way, it's easier to reuse them in another context.  So you can move that search bar to a new page.  It's now easier.  So we can set down some ground rules for our modules.  So just a couple of minor rules.  And the first one is that they should never access the DOM outside the module.  As soon as you do this, it's far easier to reason about your code because this module works on this part of the site.  It doesn't suddenly work on a totally other place on the site.  So this might seem like a natural thing to do, but it's so easy to start working on other places on the site.  They also shouldn't create global variables.  As soon as it creates global variables, someone else could work on those global variables and could put your system in the wrong state.  So generally, try to keep away from global variables as much as possible.  This is perhaps one of the most important things to think about when working with larger applications.  Because the global variables are really hard to track what state they're in.  And also don't access global variables, except, of course, the native built-in ones in JavaScript.  So keeping these things in mind should really get you a long way.  There's just a couple of simple steps, a couple of simple things to think about.  So you have mvstar, just split, separate the DOM from your state or your data.  And then you have modules.  It's just a way to think about how you separate those things.  So you create these small, bite-sized pieces of your application instead of having these large JavaScript files.  So next up then is how we handle dependencies.  This is also quite hard in JavaScript.  We have a lot of different types of dependencies.  We might depend on other code.  So that's when the application runs.  We actually need to have access to some objects.  We need to have access to a place in the DOM where we should place our module.  And there's a lot of dependencies you have in your code.  You also tend to depend on other files.  As soon as we start thinking about modules, you usually need to depend on something else.  So you need to get that into your application in some way.  And we also depend on third-party libraries.  So things like jQuery, Backbone, Angular, and those things.  So if we're just going to look at one of them at a time,  so the first one is that we depend on other code.  And the problem here is that we tend to depend far too much on global stuff  because JavaScript makes this really easy.  So here's an example of that.  We have this small bit of jQuery.  We find an element in the DOM.  We click that element.  We wait for a click on that element.  And then we do an IAX request.  And on success, we update the DOM.  So just a minor piece of code.  The problem here is that we have a bit of code that can't be reused.  We have this anonymous function that is called directly.  So this is really hard to reuse in another context.  What if you need to fetch the same data?  It's also difficult to test because you need to actually have the DOM available  and you need to bind this click event and then click on the button.  And that's really difficult.  You don't need that at all.  You shouldn't need that at all.  And you also have a couple of globals.  So here we're using jQuery.  So jQuery.get.  We do an IAX request.  And we also update the DOM directly here.  So by doing some changes here, we can really make this far better.  So the first step could just be to name that function  and take it out of the click handler.  So now we have a method here that we can call directly.  So now you don't have to create the DOM with the click handler, the click event.  So now you can call the function directly, but you still have a couple of problems.  You have this IAX request that is difficult to test.  So now when you call this bit of code, it will do an IAX request.  So you need to either muck out jQuery or you need to muck out all IAX requests.  You shouldn't need to do that.  You also still have the globals.  You rely on having jQuery available in the global namespace.  And you also depend on having the DOM.  So we kind of got away from the DOM problem, but we also still have a DOM problem.  So here we're just changing it up a little bit.  Instead of relying on jQuery being available in the global namespace,  we send it into the function.  So we could have called in something else here, like just jQuery, when we sent it in.  But here I called it IAX, so we can see IAX.get.  So now if we want to test this, this is far easier.  Now we can muck this out.  We can send in something that has a get method that does just what we want to do.  We have also sent in the HTML element where results should live.  So now we can create an empty element with jQuery and send that in, for example.  So you don't need to set up the entire DOM anymore.  So now this is far easier to test, and this is actually far easier to reuse in other contexts.  If you need to do the same IAX request and update another place in the DOM,  that is really easy now.  So the thing is that now we can control the dependencies.  So we're still at a reasonably low level of abstraction here.  So we could have had higher level abstractions for IAX and everything.  We're still just working with jQuery itself.  It's just that we're using jQuery a little bit differently than we usually do.  So I wrote a blog post about this.  I think it's about a year ago where I talked a little bit about a view abstraction based on backbone.  So just using pure JavaScript instead of relying on backbone,  which even though it's small, it's a thousand lines of code.  So you could create, say that you have a view with just some subsection of your site,  for example, the search bar that we saw on Facebook.  So you could have a user view where you say that you want to receive, do I have the mouse here?  Maybe?  So you say here that you want to have an HTML element that you receive,  and you also want to receive a user.  So with user view now, we can, this is on the bottom here,  we can say that we want to create an instance of this.  So with JavaScript you can create an instance of your function,  and then this will be for that instance.  So we put the HTML element that we send in,  which is here something we found in the DOM,  and we also send in user and put that on the instance.  So now we can create methods on this view.  So we can create a showImageInstant method that just appends an image to the DOM.  And now we can call that method.  So the thing is now that we have another way to think about how we use jQuery.  Now we send it in, now we have wrapped it in a reasonably simple abstraction.  So this is no library at all, this is just a way to use pure JavaScript.  And this is basically what, for example, Backbone.  It's just a couple of helpers in addition to this.  So what I would say is always inject your dependencies.  This really helps with structure, it helps with testing,  and it helps with creating larger applications.  And an example of someone who has really embraced this is AngularJS.  Here's an example of some AngularJS code.  You have a controller, which is the thing that puts stuff, for example,  on or makes stuff available for your DOM, for your HTML.  So the thing that you put on scope here, which is something you get into your controller,  will be available for your template.  So here you can see that.  So let's see if we have the mouse here.  What you put on scope here will be available.  So now we can do an IX request.  And then on success, we update the scope, which then makes it available for the template.  The thing is that it doesn't matter which way you order your arguments here.  If you just change that around, Angular will still handle this with no problem.  So they have a built-in service locator.  So they see the function, they can see the name of the parameters,  and then they go looking for that.  So here it's built in functionality in AngularJS, the HTTP and the scope.  But you can also register your own bits of code here.  So to take an example from AngularJS, you could do something like this.  You say that you want to create a module.  Here it's named NDC, and it has no dependencies.  That's the empty array.  Then you can say that you want to create a function, which is named awesome.  And that function should return awesome conference.  And then you can create an injector.  And in your injector, you say that you depend on NDC, which is your module,  and you also depend on NG, which is a built-in Angular module.  So this happens behind the scenes in AngularJS itself,  but it's quite easy to do with the built-in stuff.  So here we see that the NDC is just the NDC module.  And now we can invoke a function on that injector.  So here we see that we have a function, which takes in awesome, and then it console logs.  So this will actually console log awesome conference.  So Angular finds the name here, and it finds it in the things you have registered on the NDC module.  Another cool thing that is going on in a lot of new libraries and frameworks is two-way binding.  So this is a really cool way to work with the DOM.  It's done in, amongst others, knockout, Angular, and Ember.  So if you look at an example from EmberJS, we have a template at the top here,  where it says if it's expanded, then it should show some data.  And it should also show a button where you can press on to contract the input.  And then if it is expanded, it should show a button where it says show more.  So if you look at the post controller, which is the thing that gives the data,  give the template its data, you can see that it's expanded defaults to being false.  So the first time you render this template, it will go to showing the button that says show more.  And on the show more button, you have this action where it say expand.  So when you click on expand, click on this button, it calls the expand action,  which is on the post controller.  So here we see that we update is expanded to true.  So now it will automatically update the template.  So the thing is here, we have no jQuery at all, but we're still working with the DOM.  So this is just another way to work with the DOM that is really,  it's used quite a bit in the three libraries I mentioned.  So you get rid of event handlers, you get rid of manual DOM updates,  and you can ensure that your template is in sync with your data  without having all of this jQuery hacking.  So that was the first thing.  Now we have another way to work on, work with other code.  So now if we go on to look on other files,  the problem now is that we depend on the order of script elements in our index HTML.  And when you get hundreds of files, this is really a problem.  It becomes unmaintainable, and it also becomes really brittle.  And it's hard to know what needs to be there for something to function,  especially when you get, say that you have 100 files, this is really hard.  So we want to get away from this.  We need something better.  Because the thing is that we introduced lots of globals here.  So we pull in jQuery, then we have jQuery as a global.  We pull in Backbone, then we have Backbone on the global namespace.  We have our own code that is on the global namespace.  So you have a lot of weakly stated dependencies.  It's hard to see what needs to be there for something else to function.  So we need a better way to refer to other files.  Two ways of doing this is commonJS and AMD.  CommonJS is the way that Node.js solves this.  It looks something like this.  We say that we require jQuery, and we take the results from that require,  and we put it on the dollar.  And then we can also say that we export something from our current file.  So here we export my function, so that when other require our file, that file,  it has my function available.  AMD is another way of doing this.  Here you see that when you define this bit of functionality, you depend on jQuery.  And it's also put in here into the dollar.  And what you return from this function will be available when someone else  say that they depend on this file.  So now we can avoid globals.  Now we can get away from having jQuery on the global namespace.  And we don't need to use backbone from the global namespace.  So now it's really easy to see dependencies.  When you open a file, you can see it at the top of the file, what this bit of code depend on.  And that makes your code, I would say, easier to understand.  You can see right then and there what you depend on.  And you can also see that you depend on too much stuff.  When you have 50 things you depend on, that's usually a sign that you need to do something.  So this is also a way to help you.  And I'll split your code into new modules to split that file into something less than the 5,000 lines of code.  Right now AMD works better in current browsers.  You have some ways of running common JS, such as a library called Browserify.  But right now AMD works a little bit better because it's asynchronous.  So what happens here is that usually when you say that you depend on jQuery,  the library can see have you loaded jQuery already?  No, you haven't. Okay, then I go dynamically look for jQuery on the server.  And you can also minify that into one file for production.  For example, with something like RequireJS, there's a lot of implementations of AMD right now.  I've used RequireJS quite a bit.  And it works quite good.  Of course there's a bit of complexity when you pull in something like this  because this is a central part of your site, how you handle modules.  So RequireJS has worked quite good for me.  It also has a minifier.  So you can have 200 files in development and you only want one file in production.  There's a minifier there that you can run to get this one file.  So now we have handled how we depend on other files.  So now we can see that there isn't really a problem depending on 200 other files  when you have something that can solve it for you.  There's also stuff going on here for a new versions of JavaScript.  So there will be some built-in modules stuff coming in a new version of JavaScript.  So the last thing for dependencies is that we depend on third-party libraries.  And we need to keep this up to date.  Now you usually need to go to jQuery.com, see that, okay, jQuery is updated,  and then you need to download it and everything.  And it's also really hard to see, is it updated, without going to all of these pages to see.  So you can use a package manager like Component or Bower.  This is still, I don't know, quite...  This is new stuff. This is happening right now.  Twitter created Bower or released Bower.  It's not a long time ago.  So this is quite new and it might not be the perfect solution yet.  And there's a lot of difficult problems to solve here.  But with Bower, for example, you can do something like this.  You say that in your Bower.json file, you can say that you depend on AngularJS,  you depend on AngularResource, and you can see that we depend here on version 107.  We can also say that we depend on jQuery.  And now, when you type Bower install, it will go looking for those on GitHub.  And the nice thing now is that if you type BowerList, you can see that you have downloaded Angular,  AngularResource, which depend on Angular, and you can also see that jQuery is not up to date.  So there are new versions available.  And then you have 20, 30, 40 dependencies. This makes your code really easy.  It makes it far easier to update dependencies regularly.  So, for example, here, if we have written BowerUpdate, we could update these dependencies.  And to save a new dependency into our Bower.json file, we can say Bower install,  dash dash save bootstrap, for example, to download Twitter Bootstrap.  So this is somewhat similar to NPM for those that I've played with Node.js.  So, we've been through quite a lot of stuff.  There's a lot of new stuff going on in JavaScript right now, and there has been for a couple of years,  and there will happen a lot of stuff the next couple of years.  But your question might be, seriously, do you need to do all this stuff?  And I think it depends on the type of application you're making.  So usually, we have applications that need to live for a long time.  You have a lot of unforeseen requirements.  You need to adapt to what's going on.  So there's this big push for having even more interactive stuff on the front end.  So we have more and more complex applications.  And when we have more and more complex applications, we really need to think about how we develop our applications.  So I absolutely think it's necessary to start thinking about these things and learn more about them.  And not all of them are ready for production use if you're not ready to debug it and stuff.  I've been debugging Bower somewhat lately. It doesn't run that good on Windows.  So I think we need it, but we're still not quite there yet.  For some of the things we are, so you should absolutely go check out Angular and Ember and Knockout and Backbone at least.  There's a lot of cool stuff going on.  So if you want to learn more about this, me and a couple of friends created this resource for finding good articles and videos on this stuff  because we thought it was really difficult to find good stuff on JavaScript.  So if you have any good ideas for articles, videos, presentations, whatever, just send us an email  because we really want to have a really good resource for people to find good stuff for creating large JavaScript applications.  So that's all. I've put the presentation online so you can find it at Speaker Deck.  So any questions?  And if you don't have any questions right now, you can also find me afterwards.  I will also be here on both tomorrow and on Friday, so you can just come by and talk about JavaScript.  So if there are no questions, I think we're done. Thank you.
Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_0_59.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_60_107.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_108_683.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_684_2819.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_2820_3107.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_3108_3599.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_3600_4499.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_4500_5951.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_5952_7283.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_7284_8579.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_8580_9167.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_9168_9335.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_9336_9767.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_9768_10103.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_10104_11135.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_11136_11675.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_11676_12131.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_12132_13379.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_13380_14051.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_14052_16943.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_16944_17543.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_17544_17675.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_17676_17987.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_17988_18347.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_18348_18623.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_18624_21407.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_21408_21683.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_21684_22355.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_22356_26591.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_26592_27611.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_27612_29255.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_29256_31679.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_31680_31763.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_31764_37007.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_37008_37415.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_37416_46067.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_46068_46751.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_46752_58955.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_58956_59675.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_59676_60491.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_60492_60539.png Kim_Joar_Bekkelund_-_Patterns_of_large-scale_JavaScript_applications-68236481_60540_60585.png
Yes, should we get started? So today we're going to talk a little bit about patterns of large scale JavaScript applications. There's been a lot of cool stuff going on these last few years. So I want to talk a little bit about some of the things that I've experienced and learned. Hello, my name is Kim. I work at Beck Consulting here in Oslo. And you can find more about me here. So we're going to talk about large JavaScript applications. But people tend to think that I'm talking about really large JavaScript applications when I say this. 20,000 lines of code or more. And it doesn't really have to be that way. All of these things that I'm talking about also the same concept apply for smaller applications. They can be a thousand lines of code. They can be a couple of thousand lines of code. And a lot of the patterns are still important to think about. So the main thing usually is that these are applications that need to live for a long time. And also applications that have unforeseen requirements. So most of this will apply to all of the JavaScript apps we write, at least for work. So the problem with JavaScript is that we tend to start out with something like this. It's a small feature. We send all the HTML from the server to the browser. And then we start creating a couple of small features. Five lines, 10 lines, 20 lines of code. And as the time goes by, we need more and more functionality. And then we suddenly have 5,000 lines of code in one file. And this happens all the time. I've been on several projects where this happened and it happens to everyone. I think this is a good term for it, jQuery soup. We get this brittle, tangled mess. And it's really hard to understand and hard to code in those apps. And maybe the worst thing is that it's hard to reason about the code. So that makes it difficult to create new features. So if you go back a couple of years to around 2006, we had an obtrusive JavaScript. We sent HTML to the browser and we used JavaScript to obtrusively add functionality. So it worked without JavaScript and we added functionality with JavaScript. And this year we also got jQuery. So the browsers back then was IE6. IE7 came out in 2006. We had Firefox 2. And the state of the web was different back then. So JavaScript was an amazing library. We really needed jQuery. The thing is that it's an amazing library for DOM manipulation. But we tend to follow a lot of anti-patterns when our apps grow. So we have things like coupling ourselves too much to the DOM. So when you change the HTML, you actually end up breaking your JavaScript. You put everything inside document ready. So whenever the code loads, it starts the entire thing and you have no way of testing it. And it's also hard to reuse. And also everything tends to end up in a jQuery plugin. And also all the things that don't have anything to do with jQuery. And maybe the worst thing is that too often the business logic is just hidden inside this brittle, tangled mess of a code base. So if we just look at a reasonably simple example, we have a form here. And when you submit the form, we do an IEX request. And on success on that request, we actually receive some... We get the data and we put it into the DOM. But the thing is that right here there's a lot of stuff going on. We have page events and user events and network IEO. We have network events. We do templating, parsing of the response and a lot of stuff. And we also get callback L. We got these structures that just go in and in and in. So there's callbacks inside, callbacks inside, callbacks. And what happens then when we have this considerable increase in the amount of JavaScript? We have a lot of more interactive pages than before. And there's a lot of complexity just in the user interfaces. We could have something like the new Google Maps. So here we require JavaScript for anything to function at all. So we've come a long way. And what has happened is that we're pushing more and more stuff from the back end to the front end. So a lot of the stuff that we before did on the back end, we now do dynamically with JavaScript on the front end. And also we've got a lot of new capabilities in the browsers. There's still a browser war. There's still really pushing for new stuff into the browser. So you have geolocation, local storage, canvas, and a lot of other stuff. So this means that there's more and more complexity in your front end application. So the problem is no longer the browser itself and the capabilities you have there. It's the size and complexity of our applications. I saw a great talk about a year ago from Brandon Keepers who works at GitHub. He said that the litmus test for your application is that if your site breaks, when your JavaScript fails, you should treat it like a real language. And this is mainly what we're going to focus on, what it means to treat JavaScript as a real language. Because what we want is testable, scalable, and maintainable JavaScript. This is where we want to be. The problem, though, is that this is really difficult. It takes a lot of time to learn. I've been doing this full time for a couple of years, and we've had a really hard time to get there. We're starting to get there. So the problem is that we need to unlearn a lot of stuff, because we've coded JavaScript for 10 years, a lot of us, and we're used to doing it the jQuery way. And it's so easy to create brittle systems when we're using jQuery. But the first thing is that we need to unlearn our DOM-centric approach to creating websites. So the problem with the DOM is that it's this big blob of state. So you have this stateful and global object you're working with all the time. So this makes it really hard to maintain your codebase when your application grows, because you're working on this one blob of state. And it's also really hard to test. And debugging some of these applications can also be really hard, because there's usually a lot of stuff going on, and it depends on the state you're currently in. We also need to unlearn how we write JavaScript. We need to learn from how they do it on the back end, and we need to get away from 5,000 lines of code in a single file. We also need to watch out for globals, not just the DOM itself, but also globals in general. So basically, learn more from the back end. And we need to unlearn how we use jQuery. This is a great quote from growing object-oriented software guided by tests, which is an amazing book. They say that when your code is difficult to test, it's most likely because you need to improve your design. Justin Meyer, who created a JavaScript MVC library, said this, is that to build large applications, you need to never build large applications. You need to build small pieces and assemble those into your application. So we need to get away from these big files. And these last couple of years, we've started to get some help here. We've got a lot of new libraries, so we have things like knockouts and backbone, which are reasonably small. We also have things like Ember and AngularJS, which is more full-fledged frameworks. So they're trying to help solve the problem, and they have quite different philosophies, so we're going to try to understand some of the things that they do. Because right now, it's easy to feel overwhelmed by choice. It's a daunting task just to be able to choose one of them. I would have struggled with that if I was starting a new project right now. So the thing is that we need a foundation to discuss the features they have and how they do things, and also to understand the value they add and how you can solve it yourself without using a library or framework at all. So the most important thing is to get the state away from the DOM. This is the same for all of those libraries. They all want you to get the state away from the DOM. And one way of doing this is NBC. This is what most of those libraries are inspired by. So if we just look at these last few years, we have from 2005 here, and this is a Google search for JavaScript NBC, or the Google Trends. And we see that this is really growing. So the main thing here about NBC, I guess a lot of you are back-end developers, so you know what NBC is. But the main point here is that you want this clean separation between your data and the DOM. So the controller here is the thing that mediates input and manipulates the model. However, you won't see that cleanly applied in a lot of front-end JavaScript frameworks and libraries. There's a lot of variation there, so you often see this instead, ModelViewStar. And this is just because a lot of them has a different take on what that last part is. So the thing is you want to split your data and the DOM, and how you do it, I think you can call it that, is quite different from library to library. So you can have something like a ModelView presenter where all the presentation logic is pushed into the presenter. So now the view doesn't know about the model at all. It's the presenter that gives the view the data. You can have something like ModelViewViewModel, where the view model is the one that exposes data to the view. So this is a quote from one of the core developers in AngularJS. He says that AngularJS is a ModelViewWhatever framework, where whatever stands for whatever works for you. They started out being mostly MVC and has gradually become more NVVM. But the thing is they also say this very clearly that it is ModelViewWhatever. So the most important thing here is separating the state and the DOM. So this is the first step. We've already gotten somewhere. So just by thinking about this, it should be easier to work with JavaScript. MVStar is not enough, though. Just having this pattern to follow still doesn't say a lot about how your code should look like and how it should work. And a lot of things that you need to think about. For example, here, Joel Hooks wrote a great blog post about a month ago, where he talked about his experiences with AngularJS. And what he says is that when he started on this project, there was a lot of monolithic single files that contained too much code. So just following MVStar itself is not enough. So he talked a lot about the importance of code organization and how important that was for actually getting an application that was easy to work with and where you can add features rapidly. And this is also a great quote by Jim Levin, who says that, just how you split your code has an impact on the maintainability of the application. So really what we want to do is have far more files and far less in the file size. So we want to get away from these 5,000 lines of code in a single file. It's better to have more files that are smaller, easier to work with. So you want more focused files. There's not enough just to split up. You need to think about how you do it. So there's two ways, for example, there are two ways we can think about this. We could package our code by feature or we could package it by layer. So if we look at this, is it large enough? So for example, if you package your code by feature, you could have a module like this, a module folder like this, where you have, you can see the high level scope of the application. So here we're working with BankID, we have some customer information and some other stuff. Or you could have package it by layer, so you have a modules, models folder, a collections folder, a views folder, a routers folder. So this is from a backbone project. So I prefer packaging by feature and I usually tend to call them modules. That was also the module name on the folder they were in. So a module is something like this. So here we see just the search bar on Facebook. That can be a module. So it's a bite-sized piece of your application. It could also be something like this, where we input the status. So it's still a small feature on Facebook itself. Or it could be something like this on the right side. So modules are the things that contain business logic. So this is mainly where you want to have all your business logic and all of the stuff that is special about your application. All the other stuff is usually mostly set up and reusable functions. So the modules should be small. They should try to solve one problem and should really try to relentlessly split into modules. Of course you shouldn't go for like 10 lines and everything, but you should really focus on what feature are we working on here. And try to separate them from the rest of the site. So this should be focused and they should really do one thing really well. I also think that modules should be decoupled from other modules. So you don't need absolute separation, but they really should be separated from each other. So you can place it in a new context. You can move it around on the site and it doesn't break anything. That's not easy in a lot of jQuery apps. If you move it around, you change the DOM structure so your application breaks. At least it's more difficult to do this in a jQuery application. And also it should, just doing this should make your app easier to reason about. It's easier to understand what's happening where. And I think that they should preferably be reusable. This is of course not a requirement, but it's a good thing to have in mind because it puts you in another mindset. So you start thinking about the things you're creating. So I tend to think a lot about this when I work with what I call modules. And I also think that they should be easily testable and they will be more easily testable just by creating these smaller, more focused modules instead of having your big application. So just by following the steps themselves, it should be easier to test your modules and your application in general. So there's still a lot of things we need to think about to get testable JavaScript because testable JavaScript is really hard. There's a lot of books written about it recently. There's a lot of presentations about it. There's a lot of people talking about it, but still it's quite hard to get there. We've been working quite hard with it for a couple of years and we're still struggling with really testing our JavaScript in a good way. And one of the core problems here is that we need to work with a DOM that is actually stateful and that is the big blob. So testing JavaScript is hard. Again, from growing object oriented software, they say that your system is easier to change if its objects are context independent. So that's precisely what we want to do with our modules. So the object shouldn't have any built-in knowledge about the system in which it executes. So now moving them around is far easier. If they don't depend on the DOM in the same way, it's easier to reuse them in another context. So you can move that search bar to a new page. It's now easier. So we can set down some ground rules for our modules. So just a couple of minor rules. And the first one is that they should never access the DOM outside the module. As soon as you do this, it's far easier to reason about your code because this module works on this part of the site. It doesn't suddenly work on a totally other place on the site. So this might seem like a natural thing to do, but it's so easy to start working on other places on the site. They also shouldn't create global variables. As soon as it creates global variables, someone else could work on those global variables and could put your system in the wrong state. So generally, try to keep away from global variables as much as possible. This is perhaps one of the most important things to think about when working with larger applications. Because the global variables are really hard to track what state they're in. And also don't access global variables, except, of course, the native built-in ones in JavaScript. So keeping these things in mind should really get you a long way. There's just a couple of simple steps, a couple of simple things to think about. So you have mvstar, just split, separate the DOM from your state or your data. And then you have modules. It's just a way to think about how you separate those things. So you create these small, bite-sized pieces of your application instead of having these large JavaScript files. So next up then is how we handle dependencies. This is also quite hard in JavaScript. We have a lot of different types of dependencies. We might depend on other code. So that's when the application runs. We actually need to have access to some objects. We need to have access to a place in the DOM where we should place our module. And there's a lot of dependencies you have in your code. You also tend to depend on other files. As soon as we start thinking about modules, you usually need to depend on something else. So you need to get that into your application in some way. And we also depend on third-party libraries. So things like jQuery, Backbone, Angular, and those things. So if we're just going to look at one of them at a time, so the first one is that we depend on other code. And the problem here is that we tend to depend far too much on global stuff because JavaScript makes this really easy. So here's an example of that. We have this small bit of jQuery. We find an element in the DOM. We click that element. We wait for a click on that element. And then we do an IAX request. And on success, we update the DOM. So just a minor piece of code. The problem here is that we have a bit of code that can't be reused. We have this anonymous function that is called directly. So this is really hard to reuse in another context. What if you need to fetch the same data? It's also difficult to test because you need to actually have the DOM available and you need to bind this click event and then click on the button. And that's really difficult. You don't need that at all. You shouldn't need that at all. And you also have a couple of globals. So here we're using jQuery. So jQuery.get. We do an IAX request. And we also update the DOM directly here. So by doing some changes here, we can really make this far better. So the first step could just be to name that function and take it out of the click handler. So now we have a method here that we can call directly. So now you don't have to create the DOM with the click handler, the click event. So now you can call the function directly, but you still have a couple of problems. You have this IAX request that is difficult to test. So now when you call this bit of code, it will do an IAX request. So you need to either muck out jQuery or you need to muck out all IAX requests. You shouldn't need to do that. You also still have the globals. You rely on having jQuery available in the global namespace. And you also depend on having the DOM. So we kind of got away from the DOM problem, but we also still have a DOM problem. So here we're just changing it up a little bit. Instead of relying on jQuery being available in the global namespace, we send it into the function. So we could have called in something else here, like just jQuery, when we sent it in. But here I called it IAX, so we can see IAX.get. So now if we want to test this, this is far easier. Now we can muck this out. We can send in something that has a get method that does just what we want to do. We have also sent in the HTML element where results should live. So now we can create an empty element with jQuery and send that in, for example. So you don't need to set up the entire DOM anymore. So now this is far easier to test, and this is actually far easier to reuse in other contexts. If you need to do the same IAX request and update another place in the DOM, that is really easy now. So the thing is that now we can control the dependencies. So we're still at a reasonably low level of abstraction here. So we could have had higher level abstractions for IAX and everything. We're still just working with jQuery itself. It's just that we're using jQuery a little bit differently than we usually do. So I wrote a blog post about this. I think it's about a year ago where I talked a little bit about a view abstraction based on backbone. So just using pure JavaScript instead of relying on backbone, which even though it's small, it's a thousand lines of code. So you could create, say that you have a view with just some subsection of your site, for example, the search bar that we saw on Facebook. So you could have a user view where you say that you want to receive, do I have the mouse here? Maybe? So you say here that you want to have an HTML element that you receive, and you also want to receive a user. So with user view now, we can, this is on the bottom here, we can say that we want to create an instance of this. So with JavaScript you can create an instance of your function, and then this will be for that instance. So we put the HTML element that we send in, which is here something we found in the DOM, and we also send in user and put that on the instance. So now we can create methods on this view. So we can create a showImageInstant method that just appends an image to the DOM. And now we can call that method. So the thing is now that we have another way to think about how we use jQuery. Now we send it in, now we have wrapped it in a reasonably simple abstraction. So this is no library at all, this is just a way to use pure JavaScript. And this is basically what, for example, Backbone. It's just a couple of helpers in addition to this. So what I would say is always inject your dependencies. This really helps with structure, it helps with testing, and it helps with creating larger applications. And an example of someone who has really embraced this is AngularJS. Here's an example of some AngularJS code. You have a controller, which is the thing that puts stuff, for example, on or makes stuff available for your DOM, for your HTML. So the thing that you put on scope here, which is something you get into your controller, will be available for your template. So here you can see that. So let's see if we have the mouse here. What you put on scope here will be available. So now we can do an IX request. And then on success, we update the scope, which then makes it available for the template. The thing is that it doesn't matter which way you order your arguments here. If you just change that around, Angular will still handle this with no problem. So they have a built-in service locator. So they see the function, they can see the name of the parameters, and then they go looking for that. So here it's built in functionality in AngularJS, the HTTP and the scope. But you can also register your own bits of code here. So to take an example from AngularJS, you could do something like this. You say that you want to create a module. Here it's named NDC, and it has no dependencies. That's the empty array. Then you can say that you want to create a function, which is named awesome. And that function should return awesome conference. And then you can create an injector. And in your injector, you say that you depend on NDC, which is your module, and you also depend on NG, which is a built-in Angular module. So this happens behind the scenes in AngularJS itself, but it's quite easy to do with the built-in stuff. So here we see that the NDC is just the NDC module. And now we can invoke a function on that injector. So here we see that we have a function, which takes in awesome, and then it console logs. So this will actually console log awesome conference. So Angular finds the name here, and it finds it in the things you have registered on the NDC module. Another cool thing that is going on in a lot of new libraries and frameworks is two-way binding. So this is a really cool way to work with the DOM. It's done in, amongst others, knockout, Angular, and Ember. So if you look at an example from EmberJS, we have a template at the top here, where it says if it's expanded, then it should show some data. And it should also show a button where you can press on to contract the input. And then if it is expanded, it should show a button where it says show more. So if you look at the post controller, which is the thing that gives the data, give the template its data, you can see that it's expanded defaults to being false. So the first time you render this template, it will go to showing the button that says show more. And on the show more button, you have this action where it say expand. So when you click on expand, click on this button, it calls the expand action, which is on the post controller. So here we see that we update is expanded to true. So now it will automatically update the template. So the thing is here, we have no jQuery at all, but we're still working with the DOM. So this is just another way to work with the DOM that is really, it's used quite a bit in the three libraries I mentioned. So you get rid of event handlers, you get rid of manual DOM updates, and you can ensure that your template is in sync with your data without having all of this jQuery hacking. So that was the first thing. Now we have another way to work on, work with other code. So now if we go on to look on other files, the problem now is that we depend on the order of script elements in our index HTML. And when you get hundreds of files, this is really a problem. It becomes unmaintainable, and it also becomes really brittle. And it's hard to know what needs to be there for something to function, especially when you get, say that you have 100 files, this is really hard. So we want to get away from this. We need something better. Because the thing is that we introduced lots of globals here. So we pull in jQuery, then we have jQuery as a global. We pull in Backbone, then we have Backbone on the global namespace. We have our own code that is on the global namespace. So you have a lot of weakly stated dependencies. It's hard to see what needs to be there for something else to function. So we need a better way to refer to other files. Two ways of doing this is commonJS and AMD. CommonJS is the way that Node.js solves this. It looks something like this. We say that we require jQuery, and we take the results from that require, and we put it on the dollar. And then we can also say that we export something from our current file. So here we export my function, so that when other require our file, that file, it has my function available. AMD is another way of doing this. Here you see that when you define this bit of functionality, you depend on jQuery. And it's also put in here into the dollar. And what you return from this function will be available when someone else say that they depend on this file. So now we can avoid globals. Now we can get away from having jQuery on the global namespace. And we don't need to use backbone from the global namespace. So now it's really easy to see dependencies. When you open a file, you can see it at the top of the file, what this bit of code depend on. And that makes your code, I would say, easier to understand. You can see right then and there what you depend on. And you can also see that you depend on too much stuff. When you have 50 things you depend on, that's usually a sign that you need to do something. So this is also a way to help you. And I'll split your code into new modules to split that file into something less than the 5,000 lines of code. Right now AMD works better in current browsers. You have some ways of running common JS, such as a library called Browserify. But right now AMD works a little bit better because it's asynchronous. So what happens here is that usually when you say that you depend on jQuery, the library can see have you loaded jQuery already? No, you haven't. Okay, then I go dynamically look for jQuery on the server. And you can also minify that into one file for production. For example, with something like RequireJS, there's a lot of implementations of AMD right now. I've used RequireJS quite a bit. And it works quite good. Of course there's a bit of complexity when you pull in something like this because this is a central part of your site, how you handle modules. So RequireJS has worked quite good for me. It also has a minifier. So you can have 200 files in development and you only want one file in production. There's a minifier there that you can run to get this one file. So now we have handled how we depend on other files. So now we can see that there isn't really a problem depending on 200 other files when you have something that can solve it for you. There's also stuff going on here for a new versions of JavaScript. So there will be some built-in modules stuff coming in a new version of JavaScript. So the last thing for dependencies is that we depend on third-party libraries. And we need to keep this up to date. Now you usually need to go to jQuery.com, see that, okay, jQuery is updated, and then you need to download it and everything. And it's also really hard to see, is it updated, without going to all of these pages to see. So you can use a package manager like Component or Bower. This is still, I don't know, quite... This is new stuff. This is happening right now. Twitter created Bower or released Bower. It's not a long time ago. So this is quite new and it might not be the perfect solution yet. And there's a lot of difficult problems to solve here. But with Bower, for example, you can do something like this. You say that in your Bower.json file, you can say that you depend on AngularJS, you depend on AngularResource, and you can see that we depend here on version 107. We can also say that we depend on jQuery. And now, when you type Bower install, it will go looking for those on GitHub. And the nice thing now is that if you type BowerList, you can see that you have downloaded Angular, AngularResource, which depend on Angular, and you can also see that jQuery is not up to date. So there are new versions available. And then you have 20, 30, 40 dependencies. This makes your code really easy. It makes it far easier to update dependencies regularly. So, for example, here, if we have written BowerUpdate, we could update these dependencies. And to save a new dependency into our Bower.json file, we can say Bower install, dash dash save bootstrap, for example, to download Twitter Bootstrap. So this is somewhat similar to NPM for those that I've played with Node.js. So, we've been through quite a lot of stuff. There's a lot of new stuff going on in JavaScript right now, and there has been for a couple of years, and there will happen a lot of stuff the next couple of years. But your question might be, seriously, do you need to do all this stuff? And I think it depends on the type of application you're making. So usually, we have applications that need to live for a long time. You have a lot of unforeseen requirements. You need to adapt to what's going on. So there's this big push for having even more interactive stuff on the front end. So we have more and more complex applications. And when we have more and more complex applications, we really need to think about how we develop our applications. So I absolutely think it's necessary to start thinking about these things and learn more about them. And not all of them are ready for production use if you're not ready to debug it and stuff. I've been debugging Bower somewhat lately. It doesn't run that good on Windows. So I think we need it, but we're still not quite there yet. For some of the things we are, so you should absolutely go check out Angular and Ember and Knockout and Backbone at least. There's a lot of cool stuff going on. So if you want to learn more about this, me and a couple of friends created this resource for finding good articles and videos on this stuff because we thought it was really difficult to find good stuff on JavaScript. So if you have any good ideas for articles, videos, presentations, whatever, just send us an email because we really want to have a really good resource for people to find good stuff for creating large JavaScript applications. So that's all. I've put the presentation online so you can find it at Speaker Deck. So any questions? And if you don't have any questions right now, you can also find me afterwards. I will also be here on both tomorrow and on Friday, so you can just come by and talk about JavaScript. So if there are no questions, I think we're done. Thank you.
send all the HTML from the server to the browser. function at all. So we've come a long way. And So this means that there's more and more problem is no longer the browser itself and the complexity of our applications. I saw a great to unlearn our DOM-centric approach to creating couple of years, we've started to get some help thing is that we need a foundation to discuss the look at these last few years, we have from 2005 everything, but you should really focus on what JavaScript. And this is basically what, for in Backbone, then we have Backbone on the global can get away from having jQuery on the global namespace. And we don't need to use backbone from