How to Solve Django's Top 5 Enterprise Headaches
Makai, Matt
The top five Django problems in large enterprise organizations are integrating with Active Directory, passing security audits, transferring data from legacy systems, installing packages from PyPI through proxy servers and combating misperceptions around dynamically typed programming languages. We'll solve these problems with code and resources to back up arguments to enterprise stakeholders.
 Music Good afternoon everyone. My name is Matt McKay and I am a developer evangelist for Twilio. But before I started at Twilio, I was doing consulting work in large enterprises across government, nonprofit and commercial organizations. And I discovered that there were a lot of things that came up on every project, drove me nuts, were incredibly frustrating. And when I heard that there was going to be an enterprise related track at DjangoCon, I thought this would be the time for me to vent my frustration into a presentation and then also hopefully help you guys in these situations to solve some of these problems. So like a good consultant, I decided I was going to whiteboard this out, but there's no whiteboards in here so I just drew this ahead of time. So this is my whiteboard version of my presentation. And I've also drawn some pictures to go along with this and I am going to use the term client to represent the sort of boss figure, the person that is driving what the feature requests are. So for people that are staff, it's probably just the boss. For me in a client consulting relationship, I'm just going to say client so just so you know what it is. And so the things that I would get requested all the time from clients, here's the top five things that I've on every project and it would drive me nuts. The first one was integrating with Active Directory. It's everywhere. It literally is in every enterprise and it was always at the last minute, oh, we're getting ready to deploy to production. This integrates with Active Directory, right? I'm like, yeah, I, well, let me just check on that for a second. So I would make sure that I would integrate with Active Directory beforehand, right, instead of doing it right before production. The second thing is passing security audits. How do you go about convincing security that this crazy framework called Django, which they always seem to pronounce the Django, is not something that is any more inherently insecure than the spring NBC framework that they're used to doing audits on. The third thing is ingesting legacy data. So in any enterprise, there are existing systems. How do you interact with those existing systems or take over existing databases that you are now putting a new front end on? The fourth thing is securing dependency installations. So we now in the Python community have the ability to do pip install, whatever framework, pip install Django. And we're so used to doing that, but in enterprise environment, sometimes they think, well, that's strange. We don't control that repository of libraries. We need something else. We need to secure that connection somehow. And then the fifth thing is correcting. This isn't something necessarily specific to Django, but it's something specific to Python and any sort of dynamic language. It's amazing to me the number of conversations I've had with literally CIOs or executive level people that are like this dynamic typing thing. And I'm like, this is not something you should be concerning yourself with as a CIO, but it comes up over and over again. And there is a certain way that I have, through much trial and error, been able to convince many people on the executive level that this is not something that they should be concerning themselves with. So the first thing is the boss man, the client guy says, Active Directory integration. Let's make sure we can integrate with our Active Directory system. So when I hear Active Directory, I'm like, this, my eyes just glaze over. And I'm thinking, like, this is something, some enterprisey thing that, like, there's an enterprise diagram that some enterprise architect drew by hand over a course of a year to just, like, figure out where all the systems are. And I just, like, that's not what excites me as a software developer. This is something that I just want to do. I want to get it over with. And I just don't want to think about ever again. I just want to do it once. And actually, Active Directory is actually not that complicated. By and large, there's a few different setups you can have. But by and large, what they really say when they want you to integrate with Active Directory is, look, we've already stored emails, first names, last names, a bunch of information about our employees and the people in our organization. We just want to make sure that they don't have to, like, plug that information in again. And so that when they log into their system, they don't have to constantly reenter that information. They're logging into our Django app for the first time. And it says, welcome, Kate. Here's the application. They don't have to punch in. Like, it doesn't ask them, like, who are you? What's your user ID? Like, that information, you should be able to look up. Unfortunately, Active Directory actually supplies that. So the way that you would handle this is, so we can look at it as any sort of back end, as if we were doing a model back end, which is the standard one that we'd be looking up for the user and the database. So instead of having a model back end, we can have an active directory back end. The way that we would do this, fortunately, is with Python LDAP. So any slide with a green background is a library that you can install. So we can do pip install Python LDAP. And this takes away some of the complexity from us. But how do we actually integrate this general Python LDAP library into our Django application? Fortunately, it actually doesn't require that much work. The first thing would be to add some things in our settings.py file, just like if we're using any sort of dependency. Now, I don't want you to have to worry too much about the details of what this looks like. First off, my handwriting is pretty terrible. And second of all, this is just a very abstract sort of pseudo-code way of looking at it. At the end of the presentation, I will give you all of the resources that I mentioned today. I wrote a blog post specifically for this talk. It has links to all of the resources I mentioned in the presentation, all the libraries. It even has the source code to the presentation. So you'll get everything at the end. First, you just look through this stuff and remember that you'll be getting the resources at the end. So the way that we look at this in the settings.py file is we have a DNS name, which is our server for Active Directory. We have a port number that it's associated with that in general. That is 389. We have some search fields. These search fields are really what we're looking up. So we want to know what the email address is of the user who's logging in or who's using our application. What is our first name, last name? And this is going to be dependent upon your organization, what they decide to store in Active Directory. So that is one question you're going to want to ask ahead of time. What do we store in Active Directory? Do we just use it for basic user information? Or do we actually have roles and different privileges that are stored in Active Directory that we should be using? Or do we need to rebuild that in our application? And just because you have this information here doesn't mean you can't also have some other username password or some other authentication scheme. This is most of the time, at the very least, just you so you don't have to have people re-enter the information. With the LDAP URL, most of the time you'd construct that using the DNS name and the LDAP port. I just hard-coded it there. And then you have some domain. So you add these things into your settings.py based on how your LDAP is set up. And someone in the organization should be able to supply you with that information. And then you create a class. It is a separate backend class. Fortunately, this is something that is a standardized get user, authenticate, create user. There's a bunch of functions that you write for a backend. And people have already written gists and libraries that will encapsulate this. The idea here is that you should be able to take this backend and turn it on when you want it. And for your own development purposes, turn it off when you don't want it. So from our own local development, we don't want to authenticate against LDAP. We're just doing our own development work. We want to authenticate against a model backend. So you should just be able to flip that in your settings.py file so that when you're ready to deploy to your test or your production environment, you'd flip this on. So that covers Active Directory integration. It should be something that isn't scary anymore. It should be something that, okay, I write it once and I'm done with it. The second thing is passing security audits. And this is obviously not just a technical problem. There's not like just a few lines of code that you can write. It's not like you can do pip install security audit. Well, that's actually a really good idea for a library. But there's a bunch of things that you need to know and explain to a security team. So one of the things is as developers, we have our heads down in the code and we're thinking so much about what problems are we solving, implementing new features, getting the system up and running, that we're speaking a completely different language than the security team. One of the things that I found so amazing was I expected the security team to be just as technical as I was as a software developer. But in general, in large organizations, they're not. They actually sometimes don't even have a software development background. We're speaking completely different languages. And when you go into a meeting and you start talking about, well, here's how we're implementing things from a security perspective and you literally hand them piles of code on printed out on pieces of paper, they're like, these guys don't know what they're talking about. They're not speaking the same language as us. So the first thing to do if there's a security team is find out how technical are they. And generally try to find the most technical person or the person who's the most interested in the software development aspects and then work with them to figure out what standards are they using. One of the most common standards for web applications is called the Open Web Application Security Project. They have a top ten list of security vulnerabilities in web applications. And this is probably something that is the input to a checklist. They're trying to check off boxes. The security team knows there is no application that is completely secure. They want to make sure that the bases are covered and they generally take those bases, the boxes they're trying to check based on some standard open security protocol like this or security checklist like this. And it's looking for certain things that as developers we should know at least a little something about. The first thing would be injection, particularly SQL injection. So coming into the meeting knowing what SQL injection is and understanding exactly how Django prevents that with the ORM and highlighting if we're using SQL, if we drop down into the SQL and bypass the Django ORM we call that out in our own information that we hand out, hand to the security team. The second thing would be things like cross-site scripting and then also cross-site request forgery. These are things that Django has pages on. And the great part is about these is that each one of these top 10 items we don't have to reinvent the wheel as Django developers. There's a great talk by Jacob Kaplan Moss that talks about every single one of these security vulnerabilities particularly in the context of Django and how Django handles them. So you can watch that talk and read a blog post which I'll give to you which maps these security vulnerabilities versus what Django does. And obviously you're going to have to adapt it to your own code that you've written but what that means is that a lot of the heavy lifting has been done by people who've solved these problems before. So that is in general how I approach security audits. Again, remember, a security team is often not nearly as technical as we are and therefore we want to be speaking their language so that we don't end up in an adversarial relationship with them. Another thing is transferring legacy data. This happens constantly. We're building new systems in an ongoing ecosystem. The first situation which can often be the cleanest is we're just building a new application on a legacy database. There's already stuff out there and we don't have to worry about another application touching that database. It is our database that we're working with. It has existing data, it has existing relationships. We just need to know how to map our Django application to that. Fortunately we do not have to do this manually. There's something called InspectDB. So once we've configured our Django app to read from a database, we can do InspectDB and we can pipe that into a or redirect that into a models.py file and then it will generate the schema for us. Now we may not know exactly what's in that models.py file so we could look through it and we could read through everything which certainly we're going to have to do at some point and figure out the structure of the database. But I'm a visual person and so there's another thing that we can use which is we can do a pip install Django extensions and then we can run a manage.py graph models with some output and it will generate, getting off of our whiteboard for a second, it will generate a file, a picture that looks like this that actually shows us the structure of the database. And this is fantastic for getting up to speed on what is already out there. We don't have to hand figure out, okay, what table connects to what other table, what are the foreign key relationships, we can literally visualize this and generate our models.py file. Now obviously that's the beginning of the journey. We'll need to migrate tables, we'll need to figure out if that structure is appropriate for our application but it certainly is a starting point that we can get to very quickly and start understanding what that legacy database contains for our application. Now this situation is not always that simple. Sometimes there is an existing application and the requirement is well we just need to get some of the data out of this existing legacy database. So there can be a temptation because Django, believe it was as a 1.3 or 1.4, you can use multiple databases. So you may be tempted to say well we have our own database, we're also going to have our legacy database, we'll just use two databases for our Django application. However, I recommend against doing this, if at all possible, in enterprises weird stuff happens, you don't always have control over the situation, sometimes there's some sort of political battle that is preventing you from the optimal scenario and you may have to directly connect to the database. You can do this, but what I generally recommend is having some sort of wrapper around that existing application. If there's already a team that is developing software for that other application, have them generate an interface, some sort of API that allows you to pull whatever data you need out of it on a regular basis. The danger if you do what I've xed out here is that other systems make, other teams and systems may come along and say oh you've got access to that data, can you write me an API? But you don't actually have control over that database. Someone else is touching that database and so what ends up happening is you become this dependency that you never intended to be. So it's better to push it down onto the application that it really does have control over that database. Now, there's one of my other favorite applications. At my first job they were called end user computing applications and they looked something like this. They were an Excel spreadsheet. The great part was that the company I was working for, a very large company, they had so many, literally hundreds of these things and they had a lot of errors in them. There's no unit testing of Excel spreadsheets and what ended up happening was they had all these cascading errors and they had to restate financials to the tune of hundreds of millions of dollars because they kept having errors propagate throughout all of their financial systems that were basically this. We're talking about major financial institution in the country. Enterprises are very strange places. Now fortunately some environments recognize this is not a sustainable situation. So they'll ask you as a developer to come in and build a system that replaces a spreadsheet. But the caveat is, we want all the data that's in this spreadsheet to be in the system on day one, so we have a hard cut over. So when we log into that system, it has all the data that we're expecting in our Excel spreadsheet. There are some tools that will help us do this. Excel RD and Excel WT, and I'll tell you exactly how you should use them. The first one is for reading Excel spreadsheets, so you can read Excel spreadsheets directly. But I recommend, strongly, strongly recommend that you have a lot of data validation and that you agree beforehand, you say what is the input that we should be taking in, and we will reject anything that does not match that criteria. Column B should have only numbers, and if there are any letters, we reject that, and we will spit that back out as an Excel spreadsheet with Excel WT. So we'll give you back an Excel spreadsheet of all the stuff that does not pass data validation. We're not going to continue to re-ingest this information, but we will suck in as much as we can, and then your team who handles the business side of things, you can figure out what you need to correct. So you push the data correction onto the team that has to hand figure out, well, why did we put, you know, a number in a sal... or why did we put a bunch of letters in a salary column? I mean, this is literally the type of errors that happen in Excel spreadsheet. There's just no validation. So you have validation on your side when you suck in the data, and then you just say, you have exception scenarios where you just pump out and Excel spreadsheet says, you figure this out, and we're happy to take it in when you've fixed the problems. That is by far the best way, and probably one of the only ways to make this a sustainable situation where you can cut over on day one. All right. So another thing is securing packages and installations. There's two things I want to talk about here. Number one is I've gotten on client site, and I've done pip install Django, and the thing fails. Well, often we're going through a proxy server, and they're funneling all traffic through that proxy server. Now, fortunately, pip has a setting. It has a setting where you can specify a proxy server, but also as a leader versions, it will respect this environment variable. So you can export the HTTPS on HTTPS underscore proxy environment variable, and pip will respect that. It'll say, okay, I'm going to go through this proxy server, and then it won't begin installing your packages. Just make sure that you set this every time. I've set it before, and then I forget to set it in my environment variables, and then my pip will fail, and I'm like, what's the proxy server again? So just make sure that if you're going to set this, just set it somewhere where you know what the proxy server is before you use pip every time, otherwise it can fail. Another way is this is our standard setup. So we're used to as developers, you know, we got to PyPI, or PyPI, rather, and we download the packages, and we have them installed in our virtual M, and we're good to go. But this can be very risky for enterprises. They say, we don't have control over this PyPI environment. We don't really care that it's a, you know, a community resource or anything like that. What they care about is like, they want to control that. They want to run their security audit process, and they want to run their scans on a central repository, and that is understandable. There are some malicious things that can happen to central repositories that you do not own. And so what you really want is something more like this. This is all self-contained in the enterprise, and you can certainly do this. In fact, it actually does not take that long to set up a PyPI environment of your own. The only thing is you're going to have to upload and establish all of the packages that you need there, and then make sure the developers are not installing from the central PyPI that all developers in the enterprise know that this is a centralized repository in the enterprise itself. So the only thing about this is if you've got one team that's working on a Django project, and they have to be the ones who are uploading all the packages and making sure everything is scanned, it can be a lot of overhead. So I would fight back against this unless you have multiple teams that are willing to share the burden of hosting your own PyPI server. But it is certainly doable. And then the final thing is what are these dynamically typed languages? As I said in the beginning, it's always amazing to me when CIOs are like, I don't know about this whole Python thing. It's dynamically typed. I'm like, you haven't programmed a lot of code in 20 years. What does it matter to you? But it's so funny. So what I used to do is I would always say, look, if Python is good enough for Google, it's good enough for this big company. And the CIO would be like, but we're not Google. And I found that there was a lot of pushback. They're like, we would like to be that place eventually, but we're not there right now. And so the way to mitigate this, the way to mitigate this with an executive team or people that are pushing back against dynamically typed programming languages is to actually point out all the places that they're already using it. Look, we've been using Python in all these places for the past 10 years. And guess what? The enterprise has not blown up yet. So if you start going around to use, you take a look at all of the tools that your enterprise is using, that there are a lot of these are actually have Python that you're using for, you're actually using Python with them already. So for example, WebSphere uses Python to set up its configuration. You can write Python scripts to auto generate or to do the WebSphere configuration. And so, okay, there's one use case of Python already being in the environment. If you're using Ansible or using SaltStack or even if you're using Chef or Puppet, those are dynamically typed, they're built on top of dynamically typed languages. Each extraction and transformation taking data out of one system and putting it into another. You point out all the situations in which Python and dynamically typed languages, all of them are already being used in the enterprise. And that can be a powerful thing for someone who is just trying to say, well, we don't want to introduce too much change into the enterprise right now. Say you're already using it and therefore it's obviously safe for consumption because it's already been here for 10 years. Now, after that, then you talk about something like, here's the respected peer organizations that are building systems with Python and Django. And there's a bonus, if you can, if you know that there are certain organizations they bring up and they're like, you know, these are the leading organizations in our industry, not necessarily like Google or Facebook or something like that, but if they're in government space or in the financial space and you know that these group of executives always talk about, this company is doing great things, you can find out what those other companies are doing. And the majority of the time, like Bank of America is doing a ton of work with Python. They don't really talk about it too much, but it is something that is out there. So if you're in the financial industry and executives are pushing back on, well, we don't want to use dynamically typed languages or Django, but they're always talking about how great Bank of America runs some of their systems or how, you know, they're able to respond quickly with their IT systems, you say, by the way, like, they're using this already, and that may be one of the reasons why they're outflanking us here. And so that puts a little bit of pressure from the standpoint of like social proof. You're basically saying, look, you don't want to be, there's a few different risks there. Like in the first category, you're mitigating the risk of like, we don't use this already. So you're taking that consideration off the table. Then what you're doing here is you're mitigating like, look, your peers are using this, and they're outrunning you. And that is a fear that many executives have. They're worried about being overrun by other companies. And so you can point out like, look, these other organizations are already using dynamically typed languages. And finally, I do generally have a laundry list of the leading tech companies, and I talk about, here's exactly what these tech companies are using Python for, and these companies are doing really great things with small development teams. And if that's the direction of your organization, then that's probably something you should be thinking about as well. You basically want to start crossing down the list of all the rebuttals that they can have around dynamically typed languages. So we covered a few things here. First and foremost, integrating Active Directory. It's not something that should be too scary. There's some great resources out there with, and using Python LDAP. This is not something that should take you two weeks. You can generally write this stuff in about a day and be done with it and never think about it again, and reuse it on whatever project you go to. Passing security audits talk in their language. Instead of just talking in software development language, it can take a little while to get used to that, but you remove the adversarial relationship between the development team and the security team. Adjusting legacy data, there's a few scenarios here. If you're doing a hard cutover, then you can do InspectDB. You can do graph the data model and figure out how it works. And if you're talking to an existing system, I recommend using an API as opposed to using a set and just connecting directly to the database, because that can be an unsustainable situation. And then also with Excel spreadsheets, make sure you do the data validation and spit out the errors and let them correct the errors as opposed to trying to correct the errors in your own system. Securing dependency installations, PIP will respect the HTTPS proxy variable, but you can also set up your own PIPI if that's something that your enterprise is large enough to maintain. And then finally, correcting misconceptions around dynamic typing. There's a approach that tends to work very well. Obviously, every culture is different. All the pushback from executives can be different, but there is a structured approach that will allow you to just say, you know, this is something that is not too risky. This is something that allows leading institutions to be able to use Python and use it well. All right, so I told you I'd give you guys the resources. I know not everyone has a total cell phone service in here, but if you send a text message, I'll leave this up for just a minute. If you send a text message to this number, it's 503-476-3056. It will respond back with a text message. This is totally anonymous. If you could give me a score of 1 to 10, 1 being the lowest, and 10 being the highest, would you recommend this talk to a friend or a colleague? This allows me to determine, is this a talk that is something that is valuable to the audience, or is this something that maybe I need to work significantly on and maybe change out the topic so that it can be more appropriate based on the talk title? A lot of times we don't necessarily get feedback. This allows me to get feedback based on the audience's reaction. So if you send a text message, the benefit to you guys is you get the link that has all the resources to the blog post, has a link to the presentation, a link to the presentation source code, everything you need that I've talked about throughout this talk. So again, 503-476-3056, and this will be up for a while. So if you don't have cell phone service in here, just send it when you go outside and you'll get the link back. Okay, one more little quick plug. I'm the author of FullStackPython.com. If you're looking for resources around Python, this is like my passion project for 2014. I put a lot of time and effort into it. I actually just crossed 90,000 readers this year. I'm super stoked that this really helps people. This came out of me just writing emails to a lot of developers and saying, here's some great resources. I put it on the Internet for everyone in the community to be able to consume. My name is Matt McKay. I'm a developer evangelist with Twilio. Here's my contact information. Thank you all very much for having me, and I will answer questions outside afterwards. Thank you.
 Music  Good afternoon everyone. My name is Matt McKay and I am a developer evangelist for Twilio.  But before I started at Twilio, I was doing consulting work in large enterprises across government, nonprofit and commercial organizations.  And I discovered that there were a lot of things that came up on every project, drove me nuts, were incredibly frustrating.  And when I heard that there was going to be an enterprise related track at DjangoCon, I thought this would be the time for me to vent my frustration into a presentation  and then also hopefully help you guys in these situations to solve some of these problems.  So like a good consultant, I decided I was going to whiteboard this out, but there's no whiteboards in here so I just drew this ahead of time.  So this is my whiteboard version of my presentation.  And I've also drawn some pictures to go along with this and I am going to use the term client to represent the sort of boss figure, the person that is driving what the feature requests are.  So for people that are staff, it's probably just the boss.  For me in a client consulting relationship, I'm just going to say client so just so you know what it is.  And so the things that I would get requested all the time from clients, here's the top five things that I've on every project and it would drive me nuts.  The first one was integrating with Active Directory.  It's everywhere.  It literally is in every enterprise and it was always at the last minute, oh, we're getting ready to deploy to production.  This integrates with Active Directory, right?  I'm like, yeah, I, well, let me just check on that for a second.  So I would make sure that I would integrate with Active Directory beforehand, right, instead of doing it right before production.  The second thing is passing security audits.  How do you go about convincing security that this crazy framework called Django, which they always seem to pronounce the Django, is not something that is any more inherently insecure than the spring NBC framework that they're used to doing audits on.  The third thing is ingesting legacy data.  So in any enterprise, there are existing systems.  How do you interact with those existing systems or take over existing databases that you are now putting a new front end on?  The fourth thing is securing dependency installations.  So we now in the Python community have the ability to do pip install, whatever framework, pip install Django.  And we're so used to doing that, but in enterprise environment, sometimes they think, well, that's strange.  We don't control that repository of libraries.  We need something else.  We need to secure that connection somehow.  And then the fifth thing is correcting.  This isn't something necessarily specific to Django, but it's something specific to Python and any sort of dynamic language.  It's amazing to me the number of conversations I've had with literally CIOs or executive level people that are like this dynamic typing thing.  And I'm like, this is not something you should be concerning yourself with as a CIO, but it comes up over and over again.  And there is a certain way that I have, through much trial and error, been able to convince many people on the executive level that this is not something that they should be concerning themselves with.  So the first thing is the boss man, the client guy says, Active Directory integration.  Let's make sure we can integrate with our Active Directory system.  So when I hear Active Directory, I'm like, this, my eyes just glaze over.  And I'm thinking, like, this is something, some enterprisey thing that, like, there's an enterprise diagram that some enterprise architect drew by hand over a course of a year to just, like, figure out where all the systems are.  And I just, like, that's not what excites me as a software developer.  This is something that I just want to do.  I want to get it over with.  And I just don't want to think about ever again.  I just want to do it once.  And actually, Active Directory is actually not that complicated.  By and large, there's a few different setups you can have.  But by and large, what they really say when they want you to integrate with Active Directory is, look, we've already stored emails, first names, last names, a bunch of information about our employees and the people in our organization.  We just want to make sure that they don't have to, like, plug that information in again.  And so that when they log into their system, they don't have to constantly reenter that information.  They're logging into our Django app for the first time.  And it says, welcome, Kate.  Here's the application.  They don't have to punch in.  Like, it doesn't ask them, like, who are you?  What's your user ID?  Like, that information, you should be able to look up.  Unfortunately, Active Directory actually supplies that.  So the way that you would handle this is, so we can look at it as any sort of back end, as if we were doing a model back end, which is the standard one that we'd be looking up for the user and the database.  So instead of having a model back end, we can have an active directory back end.  The way that we would do this, fortunately, is with Python LDAP.  So any slide with a green background is a library that you can install.  So we can do pip install Python LDAP.  And this takes away some of the complexity from us.  But how do we actually integrate this general Python LDAP library into our Django application?  Fortunately, it actually doesn't require that much work.  The first thing would be to add some things in our settings.py file, just like if we're using any sort of dependency.  Now, I don't want you to have to worry too much about the details of what this looks like.  First off, my handwriting is pretty terrible.  And second of all, this is just a very abstract sort of pseudo-code way of looking at it.  At the end of the presentation, I will give you all of the resources that I mentioned today.  I wrote a blog post specifically for this talk.  It has links to all of the resources I mentioned in the presentation, all the libraries.  It even has the source code to the presentation.  So you'll get everything at the end.  First, you just look through this stuff and remember that you'll be getting the resources at the end.  So the way that we look at this in the settings.py file is we have a DNS name, which is our server for Active Directory.  We have a port number that it's associated with that in general.  That is 389.  We have some search fields.  These search fields are really what we're looking up.  So we want to know what the email address is of the user who's logging in or who's using our application.  What is our first name, last name?  And this is going to be dependent upon your organization, what they decide to store in Active Directory.  So that is one question you're going to want to ask ahead of time.  What do we store in Active Directory?  Do we just use it for basic user information?  Or do we actually have roles and different privileges that are stored in Active Directory that we should be using?  Or do we need to rebuild that in our application?  And just because you have this information here doesn't mean you can't also have some other username password or some other authentication scheme.  This is most of the time, at the very least, just you so you don't have to have people re-enter the information.  With the LDAP URL, most of the time you'd construct that using the DNS name and the LDAP port.  I just hard-coded it there.  And then you have some domain.  So you add these things into your settings.py based on how your LDAP is set up.  And someone in the organization should be able to supply you with that information.  And then you create a class.  It is a separate backend class.  Fortunately, this is something that is a standardized get user, authenticate, create user.  There's a bunch of functions that you write for a backend.  And people have already written gists and libraries that will encapsulate this.  The idea here is that you should be able to take this backend and turn it on when you want it.  And for your own development purposes, turn it off when you don't want it.  So from our own local development, we don't want to authenticate against LDAP.  We're just doing our own development work.  We want to authenticate against a model backend.  So you should just be able to flip that in your settings.py file  so that when you're ready to deploy to your test or your production environment,  you'd flip this on.  So that covers Active Directory integration.  It should be something that isn't scary anymore.  It should be something that, okay, I write it once and I'm done with it.  The second thing is passing security audits.  And this is obviously not just a technical problem.  There's not like just a few lines of code that you can write.  It's not like you can do pip install security audit.  Well, that's actually a really good idea for a library.  But there's a bunch of things that you need to know and explain to a security team.  So one of the things is as developers, we have our heads down in the code  and we're thinking so much about what problems are we solving,  implementing new features, getting the system up and running,  that we're speaking a completely different language than the security team.  One of the things that I found so amazing was I expected the security team  to be just as technical as I was as a software developer.  But in general, in large organizations, they're not.  They actually sometimes don't even have a software development background.  We're speaking completely different languages.  And when you go into a meeting and you start talking about,  well, here's how we're implementing things from a security perspective  and you literally hand them piles of code on printed out on pieces of paper,  they're like, these guys don't know what they're talking about.  They're not speaking the same language as us.  So the first thing to do if there's a security team is find out how technical are they.  And generally try to find the most technical person  or the person who's the most interested in the software development aspects  and then work with them to figure out what standards are they using.  One of the most common standards for web applications is called the Open Web Application Security Project.  They have a top ten list of security vulnerabilities in web applications.  And this is probably something that is the input to a checklist.  They're trying to check off boxes.  The security team knows there is no application that is completely secure.  They want to make sure that the bases are covered  and they generally take those bases, the boxes they're trying to check  based on some standard open security protocol like this or security checklist like this.  And it's looking for certain things that as developers we should know at least a little something about.  The first thing would be injection, particularly SQL injection.  So coming into the meeting knowing what SQL injection is  and understanding exactly how Django prevents that with the ORM  and highlighting if we're using SQL, if we drop down into the SQL and bypass the Django ORM  we call that out in our own information that we hand out, hand to the security team.  The second thing would be things like cross-site scripting  and then also cross-site request forgery.  These are things that Django has pages on.  And the great part is about these is that each one of these top 10 items  we don't have to reinvent the wheel as Django developers.  There's a great talk by Jacob Kaplan Moss that talks about every single one of these security vulnerabilities  particularly in the context of Django and how Django handles them.  So you can watch that talk and read a blog post which I'll give to you  which maps these security vulnerabilities versus what Django does.  And obviously you're going to have to adapt it to your own code that you've written  but what that means is that a lot of the heavy lifting has been done  by people who've solved these problems before.  So that is in general how I approach security audits.  Again, remember, a security team is often not nearly as technical as we are  and therefore we want to be speaking their language  so that we don't end up in an adversarial relationship with them.  Another thing is transferring legacy data.  This happens constantly.  We're building new systems in an ongoing ecosystem.  The first situation which can often be the cleanest is we're just building a new application on a legacy database.  There's already stuff out there and we don't have to worry about another application touching that database.  It is our database that we're working with.  It has existing data, it has existing relationships.  We just need to know how to map our Django application to that.  Fortunately we do not have to do this manually.  There's something called InspectDB.  So once we've configured our Django app to read from a database,  we can do InspectDB and we can pipe that into a or redirect that into a models.py file  and then it will generate the schema for us.  Now we may not know exactly what's in that models.py file  so we could look through it and we could read through everything  which certainly we're going to have to do at some point  and figure out the structure of the database.  But I'm a visual person and so there's another thing that we can use  which is we can do a pip install Django extensions  and then we can run a manage.py graph models with some output  and it will generate, getting off of our whiteboard for a second,  it will generate a file, a picture that looks like this  that actually shows us the structure of the database.  And this is fantastic for getting up to speed on what is already out there.  We don't have to hand figure out, okay, what table connects to what other table,  what are the foreign key relationships, we can literally visualize this  and generate our models.py file.  Now obviously that's the beginning of the journey.  We'll need to migrate tables, we'll need to figure out  if that structure is appropriate for our application  but it certainly is a starting point that we can get to very quickly  and start understanding what that legacy database contains for our application.  Now this situation is not always that simple.  Sometimes there is an existing application  and the requirement is well we just need to get some of the data  out of this existing legacy database.  So there can be a temptation because Django, believe it was as a 1.3 or 1.4,  you can use multiple databases.  So you may be tempted to say well we have our own database,  we're also going to have our legacy database,  we'll just use two databases for our Django application.  However, I recommend against doing this, if at all possible,  in enterprises weird stuff happens,  you don't always have control over the situation,  sometimes there's some sort of political battle  that is preventing you from the optimal scenario  and you may have to directly connect to the database.  You can do this, but what I generally recommend  is having some sort of wrapper around that existing application.  If there's already a team that is developing software for that other application,  have them generate an interface, some sort of API  that allows you to pull whatever data you need out of it  on a regular basis.  The danger if you do what I've xed out here  is that other systems make, other teams and systems may come along  and say oh you've got access to that data,  can you write me an API?  But you don't actually have control over that database.  Someone else is touching that database  and so what ends up happening is you become this dependency  that you never intended to be.  So it's better to push it down onto the application  that it really does have control over that database.  Now, there's one of my other favorite applications.  At my first job they were called end user computing applications  and they looked something like this.  They were an Excel spreadsheet.  The great part was that the company I was working for,  a very large company, they had so many,  literally hundreds of these things  and they had a lot of errors in them.  There's no unit testing of Excel spreadsheets  and what ended up happening was they had all these cascading errors  and they had to restate financials  to the tune of hundreds of millions of dollars  because they kept having errors  propagate throughout all of their financial systems  that were basically this.  We're talking about major financial institution in the country.  Enterprises are very strange places.  Now fortunately some environments recognize  this is not a sustainable situation.  So they'll ask you as a developer to come in  and build a system that replaces a spreadsheet.  But the caveat is, we want all the data that's in this spreadsheet  to be in the system on day one,  so we have a hard cut over.  So when we log into that system,  it has all the data that we're expecting in our Excel spreadsheet.  There are some tools that will help us do this.  Excel RD and Excel WT,  and I'll tell you exactly how you should use them.  The first one is for reading Excel spreadsheets,  so you can read Excel spreadsheets directly.  But I recommend, strongly, strongly recommend  that you have a lot of data validation  and that you agree beforehand,  you say what is the input that we should be taking in,  and we will reject anything that does not match that criteria.  Column B should have only numbers,  and if there are any letters, we reject that,  and we will spit that back out as an Excel spreadsheet with Excel WT.  So we'll give you back an Excel spreadsheet  of all the stuff that does not pass data validation.  We're not going to continue to re-ingest this information,  but we will suck in as much as we can,  and then your team who handles the business side of things,  you can figure out what you need to correct.  So you push the data correction onto the team that has to hand figure out,  well, why did we put, you know, a number in a sal...  or why did we put a bunch of letters in a salary column?  I mean, this is literally the type of errors that happen in Excel spreadsheet.  There's just no validation.  So you have validation on your side when you suck in the data,  and then you just say, you have exception scenarios where you just pump out  and Excel spreadsheet says, you figure this out,  and we're happy to take it in when you've fixed the problems.  That is by far the best way, and probably one of the only ways  to make this a sustainable situation where you can cut over on day one.  All right. So another thing is securing packages and installations.  There's two things I want to talk about here.  Number one is I've gotten on client site, and I've done pip install Django,  and the thing fails.  Well, often we're going through a proxy server,  and they're funneling all traffic through that proxy server.  Now, fortunately, pip has a setting.  It has a setting where you can specify a proxy server,  but also as a leader versions, it will respect this environment variable.  So you can export the HTTPS on HTTPS underscore proxy environment variable,  and pip will respect that.  It'll say, okay, I'm going to go through this proxy server,  and then it won't begin installing your packages.  Just make sure that you set this every time.  I've set it before, and then I forget to set it in my environment variables,  and then my pip will fail, and I'm like, what's the proxy server again?  So just make sure that if you're going to set this,  just set it somewhere where you know what the proxy server is  before you use pip every time, otherwise it can fail.  Another way is this is our standard setup.  So we're used to as developers, you know, we got to PyPI, or PyPI, rather,  and we download the packages, and we have them installed in our virtual M,  and we're good to go.  But this can be very risky for enterprises.  They say, we don't have control over this PyPI environment.  We don't really care that it's a, you know, a community resource or anything like that.  What they care about is like, they want to control that.  They want to run their security audit process,  and they want to run their scans on a central repository,  and that is understandable.  There are some malicious things that can happen to central repositories that you do not own.  And so what you really want is something more like this.  This is all self-contained in the enterprise, and you can certainly do this.  In fact, it actually does not take that long to set up a PyPI environment of your own.  The only thing is you're going to have to upload and establish all of the packages that you need there,  and then make sure the developers are not installing from the central PyPI  that all developers in the enterprise know that this is a centralized repository in the enterprise itself.  So the only thing about this is if you've got one team that's working on a Django project,  and they have to be the ones who are uploading all the packages and making sure everything is scanned,  it can be a lot of overhead.  So I would fight back against this unless you have multiple teams that are willing to share the burden  of hosting your own PyPI server.  But it is certainly doable.  And then the final thing is what are these dynamically typed languages?  As I said in the beginning, it's always amazing to me when CIOs are like,  I don't know about this whole Python thing. It's dynamically typed.  I'm like, you haven't programmed a lot of code in 20 years. What does it matter to you?  But it's so funny.  So what I used to do is I would always say, look, if Python is good enough for Google,  it's good enough for this big company.  And the CIO would be like, but we're not Google.  And I found that there was a lot of pushback. They're like, we would like to be that place eventually,  but we're not there right now.  And so the way to mitigate this, the way to mitigate this with an executive team or people that are pushing back  against dynamically typed programming languages is to actually point out all the places that they're already using it.  Look, we've been using Python in all these places for the past 10 years.  And guess what? The enterprise has not blown up yet.  So if you start going around to use, you take a look at all of the tools that your enterprise is using,  that there are a lot of these are actually have Python that you're using for,  you're actually using Python with them already.  So for example, WebSphere uses Python to set up its configuration.  You can write Python scripts to auto generate or to do the WebSphere configuration.  And so, okay, there's one use case of Python already being in the environment.  If you're using Ansible or using SaltStack or even if you're using Chef or Puppet,  those are dynamically typed, they're built on top of dynamically typed languages.  Each extraction and transformation taking data out of one system and putting it into another.  You point out all the situations in which Python and dynamically typed languages,  all of them are already being used in the enterprise.  And that can be a powerful thing for someone who is just trying to say,  well, we don't want to introduce too much change into the enterprise right now.  Say you're already using it and therefore it's obviously safe for consumption  because it's already been here for 10 years.  Now, after that, then you talk about something like,  here's the respected peer organizations that are building systems with Python and Django.  And there's a bonus, if you can, if you know that there are certain organizations they bring up  and they're like, you know, these are the leading organizations in our industry,  not necessarily like Google or Facebook or something like that,  but if they're in government space or in the financial space  and you know that these group of executives always talk about,  this company is doing great things, you can find out what those other companies are doing.  And the majority of the time, like Bank of America is doing a ton of work with Python.  They don't really talk about it too much, but it is something that is out there.  So if you're in the financial industry and executives are pushing back on,  well, we don't want to use dynamically typed languages or Django,  but they're always talking about how great Bank of America runs some of their systems  or how, you know, they're able to respond quickly with their IT systems,  you say, by the way, like, they're using this already,  and that may be one of the reasons why they're outflanking us here.  And so that puts a little bit of pressure from the standpoint of like social proof.  You're basically saying, look, you don't want to be, there's a few different risks there.  Like in the first category, you're mitigating the risk of like, we don't use this already.  So you're taking that consideration off the table.  Then what you're doing here is you're mitigating like, look, your peers are using this,  and they're outrunning you.  And that is a fear that many executives have.  They're worried about being overrun by other companies.  And so you can point out like, look, these other organizations are already using dynamically typed languages.  And finally, I do generally have a laundry list of the leading tech companies,  and I talk about, here's exactly what these tech companies are using Python for,  and these companies are doing really great things with small development teams.  And if that's the direction of your organization,  then that's probably something you should be thinking about as well.  You basically want to start crossing down the list of all the rebuttals that they can have around dynamically typed languages.  So we covered a few things here.  First and foremost, integrating Active Directory.  It's not something that should be too scary.  There's some great resources out there with, and using Python LDAP.  This is not something that should take you two weeks.  You can generally write this stuff in about a day and be done with it and never think about it again,  and reuse it on whatever project you go to.  Passing security audits talk in their language.  Instead of just talking in software development language, it can take a little while to get used to that,  but you remove the adversarial relationship between the development team and the security team.  Adjusting legacy data, there's a few scenarios here.  If you're doing a hard cutover, then you can do InspectDB.  You can do graph the data model and figure out how it works.  And if you're talking to an existing system, I recommend using an API as opposed to using a set  and just connecting directly to the database, because that can be an unsustainable situation.  And then also with Excel spreadsheets, make sure you do the data validation and spit out the errors  and let them correct the errors as opposed to trying to correct the errors in your own system.  Securing dependency installations, PIP will respect the HTTPS proxy variable,  but you can also set up your own PIPI if that's something that your enterprise is large enough to maintain.  And then finally, correcting misconceptions around dynamic typing.  There's a approach that tends to work very well.  Obviously, every culture is different.  All the pushback from executives can be different, but there is a structured approach that will allow you to just say,  you know, this is something that is not too risky.  This is something that allows leading institutions to be able to use Python and use it well.  All right, so I told you I'd give you guys the resources.  I know not everyone has a total cell phone service in here, but if you send a text message,  I'll leave this up for just a minute.  If you send a text message to this number, it's 503-476-3056.  It will respond back with a text message.  This is totally anonymous.  If you could give me a score of 1 to 10, 1 being the lowest,  and 10 being the highest, would you recommend this talk to a friend or a colleague?  This allows me to determine, is this a talk that is something that is valuable to the audience,  or is this something that maybe I need to work significantly on and maybe change out the topic  so that it can be more appropriate based on the talk title?  A lot of times we don't necessarily get feedback.  This allows me to get feedback based on the audience's reaction.  So if you send a text message, the benefit to you guys is you get the link  that has all the resources to the blog post, has a link to the presentation,  a link to the presentation source code, everything you need that I've talked about throughout this talk.  So again, 503-476-3056, and this will be up for a while.  So if you don't have cell phone service in here, just send it when you go outside  and you'll get the link back.  Okay, one more little quick plug.  I'm the author of FullStackPython.com.  If you're looking for resources around Python, this is like my passion project for 2014.  I put a lot of time and effort into it.  I actually just crossed 90,000 readers this year.  I'm super stoked that this really helps people.  This came out of me just writing emails to a lot of developers and saying,  here's some great resources.  I put it on the Internet for everyone in the community to be able to consume.  My name is Matt McKay. I'm a developer evangelist with Twilio.  Here's my contact information. Thank you all very much for having me,  and I will answer questions outside afterwards. Thank you.
DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_0_71.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_72_143.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_144_203.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_204_311.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_312_419.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_420_503.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_504_1607.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_1608_1919.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_1920_2543.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_2544_3491.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_3492_5351.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_5352_6251.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_6252_7703.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_7704_12083.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_12084_13883.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_13884_14111.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_14112_14231.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_14232_16739.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_16740_17003.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_17004_17411.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_17412_17939.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_17940_18659.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_18660_18935.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_18936_19883.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_19884_22283.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_22284_23771.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_23772_26735.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_26736_27575.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_27576_30683.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_30684_34799.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_34800_35399.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_35400_37871.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_37872_40319.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_40320_40679.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_40680_40739.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_40740_40823.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_40824_40883.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_40884_40931.png DjangoCon_2014-_How_to_Solve_Django_s_Top_5_Enterprise_Headaches_aMtiCX38w20_40932_41048.png
Music Good afternoon everyone. My name is Matt McKay and I am a developer evangelist for Twilio. But before I started at Twilio, I was doing consulting work in large enterprises across government, nonprofit and commercial organizations. And I discovered that there were a lot of things that came up on every project, drove me nuts, were incredibly frustrating. And when I heard that there was going to be an enterprise related track at DjangoCon, I thought this would be the time for me to vent my frustration into a presentation and then also hopefully help you guys in these situations to solve some of these problems. So like a good consultant, I decided I was going to whiteboard this out, but there's no whiteboards in here so I just drew this ahead of time. So this is my whiteboard version of my presentation. And I've also drawn some pictures to go along with this and I am going to use the term client to represent the sort of boss figure, the person that is driving what the feature requests are. So for people that are staff, it's probably just the boss. For me in a client consulting relationship, I'm just going to say client so just so you know what it is. And so the things that I would get requested all the time from clients, here's the top five things that I've on every project and it would drive me nuts. The first one was integrating with Active Directory. It's everywhere. It literally is in every enterprise and it was always at the last minute, oh, we're getting ready to deploy to production. This integrates with Active Directory, right? I'm like, yeah, I, well, let me just check on that for a second. So I would make sure that I would integrate with Active Directory beforehand, right, instead of doing it right before production. The second thing is passing security audits. How do you go about convincing security that this crazy framework called Django, which they always seem to pronounce the Django, is not something that is any more inherently insecure than the spring NBC framework that they're used to doing audits on. The third thing is ingesting legacy data. So in any enterprise, there are existing systems. How do you interact with those existing systems or take over existing databases that you are now putting a new front end on? The fourth thing is securing dependency installations. So we now in the Python community have the ability to do pip install, whatever framework, pip install Django. And we're so used to doing that, but in enterprise environment, sometimes they think, well, that's strange. We don't control that repository of libraries. We need something else. We need to secure that connection somehow. And then the fifth thing is correcting. This isn't something necessarily specific to Django, but it's something specific to Python and any sort of dynamic language. It's amazing to me the number of conversations I've had with literally CIOs or executive level people that are like this dynamic typing thing. And I'm like, this is not something you should be concerning yourself with as a CIO, but it comes up over and over again. And there is a certain way that I have, through much trial and error, been able to convince many people on the executive level that this is not something that they should be concerning themselves with. So the first thing is the boss man, the client guy says, Active Directory integration. Let's make sure we can integrate with our Active Directory system. So when I hear Active Directory, I'm like, this, my eyes just glaze over. And I'm thinking, like, this is something, some enterprisey thing that, like, there's an enterprise diagram that some enterprise architect drew by hand over a course of a year to just, like, figure out where all the systems are. And I just, like, that's not what excites me as a software developer. This is something that I just want to do. I want to get it over with. And I just don't want to think about ever again. I just want to do it once. And actually, Active Directory is actually not that complicated. By and large, there's a few different setups you can have. But by and large, what they really say when they want you to integrate with Active Directory is, look, we've already stored emails, first names, last names, a bunch of information about our employees and the people in our organization. We just want to make sure that they don't have to, like, plug that information in again. And so that when they log into their system, they don't have to constantly reenter that information. They're logging into our Django app for the first time. And it says, welcome, Kate. Here's the application. They don't have to punch in. Like, it doesn't ask them, like, who are you? What's your user ID? Like, that information, you should be able to look up. Unfortunately, Active Directory actually supplies that. So the way that you would handle this is, so we can look at it as any sort of back end, as if we were doing a model back end, which is the standard one that we'd be looking up for the user and the database. So instead of having a model back end, we can have an active directory back end. The way that we would do this, fortunately, is with Python LDAP. So any slide with a green background is a library that you can install. So we can do pip install Python LDAP. And this takes away some of the complexity from us. But how do we actually integrate this general Python LDAP library into our Django application? Fortunately, it actually doesn't require that much work. The first thing would be to add some things in our settings.py file, just like if we're using any sort of dependency. Now, I don't want you to have to worry too much about the details of what this looks like. First off, my handwriting is pretty terrible. And second of all, this is just a very abstract sort of pseudo-code way of looking at it. At the end of the presentation, I will give you all of the resources that I mentioned today. I wrote a blog post specifically for this talk. It has links to all of the resources I mentioned in the presentation, all the libraries. It even has the source code to the presentation. So you'll get everything at the end. First, you just look through this stuff and remember that you'll be getting the resources at the end. So the way that we look at this in the settings.py file is we have a DNS name, which is our server for Active Directory. We have a port number that it's associated with that in general. That is 389. We have some search fields. These search fields are really what we're looking up. So we want to know what the email address is of the user who's logging in or who's using our application. What is our first name, last name? And this is going to be dependent upon your organization, what they decide to store in Active Directory. So that is one question you're going to want to ask ahead of time. What do we store in Active Directory? Do we just use it for basic user information? Or do we actually have roles and different privileges that are stored in Active Directory that we should be using? Or do we need to rebuild that in our application? And just because you have this information here doesn't mean you can't also have some other username password or some other authentication scheme. This is most of the time, at the very least, just you so you don't have to have people re-enter the information. With the LDAP URL, most of the time you'd construct that using the DNS name and the LDAP port. I just hard-coded it there. And then you have some domain. So you add these things into your settings.py based on how your LDAP is set up. And someone in the organization should be able to supply you with that information. And then you create a class. It is a separate backend class. Fortunately, this is something that is a standardized get user, authenticate, create user. There's a bunch of functions that you write for a backend. And people have already written gists and libraries that will encapsulate this. The idea here is that you should be able to take this backend and turn it on when you want it. And for your own development purposes, turn it off when you don't want it. So from our own local development, we don't want to authenticate against LDAP. We're just doing our own development work. We want to authenticate against a model backend. So you should just be able to flip that in your settings.py file so that when you're ready to deploy to your test or your production environment, you'd flip this on. So that covers Active Directory integration. It should be something that isn't scary anymore. It should be something that, okay, I write it once and I'm done with it. The second thing is passing security audits. And this is obviously not just a technical problem. There's not like just a few lines of code that you can write. It's not like you can do pip install security audit. Well, that's actually a really good idea for a library. But there's a bunch of things that you need to know and explain to a security team. So one of the things is as developers, we have our heads down in the code and we're thinking so much about what problems are we solving, implementing new features, getting the system up and running, that we're speaking a completely different language than the security team. One of the things that I found so amazing was I expected the security team to be just as technical as I was as a software developer. But in general, in large organizations, they're not. They actually sometimes don't even have a software development background. We're speaking completely different languages. And when you go into a meeting and you start talking about, well, here's how we're implementing things from a security perspective and you literally hand them piles of code on printed out on pieces of paper, they're like, these guys don't know what they're talking about. They're not speaking the same language as us. So the first thing to do if there's a security team is find out how technical are they. And generally try to find the most technical person or the person who's the most interested in the software development aspects and then work with them to figure out what standards are they using. One of the most common standards for web applications is called the Open Web Application Security Project. They have a top ten list of security vulnerabilities in web applications. And this is probably something that is the input to a checklist. They're trying to check off boxes. The security team knows there is no application that is completely secure. They want to make sure that the bases are covered and they generally take those bases, the boxes they're trying to check based on some standard open security protocol like this or security checklist like this. And it's looking for certain things that as developers we should know at least a little something about. The first thing would be injection, particularly SQL injection. So coming into the meeting knowing what SQL injection is and understanding exactly how Django prevents that with the ORM and highlighting if we're using SQL, if we drop down into the SQL and bypass the Django ORM we call that out in our own information that we hand out, hand to the security team. The second thing would be things like cross-site scripting and then also cross-site request forgery. These are things that Django has pages on. And the great part is about these is that each one of these top 10 items we don't have to reinvent the wheel as Django developers. There's a great talk by Jacob Kaplan Moss that talks about every single one of these security vulnerabilities particularly in the context of Django and how Django handles them. So you can watch that talk and read a blog post which I'll give to you which maps these security vulnerabilities versus what Django does. And obviously you're going to have to adapt it to your own code that you've written but what that means is that a lot of the heavy lifting has been done by people who've solved these problems before. So that is in general how I approach security audits. Again, remember, a security team is often not nearly as technical as we are and therefore we want to be speaking their language so that we don't end up in an adversarial relationship with them. Another thing is transferring legacy data. This happens constantly. We're building new systems in an ongoing ecosystem. The first situation which can often be the cleanest is we're just building a new application on a legacy database. There's already stuff out there and we don't have to worry about another application touching that database. It is our database that we're working with. It has existing data, it has existing relationships. We just need to know how to map our Django application to that. Fortunately we do not have to do this manually. There's something called InspectDB. So once we've configured our Django app to read from a database, we can do InspectDB and we can pipe that into a or redirect that into a models.py file and then it will generate the schema for us. Now we may not know exactly what's in that models.py file so we could look through it and we could read through everything which certainly we're going to have to do at some point and figure out the structure of the database. But I'm a visual person and so there's another thing that we can use which is we can do a pip install Django extensions and then we can run a manage.py graph models with some output and it will generate, getting off of our whiteboard for a second, it will generate a file, a picture that looks like this that actually shows us the structure of the database. And this is fantastic for getting up to speed on what is already out there. We don't have to hand figure out, okay, what table connects to what other table, what are the foreign key relationships, we can literally visualize this and generate our models.py file. Now obviously that's the beginning of the journey. We'll need to migrate tables, we'll need to figure out if that structure is appropriate for our application but it certainly is a starting point that we can get to very quickly and start understanding what that legacy database contains for our application. Now this situation is not always that simple. Sometimes there is an existing application and the requirement is well we just need to get some of the data out of this existing legacy database. So there can be a temptation because Django, believe it was as a 1.3 or 1.4, you can use multiple databases. So you may be tempted to say well we have our own database, we're also going to have our legacy database, we'll just use two databases for our Django application. However, I recommend against doing this, if at all possible, in enterprises weird stuff happens, you don't always have control over the situation, sometimes there's some sort of political battle that is preventing you from the optimal scenario and you may have to directly connect to the database. You can do this, but what I generally recommend is having some sort of wrapper around that existing application. If there's already a team that is developing software for that other application, have them generate an interface, some sort of API that allows you to pull whatever data you need out of it on a regular basis. The danger if you do what I've xed out here is that other systems make, other teams and systems may come along and say oh you've got access to that data, can you write me an API? But you don't actually have control over that database. Someone else is touching that database and so what ends up happening is you become this dependency that you never intended to be. So it's better to push it down onto the application that it really does have control over that database. Now, there's one of my other favorite applications. At my first job they were called end user computing applications and they looked something like this. They were an Excel spreadsheet. The great part was that the company I was working for, a very large company, they had so many, literally hundreds of these things and they had a lot of errors in them. There's no unit testing of Excel spreadsheets and what ended up happening was they had all these cascading errors and they had to restate financials to the tune of hundreds of millions of dollars because they kept having errors propagate throughout all of their financial systems that were basically this. We're talking about major financial institution in the country. Enterprises are very strange places. Now fortunately some environments recognize this is not a sustainable situation. So they'll ask you as a developer to come in and build a system that replaces a spreadsheet. But the caveat is, we want all the data that's in this spreadsheet to be in the system on day one, so we have a hard cut over. So when we log into that system, it has all the data that we're expecting in our Excel spreadsheet. There are some tools that will help us do this. Excel RD and Excel WT, and I'll tell you exactly how you should use them. The first one is for reading Excel spreadsheets, so you can read Excel spreadsheets directly. But I recommend, strongly, strongly recommend that you have a lot of data validation and that you agree beforehand, you say what is the input that we should be taking in, and we will reject anything that does not match that criteria. Column B should have only numbers, and if there are any letters, we reject that, and we will spit that back out as an Excel spreadsheet with Excel WT. So we'll give you back an Excel spreadsheet of all the stuff that does not pass data validation. We're not going to continue to re-ingest this information, but we will suck in as much as we can, and then your team who handles the business side of things, you can figure out what you need to correct. So you push the data correction onto the team that has to hand figure out, well, why did we put, you know, a number in a sal... or why did we put a bunch of letters in a salary column? I mean, this is literally the type of errors that happen in Excel spreadsheet. There's just no validation. So you have validation on your side when you suck in the data, and then you just say, you have exception scenarios where you just pump out and Excel spreadsheet says, you figure this out, and we're happy to take it in when you've fixed the problems. That is by far the best way, and probably one of the only ways to make this a sustainable situation where you can cut over on day one. All right. So another thing is securing packages and installations. There's two things I want to talk about here. Number one is I've gotten on client site, and I've done pip install Django, and the thing fails. Well, often we're going through a proxy server, and they're funneling all traffic through that proxy server. Now, fortunately, pip has a setting. It has a setting where you can specify a proxy server, but also as a leader versions, it will respect this environment variable. So you can export the HTTPS on HTTPS underscore proxy environment variable, and pip will respect that. It'll say, okay, I'm going to go through this proxy server, and then it won't begin installing your packages. Just make sure that you set this every time. I've set it before, and then I forget to set it in my environment variables, and then my pip will fail, and I'm like, what's the proxy server again? So just make sure that if you're going to set this, just set it somewhere where you know what the proxy server is before you use pip every time, otherwise it can fail. Another way is this is our standard setup. So we're used to as developers, you know, we got to PyPI, or PyPI, rather, and we download the packages, and we have them installed in our virtual M, and we're good to go. But this can be very risky for enterprises. They say, we don't have control over this PyPI environment. We don't really care that it's a, you know, a community resource or anything like that. What they care about is like, they want to control that. They want to run their security audit process, and they want to run their scans on a central repository, and that is understandable. There are some malicious things that can happen to central repositories that you do not own. And so what you really want is something more like this. This is all self-contained in the enterprise, and you can certainly do this. In fact, it actually does not take that long to set up a PyPI environment of your own. The only thing is you're going to have to upload and establish all of the packages that you need there, and then make sure the developers are not installing from the central PyPI that all developers in the enterprise know that this is a centralized repository in the enterprise itself. So the only thing about this is if you've got one team that's working on a Django project, and they have to be the ones who are uploading all the packages and making sure everything is scanned, it can be a lot of overhead. So I would fight back against this unless you have multiple teams that are willing to share the burden of hosting your own PyPI server. But it is certainly doable. And then the final thing is what are these dynamically typed languages? As I said in the beginning, it's always amazing to me when CIOs are like, I don't know about this whole Python thing. It's dynamically typed. I'm like, you haven't programmed a lot of code in 20 years. What does it matter to you? But it's so funny. So what I used to do is I would always say, look, if Python is good enough for Google, it's good enough for this big company. And the CIO would be like, but we're not Google. And I found that there was a lot of pushback. They're like, we would like to be that place eventually, but we're not there right now. And so the way to mitigate this, the way to mitigate this with an executive team or people that are pushing back against dynamically typed programming languages is to actually point out all the places that they're already using it. Look, we've been using Python in all these places for the past 10 years. And guess what? The enterprise has not blown up yet. So if you start going around to use, you take a look at all of the tools that your enterprise is using, that there are a lot of these are actually have Python that you're using for, you're actually using Python with them already. So for example, WebSphere uses Python to set up its configuration. You can write Python scripts to auto generate or to do the WebSphere configuration. And so, okay, there's one use case of Python already being in the environment. If you're using Ansible or using SaltStack or even if you're using Chef or Puppet, those are dynamically typed, they're built on top of dynamically typed languages. Each extraction and transformation taking data out of one system and putting it into another. You point out all the situations in which Python and dynamically typed languages, all of them are already being used in the enterprise. And that can be a powerful thing for someone who is just trying to say, well, we don't want to introduce too much change into the enterprise right now. Say you're already using it and therefore it's obviously safe for consumption because it's already been here for 10 years. Now, after that, then you talk about something like, here's the respected peer organizations that are building systems with Python and Django. And there's a bonus, if you can, if you know that there are certain organizations they bring up and they're like, you know, these are the leading organizations in our industry, not necessarily like Google or Facebook or something like that, but if they're in government space or in the financial space and you know that these group of executives always talk about, this company is doing great things, you can find out what those other companies are doing. And the majority of the time, like Bank of America is doing a ton of work with Python. They don't really talk about it too much, but it is something that is out there. So if you're in the financial industry and executives are pushing back on, well, we don't want to use dynamically typed languages or Django, but they're always talking about how great Bank of America runs some of their systems or how, you know, they're able to respond quickly with their IT systems, you say, by the way, like, they're using this already, and that may be one of the reasons why they're outflanking us here. And so that puts a little bit of pressure from the standpoint of like social proof. You're basically saying, look, you don't want to be, there's a few different risks there. Like in the first category, you're mitigating the risk of like, we don't use this already. So you're taking that consideration off the table. Then what you're doing here is you're mitigating like, look, your peers are using this, and they're outrunning you. And that is a fear that many executives have. They're worried about being overrun by other companies. And so you can point out like, look, these other organizations are already using dynamically typed languages. And finally, I do generally have a laundry list of the leading tech companies, and I talk about, here's exactly what these tech companies are using Python for, and these companies are doing really great things with small development teams. And if that's the direction of your organization, then that's probably something you should be thinking about as well. You basically want to start crossing down the list of all the rebuttals that they can have around dynamically typed languages. So we covered a few things here. First and foremost, integrating Active Directory. It's not something that should be too scary. There's some great resources out there with, and using Python LDAP. This is not something that should take you two weeks. You can generally write this stuff in about a day and be done with it and never think about it again, and reuse it on whatever project you go to. Passing security audits talk in their language. Instead of just talking in software development language, it can take a little while to get used to that, but you remove the adversarial relationship between the development team and the security team. Adjusting legacy data, there's a few scenarios here. If you're doing a hard cutover, then you can do InspectDB. You can do graph the data model and figure out how it works. And if you're talking to an existing system, I recommend using an API as opposed to using a set and just connecting directly to the database, because that can be an unsustainable situation. And then also with Excel spreadsheets, make sure you do the data validation and spit out the errors and let them correct the errors as opposed to trying to correct the errors in your own system. Securing dependency installations, PIP will respect the HTTPS proxy variable, but you can also set up your own PIPI if that's something that your enterprise is large enough to maintain. And then finally, correcting misconceptions around dynamic typing. There's a approach that tends to work very well. Obviously, every culture is different. All the pushback from executives can be different, but there is a structured approach that will allow you to just say, you know, this is something that is not too risky. This is something that allows leading institutions to be able to use Python and use it well. All right, so I told you I'd give you guys the resources. I know not everyone has a total cell phone service in here, but if you send a text message, I'll leave this up for just a minute. If you send a text message to this number, it's 503-476-3056. It will respond back with a text message. This is totally anonymous. If you could give me a score of 1 to 10, 1 being the lowest, and 10 being the highest, would you recommend this talk to a friend or a colleague? This allows me to determine, is this a talk that is something that is valuable to the audience, or is this something that maybe I need to work significantly on and maybe change out the topic so that it can be more appropriate based on the talk title? A lot of times we don't necessarily get feedback. This allows me to get feedback based on the audience's reaction. So if you send a text message, the benefit to you guys is you get the link that has all the resources to the blog post, has a link to the presentation, a link to the presentation source code, everything you need that I've talked about throughout this talk. So again, 503-476-3056, and this will be up for a while. So if you don't have cell phone service in here, just send it when you go outside and you'll get the link back. Okay, one more little quick plug. I'm the author of FullStackPython.com. If you're looking for resources around Python, this is like my passion project for 2014. I put a lot of time and effort into it. I actually just crossed 90,000 readers this year. I'm super stoked that this really helps people. This came out of me just writing emails to a lot of developers and saying, here's some great resources. I put it on the Internet for everyone in the community to be able to consume. My name is Matt McKay. I'm a developer evangelist with Twilio. Here's my contact information. Thank you all very much for having me, and I will answer questions outside afterwards. Thank you.
solve some of these problems. So like a good integrating with Active Directory. It's production. The second thing is passing security developer. But in general, in large transferring legacy data. This happens typed programming languages is to actually point building systems with Python and Django. And out like, look, these other organizations are that they can have around dynamically typed