What Developers and Economists Can Learn from Each Other!
Jefferson, Rob
(Note: This talk was originally scheduled to appear at !!Con 2015.) In the past twenty-ish years, I've had a variety of professional roles: sysadmin, researcher, shipping-container-farm designer, grad student in economics, and most recently developer and devops engineer. One of my favorite things is using what I've learned in one field in others. When I began doing development, there were skills and practices I was learning that would have helped me as an economist (writing good tests, modular software design). Similarly, I learned lessons while studying economics that I've taken with me to development (recognizing endogeneity biases, making technical decisions at the margin, using feedback to automate processes). There are also things that both developers and economists can and should learn, especially about the limitations of their respective toolkits, as well as the importance of working with compassion and empathy. In this talk, I'd like to share some of these lessons, with examples of how the lessons connect with each other.
 I want to thank the Bang Bang Con committee for letting me do this talk. It was originally scheduled for last year, but last year at this time I was in bed with a two week migraine. So today is vastly better. So I want to start out with saying, well, this isn't an Econ conference, so why am I talking about programmers and economists? Well, economists do a ton of programming from the first class to dissertation work, to current research, you're doing a lot of programming as an economist. And the programs often have serious policy implications because fairly frequently you're generating statistical models, those results get published in papers, or they're part of policy analyses, and also sorts of important decisions are made based on the output of these programs. And I find that tech and econ cultures have a lot of similarities, and I'll get to that in a bit. Finally, some parts of the economics philosophy have changed my views on programming and vice versa. A little bit about me. I was at Carnegie Mellon for two years, studied CS, but I can only remember about 10% of it on a good day, and I left for reasons. And then I worked for a while doing business analysis, consulting, I was an assistant man. I went back to CMU, studied economics this time, finally got my degree after a while, and I was accepted in grad school when I went to Northwestern for a bit, started an econ PhD program, spent about two years there, and left again for reasons. And then I worked for Solutions for Progress in Philly, first as a research assistant. If you're interested, ask me some time about the farm. And after that, a software engineer, because the company needed someone in the systems group who could code Perl, and I could do so poorly, so it was better than working on a farm. And now I'm working for Comcast as a DevOps engineer or as I prefer calling it dev oops. So I want to begin with what economists can learn from programmers or developers or sysadmins or what have you. So sustainable development, as I said before, modern econ is programming, statistical modeling, so much of it is part of empirical work. Empirical work is one of the standard things in economics nowadays. Computational econ is increasingly popular. And so this is basically simulating economic agents using AI techniques. You might have heard of agent-based modeling. General purpose graphics processing units, they're available, they're getting cheaper. They make large-scale optimization problems faster and cheaper to solve because most of what economists do boils down to linear algebra. And GPUs are excellent at doing linear algebra really, really fast. Finally, algorithmic game theory. It's of considerable interest in both CS and economics, and there's a lot of overlap in that field. And you find this a lot in things like auctions, like eBay or all-pay auction sites. You even find algorithmic game theory in things like load balancing and routing. But it turns out that coding gets short-shrift. When you're a grad student, you're spending most of your time in classes. You don't really have actual time to do it right. You just have to get some results. There's no formal training. You're thrown in. You get a problem set. There's very little direction. You wind up using a lot of code written by a handful of people with no documentation, but someone in a previous class or in another group told you, yeah, this is what you need to get it to run. Programming, data munging, debugging. If you're lucky, you can get a research assistant to do it. But of course, when you are that research assistant, you got some problems, possibly. You got no time to learn FORTRAN. And there's a bunch of econ code out there written in FORTRAN. But there's no time to rewrite it in Python, Ivor. So what do you do? And if you're real unlucky, you might just get an executable. And you've got this black box. And of course, there's no time to reverse engineer it. So you work, work, work, and a few years later, I don't remember making this, but it works. Why hope it works? You don't know. However, there's hope. There's something called a Quanticon project, which has blossomed since last year when I was to give this talk. It's basically open source Python and Julia libraries. There's a bunch of tests. It's got backing from various research groups. It's fantastic so far. The software carpentry. Basically teaching general skills of writing with modules, writing sustainable code. And then there's FedRed, which is the Fed Reserve economic data website in Kwandol. That's a site with pay and free and paid data sets. And these offer standardized API. So you have to do a little less munging than you previously had to. And the Open Knowledge Foundation, which has all sorts of resources for people who never had the data courses or the data analysis courses, but it's an easy way to get started on that. Next thing to learn, algorithms. And you don't need to be an algorithms expert. You don't need your intro to algorithms book from undergrad if you've had it or whatnot. But it's kind of good at the grasp on asymptotic behavior because data sets are often bigger than you expect. But they're usually smaller than you hope. And you have to be careful about you've got an N squared algorithm. You're running a huge data set. It could very well take a while. And if you've taken econometrics courses, you find that some of the ideas of statistical asymptotic behavior move over to algorithmic asymptotic behavior. When I was doing undergrad research, one of the things that I found myself doing over and over and over was nesting loops, sometimes three, four, five layers deep. Time to take a step back because I'm probably duplicating things. I can probably think about some shortcuts to make the calculations a little bit faster. And finally, don't do more than you have to. Use techniques to pre-compute things like memoization, dynamic programming. Really testing. You've got results, but are they consistent? I don't know. Did you make sure you have your controls? I think so. You made a complicated model. Are its implications what you thought they would be? Well, I wrote them on paper, but I don't know. And finally, you're using your colleagues' libraries and their floating point and your colleague doesn't really understand IEEE 754. So what you think you might be getting isn't necessarily what you're actually going to get. So how bad could it be? Well, here's a few disasters. One is Donahue and Leavitt paper, the abortions and crime paper, basically showing, trying to demonstrate that there is a link between abortions in the 70s and declining crime rate 20 years later. It turns out that the code neglected to include the controls. So forgot to control for various things. And once you corrected that, the impact was just about zero. There was another paper from Marty Feldstein claiming that social security depressed the savings. There is an error in that that once, in the code that once fixed, reduced that to virtually zero. And finally, most recently, the Reinhardt and Rogoff paper claiming that high debt levels reduced growth. This is important in part because it has significant policy implications for EU austerity. And in fact, it was a paper used to justify it. But it turned out that there were significant Excel copy-paste errors, which totally overstated the relationship. Whoops. So what can we do? We can learn, economists should learn to write tests, unit tests for small model components, regression tests for full models, and regression tests to make sure that the regressions as where you developed actually, you know, the results don't change dramatically if you change small features in your code. You standard libraries, they're out there. They work most of the time. There's usually documentation, explicit checklist for control variables. You think you've included all of them? You probably haven't. Finally use GitHub or GitLab or some collaborative system because you can get other people looking at your code, you can do code reviews, and finally you get version control in the deal, which when I was studying economics, most people didn't use it. Now I want to turn to what programmers can learn from economists. First causation. This says lurking variable kitty where you least expect it. So modeling in one slide. So in any sort of model, you've got factors X, you want to see how they, what impact they have on some other variable Y. You build models to estimate those effects. One of the core assumptions is that X is uncorrelated with Y. Frequently it's not true. We hope it's true. When it doesn't happen, you have endogeneity bias. The most popular way to deal with that is replace that X with a never variable that's correlated with X, but not with Y, and that's called an instrument, then you rerun the regression. I want to use this to troubleshoot like an economist, but wonders, endogeneity bias everywhere. It's more difficult to avoid and tech debugging in part because instruments are terribly difficult to find. You can't really ignore institutional effects. It turns out that seemingly technical causes are actually policy related. A problem, a programming problem I had at work a couple of weeks ago was actually due to the code. The code I was dealing with was actually correct, but I was attempting to place, but it apparently violated certain policy rules that went into developing the program. This was something that took me a long time to figure out because I was focused on the technology, but not the policy. Finally, in tech and economic modeling, it's important to understand the cultural and historical context. Economic models make all sorts of assumptions about culture and history. It doesn't necessarily look like it a simple regression, but we frequently assume that these have normal growth and spending patterns and any deviations or what we're looking for. We tacitly assume some sort of normal. Similarly in programming, we make all sorts of assumptions even beyond internationalization and localization. The classic example being assuming that someone has both a first and last name. Of course, there's many other assumptions that we make and get broken at our peril. Next I want to talk about specialization in trade, which is something that I think we can learn about. It's a little contentious because I think generalists are awesome, but life's too short to do everything. Do you really want to write your own network stack for production? I don't. Even a skilled generalist can benefit from specializing and novices can help with that too. Specializing is a lot less useful without trade. If no one wants your cobalt expertise, there's little benefit in focusing on it. We come to comparative advantage. This is David Ricardo who first developed the idea in the late 18th century. I want first absolute advantage. Alice is better at writing network code than Barb. And Barb is better at building numerical models. Hence they should just do what they do best. On the other hand, comparative advantage, Alice can write network code and numerical models better than Barb. But Alice is better at numerical models than she is at network code. Hence Alice should take on the numerical models, Barb writes the network code. They can both work at the same time and so they can get more done than I've ever alone. But this is a hasty generalization. Comparative advantage demonstrates what's possible, not necessarily what's best. Finally, I want to talk about what both economists and developers can learn and have to learn. One of them, and really the main thing is community reflection and action. This is where the communities are similar. This is the classic XKCD comic about fields arranged by purity. Economics wants to be somewhere around mathematicians. In terms of practice, it's somewhere between sociologists and psychologists. There's a premium on the hardcore. Both tech and econ spend a lot of time on hierarchies. In tech, OS hackers are hardcore and real. Web devs are not. In economics, the hierarchies theorists are at the top. Then there's industrial organization people and economic historians, such as myself, are not. Both have forums that advance these sorts of ideas, in tech there's hacker news and in econ there's econ job market rumors, which is basically the 4chan of economics. Take that how you will. We are only as good as what we tolerate. We need to critically examine our fields. We need to look at these arbitrary classes of real programmers and real economists, reevaluating privileging theory over what actually happens. In tech, we need to reassess our privileging of tools over people. By people, I mean devs and users support. Minorities in both fields are underrepresented and discouraged. I was the only, when I went to grad school, I was the only active African American in my department, that student faculty and I think staff. There was one professor who had retired a couple of years before. I felt very out of place. I also felt largely out of place until relatively recently in tech. In both fields, racism, sexism, homophobia, transphobia are rampant. I can talk about things I've seen in econ offline. But what can we do? One is more inclusive events, not just across race, gender and class, but also skills and interests. Perhaps there could be an econ equivalent of bang bang con sometime. Make it easier to get into technical and economic discourse. Blogs and Twitter have made a significant shift in voices and econ over the past few years. Be patient and listen. As Katzmall noted yesterday, that's to refrain from approaching a problem tools first, whether you're trying to impose an economic model on your observations or a programming language or libraries on a programming task. And finally, this talk is dedicated to my colleague Tiago Perez who passed away two weeks ago. We were classmates at Northwestern and his birthday was on May 4th, so May 4th always be with him. Thank you. I hope you enjoyed the video.
 I want to thank the Bang Bang Con committee for letting me do this talk.  It was originally scheduled for last year, but last year at this time I was in bed with  a two week migraine.  So today is vastly better.  So I want to start out with saying, well, this isn't an Econ conference, so why am I  talking about programmers and economists?  Well, economists do a ton of programming from the first class to dissertation work, to current  research, you're doing a lot of programming as an economist.  And the programs often have serious policy implications because fairly frequently you're  generating statistical models, those results get published in papers, or they're part of  policy analyses, and also sorts of important decisions are made based on the output of  these programs.  And I find that tech and econ cultures have a lot of similarities, and I'll get to that  in a bit.  Finally, some parts of the economics philosophy have changed my views on programming and vice  versa.  A little bit about me.  I was at Carnegie Mellon for two years, studied CS, but I can only remember about 10% of it  on a good day, and I left for reasons.  And then I worked for a while doing business analysis, consulting, I was an assistant man.  I went back to CMU, studied economics this time, finally got my degree after a while,  and I was accepted in grad school when I went to Northwestern for a bit, started an econ  PhD program, spent about two years there, and left again for reasons.  And then I worked for Solutions for Progress in Philly, first as a research assistant.  If you're interested, ask me some time about the farm.  And after that, a software engineer, because the company needed someone in the systems  group who could code Perl, and I could do so poorly, so it was better than working on  a farm.  And now I'm working for Comcast as a DevOps engineer or as I prefer calling it dev oops.  So I want to begin with what economists can learn from programmers or developers or sysadmins  or what have you.  So sustainable development, as I said before, modern econ is programming, statistical modeling,  so much of it is part of empirical work.  Empirical work is one of the standard things in economics nowadays.  Computational econ is increasingly popular.  And so this is basically simulating economic agents using AI techniques.  You might have heard of agent-based modeling.  General purpose graphics processing units, they're available, they're getting cheaper.  They make large-scale optimization problems faster and cheaper to solve because most of  what economists do boils down to linear algebra.  And GPUs are excellent at doing linear algebra really, really fast.  Finally, algorithmic game theory.  It's of considerable interest in both CS and economics, and there's a lot of overlap in  that field.  And you find this a lot in things like auctions, like eBay or all-pay auction sites.  You even find algorithmic game theory in things like load balancing and routing.  But it turns out that coding gets short-shrift.  When you're a grad student, you're spending most of your time in classes.  You don't really have actual time to do it right.  You just have to get some results.  There's no formal training.  You're thrown in.  You get a problem set.  There's very little direction.  You wind up using a lot of code written by a handful of people with no documentation,  but someone in a previous class or in another group told you, yeah, this is what you need  to get it to run.  Programming, data munging, debugging.  If you're lucky, you can get a research assistant to do it.  But of course, when you are that research assistant, you got some problems, possibly.  You got no time to learn FORTRAN.  And there's a bunch of econ code out there written in FORTRAN.  But there's no time to rewrite it in Python, Ivor.  So what do you do?  And if you're real unlucky, you might just get an executable.  And you've got this black box.  And of course, there's no time to reverse engineer it.  So you work, work, work, and a few years later, I don't remember making this, but it works.  Why hope it works?  You don't know.  However, there's hope.  There's something called a Quanticon project, which has blossomed since last year when  I was to give this talk.  It's basically open source Python and Julia libraries.  There's a bunch of tests.  It's got backing from various research groups.  It's fantastic so far.  The software carpentry.  Basically teaching general skills of writing with modules, writing sustainable code.  And then there's FedRed, which is the Fed Reserve economic data website in Kwandol.  That's a site with pay and free and paid data sets.  And these offer standardized API.  So you have to do a little less munging than you previously had to.  And the Open Knowledge Foundation, which has all sorts of resources for people who never  had the data courses or the data analysis courses, but it's an easy way to get started  on that.  Next thing to learn, algorithms.  And you don't need to be an algorithms expert.  You don't need your intro to algorithms book from undergrad if you've had it or whatnot.  But it's kind of good at the grasp on asymptotic behavior because data sets are often bigger  than you expect.  But they're usually smaller than you hope.  And you have to be careful about you've got an N squared algorithm.  You're running a huge data set.  It could very well take a while.  And if you've taken econometrics courses, you find that some of the ideas of statistical  asymptotic behavior move over to algorithmic asymptotic behavior.  When I was doing undergrad research, one of the things that I found myself doing over and  over and over was nesting loops, sometimes three, four, five layers deep.  Time to take a step back because I'm probably duplicating things.  I can probably think about some shortcuts to make the calculations a little bit faster.  And finally, don't do more than you have to.  Use techniques to pre-compute things like memoization, dynamic programming.  Really testing.  You've got results, but are they consistent?  I don't know.  Did you make sure you have your controls?  I think so.  You made a complicated model.  Are its implications what you thought they would be?  Well, I wrote them on paper, but I don't know.  And finally, you're using your colleagues' libraries and their floating point and your  colleague doesn't really understand IEEE 754.  So what you think you might be getting isn't necessarily what you're actually going to  get.  So how bad could it be?  Well, here's a few disasters.  One is Donahue and Leavitt paper, the abortions and crime paper, basically showing, trying  to demonstrate that there is a link between abortions in the 70s and declining crime rate  20 years later.  It turns out that the code neglected to include the controls.  So forgot to control for various things.  And once you corrected that, the impact was just about zero.  There was another paper from Marty Feldstein claiming that social security depressed the  savings.  There is an error in that that once, in the code that once fixed, reduced that to virtually  zero.  And finally, most recently, the Reinhardt and Rogoff paper claiming that high debt levels  reduced growth.  This is important in part because it has significant policy implications for EU austerity.  And in fact, it was a paper used to justify it.  But it turned out that there were significant Excel copy-paste errors, which totally overstated  the relationship.  Whoops.  So what can we do?  We can learn, economists should learn to write tests, unit tests for small model components,  regression tests for full models, and regression tests to make sure that the regressions as  where you developed actually, you know, the results don't change dramatically if you change  small features in your code.  You standard libraries, they're out there.  They work most of the time.  There's usually documentation, explicit checklist for control variables.  You think you've included all of them?  You probably haven't.  Finally use GitHub or GitLab or some collaborative system because you can get other people looking  at your code, you can do code reviews, and finally you get version control in the deal,  which when I was studying economics, most people didn't use it.  Now I want to turn to what programmers can learn from economists.  First causation.  This says lurking variable kitty where you least expect it.  So modeling in one slide.  So in any sort of model, you've got factors X, you want to see how they, what impact they  have on some other variable Y.  You build models to estimate those effects.  One of the core assumptions is that X is uncorrelated with Y.  Frequently it's not true.  We hope it's true.  When it doesn't happen, you have endogeneity bias.  The most popular way to deal with that is replace that X with a never variable that's correlated  with X, but not with Y, and that's called an instrument, then you rerun the regression.  I want to use this to troubleshoot like an economist, but wonders, endogeneity bias everywhere.  It's more difficult to avoid and tech debugging in part because instruments are terribly difficult  to find.  You can't really ignore institutional effects.  It turns out that seemingly technical causes are actually policy related.  A problem, a programming problem I had at work a couple of weeks ago was actually due  to the code.  The code I was dealing with was actually correct, but I was attempting to place, but it apparently  violated certain policy rules that went into developing the program.  This was something that took me a long time to figure out because I was focused on the  technology, but not the policy.  Finally, in tech and economic modeling, it's important to understand the cultural and historical  context.  Economic models make all sorts of assumptions about culture and history.  It doesn't necessarily look like it a simple regression, but we frequently assume that  these have normal growth and spending patterns and any deviations or what we're looking for.  We tacitly assume some sort of normal.  Similarly in programming, we make all sorts of assumptions even beyond internationalization  and localization.  The classic example being assuming that someone has both a first and last name.  Of course, there's many other assumptions that we make and get broken at our peril.  Next I want to talk about specialization in trade, which is something that I think we  can learn about.  It's a little contentious because I think generalists are awesome, but life's too short  to do everything.  Do you really want to write your own network stack for production?  I don't.  Even a skilled generalist can benefit from specializing and novices can help with that  too.  Specializing is a lot less useful without trade.  If no one wants your cobalt expertise, there's little benefit in focusing on it.  We come to comparative advantage.  This is David Ricardo who first developed the idea in the late 18th century.  I want first absolute advantage.  Alice is better at writing network code than Barb.  And Barb is better at building numerical models.  Hence they should just do what they do best.  On the other hand, comparative advantage, Alice can write network code and numerical  models better than Barb.  But Alice is better at numerical models than she is at network code.  Hence Alice should take on the numerical models, Barb writes the network code.  They can both work at the same time and so they can get more done than I've ever alone.  But this is a hasty generalization.  Comparative advantage demonstrates what's possible, not necessarily what's best.  Finally, I want to talk about what both economists and developers can learn and have to learn.  One of them, and really the main thing is community reflection and action.  This is where the communities are similar.  This is the classic XKCD comic about fields arranged by purity.  Economics wants to be somewhere around mathematicians.  In terms of practice, it's somewhere between sociologists and psychologists.  There's a premium on the hardcore.  Both tech and econ spend a lot of time on hierarchies.  In tech, OS hackers are hardcore and real.  Web devs are not.  In economics, the hierarchies theorists are at the top.  Then there's industrial organization people and economic historians, such as myself, are  not.  Both have forums that advance these sorts of ideas, in tech there's hacker news and  in econ there's econ job market rumors, which is basically the 4chan of economics.  Take that how you will.  We are only as good as what we tolerate.  We need to critically examine our fields.  We need to look at these arbitrary classes of real programmers and real economists, reevaluating  privileging theory over what actually happens.  In tech, we need to reassess our privileging of tools over people.  By people, I mean devs and users support.  Minorities in both fields are underrepresented and discouraged.  I was the only, when I went to grad school, I was the only active African American in  my department, that student faculty and I think staff.  There was one professor who had retired a couple of years before.  I felt very out of place.  I also felt largely out of place until relatively recently in tech.  In both fields, racism, sexism, homophobia, transphobia are rampant.  I can talk about things I've seen in econ offline.  But what can we do?  One is more inclusive events, not just across race, gender and class, but also skills and  interests.  Perhaps there could be an econ equivalent of bang bang con sometime.  Make it easier to get into technical and economic discourse.  Blogs and Twitter have made a significant shift in voices and econ over the past few  years.  Be patient and listen.  As Katzmall noted yesterday, that's to refrain from approaching a problem tools first, whether  you're trying to impose an economic model on your observations or a programming language  or libraries on a programming task.  And finally, this talk is dedicated to my colleague Tiago Perez who passed away two  weeks ago.  We were classmates at Northwestern and his birthday was on May 4th, so May 4th always  be with him.  Thank you.  I hope you enjoyed the video.
__Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_0_71.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_72_119.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_120_287.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_288_383.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_384_479.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_480_2735.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_2736_5351.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_5352_7703.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_7704_9215.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_9216_9503.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_9504_9623.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_9624_13931.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_13932_16919.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_16920_18695.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_18696_18827.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_18828_18887.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_18888_20339.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_20340_23867.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_23868_24839.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_24840_25823.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_25824_26051.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_26052_28199.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_28200_30023.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_30024_31919.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_31920_31967.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_31968_32015.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_32016_32231.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_32232_32351.png __Con_2016_-_What_Developers_and_Economists_Can_Learn_from_Each_Other__By_Rob_Jefferson_mwnOiblMWC0_32352_32401.png
I want to thank the Bang Bang Con committee for letting me do this talk. It was originally scheduled for last year, but last year at this time I was in bed with a two week migraine. So today is vastly better. So I want to start out with saying, well, this isn't an Econ conference, so why am I talking about programmers and economists? Well, economists do a ton of programming from the first class to dissertation work, to current research, you're doing a lot of programming as an economist. And the programs often have serious policy implications because fairly frequently you're generating statistical models, those results get published in papers, or they're part of policy analyses, and also sorts of important decisions are made based on the output of these programs. And I find that tech and econ cultures have a lot of similarities, and I'll get to that in a bit. Finally, some parts of the economics philosophy have changed my views on programming and vice versa. A little bit about me. I was at Carnegie Mellon for two years, studied CS, but I can only remember about 10% of it on a good day, and I left for reasons. And then I worked for a while doing business analysis, consulting, I was an assistant man. I went back to CMU, studied economics this time, finally got my degree after a while, and I was accepted in grad school when I went to Northwestern for a bit, started an econ PhD program, spent about two years there, and left again for reasons. And then I worked for Solutions for Progress in Philly, first as a research assistant. If you're interested, ask me some time about the farm. And after that, a software engineer, because the company needed someone in the systems group who could code Perl, and I could do so poorly, so it was better than working on a farm. And now I'm working for Comcast as a DevOps engineer or as I prefer calling it dev oops. So I want to begin with what economists can learn from programmers or developers or sysadmins or what have you. So sustainable development, as I said before, modern econ is programming, statistical modeling, so much of it is part of empirical work. Empirical work is one of the standard things in economics nowadays. Computational econ is increasingly popular. And so this is basically simulating economic agents using AI techniques. You might have heard of agent-based modeling. General purpose graphics processing units, they're available, they're getting cheaper. They make large-scale optimization problems faster and cheaper to solve because most of what economists do boils down to linear algebra. And GPUs are excellent at doing linear algebra really, really fast. Finally, algorithmic game theory. It's of considerable interest in both CS and economics, and there's a lot of overlap in that field. And you find this a lot in things like auctions, like eBay or all-pay auction sites. You even find algorithmic game theory in things like load balancing and routing. But it turns out that coding gets short-shrift. When you're a grad student, you're spending most of your time in classes. You don't really have actual time to do it right. You just have to get some results. There's no formal training. You're thrown in. You get a problem set. There's very little direction. You wind up using a lot of code written by a handful of people with no documentation, but someone in a previous class or in another group told you, yeah, this is what you need to get it to run. Programming, data munging, debugging. If you're lucky, you can get a research assistant to do it. But of course, when you are that research assistant, you got some problems, possibly. You got no time to learn FORTRAN. And there's a bunch of econ code out there written in FORTRAN. But there's no time to rewrite it in Python, Ivor. So what do you do? And if you're real unlucky, you might just get an executable. And you've got this black box. And of course, there's no time to reverse engineer it. So you work, work, work, and a few years later, I don't remember making this, but it works. Why hope it works? You don't know. However, there's hope. There's something called a Quanticon project, which has blossomed since last year when I was to give this talk. It's basically open source Python and Julia libraries. There's a bunch of tests. It's got backing from various research groups. It's fantastic so far. The software carpentry. Basically teaching general skills of writing with modules, writing sustainable code. And then there's FedRed, which is the Fed Reserve economic data website in Kwandol. That's a site with pay and free and paid data sets. And these offer standardized API. So you have to do a little less munging than you previously had to. And the Open Knowledge Foundation, which has all sorts of resources for people who never had the data courses or the data analysis courses, but it's an easy way to get started on that. Next thing to learn, algorithms. And you don't need to be an algorithms expert. You don't need your intro to algorithms book from undergrad if you've had it or whatnot. But it's kind of good at the grasp on asymptotic behavior because data sets are often bigger than you expect. But they're usually smaller than you hope. And you have to be careful about you've got an N squared algorithm. You're running a huge data set. It could very well take a while. And if you've taken econometrics courses, you find that some of the ideas of statistical asymptotic behavior move over to algorithmic asymptotic behavior. When I was doing undergrad research, one of the things that I found myself doing over and over and over was nesting loops, sometimes three, four, five layers deep. Time to take a step back because I'm probably duplicating things. I can probably think about some shortcuts to make the calculations a little bit faster. And finally, don't do more than you have to. Use techniques to pre-compute things like memoization, dynamic programming. Really testing. You've got results, but are they consistent? I don't know. Did you make sure you have your controls? I think so. You made a complicated model. Are its implications what you thought they would be? Well, I wrote them on paper, but I don't know. And finally, you're using your colleagues' libraries and their floating point and your colleague doesn't really understand IEEE 754. So what you think you might be getting isn't necessarily what you're actually going to get. So how bad could it be? Well, here's a few disasters. One is Donahue and Leavitt paper, the abortions and crime paper, basically showing, trying to demonstrate that there is a link between abortions in the 70s and declining crime rate 20 years later. It turns out that the code neglected to include the controls. So forgot to control for various things. And once you corrected that, the impact was just about zero. There was another paper from Marty Feldstein claiming that social security depressed the savings. There is an error in that that once, in the code that once fixed, reduced that to virtually zero. And finally, most recently, the Reinhardt and Rogoff paper claiming that high debt levels reduced growth. This is important in part because it has significant policy implications for EU austerity. And in fact, it was a paper used to justify it. But it turned out that there were significant Excel copy-paste errors, which totally overstated the relationship. Whoops. So what can we do? We can learn, economists should learn to write tests, unit tests for small model components, regression tests for full models, and regression tests to make sure that the regressions as where you developed actually, you know, the results don't change dramatically if you change small features in your code. You standard libraries, they're out there. They work most of the time. There's usually documentation, explicit checklist for control variables. You think you've included all of them? You probably haven't. Finally use GitHub or GitLab or some collaborative system because you can get other people looking at your code, you can do code reviews, and finally you get version control in the deal, which when I was studying economics, most people didn't use it. Now I want to turn to what programmers can learn from economists. First causation. This says lurking variable kitty where you least expect it. So modeling in one slide. So in any sort of model, you've got factors X, you want to see how they, what impact they have on some other variable Y. You build models to estimate those effects. One of the core assumptions is that X is uncorrelated with Y. Frequently it's not true. We hope it's true. When it doesn't happen, you have endogeneity bias. The most popular way to deal with that is replace that X with a never variable that's correlated with X, but not with Y, and that's called an instrument, then you rerun the regression. I want to use this to troubleshoot like an economist, but wonders, endogeneity bias everywhere. It's more difficult to avoid and tech debugging in part because instruments are terribly difficult to find. You can't really ignore institutional effects. It turns out that seemingly technical causes are actually policy related. A problem, a programming problem I had at work a couple of weeks ago was actually due to the code. The code I was dealing with was actually correct, but I was attempting to place, but it apparently violated certain policy rules that went into developing the program. This was something that took me a long time to figure out because I was focused on the technology, but not the policy. Finally, in tech and economic modeling, it's important to understand the cultural and historical context. Economic models make all sorts of assumptions about culture and history. It doesn't necessarily look like it a simple regression, but we frequently assume that these have normal growth and spending patterns and any deviations or what we're looking for. We tacitly assume some sort of normal. Similarly in programming, we make all sorts of assumptions even beyond internationalization and localization. The classic example being assuming that someone has both a first and last name. Of course, there's many other assumptions that we make and get broken at our peril. Next I want to talk about specialization in trade, which is something that I think we can learn about. It's a little contentious because I think generalists are awesome, but life's too short to do everything. Do you really want to write your own network stack for production? I don't. Even a skilled generalist can benefit from specializing and novices can help with that too. Specializing is a lot less useful without trade. If no one wants your cobalt expertise, there's little benefit in focusing on it. We come to comparative advantage. This is David Ricardo who first developed the idea in the late 18th century. I want first absolute advantage. Alice is better at writing network code than Barb. And Barb is better at building numerical models. Hence they should just do what they do best. On the other hand, comparative advantage, Alice can write network code and numerical models better than Barb. But Alice is better at numerical models than she is at network code. Hence Alice should take on the numerical models, Barb writes the network code. They can both work at the same time and so they can get more done than I've ever alone. But this is a hasty generalization. Comparative advantage demonstrates what's possible, not necessarily what's best. Finally, I want to talk about what both economists and developers can learn and have to learn. One of them, and really the main thing is community reflection and action. This is where the communities are similar. This is the classic XKCD comic about fields arranged by purity. Economics wants to be somewhere around mathematicians. In terms of practice, it's somewhere between sociologists and psychologists. There's a premium on the hardcore. Both tech and econ spend a lot of time on hierarchies. In tech, OS hackers are hardcore and real. Web devs are not. In economics, the hierarchies theorists are at the top. Then there's industrial organization people and economic historians, such as myself, are not. Both have forums that advance these sorts of ideas, in tech there's hacker news and in econ there's econ job market rumors, which is basically the 4chan of economics. Take that how you will. We are only as good as what we tolerate. We need to critically examine our fields. We need to look at these arbitrary classes of real programmers and real economists, reevaluating privileging theory over what actually happens. In tech, we need to reassess our privileging of tools over people. By people, I mean devs and users support. Minorities in both fields are underrepresented and discouraged. I was the only, when I went to grad school, I was the only active African American in my department, that student faculty and I think staff. There was one professor who had retired a couple of years before. I felt very out of place. I also felt largely out of place until relatively recently in tech. In both fields, racism, sexism, homophobia, transphobia are rampant. I can talk about things I've seen in econ offline. But what can we do? One is more inclusive events, not just across race, gender and class, but also skills and interests. Perhaps there could be an econ equivalent of bang bang con sometime. Make it easier to get into technical and economic discourse. Blogs and Twitter have made a significant shift in voices and econ over the past few years. Be patient and listen. As Katzmall noted yesterday, that's to refrain from approaching a problem tools first, whether you're trying to impose an economic model on your observations or a programming language or libraries on a programming task. And finally, this talk is dedicated to my colleague Tiago Perez who passed away two weeks ago. We were classmates at Northwestern and his birthday was on May 4th, so May 4th always be with him. Thank you. I hope you enjoyed the video.
letting me do this talk. It was originally programming as an economist. And the programs output of these programs. And I find that tech finally got my degree after a while, and I was some time about the farm. And after that, a software engineer, because the company needed work is one of the standard things in economics a few years later, I don't remember making this, skills of writing with modules, writing behavior. When I was doing undergrad research, virtually zero. And finally, most recently, the that there were significant Excel copy-paste learn, economists should learn to write tests, studying economics, most people didn't use it. like an economist, but wonders, endogeneity bias want to talk about what both economists and developers can learn and have to learn. One of economics, the hierarchies theorists are at the staff. There was one professor who had retired a