- D. Hillard, Stephen Purpura, J. Wilkerson, B. Jones, F. Baumgartner, R. Zeckhauser, Jesse Shapiro, Claire Cardie, E. Hovy, D. Lazer, Michael Neblo, K. Esterling, Aleks Jakulin, M. Baum, Jamie Callan, Micah Altman, David King, James E. Purpura, A. Gibson, J. Rigg, Wilkerson Hillard. 2007. An Active Learning Framework for Classifying Political Text. Abstract: We develop a framework and tools for applying a computer‐assisted context analysis system and find that it achieves levels of accuracy comparable to humans for about 80% less effort when starting from scratch (no labeled examples). The system is presented using a case study of Congressional bill titles as a proxy for the full text of Congressional bills. We also demonstrate that the system can use information learned from previous experiments to reduce the labeling requirements still further to over 90% savings of the current human effort. This study assumes that social scientists have a need to locate individual documents in a subject area. To support this need, "Topic classification," where documents are coded according to some organizing framework, is used to facilitate search and summarization. Our proposed framework for effectively employing machine learning methods mitigates the high costs of the standard method of topic classification ‐ human labeling. We scientifically evaluate the efficacy and accuracy of the automated approach using a large corpus of 380,000 human‐labeled events and a classification system that includes 20 major policy topics, 226 subtopics, and a demonstrably strong level of human inter‐ coder agreement.
