Academic Placement Data and Analysis: 2016 Final Report Carolyn Dicey Jennings∗1, Patrice Cobb1, Chelsea Gordon1, Bryan Kerster1, Angelo Kyrilov1, Evette Montes1, Sam Spevack1, David W. Vinson1 and Justin Vlasits2 1University of California, Merced 2University of California, Berkeley Contents 1 Funding Details 2 2 Overall Progress 2 3 Progress on Survey 3 3.1 Hiring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3.2 Survey Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3.3 Survey Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3.4 Survey Data Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 ∗contact: cjennings3@ucmerced.edu 1 4 Progress on Database Interface 4 4.1 Hiring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4.2 Database Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.3 Individual Editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.4 Project Personnel Editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 State of the Database 5 6 Evaluation and Outlook 6 1 Funding Details In June 2015 Carolyn Dicey Jennings applied for a $4,778 grant from the American Philosophical Association Small Grant Fund to sponsor graduate student research assistants. The proposed duties of the graduate student research assistants were to add an administrative interface to the website and run a qualitative survey. In December 2015 Jennings was notied that the board approved these funds. The funds were received at University of California, Merced in January 2016, at which point Jennings hired ve graduate student research assistants (in alphabetical order, with dedicated project hours in parentheses): Patrice Cobb, a graduate student from Psychological Sciences (40 hours); Chelsea Gordon, a graduate student from Cognitive and Information Sciences (40 hours); Bryan Kerster, a graduate student from Cognitive and Information Sciences (25 hours); Angelo Kyrilov, a graduate student from Electrical Engineering and Computer Science (65 hours); and Sam Spevack, a graduate student from Cognitive and Information Sciences (40 hours). From her own research funds Jennings was able to sponsor an undergraduate research assistant, Evette Montes. 2 Overall Progress This year we released a qualitative survey. We have the initial results from that survey and are currently analysing the results for publication. We also updated the database interface for the sake of eciency and project sustainability. Interim progress was reported in an update, partially described in a post at the APA blog. 2 3 Progress on Survey 3.1 Hiring Chelsea Gordon and Sam Spevack were hired in February to run the qualitative survey with the help of David Vinson. 3.2 Survey Design In March Chelsea Gordon, Sam Spevack, and David Vinson designed a survey in order to collect response data from recent philosophy graduates that reected satisfaction and self-description of their programs as well as satisfaction of job placement. They implemented a couple of versions of this study in April and May, collecting pilot data in May to ensure that they were eliciting useful responses from survey respondents. Based on the responses in the pilot study (distributed to graduate students in the Social Sciences, Humanities, and Arts program at UC Merced), they implemented a few changes to the survey and created a nal version to send to those individuals in the APDA database. 3.3 Survey Data Collection In June, Chelsea Gordon and Sam Spevack began to set up the data for analysis and wrote a few scripts that would take care of data pre-processing in order to nalize it for analysis. This involved things like stripping out punctuation, capitalization, and stop words (content-less words such as "the", etc.). Then they made a dictionary of all of the individual words in the dataset and pulled out the frequency for each of these terms, to get an idea of what key words or themes might be good to use in the nal analysis. Based on this content, they came up with a classication/coding scheme to pull out various sentences related to dierent themes in order to do theme-specic analyses (i.e., faculty-related, community-related, funding-related, content-related, etc.). While the number of full responses that we have currently is too low to make use of these theme-specic analyses, this scheme could prove very useful in the future once more data has been collected. 3 3.4 Survey Data Analyses Chelsea Gordon and Sam Spevack have a number of initial ndings from their work in June, which they are currently developing for publication. These include the following: • Gender is a signicant predictor of how participants answer the question "how likely are you to recommend this program to other prospective students?", with women being less likely to recommend their program. This is a signicant eect (p .01). • Those who have permanent placements rate their PhD programs higher (are more likely to recommend them) than those with temporary placement positions. (p < .01). Postdocs tend to rate in between. Interestingly, this could be an eect of better programs providing better placements, or of people with permanent positions judging their previous programs as better because of their positive outcome and higher level of satisfaction. • Using LIWC (Linguistic Inquiry and Word Count) for the free response questions in the survey, those with temporary positions talk about money more, and women talk about aliation more (and these reach significance in a linear mixed model). • For all programs with 5 or more respondents, we have a set of key words that graduates tended to use to describe that program. We will make this data public in the near future to help provide a snapshot of graduate programs, especially for the sake of prospective graduate students. • There is a signicant interaction between response on condence rating and gender in predicting recommendation likelihood, (p < .04). Women with higher condence are more likely to recommend their graduate programs, while men with higher condence are less likely to recommend theirs. 4 Progress on Database Interface 4.1 Hiring Angelo Kyrilov and Bryan Kerster were hired in March to work on the administrative interface and other software issues. 4 4.2 Database Improvements Angelo Kyrilov provided several technical improvements to the database that aid in eciency and security in March and April. He also worked to update our data with ndings from the previous months, including the removal of duplicates and the addition of new gender and AOS data (provided by Carolyn Jennings and Justin Vlasits). He worked with Bryan Kerster to create a canonical university list from the new Carnegie Classication les, released in late 2015, and our own database. He worked to help Patrice Cobb to put together program-specic analyses, which were released in April as a report update. 4.3 Individual Editing The most important database interface change was to enable individual editing. Angelo Kyrilov worked on this from April to June. Angelo's design allowed us to send out the survey at the same time that we allowed individuals access to edit their own data through a unique link. 4.4 Project Personnel Editing Angelo Kyrilov also set up an interface for project personnel to edit program contacts and send an email to this list or to the list of individuals in the database. This will help to streamline the project for future maintenance and use. 5 State of the Database The database has more than doubled in size since the time prior to APA funding. We had 1,862 graduates in June 2015 and we now have 4,337 graduates, in August 2016 (and 2,659 placements in June 2015 to 5,761 placements today). We have added an app to the website (created by Patrice Cobb) that will enable more direct access to the database, and we hope to develop more such apps in the near future, here. Some characteristics of the overall database are below: • 66% of the graduates are men (2845), 27% are women (1160), 330 are unknown (8%), and a fraction of a percentage are "other" (1) or "prefer not to answer" (1) 5 • 93% of the graduates have an ethnicity (4040) and race (4025) that are currently unknown or "prefer not to answer," with 6% white, non-hispanic (279) and the other percent asian, non-hispanic (20) or "two or more races," non-hispanic (13) • 27% of the graduates have an unknown AOS category (1150), with 21% in LEMM (914), 24% in Value Theory (1056), 18% in History and Traditions (779), and 10% in Science, Logic, and Math (438). • Other than those graduates with unknown primary AOS, most were in Ethics (11%, 495), followed by Social/Political Philosophy (6%, 241), Philosophy of Mind (5%, 228), Epistemology (5%, 221), Ancient Philosophy (4%, 191), Metaphysics (4%, 188), Philosophy of Science (4%, 187), Continental Philosophy (4%, 185), Modern Philosophy (4%, 175), Philosophy of Language (3%, 151), Applied Ethics (3%, 117), Philosophy of Cognitive Science/Psychology/Neuroscience/Linguistics (2%, 105), Logic (1%, 62), 19th/20th Century Philosophy (1%, 57), Philosophy of Art/Aesthetics (1%, 54), Metaethics (1%, 53), Asian Philosophy (1%, 51), and Philosophy of Biology (1%, 45). (19 other elds yield less than 1% of the total graduates.) • Graduates in our database span graduation years from 1991 to 2016 (with 7% having unknown graduation year), but the best represented years are 2011-2014. Starting with years in which there are more than 100 graduates in the database, we have 137 graduates in 2005, 186 in 2006, 189 in 2007, 220 in 2008, 228 in 2009, 309 in 2010, 441 in 2011, 511 in 2012, 469 in 2013, 460 in 2014, 298 in 2015, and 119 in 2016. • Similarly, 15% of placement years are unknown, with the best represented years 2012-2015. Focusing on those years, there are 384 placements in 2012, 464 in 2013, 587 in 2014, and 573 in 2015. (288 so far in 2016.) • 9% of those in the database have no recorded placement (369), 60% have a permanent academic position (2617; 55% tenure-track), 28% have a temporary academic position (1216), and 3% have a non-academic placement (135). 6 Evaluation and Outlook Our primary goals this year were to conduct a qualitative survey and to improve the database interface. We achieved both of these goals. The project is now at a point where it can be continued without major funding, which is an 6 important milestone. We hope to publish one or more papers over the next year on the ndings from this project so that we can make our work public in a timely and responsible manner. We also hope to provide more direct access to the data, and will continue to work on an application page on the website that will allow such access. In the next year we hope to work with undergraduate research assistants to collect as much data as possible on nonacademic placement–a topic that is currently underserved by our project. We would also like to run a new survey designed especially for this group. Thanks are due to the American Philosophical Association for supporting this work and for the many members of the profession who either contributed to the project or reached out to us to lend their support. Our work this year would not have been possible without all of you and we hope for your continued support.