On the Variability of Software Engineering Needs for Deep Learning: Stages, Trends, and Application Types

—The wide use of Deep Learning (DL) has not been followed by the corresponding advances in software engineering (SE) for DL. Research shows that developers writing DL software have speciﬁc development stages (i.e., SE4DL stages) and face new DL-speciﬁc problems. Despite substantial research, it is not clear how such needs vary over stages, DL application types, or if they change over time. To help focus research and development efforts on DL-development challenges, we analyze 92,830 Stack Overﬂow (SO) questions and 227,756 READMEs of public repositories related to DL. Latent Dirichlet Allocation (LDA) reveals 27 topics for the SO questions with 19 (70.4%) topics primarily relating to a single SE4DL stage and eight topics spanning multiple stages. Most questions concern Data Preparation and Model Setup stages. The relative rates of questions for 11 topics have increased, for eight topics decreased over time. Questions for the former 11 topics had a lower percentage of having an accepted answer than for the remaining topics. LDA on README ﬁles reveals 16 distinct application types for the 227k repositories. We apply the LDA model ﬁtted on READMEs to the 92,830 SO questions and ﬁnd that 27% of the questions are related to the 16 DL application types. The distribution of topics with the most questions vary with application types, with half topics relating to the second and third stages. Speciﬁcally, developers ask the most questions about topics relating to Data Preparation (2nd) stage for four mature application types such as Image Segmentation, and topics relating to Model Setup (3rd) stage for four application types concerning emerging methods such as Transfer Learning. Based on our ﬁndings, we distill several actionable insights for SE4DL research, practice, and education such as better support on using trained models, application-type speciﬁc tools and teaching materials.