Performance of Lung-RADS in the National Lung Screening Trial: a retrospective assessment.
Editors' Notes Context The definitions used to classify low-dose computed tomography findings may markedly influence the benefits and harms of lung cancer screening. Contribution This analysis of data from a large screening trial found that using the recently proposed Lung-RADS approach to classifying low-dose computed tomography findings substantially decreased the false-positive result rate but with a concomitant decrease in sensitivity. Implication Adopting the Lung-RADS classification system may improve the results of lung cancer screening programs. The U.S. Preventive Services Task Force recently recommended (grade B) lung cancer screening with low-dose computed tomography (LDCT) for high-risk current and former smokers (1). The primary evidence used by the Task Force was the National Lung Screening Trial (NLST), which reported a 20% reduction in lung cancerspecific death associated with LDCT screening (2). Important considerations for widespread use of LDCT lung cancer screening in clinical practice include the definition of a positive result in computed tomography (CT) screening and the appropriate management of positive screening results. Much knowledge has accumulated since the NLST was designed in 2002. In this trial, the definition of a positive screening result was a nodule of 4 mm or greater in the longest diameter that had no specific benign calcification patterns. In addition, the NLST achieved its results without a trial-wide specified protocol for diagnostic management for positive screening results. A recent reanalysis of the NLST examined the effect of different cutoffs defining a positive screening result and found that increasing the threshold to 6 or 8 mm would have resulted in substantial decreases in the false-positive result rate with only small corresponding decreases in sensitivity (3). The International Early Lung Cancer Action Program reported similar results for baseline LDCT screenings, showing a substantial reduction in the positivity rate of screening results with increasing size cutoffs and only a few resultant missed cancer cases (4). Over this period, several professional organizations have promulgated lung cancer screening guidelines, many of which define a positive screening result and include nodule management (57). The American College of Radiology recently began efforts to standardize the reporting of LDCT screening results in a manner analogous to the use of the Breast Imaging Reporting and Data System for mammography, based on the best available data. This effort included defining a positive result on lung cancer screening CT in the most effective manner, attempting to reduce the substantial false-positive result rate while having the least possible effect on test sensitivity, and suggesting management recommendations based on lung cancer risk. Published data from several LDCT screening studies, including the NLST, the International Early Lung Cancer Action Program, and the European NELSON (Nederlands-Leuvens Longkanker Screenings Onderzoek) trial, were used by a consensus panel to help derive positivity criteria for Lung-RADS (4, 810), which was officially released in May 2014 (11). Compared with the NLST criteria, Lung-RADS increases the size threshold for a positive baseline screening result from a 4-mm greatest transverse diameter to a 6-mm transverse bidimensional average (and to 20 mm for nonsolid nodules) and requires growth for preexisting nodules. Although data from the NLST, in part, were used to develop the Lung-RADS criteria, only published summary-level data were considered. These data were sufficient to give an approximate positivity rate for Lung-RADS as applied to the NLST but not to give an exact distribution of Lung-RADS scores. This is especially the case for screenings after baseline, where the individual nodule history over time is critical in defining the Lung-RADS category. We used participant- and nodule-level data to retrospectively apply the Lung-RADS criteria to the NLST. We evaluate the effect of Lung-RADS on the performance characteristics of LDCT screening, including sensitivity, false-positive result rate, positive predictive value (PPV), and negative predictive value (NPV). In addition, we compare the characteristics of the cancer cases detectable by Lung-RADS with those that it would have missed. Methods NLST Design The NLST randomly assigned participants aged 55 to 74 years to LDCT or chest radiography screening. Eligibility criteria included 30 pack-years of smoking or greater and current smoking status or having quit within the past 15 years (12). Participants were recruited at 33 U.S. centers from 2002 to 2004 and received either LDCT or chest radiography over 3 annual screening rounds (denoted T0, T1, and T2). The NLST was approved by the institutional review board at each screening center, and all participants provided informed consent. The NLST study protocol defined a noncalcified nodule (NCN) of 4 mm or greater in the longest transverse diameter as a positive screening result. For each NCN that was 4 mm or greater, radiologists used standardized forms to report location, greatest transverse and perpendicular diameters, margins, and attenuation characteristics. At T1 and T2, they reported whether the abnormality was preexisting or new based on examinations of previous images and, if preexisting, whether it had grown and whether a suspicious change in attenuation had occurred since past screenings. Noncalcified nodules that were unchanged from T0 to T2, representing stability for 2 years, could be considered benign and constitute a negative screening result at the radiologist's discretion. Other abnormalities, including adenopathy or effusion, could also trigger positive screening results. Positive results were tracked for resultant diagnostic procedures and lung cancer diagnoses. In addition, participants were followed with annual surveys to ascertain incident cancer cases. All reported cancer cases were verified with medical records, with stage and histologic characteristics recorded. Deaths were tracked with the annual surveys and supplemented by National Death Index searches. Lung-RADS Table 1 describes the primary criteria for defining Lung-RADS categories. Categories 1 (negative) and 2 (benign appearance) correspond to negative screening results, and categories 3 (probably benign) and 4 (suspicious) correspond to positive screening results. Category 4 is further divided into 4A, 4B, and 4X (8). In the context of annual screening, a negative screening result assumes that reevaluation will occur at the next annual screening, whereas a positive screening result means that additional evaluation is recommended before the next annual screening. The distinctions between the positive screening categories are important because Lung-RADS management guidelines differ substantially across categories, ranging from follow-up CT at 6 months for category 3 to positron emission tomography and CT or biopsy for 4B. In addition, category 2 involves tracking small nodules on the next annual screening, and category 1 does not involve tracking nodules. Table 1. Summary of Lung-RADS Classification Lung-RADS criteria distinguish between baseline (first) and subsequent screenings. For baseline screenings (generally lacking comparison examinations), the criteria are based on nodule size, as measured by average diameter, and nodule attenuation (solid, part-solid, or nonsolid). For subsequent screenings, the criteria also consider the preexistence and growth of the nodule. For baseline screenings, positive screening results for solid and part-solid nodules require a size of 6 mm, and 20 mm is required for nonsolid (that is, ground-glass) nodules. For positivity on subsequent screenings, 4 mm is required for new (solid or part-solid) nodules, and preexisting nodules must show growth, defined as an increase in size of greater than 1.5 mm. New or growing nonsolid nodules still must meet the 20-mm size requirement. For part-solid nodules, the size and/or growth of the solid component is also considered. The overall Lung-RADS screening category is determined by the nodule with the highest individual Lung-RADS score. Category 3 or 4 nodules with additional features (such as spiculation) or imaging findings that increase suspicion for cancer (such as enlarged lymph nodes) can qualify as category 4X. Applying Lung-RADS to the NLST The average diameter for NLST nodules was computed as the mean of the longest diameter and the longest perpendicular diameter. The NLST attenuation classifications of soft tissue, ground glass, and mixed were mapped to the Lung-RADS classifications of solid, nonsolid, and part-solid, respectively. The NLST did not report the amount of growth but only whether growth occurred; therefore, report of growth in the NLST was considered nodule growth for Lung-RADS. For part-solid nodules, the NLST did not report the size of the solid component, which may be required to distinguish among categories 3, 4A, and 4B. Therefore, if NLST data for a part-solid nodule were consistent with 2 or more categories of 3 or higher, a range (such as 3 to 4B) instead of a single category was denoted for our analysis. These category ranges for part-solid nodules were used only if they constituted (at the upper limit of their range) the nodule with the highest degree of suspicion. In addition, the solid component was assumed to be growing if the nodule as a whole was reported as growing or there was a suspicious change in attenuation; growth specifically of the solid component was not recorded in the NLST. If other suspicious findings, in the absence of any nodules measuring 4 mm or greater, constituted a positive screening result in the NLST, this was classified as category 4X for Lung-RADS. Quantitative Methods Lung cancer was deemed to be present at a screening if it was diagnosed within 1 year or before the next screening (whichever came first) or, for po