text: Brissenden, R. J. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, eds. F. R. Harnden, Jr., F. A. Primini, & H. E. Payne (San Francisco: ASP), 22
  Chandra X-ray Observatory Operations
  Roger J. Brissenden
  Harvard-Smithsonian Center for Astrophysics
  Abstract:
  The Chandra X-ray Observatory was launched from the Space Shuttle on 23 July 1999 and has completed the first year of operations with outstanding results. We present a description of the Chandra Observatory, the Chandra mission operations concept, ground system architecture, selected operations metrics and an example where operational processes have required modification due to on-orbit events and experience.
  1. Introduction
  The Chandra X-ray Observatory (CXO) (formerly the Advanced X-ray Astrophysics Facility, or AXAF) is the third of NASA's Great Observatory missions; it follows the Hubble Space Telescope (HST; 1990-) and the Compton Gamma-Ray Observatory (CGRO; 1991-2000) and precedes the Space Infrared Telescope (SIRTF; 2002). Chandra is a space-based Observatory containing a high resolution (0 5) X-ray telescope responsive to the energy range 0.1-10keV and a complementary set of imaging and spectroscopic instruments. The mission was designed with a minimum 5-year lifetime and a goal of 10+ years, and provides an order-of-magnitude advance in spatial and spectral resolution over previous X-ray telescopes.
  The Chandra Program is managed by NASA's Marshall Space Flight Center. Science and mission operations for the Program are carried out at the Chandra X-ray Center (CXC) and Operations Control Center (OCC) located in Cambridge, MA, using facilities of the Smithsonian Astrophysical Observatory (SAO) and the Massachusetts Institute of Technology (MIT). Observing time is awarded through an annual peer review. Selected targets are scheduled in weekly segments and command loads to implement the mission schedule are uplinked to the spacecraft from the OCC via NASA's Deep Space Network. Telemetry and data are downlinked approximately every 8 hours, monitored for state of health at the OCC, and passed to the CXC for science processing, archiving, and distribution to the Observer. In addition, CXC also provides an Education and Outreach program, and administers the Chandra Grants and Fellowship programs.
  The Chandra Observatory was launched on the Space Shuttle Columbia (STS-93) on 23 July 1999. Following launch and orbital insertion, Chandra underwent a 2 month Orbital Activation and Checkout phase before starting 2 months of Guaranteed Time Observations in September 1999. The first cycle of General Observer observations began in November 1999, the second cycle in November 2000, and the third is scheduled for November 2001.
  In this paper, we provide a brief overview of the Chandra Observatory and discuss the launch, orbital insertion, and activation (§2), describe the science-phase mission operations concept (§3), and give a summary of selected operational metrics used to monitor mission progress and efficiency (§4). In §5 we provide an example of how an on-orbit anomaly can impact all of these systems and processes, and discuss the importance of a system-wide approach (for both the space and ground segments) to problem response. For a discussion of the scientific aspects of the mission, see Murray (2001).
  2. Mission and Observatory Description
  2.1. Observatory Description
  Figure 1 shows Chandra in its deployed configuration with selected components labeled.
  Figure 1: Chandra X-ray Observatory showing selected components.
  The Observatory consists of the telescope system, the science instruments, the Command, Control, and Data Management system (CCDM), the Pointing, Control, and Attitude Determination system (PCAD), the Electrical Power System (EPS), thermal control, and propulsion systems. Chandra is a physically large spacecraft with a wing-span of 19.5m, length with sun-shade door open of 11.8m and an on-orbit mass of 4800kg.
  The principal components of the telescope system are the High Resolution Mirror Assembly (HRMA), which consists of four pairs of grazing incidence Wolter Type I mirrors with focal length 10m, and an optical bench assembly that connects the mirror assembly to the Integrated Science Instrument Module (ISIM) that houses the focal plane science instruments. The mirror assembly has an effective area of 800, 400, and 100 cm at 0.25, 5.0, and 8.0keV, respectively. It provides a ghost-free field of view of 30 diameter with a plate scale of 48.8 marcsec , and a point spread function whose full width at half maximum including detector effects is 0 5.
  X-rays are focused onto one of two selectable focal plane instruments, the Advanced CCD Imaging Spectrometer (ACIS) and the High Resolution Camera (HRC), which are optimized for the higher and lower portions of the energy range, respectively. The ACIS focal plane is populated by ten CCDs each format with 24 m pixel size, configured as a imaging array and a spectroscopic array. Two of the ACIS CCDs are back illuminated and eight are front illuminated. Image data from the ACIS CCD chips are acquired and processed on-board by the ACIS flight software. The software is hosted by a Mongoose controller based on a MIPS R3000 chip set. By means of ground specified parameter blocks, it supports configurable science runs with a variety of observing modes including timed exposure, continuous clocking, spectroscopy, calibration, and diagnostic.
  The HRC is a microchannel plate instrument with two detector regions configured as imaging and spectroscopy detectors. The HRC provides time resolution up to 16 s. ACIS and HRC are complemented by two movable gratings, the High Energy Transmission Grating (HETG) and the Low Energy Transmission Grating (LETG), that can be inserted into the optical path to disperse a spectrum across one of the spectroscopic detectors. The HETG contains two sets of gratings, each with a different period, mounted on the same structure. The outer set disperses X-rays from the outer two mirror pairs, and the inner set disperses from the inner two mirrors pairs. The spectra form an X shaped pattern centered on the undispersed zeroth order. The HETG is matched with the ACIS to provide high resolution spectroscopy with up to 1000 over 0.4-10keV. The LETG is matched with the HRC to provide high resolution spectroscopy with over 0.07-7.29keV.
  The CCDM system provides the command, telemetry, and data management functions for the spacecraft. The major components include two low gain antennas, two S-band transponders, two 1.8 Gb (giga-bit) solid state recorders, redundant 16-bit CDI 1750A On-Board Computers (OBC), redundant LSI 16-bit 1750A computers for control during safe mode, and redundant Interface Units, Command and Telemetry Units, and Remote Command and Telemetry Units for communications with the spacecraft systems. The principal telemetry rate is 32 kbps (kilo-bits per second), with 24 kbps allocated to the science instruments and 8 kbps to spacecraft engineering data. The spacecraft supports a command uplink rate of 2 kbps and a downlink range of 32-1024 kbps. The OBC flight program consists of an executive responsible for state control, interrupts, and task control, and a set of functional software systems for the control of PCAD, CCDM, EPS, Telescope (thermal and gratings), radiation monitor and health, and status. The CCDM function handles the Science Instrument telemetry.
  The PCAD system provides the sensors and control hardware used to point the observatory, slew to new targets, and perform solar array positioning and momentum unloading. Chandra's pointing requirements are modest (30 ) compared with the Hubble Space Telescope (for example) due to the photon counting nature of the detectors. The pointing direction is obtained from the gyroscopes and aspect camera as it tracks 5-8 optical stars pre-selected for each target; during ground processing the position of each photon is transformed from detector to sky coordinates using the aspect camera star data (``image reconstruction''). The performance of the aspect system on-orbit yields an absolute celestial pointing of 3 , an image reconstruction of 0 3 and celestial location of 0 76. Other components of the PCAD system include six reaction wheels for attitude control, and coarse and fine sun sensors used for pointing control modes that do not use the aspect camera.
  The Propulsion System consists of the Integral Propulsion System (IPS) used during the transfer orbit and now deactivated, and the Momentum Unloading Propulsion System (MUPS) used to unload momentum from the reaction wheels. Momentum build-up occurs due to the gravity gradient of the Earth as the spacecraft passes through perigee, and from solar pressure. The IPS system includes both 105lbf liquid apogee engines and 20lbf Reaction Control Thrusters (RCS), while the MUPS system contains 0.2lbf thrusters. Both the RCS and MUPS systems use liquid fuel hydrazine. The fuel is the only mission expendable. Projections based on usage during the first year of operations indicate sufficient fuel for more than 20 years of operation.
  The Electrical Power System generates, stores, and distributes electrical power to the spacecraft. The major components are the solar arrays (two 3-panel wings), which provide 2112 watts of power, and three NiH batteries with 120 A-hr capacity. The batteries provide power during earth eclipses and occasional lunar eclipses. Chandra's orbit was chosen to ensure a battery depth of discharge no more than 80% for any eclipse. This requirements results in eclipses generally shorter than 2 hours.
  The Thermal Control System contains passive elements such as multi-layer insulation blankets and a range of radiator materials, and active elements including temperature sensors, thermostats, and heaters. The on-board computer controls the active heaters to maintain the mirror assembly temperature at F, a key requirement for maintaining the image quality.
  2.2. Chandra Launch, Orbital Insertion, and Activation
  Following Chandra's 23 July 1999 launch on the Space Shuttle Columbia, the spacecraft was deployed together with its attached two-stage Inertial Upper Stage (IUS) rocket motor. The IUS took Chandra to its transfer orbit before separating from the satellite approximately 11 hours after launch. Chandra traveled to its final orbit via a series of five firings of its own onboard propulsion system (IPS). The final orbit of km was reached on 7 August 1999 with Keplerian orbital parameters as shown in Table 1.
  Table 1: Chandra Keplerian Orbital Element Comparison.
  Parameter 7 Aug 1999 1 Jan 2001
  Semi major axis 80798.5km 80790.1km
  Eccentricity 0.802 0.756
  Inclination 28.5 35.5
  Right Ascension of Ascending Node 194.1 148.9
  Argument of Perigee 271.1 305.1
  True Anomaly 180.1 172.0
  Period 63.491 hrs 63.481 hrs
  For comparison, the parameters are shown as of 1 Jan 2001 and are consistent with a predicted orbit circularization trend through 2005. The orbit provides approximately 80% viewing time above the radiation belts that extend to km altitude around the earth. The orbital parameters resulting from the fourth burn were propagated over a 10 year mission using multiple computers running copies of the same code. The calculation was time critical since the fifth (trim) burn parameters were required within 24 hours in order to be able to uplink and execute the burn. The final orbit parameters were adjusted by the fifth burn to ensure an optimal eclipse duration mission profile. The propagation calculation yielded a clear best choice resulting in a final orbit with no earth or lunar eclipse exceeding a battery depth of discharge of 80%.
  Once final orbit was attained, the spacecraft systems were activated over several weeks. Following the opening of the sunshade door on 8/12/99, the pointing system was activated, and the official first light image of Casseopia A was taken on 8/19/99. Other observations taken during the activation phase included aspect camera field distortion measures, and science instrument calibrations to measure the focus, determine bore-sight and optical axis, characterize the on-axis and off-axis response, effective area and plate scales, and measure standard candles.
  3. Chandra Operations Concept
  3.1. Principal Operations Thread
  The principal operational thread for the CXC is shown in Figure 2 and shows the end-to-end flow for a Chandra proposal submitted by an observer from top-to-bottom in the center of the figure. Rectangular boxes map to functional components of the CXC Data System (CXCDS).
  Figure 2: CXC Operational Thread showing Proposal Path.
  In response to an annual NASA Research Announcement, an observer prepares and submits a proposal through Proposal Support (Figure 2) using observation modeling software and remote proposal submission tools. Following the annual peer review, the targets from the accepted proposals (Proposal Data) populate the Observing Catalog (OCAT) and form the input for the long term schedule that is generated by the Science Mission Planning Team using the SPIKE scheduling software. A weekly oversubscribed target list is submitted to the OCC as an Observation Request (OR) list. The OR list includes calibration targets, which account for an average of 8% of the observing time.
  The Flight Mission Planning team incorporates any required engineering activities and develops an optimized operations schedule using the Off-Line System (OFLS) software. The OFLS includes software for mission planning and scheduling, command load generation, attitude determination, sensor calibration, ephemeris and orbit events generation, and engineering analysis. The schedule of spacecraft slews, mechanism motions, momentum dumps, and other actions is generated to ensure that none of over 800 pre-defined constraints are violated. Command loads are generated using the OFLS, and a series of additional constraint and timing checks are performed by the flight and science team prior to uplink to the spacecraft. Command loads typically run for 2 days, and three loads at a time are placed on-board for sequential execution. As the OBC executes the sequence of maneuvers, instrument movements, and configurations, telemetry is recorded at 32kbps on the spacecraft solid state recorder. The data are downlinked during a real-time pass, usually at 512kbps. Chandra is operated with three passes of 1-2 hours per day through the DSN. Two types of telemetry reach the OCC: ``real-time'' telemetry of science and engineering data that are generated on the spacecraft while it is in contact with the OCC; and ``dump'' telemetry of data that were stored on the spacecraft solid state recorder between contacts.
  Real-time commanding and monitoring are performed using the On-line System (ONLS) and GRETA1 software systems. The ONLS includes software to uplink commands and data in real-time to the spacecraft and to monitor downlink telemetry. GRETA provides software for real-time monitoring and long-term trending of engineering data. Engineers check the state of the spacecraft against the predicted state at each pass; real-time telemetry points are checked against a limit data base with automatic out-of-limit notification. Dump data are transferred from the DSN to the OCC within a few hours of each pass and then to the CXC for science processing.
  Chandra dump data are processed by the CXCDS Standard Data Processing function through a series of levels using an Automated Processing (AP) system. Level 0 decommutates telemetry and processes ancillary data, Level 1 performs event processing, standard filtering, and aspect reconstruction, Level 2 performs source detection and generates derived source properties, and Level 3 will create catalogs spanning multiple objects and observations. A determination of the useful science time on a target (``good time'') is made during AP and fed back to the mission planning function to trigger additional observations if needed. An example of ``bad time'' is a period of unacceptable aspect solution. See Plummer & Subramanian (2001) for a detailed discussion of the CXCDS Automated Processing system.
  The Data Quality Assessment function includes both automated and manual checks of the data products generated by AP, including quality of aspect solution, background rates, observation and instrument configuration parameters. Following a successful data quality check (called Verification and Validation, or V&V), L0-L2 data products are made available to the observer through the archive and on distribution media. Data have a 1 year proprietary period and are available through the archive only to the observer during that period. Non-proprietary data (e.g., calibration observations) are publically available once V&V is complete. Data analysis by observers is supported by the Data Analysis Software function with the Chandra Interactive Analysis of Observations ( CIAO) software system (Doe, Noble, & Smith 2001). CIAO provides a package of data analysis tools that include an interactive fitting and plotting environment ( sherpa), an interactive filter composition ( filterwindow), quick interaction with observation files ( firstlook), a plotting and imaging package ( chips), a file browser ( prism), and tools for high resolution spectral analysis ( guide).
  In addition to carrying out the primary proposal thread shown as the central path through Figure 2, the CXC performs other functions. These include science monitoring and trending of the science instruments and spacecraft subsystems (Science Flight), maintaining the evolving calibration of the optics and science instruments (Calibration), providing software in support of the annual peer review and other Chandra programs, and providing Education and public Outreach (General User Support).
  4. Mission Metrics
  A set of mission metrics were defined prior to launch and have proven valuable during the first year and a half of operations in detecting process bottle-necks, taking corrective action, and monitoring their resolution. The specific metrics, shown in Table 2, were chosen to cover mission expendables, end-to-end data throughput, observing efficiency, and user interaction.
  Table 2: Chandra Mission Metrics.
  A breakout of the key metric Observing Efficiency, which is defined as time collecting science photons as a fraction of total orbit time, is shown in Table 3 for Cycle 1. Chandra cannot observe in the radiation belts or during the Charge Transfer Inefficiency (CTI) measurements (see §5), so the actual on-target observing efficiency of 66% is out of a possible maximum of 75%. The present efficiency is within 10% of optimal; however effort is on-going to increase the efficiency by optimizing the target acquisition time and reducing the CTI measurement time through refined modeling of the radiation belts.
  Table 3: Chandra Cycle 1 Observing Efficiency.
  5. Operational Process Example: Reacting to Instrument Damage from the Radiation Belts
  The Problem. Shortly after the start of science observations, a sudden and unexpected degradation was detected in the energy resolution of the front-side illuminated CCD chips of the ACIS instrument. The energy resolution as characterized by the Charge Transfer Inefficiency (CTI) was seen to have become a function of row number, being nearer the pre-launch value in rows close to the frame store portion of the array and substantially degraded in the farthest row.
  The Cause. Analysis of on-orbit data and operational conditions, and ground tests of similar CCDs pointed to the likely cause as damage due to low energy protons encountered during the radiation belt passages reflecting off the X-ray mirrors and onto the focal plane. The back-side illuminated chips were not damaged, as would be expected for the soft proton scenario given the depth of their buried channels.
  The Operational Response. Immediately following the identification of the cause, operational procedures were modified to ensure that ACIS was moved out of the focal plane during radiation belt passage. In addition, measurements of the internal ACIS calibration source were scheduled during a period before and after each belt entry, to ensure that no further damage was sustained and to monitor degradation levels. These calibration measurements provide a measure of the CTI each orbit and are included in Table 3. Since this operational change, no further degradation has been detected and observations with ACIS have continued effectively through use of both the undamaged back-side chips and the front-side chips.
  Mitigation Approach. A series of mitigation approaches are being investigated to improve the energy resolution of the front-side chips, including running the ACIS focal plane at the lowest possible temperature (implemented) and clocking the chips using a new mode that distributes particle induced charge to fill the radiation-induced charge traps.
  The Operational Impacts. A systems approach was taken in responding to the CTI problem that involved all groups within the Chandra Program. The full range of impacts was unexpected and illustrates the importance of taking a systems approach when dealing with operational issues. The following steps were taken:
  The mission scheduling approach was modified to ensure that ACIS was removed from the focal plane before, during, and immediately after radiation belt passage.
  ACIS CTI calibration measurements were defined and added before and after belt passages. Analysis and monitoring software were developed to perform on-going analysis and trends.
  Spacecraft software was changed to protect ACIS in the event of a safing action. The original software was designed to leave ACIS at the mid-point between the focal point and the newly determined safe position. The flight software modifications now ensure that ACIS is always moved fully to the safe position for safing actions.
  ACIS processing, monitoring, and analysis software were modified to accommodate CTI degradation.
  A ground and in-orbit test program was undertaken to develop mitigation techniques.
  The Cycle 1 observing program was re-organized to accommodate the new instrument performance. All observers were contacted and new chip configurations were determined, observations were rescheduled and documentation was changed.
  A new calibration program was developed, planned, and implemented to characterize ACIS at a new operating temperature, and to fully characterize the chip response. Further calibration will be required in the event of implementation of other mitigation approaches.
  The Chandra team was well organized to respond to this anomaly effectively. The ACIS instrument team reproduced the problem on the ground and developed possible mitigation approaches; CXC and MSFC science and data system staff modeled the radiation belts and provided operational parameters for safe operation; the spacecraft team rapidly and safely modified and verified complex safing flight software; the mission planning staff and system responded with essentially no loss in observing efficiency; and a set of existing management processes ensured that the multiple impacts were assessed and responded to on a system-wide basis. The latter was accomplished through the use of a configuration management control board with membership from all the teams (e.g., spacecraft engineering, flight operations, science, and ground data system teams), and ensured that changes were made in a controlled and coordinated manner throughout the system.
  6. Conclusion
  We have described the Chandra mission, the Observatory, the operations concept, mission metrics, and an example of a single anomaly with system-wide impacts that the team has responded to effectively. Where possible, future missions should design processes prior to launch that assume system level responses will be required.
  Acknowledgments
  The success of the Chandra mission is due to the hard work and creativity of a large team of dedicated people at many institutions including MIT, NASA, PSU, SAO, SRON, TRW, and their contractors. This paper summarizes their outstanding work.
  This work is supported by the Chandra X-ray Center under NASA contract NAS8-39073.
  References
  Doe, S., Noble, M., & Smith, R. 2001, this volume, 310
  Murray, S. 2001, this volume, 13
  Plummer, D. & Subramanian, S. 2001, this volume, 475
  Chandra Proposers' Observatory Guide, Version 3.0, 28 December 2000:
  http://asc.harvard.edu/udocs/docs/POG/MPOG/index.html
  CIAO web location: http://asc.harvard.edu/ciao/
  Chandra X-ray Center web site: http://asc.harvard.edu/
  Footnotes
  ... GRETA1
  Heritage code from the CGRO mission, modified for use with the Chandra Program.
  ... 2
  Guaranteed Time Observer/General Observer/Director's Discretionary Time
  © Copyright 2001 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA