Abstract Simultaneous PET/MR can be considered as an integrated imaging modality only if the information of both modalities is integrated together. In current routine PET/MR applications, the PET and MR scans are performed separately, and the images are reconstructed separately as well. The information is integrated only at the application level. Here we propose unified methodologies of joint PET/MR image reconstruction, a paradigm shifting new way to integrate information of PET and MR to significantly maximize the outcome of PET/MR. The PET and MR scanners indeed measure different physical or physiological signals, but there are still redundant information (e.g. tumor boundary and mutual information) between the images obtained with the two modalities that can be utilized to build connection between PET and MR images in a potential joint reconstruction. In addition, if the compartmental model is taken into account, the physiological parameters estimated from PET and MR can have overlaps, and therefore the parametric image (voxel-wise kinetic parameters) estimated from one modality could be directly used to help the estimation of the parametric image of the other modality. Therefore, there are inter- connections between these two modalities that we can use to develop elegant methods of joint reconstruction. We will first take advantage of the simultaneous acquisition of PET/MR to develop a static image reconstruction with anatomic prior derived from MR images, and to develop methods to jointly reconstruct gated PET images using a motion field computed from MR images. We believe in both cases, the quality of PET images will be significantly improved compared to traditional approaches. For PET/MR, there are many novel ways to jointly model the dynamic PET and MR images. We will thus develop an alternating direction method of multipliers (ADMM) to directly estimate the voxel-wise kinetic parameters of dynamic PET and dynamic MR together from raw data. This will achieve the maximum signal noise ratio of parametric images for both dynamic PET and MR. We will also investigate novel approaches to parametric imaging of non-stationary kinetic modeling in which not only the images are estimated but also the uncertainty on those estimates of the parametric images. The knowledge of uncertainty is important when making decisions about progression/regression of the disease, signal detection, etc. We will use a method developed in our laboratory in which the noise in raw PET data will be transferred to parameter images using origin ensemble algorithm.