deep learning super resolution hyper spectral
215835628
Hyperspectral image super resolution combining with deep learning and spectral unmixing
Abstract In recent years, hyperspectral image super resolution has attracted the attention of many researchers and has become a hot topic in the field of computer vision. However, it is difficult to obtain high resolution images due to imaging hardware devices. At present, many existing hyperspectral image super resolution methods have not achieved good results. In this paper, we propose a hyperspectral image super resolution method combining with deep residual convolutional neural network (DRCNN) and spectral unmixing. Firstly, the spatial resolution of the image is enhanced by learning a priori knowledge of natural images. The DRCNN reconstructs high spatial resolution hyperspectral images by concatenating multiple residual blocks, each containing two convolutional layers. Secondly, the spectral features of low resolution and high resolution hyperspectral images are linked by spectral unmixing. This approach aims to obtain the endmember matrix and the abundance matrix. The final reconstruction result is obtained by multiplying the endmember matrix and the abundance matrix. In addition, in order to improve the visual effect of the reconstructed image, the total variation regularity is used to impose constraints on the abundance matrix to enhance the relationship between the pixels. The experimental results of remote sensing data based on ground facts show that the proposed method has good performance and preserves spatial information and spectral information without the need for auxiliary images.
Signal Process. Image Commun.
Changzhong Zou Xusheng Huang
219558922
Super resolution hyper spectral imaging for the direct visualization of local bandgap heterogeneity
Optical hyperspectral imaging based on absorption and scattering of photons at the visible and adjacent frequencies denotes one of the most informative and inclusive characterization methods in material research. Unfortunately, restricted by the diffraction limit of light, it is unable to resolve the nanoscale inhomogeneity in light matter interactions, which is diagnostic of the local modulation in material structure and properties. Moreover, many nanomaterials have highly anisotropic optical properties that are outstandingly appealing yet hard to characterize through conventional optical methods. Therefore, there has been a pressing demand in the diverse fields including electronics, photonics, physics, and materials science to extend the optical hyperspectral imaging into the nanometer length scale. In this work, we report a super resolution hyperspectral imaging technique that simultaneously measures optical absorption and scattering spectra with the illumination from a tungsten halogen lamp. We demonstrated sub 5 nm spatial resolution in both visible and near infrared wavelengths (415 to 980 nm) for the hyperspectral imaging of strained single walled carbon nanotubes (SWNT) and reconstructed true color images to reveal the longitudinal and transverse optical transition induced light absorption and scattering in the SWNTs. This is the first time transverse optical absorption in SWNTs were clearly observed experimentally. The new technique provides rich near field spectroscopic information that had made it possible to analyze the spatial modulation of band structure along a single SWNT induced through strain engineering.
Xuezhi Ma Qiushi Liu Ning Yu Da Xu Sanggon Kim Zebin Liu Kaili Jiang Bryan M Wong Ruoxue Yan Ming Liu
213810564
Widefield light sheet microscopy using an Airy beam combined with deep learning super resolution
Imaging across length scales and in depth has been an important pursuit of widefield optical imaging. This promises to reveal fine cellular detail within a widefield snapshot of a tissue sample. Current advances often sacrifice resolution through selective sub sampling to provide a wide field of view in a reasonable time scale. We demonstrate a new avenue for recovering high resolution images from sub sampled data in light sheet microscopy using deep learning super resolution. We combine this with the use of a widefield Airy beam to achieve high resolution imaging over extended fields of view and depths. We characterise our method on fluorescent beads as test targets. We then demonstrate improvements in imaging amyloid plaques in a cleared brain from a mouse model of Alzheimer's disease, and in excised healthy and cancerous colon and breast tissues. This development can be widely applied in all forms of light sheet microscopy to provide a two fold increase in the dynamic range of the imaged length scale. It has the potential to provide further insight into neuroscience, developmental biology and histopathology.
Stella Corsetti Philip Wijesinghe Persephone B Poulton Shuzo Sakata Khushi Vyas C Simon Herrington Jonathan Nylk Federico M Gasparoli Kishan Dholakia
216283149
Widefield light sheet microscopy using an Airy beam combined with deep learning super resolution
Imaging across length scales and in depth has been an important pursuit of widefield optical imaging. This promises to reveal fine cellular detail within a widefield snapshot of a tissue sample. Current advances often sacrifice resolution through selective sub sampling to provide a wide field of view in a reasonable time scale. We demonstrate a new avenue for recovering high resolution images from sub sampled data in light sheet microscopy using deep learning super resolution. We combine this with the use of a widefield Airy beam to achieve high resolution imaging over extended fields of view and depths. We characterise our method on fluorescent beads as test targets. We then demonstrate improvements in imaging amyloid plaques in a cleared brain from a mouse model of Alzheimer's disease, and in excised healthy and cancerous colon and breast tissues. This development can be widely applied in all forms of light sheet microscopy to provide a two fold increase in the dynamic range of the imaged length scale. It has the potential to provide further insight into neuroscience, developmental biology, and histopathology.
Stella Corsetti Philip Wijesinghe Persephone B Poulton Shuzo Sakata Khushi Vyas C Simon Herrington Jonathan Nylk Federico M Gasparoli Kishan Dholakia
221044582
Monitoring early stage invasion of exotic Spartina alterniflora using deep learning super resolution techniques based on multisource high resolution satellite imagery: A case study in the Yellow River Delta, China
Abstract Over the past decades, Spartina alterniflora, one of the top exotic invasive plants in China, has expanded throughout coastal China. In the Yellow River Delta (YRD) the rapid expansion of S. alterniflora has caused serious negative ecological effects. Current studies have concentrated primarily on mapping the distribution of S. alterniflora with medium resolution satellite imagery at the regional or landscape scale, which have a limited capability in early detection and monitoring of the invasive process at the patch scale. In this study, we proposed a framework for monitoring the early stage invasion of S. alterniflora patches in the YRD using multiyear multisource high spatial resolution satellite imagery with various ground sampling distances (WorldView 2, SPOT 6, GaoFen 1, GaoFen 2, and GaoFen 6 from 2012 to 2019) First, we proposed to use deep learning based image super resolution models to enhance all images to submeter (0.5 m) resolution. Then, we adopted stepwise evolution analysis based image segmentation and object based classification rules to detect and delineate S. alterniflora patches from the super resolved imagery. By investigating Super Resolution Convolutional Neural Networks (SRCNN) and Fast Super Resolution Convolutional Neural Networks (FSRCNN) and comparing these methods with the conventional bicubic interpolation method for image resolution enhancement, we concluded that FSRCNN was superior in constructing spectral and structural details from the 1 m/1.5 m/2 m resolution images to 0.5 m resolution. FSRCNN, in particular, was more effective and efficient in discerning and estimating the size of small S. alterniflora patches
Int. J. Appl. Earth Obs. Geoinformation
Mengmeng Chen Yinghai Ke Junhong Bai Peng Li Mingyuan Lyu Zhaoning Gong Demin Zhou
220464399
Achieving Higher Resolution Lake Area from Remote Sensing Images Through an Unsupervised Deep Learning Super Resolution Method
Lakes have been identified as an important indicator of climate change and a finer lake area can better reflect the changes. In this paper, we propose an effective unsupervised deep gradient network (UDGN) to generate a higher resolution lake area from remote sensing images. By exploiting the power of deep learning, UDGN models the internal recurrence of information inside the single image and its corresponding gradient map to generate images with higher spatial resolution. The gradient map is derived from the input image to provide important geographical information. Since the training samples are only extracted from the input image, UDGN can adapt to different settings per image. Based on the superior adaptability of the UDGN model, two strategies are proposed for super resolution (SR) mapping of lakes from multispectral remote sensing images. Finally, Landsat 8 and MODIS (moderate resolution imaging spectroradiometer) images from two study areas on the Tibetan Plateau in China were used to evaluate the performance of UDGN. Compared with four unsupervised SR methods, UDGN obtained the best SR results as well as lake extraction results in terms of both quantitative and visual aspects. The experiments prove that our approach provides a promising way to break through the limitations of median low resolution remote sensing images in lake change monitoring, and ultimately support finer lake applications.
Remote. Sens.
Mengjiao Qin Linshu Hu Zhenhong Du Yi Gao Lianjie Qin Feng Zhang Ren-yi Liu
203705039
DEEP RESIDUAL LEARNING FOR SINGLE IMAGE SUPER RESOLUTION OF MULTI SPECTRAL SATELLITE IMAGERY
Abstract. Analyzing optical remote sensing imagery depends heavily on their spatial resolution. At the same time, this data is adversely affected by fixed sensor parameters and environmental influences. Methods for increasing the quality of such data and concomitantly optimizing its information content are, thus, in high demand. In particular, single image super resolution (SISR) approaches aim to achieve this goal solely by observing the individual images.We propose to adapt a generic deep residual neural network architecture for SISR to deal with the special properties of remote sensing satellite imagery, especially taking into account the different spatial resolutions of individual Sentinel 2 bands, i.e. ground sampling distances of 20 m and 10 m. As a result, this method is able to increase the perceived resolution of the 20 m channels and mesh all spectral bands. Experimental evaluation and ablation studies on large datasets have shown superior performance compared to the state of the art and that the model is not bound by its capacity.
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
L Wagner Lukas Liebel Marco Korner
231599897
A Review on Deep Learning Super Resolution Techniques
Super resolution techniques are used to reconstruct the detail of high resolution image from low resolution lossy image. The introduction of a deep learning approach in super resolution has drawn a lot of research interest in recent years due to its learning capability and noise immunity. The applications of deep learning super resolution can mainly be found in image recovery, medical imaging, and microscopy. In this paper, the deep learning super resolutions are explored in detail based on its models and architecture. They can be classified into three neural network (NN) models, i.e. Convolutional NN based models, Recursive NN based models, and Adversarial Network based models. CNN based models apply convolution operation to embed the latent feature and subsequently decode it with deconvolution operation to achieve a higher dimension. RNN based models integrate the recursive depth model to enhance recursive learning with past memory. On the other hand, adversarial network based models apply the generative manner to learn the probability of the input pattern to forecast the possible high dimension of output information. The details of each unsupervised model are discussed in this paper to highlight its advantages and limitations. The measurement metrics such as Peak Signal to Noise Ratio (PSNR) structural similarity (SSIM) and Mean opinion score (MOS) are highlighted for performance evaluation of each super resolution models. The significance of this study provide a compact review of the current development and trend in super resolution using various deep learning models.
2020 IEEE 8th Conference on Systems, Process and Control (ICSPC)
John Julius Danker Khoo K Lim Jonathan Then Sien Phang
197422494
Utility of deep learning super resolution in the context of osteoarthritis MRI biomarkers
Super resolution is an emerging method for enhancing MRI resolution; however, its impact on image quality is still unknown.
Journal of magnetic resonance imaging JMRI
Akshay S Chaudhari Kathryn J Stevens Jeff P Wood Amit Chakraborty Eric K Gibbons Zhongnan Fang Arjun D Desai Jin Hyung Lee Garry E Gold Brian A Hargreaves
203850716
Deep Learning Super Resolution MRI: Getting Something From Nothing
The well worn narrative of artificial intelligence (AI) as a competitive replacement to diagnostic radiologists is by now a tired trope, with opinions rapidly shifting away from hype towards concrete applications. A revolution that began with deep convolutional neural networks has paved the way for a much more nuanced discussion of the impact of AI on radiology workflow: its possibilities and its limitations. While this debate continues, repeating the computer aided detection narrative ignores the much wider scope of what is possible with machine learning. In the March 2020 issue of JMRI, Chaudhari et al employ one of the most exciting recent developments in deep learning, Super Resolution, and apply it to the reconstruction of knee MRIs performed to diagnose early changes of osteoarthritis in their article entitled "Utility of Deep Learning Super Resolution in the Context of Osteoarthritis MRI Biomarkers." The technical challenge has long been balancing the need for high resolution images against the longer acquisition times of 3D sequences. Using a multi reviewer dataset from the Osteoarthritis Initiative, the authors processed 124 3D high resolution double echo steady state (DESS) acquisitions, at varying levels of disease severity. Unlike many other supervised machine learning studies, which ordinarily require the curation of manual training labels, the original DESS acquisition, acquired at 0.7 mm thickness, was degraded using an antialiasing filter to a 3x thicker slice at 2.1 mm, in effect providing matched pairs of highand lowquality images. It is these pairs of images that are then used to train the Super Resolution network, DeepResolve. From this, the evaluation task becomes the question: Starting from the low quality image, is it possible to reconstruct a high quality image? It was performed by comparing the image quality and automated cartilage segmentation performance on Super Resolution (SR) vs. classical Tricubic Interpolation (TCI) images. In multireader image quality evaluation, the readers consistently felt that the SR DeepResolve images had higher cartilage sharpness and contrast scores than TCI. Moreover, performing automated cartilage segmentation using a 3D U Net segmentation network upon DeepResolve SR images showed no statistically significant differences with the original high resolution images in Dice coefficient (DC) volumetric overlap error (VOE) and absolute volume differences, vs. statistically significant reductions in DC and VOE when segmenting on TCI images. This article applies several unique methods that are worthy of highlighting. The first is the efficient usage of simulated training labels. By having the higher quality image be the desired output, and a degraded lower quality image act as an input to the SR network, preexisting raw imaging data are efficiently leveraged, minimizing the bottleneck of manually generating training labels. Where this article takes the methodology further is in the detailed manner it evaluates the SR network towards the specific clinical application of osteoarthritis evaluation. SR images consistently outperformed TCI, with respect to many segmentation metrics and in the clinician detection and evaluation of osteophyte grading. It is this painstaking process of evaluating a new technique, across all subjective, objective, and clinical performance metrics, relative to existing practices, that is the true value of this article. Image SR, the idea of transforming low resolution into high resolution images and accurately recovering detail from sparse information, is perhaps a bit of a holy grail of image reconstruction, sounding like something straight out of science fiction, but it is in fact a field with a long history, albeit a more nascent history in Deep Learning. This technique is being applied in many other contexts, from improving the image quality of heart coronary MRAs, to adaptive MRI radiotherapy, to improving the appearance of multiple sclerosis white matter lesions on FLAIR images. However, unlike more generic methods like classical interpolation, the improved performance of many of these deep learning SR methods may be bound to a specific examination, protocol, or clinical application. Because of the reduced model transparency of such deep learning methods, continuous realworld evaluation is required, backed by prospective trials evaluating reconstruction from original low resolution images at the scanner, to establish a safe and reliable evidence base for usage. Nonetheless, while it may be hyperbolic to say that SR gives us the ability to reconstruct something from nothing, the idea that adapting interpolation algorithms to a specific exam or body part could increase performance, offers exciting See related article in our March issue: Chaudhari AS, Stevens KJ, Wood JP, et al. Utility of deep learning super resolution in the context of osteoarthritis MRI biomarkers. J Magn Reson Imaging 2020;51:768 779.
Journal of magnetic resonance imaging JMRI
Jaron Chong