lidar technology for depth
56177594
Pseudo LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
3D object detection is an essential task in autonomous driving. Recent techniques excel with highly accurate detection rates, provided the 3D input data is obtained from precise but expensive LiDAR technology. Approaches based on cheaper monocular or stereo imagery data have, until now, resulted in drastically lower accuracies a gap that is commonly attributed to poor image based depth estimation. However, in this paper we argue that it is not the quality of the data but its representation that accounts for the majority of the difference. Taking the inner workings of convolutional neural networks into consideration, we propose to convert image based depth maps to pseudo LiDAR representations essentially mimicking the LiDAR signal. With this representation we can apply different existing LiDAR based detection algorithms. On the popular KITTI benchmark, our approach achieves impressive improvements over the existing state of the art in image based performance raising the detection accuracy of objects within the 30m range from the previous state of the art of 22% to an unprecedented 74% At the time of submission our algorithm holds the highest entry on the KITTI 3D object detection leaderboard for stereo image based approaches.
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Yan Wang Wei-Lun Chao Divyansh Garg Bharath Hariharan Mark E Campbell Kilian Q Weinberger
202091088
Recent development of automotive LiDAR technology, industry and trends
Autonomous driving technology is one of the core technologies for intelligent connected vehicles. For autonomous vehicles with L3 and higher level capabilities, LiDAR is indispensable. Based on the substantial industry information of the sector and technical development, this paper systematically sorts through and analyzes a variety of LiDAR sensor technologies, the global automotive LiDAR industry ecosystem, major startups, the state of the art products and so on. In addition, this paper provides in depth research of the automotive LiDAR development trends and judgment on the prospects of automotive LiDAR integrated with the intelligent connectivity industry development trends. This paper considers that the LiDAR sensor has become a bottleneck hindering autonomous vehicles deployment, and predicts the future development paths of automotive LiDAR.
International Conference on Digital Image Processing
Fuquan Zhao Hao Jiang Zongwei Liu
210116059
Analysis of a modular SPAD based direct time of flight depth sensor architecture for wide dynamic range scenes in a LiDAR system
architecture for wide dynamic range scenes in a LiDAR system Preethi Padmanabhan1, Chao Zhang2 and Edoardo Charbon1,2 1. Advanced Quantum Architecture Laboratory (AQUA) EPFL, Neuchatel, Switzerland 2. Applied Quantum Architecture Laboratory (AQUA) Delft University of Technology, The Netherlands Email: preethi.padmanabhan@epfl.ch, c.zhang 10@tudelft.nl, edoardo.charbon@epfl.ch Phone: +41216954413, +31626585961, +41216954409
Preethi Padmanabhan Chong Zhang Edoardo Charbon
52020744
Mixed Design of SPAD Array Based TOF for Depth Camera and Unmanned Vehicle Applications
Depends on the increasing requirements of unmanned vehicle mobile Lidar systems, this paper presents a system design based on pulsed TOF depth image cameras. The design has a detection range of 300m and detecting resolution of 1.5cm. Pixels are made of SPAD. Meanwhile, innovative structure of multi pixel sharing TDC is proposed, chip area is greatly reduced and the fill factor of light sensing surface area is optimized. The TCSPC module with the functionalities of receiving each photon, measuring photon flight time and processing depth information is also included in the chip design. Based on 0.13mm CIS CMOS technology, the entire design indicates high integration with low power and fast response features.
2018 15th International Conference on Synthesis, Modeling, Analysis and Simulation Methods and Applications to Circuit Design (SMACD)
Weiwei Shi An Pan
207810189
Real time 3D reconstruction from single photon lidar data using plug and play point cloud denoisers
Single photon lidar has emerged as a prime candidate technology for depth imaging through challenging environments. Until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Here we show a new computational framework for real time three dimensional (3D) scene reconstruction from single photon data. By combining statistical models with highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of complex outdoor scenes with processing times of the order of 20 ms, where the lidar data was acquired in broad daylight from distances up to 320 metres. The proposed method can handle an unknown number of surfaces in each pixel, allowing for target detection and imaging through cluttered scenes. This enables robust, real time target reconstruction of complex moving scenes, paving the way for single photon lidar at video rates for practical 3D imaging applications. The use of single photon data has been limited by time consuming reconstruction algorithms. Here, the authors combine statistical models and computational tools known from computer graphics and show real time reconstruction of moving scenes.
Nature Communications
Julian Tachella Yoann Altmann Nicolas Mellado Aongus Mccarthy Rachael Tobin Gerald S Buller Jean-Yves Tourneret Stephen Mclaughlin
128133554
Design and evaluation of a full wave surface and bottom detection algorithm for LiDAR bathymetry of very shallow waters
Abstract Airborne Laser Bathymetry (ALB) is an attractive technology for the measurement of shallow water bodies because of the high acquisition rate and high point densities that can be achieved. Of special interest is the application of ALB in non navigable areas where the only alternatives are conventional terrestrial surveying by wading with a pole, multi media photogrammetry, or spectrally based depth retrieval. The challenge for laser based approaches in such very shallow waters 532 nm sensor. The accuracy of the algorithm is cross validated against reference measurements obtained from terrestrial survey with a total station and shows negligible bias and virtually no depth dependence for the experimental dataset.
ISPRS Journal of Photogrammetry and Remote Sensing
Roland Schwarz Gottfried Mandlburger Martin Pfennigbauer Norbert Pfeifer
155100322
Real time 3D reconstruction of complex scenes using single photon lidar: when image processing meets computer graphics
Single photon lidar has emerged as a prime candidate technology for depth imaging in extreme scenarios, such as through fog, clutter or in underwater environments. Until now, a major challenge has been the rapid reconstruction of single photon data, to permit the real time analysis of complex moving scenes. Here we show a new computational framework for real time three dimensional (3D) reconstruction of complex scenes from singlephoton data, taking full advantage of the rapid developments in single photon avalanche diode detector array technology. By combining statistical models and highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of multiple surface scenes with processing times reduced by a factor of up to 10 compared with state of the art algorithms. This has enabled robust, real time target reconstruction of complex scenes, paving the way for single photon lidar at video rates for practical 3D imaging applications, including automotive, defence and underwater imaging.
Julian Tachella Yoann Altmann Nicolas Mellado Aongus Mccarthy Rachael Tobin Gerald S Buller Jean-Yves Tourneret Stephen Mclaughlin
213251599
Deep learning optimized single pixel LiDAR
Interest in autonomous transport has led to a demand for 3D imaging technologies capable of resolving fine details at long range. Light detection and ranging (LiDAR) systems have become a key technology in this area, with depth information typically gained through time of flight photon counting measurements of a scanned laser spot. Single pixel imaging methods offer an alternative approach to spot scanning, which allows a choice of sampling basis. In this work, we present a prototype LiDAR system, which compressively samples the scene using a deep learning optimized sampling basis and reconstruction algorithms. We demonstrate that this approach improves scene reconstruction quality compared to an orthogonal sampling method, with reflectivity and depth accuracy improvements of 57% and 16% respectively, for one frame per second acquisition rates. This method may pave the way for improved scan free LiDAR systems for driverless cars and for fully optimized sampling to decision making pipelines.
Neal Radwell Steven D Johnson Matthew P Edgar Catherine F Higham Roderick Murray-Smith Miles J Padgett
210692595
Barrier Detection and Tracking from Parameterized Lidar Data
Barrier detection and tracking is a key technique in augmented reality(AR) By adding up the sense of objects, users are able to safely observe or avoid moving objects in the view. With the development of 3D Lidar technology, the acquisition of 3D points in the scene is getting more efficient. In comparison with image data, the 3D points from Lidar contain reliable depth information in a large range. However, processing unstructured point cloud is less efficient for augmented reality applications. By noticing the structure of the 3D points captured from Lidar, we propose to parameterize the data from Lidar. In this way, we are able to reuse detection and tracking methods from images for a simple solution in barrier detection and tracking from Lidar data. We test our method by using the Lidar data captured from an unmanned ship. The results show that our method can quickly detect the barriers with bounding boxes, indicating the distance, direction and size of the barrier in the scene.
2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR Adjunct)
Wen Xing Lifeng Zhu Aiguo Song
126124425
Long range depth imaging using a single photon detector array and non local data fusion
The ability to measure and record high resolution depth images at long stand off distances is important for a wide range of applications, including connected and automotive vehicles, defense and security, and agriculture and mining. In LIDAR (light detection and ranging) applications, single photon sensitive detection is an emerging approach, offering high sensitivity to light and picosecond temporal resolution, and consequently excellent surface to surface resolution. The use of large format CMOS (complementary metal oxide semiconductor) single photon detector arrays provides high spatial resolution and allows the timing information to be acquired simultaneously across many pixels. In this work, we combine state of the art single photon detector array technology with non local data fusion to generate high resolution three dimensional depth information of long range targets. The system is based on a visible pulsed illumination system at a wavelength of 670 nm and a 240 x 320 array sensor, achieving sub centimeter precision in all three spatial dimensions at a distance of 150 meters. The non local data fusion combines information from an optical image with sparse sampling of the single photon array data, providing accurate depth information at low signature regions of the target.
Scientific Reports
Susan Chan Abderrahim Halimi Fengzheng Zhu Istvan Gyongy Robert K Henderson Richard W Bowman Stephen Mclaughlin Gerald S Buller Jonathan Leach