depth perception by  Stereo Cameras
221911576
A Real Time Infrared Stereo Matching Algorithm for RGB D Cameras' Indoor 3D Perception
Low cost, commercial RGB D cameras have become one of the main sensors for indoor scene 3D perception and robot navigation and localization. In these studies, the Intel RealSense R200 sensor (R200) is popular among many researchers, but its integrated commercial stereo matching algorithm has a small detection range, short measurement distance and low depth map resolution, which severely restrict its usage scenarios and service life. For these problems, on the basis of the existing research, a novel infrared stereo matching algorithm that combines the idea of the semi global method and sliding window is proposed in this paper. First, the R200 is calibrated. Then, through Gaussian filtering, the mutual information and correlation between the left and right stereo infrared images are enhanced. According to mutual information, the dynamic threshold selection in matching is realized, so the adaptability to different scenes is improved. Meanwhile, the robustness of the algorithm is improved by the Sobel operators in the cost calculation of the energy function. In addition, the accuracy and quality of disparity values are improved through a uniqueness test and sub pixel interpolation. Finally, the BundleFusion algorithm is used to reconstruct indoor 3D surface models in different scenarios, which proved the effectiveness and superiority of the stereo matching algorithm proposed in this paper.
ISPRS Int. J. Geo Inf.
Jiageng Zhong Ming Li Xuan Liao Jiangying Qin
210700933
Correcting Decalibration of Stereo Cameras in Self Driving Vehicles
Camera systems in autonomous vehicles are subject to various sources of anticipated and unanticipated mechanical stress (vibration, rough handling, collisions) in real world conditions. Even moderate changes in camera geometry due to mechanical stress decalibrate multi camera systems and corrupt downstream applications like depth perception. We propose an on the fly stereo recalibration method applicable in real world autonomous vehicles. The method is comprised of two parts. First, in optimization step, external camera parameters are optimized with the goal to maximise the amount of recovered depth pixels. In the second step, external sensor is used to adjust the scaling of the optimized camera model. The method is lightweight and fast enough to run in parallel with stereo estimation, thus allowing an on the fly recalibration. Our extensive experimental analysis shows that our method achieves stereo reconstruction better or on par with manual calibration. If our method is used on a sequence of images, the quality of calibration can be improved even further.
Sensors
Jon Muhovic Janez Pers
12059486
The Role of Vertical Disparity in Distance and Depth Perception as Revealed by Different Stereo Camera Configurations
Vertical binocular disparity is a source of distance information allowing the portrayal of the layout and 3D metrics of the visual space. The role of vertical disparity in the perception of depth, size, curvature, or slant of surfaces was revealed in several previous studies using cue conflict paradigms. In this study, we varied the configuration of stereo cameras to investigate how changes in the horizontal and vertical disparity fields, conflicting with the vergence cue, affect perceived distance and depth. In four experiments, observers judged the distance of a cylinder displayed in front of a large fronto parallel surface. Experiment 1 revealed that the presence of a background surface decreases the uncertainty in judgments of distance, suggesting that observers use the relative horizontal disparity between the target and the background as a cue to distance. Two other experiments showed that manipulating the pattern of vertical disparity affected both distance and depth perception. When vertical disparity specified a nearer distance than vergence (convergent cameras) perceived distance and depth were underestimated as compared with the condition where vertical disparity was congruent with vergence cues (parallel cameras) When vertical disparity specified a further distance than vergence, namely an infinite distance, distance and depth were overestimated. The removal of the vertical distortion lessened the effect on perceived distance. Overall, the results suggest that the vertical disparity introduced by the specific camera configuration is mainly responsible for the effect. These findings outline the role of vertical disparity in distance and depth perception and support the use of parallel cameras for designing stereograms.
i Perception
Cyril Vienne Justin Plantier Pascaline Neveu Anne-Emmanuelle Priot
221847248
Evaluation of Increasing Camera Baseline on Depth Perception in Surgical Robotics
In this paper, we evaluate the effect of increasing camera baselines on depth perception in robot assisted surgery. Restricted by the diameter of the surgical trocar through which they are inserted, current clinical stereo endoscopes have a fixed baseline of 5.5 mm. To overcome this restriction, we propose using a stereoscopic "pickup" camera with a side firing design that allows for larger baselines. We conducted a user study with baselines of 10 mm, 15 mm, 20 mm, and 30 mm to evaluate the effect of increasing baseline on depth perception when used with the da Vinci surgical system. Subjects (N=28) were recruited and asked to rank differently sized poles, mounted at a distance of 200 mm from the cameras, according to their increasing order of height when viewed under different baseline conditions. The results showed that subjects performed better as the baseline was increased with the best performance at a 20 mm baseline. This preliminary proof of concept study shows that there is opportunity to improve depth perception in robot assisted surgical systems with a change in endoscope design philosophy. In this paper, we present this change with our side firing "pickup" camera and its flexible baseline design. Ultimately, this serves as the first step towards an adaptive baseline camera design that maximizes depth perception in surgery.
2020 IEEE International Conference on Robotics and Automation (ICRA)
Apeksha Avinash Alaa Eldin Abdelaal Septimiu E Salcudean
219530263
Depth Perception in a Single RGB Camera Using Body Dimensions and Centroid Property
Received: 18 November 2019 Accepted: 20 February 2020 Infrastructure Supervision is a compelling need for buildings and open areas. It is facilitated through the joint use of stereo vision cameras, techniques and algorithms. This Stereoscopic assessment helps monitoring systems to reconstruct people's visible surface and also provides a robust estimation of the position and posture of the person that allows 3D scene activities and interactions. In practice, in occluded fields, the correspondence between pixels and pixels interferes with the flow of data in surveillance. Structured light ToF imaging and Light Field imaging sensors came into being considering the restriction. These techniques, however failed in addressing the inaccuracies and noise introduced in the phase of profound capture. Based on the Human Anthropometric research, we suggested a technique for estimating the depth of an individual from a single RGB camera. As we deal with moving objects in a scene, also consideration is given to centroid ownership. The system is trained by feeding stature, body width and centroid as inputs to estimate a person's actual height using gradient boosting model. And a person's further anticipated height and actual height are used to predict distance. After taking actual depth (camera to person distance) and real height as ground truth, the suggested model is validated and it is inferred that the camera to person distance anticipated (Preddist) from estimated Real height is 95% correlated with actual Camera to Person distance (or depth) at a confidence level of 99.9% with RMSE of 0.092.
Traitement du Signal
P J A Alphonse K V Sriharsha
88494474
Object Detection, Classification and Localization by Infrastructural Stereo Cameras
In the future, autonomously driving vehicles have to navigate in challenging environments. In some situations, their perception capabilities are not able to generate a reliable overview of the environment, by reason of occlusions. In this contribution, an infrastructural stereo camera system for environment perception is proposed. Similar existing systems only detect moving objects by background subtraction algorithms and monocular cameras. In contrast, the proposed approach fuses three different algorithms for object detection and classification and uses stereo vision for object localization. The algorithmic concept is composed of a background subtraction algorithm based on Gaussian Mixture Models, the convolutional neural network "You only look once" as well as a novel algorithm for detecting salient objects in depth maps. The combination of these complementary object detection principles allows the reliable detection of dynamic as well as static objects. An algorithm for fusing the results of the three object detection methods based on bounding boxes is introduced. The proposed fusion algorithm for bounding boxes improves the detection results and provides an information fusion. We evaluate the proposed concept on real word data. The object detection, classification and localization in the real world scenario is investigated and discussed.
VISIGRAPP
Christian Hofmann Florian Particke Markus Hiller Jorn Thielecke
220315756
A Stereo Perception Framework for Autonomous Vehicles
Stereo cameras are crucial sensors for self driving vehicles as they are low cost and can be used to estimate depth. It can be used for multiple purposes, such as object detection, depth estimation, semantic segmentation, etc. In this paper, we propose a stereo vision based perception framework for autonomous vehicles. It uses three deep neural networks simultaneously to perform free space detection, lane boundary detection, and object detection on image frames captured using the stereo camera. The depth of the detected objects from the vehicle is estimated from the disparity image computed using two stereo image frames from the stereo camera. The proposed stereo perception framework runs at 7.4 Hz on the Nvidia Drive PX 2 hardware platform, which further allows for its use in multi sensor fusion for localization, mapping, and path planning by autonomous vehicle applications.
2020 IEEE 91st Vehicular Technology Conference (VTC2020 Spring)
Narsimlu Kemsaram Anwesha Das Gijs Dubbelman
220383785
LiDAR Stereo: Dense Depth Estimation from Sparse LiDAR and Stereo Images
Obtaining accurate depth information is vital for real world 3D perception, and a combination of different sensors is frequently used for better performance. In this paper, we propose an approach for high precision depth estimation by jointly utilizing information acquired from stereo cameras and LiDAR sensors. More specifically, we design novel convolutional neural network (CNN) architecture with two novel units: monocular completion unit and stereo fusion unit, and the complementary properties of sparse LiDAR and dense stereo images are fully explored in a boosting manner. Extensive experiments on both synthetic and real datasets have shown that the proposed approach achieves significant improvements over state of the art techniques in terms of both subjective and objective evaluations.
MM 2020
Chi Li Zhiguo Cao
222416950
Comparative Analysis of Depth Detection Algorithms using Stereo Vision
In recent years, the use of unmanned aerial vehicles in various domains has increased exponentially. Drones are being extensively used in the fields of Agriculture, Transportation, military, etc. Different sensors are being integrated into the drones depending upon the application. Lately, LIDAR sensors are being integrated on the drone for acquiring depth related information. Though these sensors have advantages, they are very costly and do not perform very well under high sun angles. Stereo cameras can be mounted on drones to get depth perception of the obstacles as they are cheaper and efficient. In this paper, we have developed and compared two different algorithms Conventional and deep learning based) for realtime depth detection of the obstacle using stereo vision with the intention to mount the stereo camera on drone in future. The comparison is based on accuracy, range of operation and load incurred by the algorithm on the system. The coefficient of determination (R 2) and correlation coefficient has also been calculated which shows that Algorithm1 exhibits correlation and R 2 value of 0:9985 and 0:9971 respectively. This is considerably higher than Algorithm2 whose values are around 0:8779 and 0:7707 respectively.
2020 IEEE 6th World Forum on Internet of Things (WF IoT)
Poluri Nikhil Koundinya Yuki Ikeda NT Sanjukumar Pachamuthu Rajalakshmi Takanori Fukao
230994397
LiDAR Stereo Camera Fusion for Accurate Depth Estimation
Dense 3D reconstruction of the surrounding environment is one the fundamental way of perception for Advanced Driver Assistance Systems (ADAS) In this field, accurate 3D modeling finds applications in many areas like obstacle detection, object tracking, and remote driving. This task can be performed with different sensors like cameras, LiDARs, and radars. Each one presents some advantages and disadvantages based on the precision of the depth, the sensor cost, and the accuracy in adverse weather conditions. For this reason, many researchers have explored the fusion of multiple sources to overcome each sensor limit and provide an accurate representation of the vehicle's surroundings. This paper proposes a novel post processing method for accurate depth estimation, based on a patch wise depth correction approach, to fuse data from LiDAR and stereo camera. This solution allows for accurate edges and object boundaries preservation in multiple challenging scenarios.
2020 AEIT International Conference of Electrical and Electronic Technologies for Automotive (AEIT AUTOMOTIVE)
Hafeez Husain Cholakkal Simone Mentasti Mattia Bersani Stefano Arrigoni Matteo Matteucci Federico Cheli