\section{Abstract}
\begin{abstract}
  Drones will revolutionize 3D modeling. A 3D model represents an accurate reconstruction of an object or structure. This paper explores the design and implementation of \textsc{ares}\xspace, which provides near real-time, accurate reconstruction of 3D models using a drone-mounted LiDAR; such a capability can be useful to document construction or check aircraft integrity between flights. Accurate reconstruction requires high drone positioning accuracy, and, because GPS can be in accurate, \textsc{ares}\xspace uses SLAM. However, in doing so it must deal with several competing constraints: drone battery and compute resources, SLAM error accumulation, and LiDAR resolution. \textsc{ares}\xspace uses careful trajectory design to find a sweet spot in this constraint space, a fast reconnaissance flight to narrow the search area for structures, and offloads expensive computations to the cloud by streaming compressed LiDAR data over LTE. \textsc{ares}\xspace reconstructs large structures to within 10s of cms and incurs less than 100~ms compute latency.

\end{abstract}
\section{Appendix}
\label{s:appendix}

\subsection{Drone compute}
\begin{figure}[b]
  \centering
  \includegraphics[width=0.7\columnwidth]{figs/fig_tx2_ransac.pdf}
  \caption{Plane-fitting on a TX2}
  \label{fig:tx2ransac}
\end{figure}

We ran a plane-fitting algorithm, RANSAC, (a module that we use in our pipeline) on a real-world point cloud trace using drone compute platform (Jetson TX2). We found that (\figref{fig:tx2ransac}) it takes the TX2, on average, 0.5~seconds to process a single point cloud. The 64-beam LiDAR generates 20 point clouds per second whereas plane-fitting accounts for only 5\% of the entire execution time of our reconstruction pipeline. Thus, the TX2 will take 200~seconds to process a single second's worth of data from the 64-beam LiDAR if we ran it at 20 frames per second. To this end, we offload computations from the drone to the cloud.

\subsection{Point cloud compression}
\label{s:app_compression}
\textsc{ares}\xspace uses two techniques (\emph{i.e.,}\xspace viewpoint filtering and Octree compression) to compress LiDAR point clouds to within 1.2 to 4.0 Mbps and transmit them over LTE.

\begin{figure}[t]
  \centering
  \includegraphics[width=0.7\columnwidth]{figs/fig_real_world_top_down.pdf}
  \caption{Top down view of reconstructed 3D model for a large real-world complex}
  \label{fig:3d_real_model}
\end{figure}

\parab{Viewpoint filtering.}
The OS1-64 LiDAR has a 360$\degree$ horizontal field-of-view (FoV) and a 45$\degree$ vertical FoV. In a drone-mounted LiDAR (\figref{fig:lidar_orientation}), only a portion of the full 360$\degree$ contains useful information. Beams directed towards the sky, or towards objects beyond LiDAR range, generate \textit{zero returns}. Viewpoint filtering removes these, and also removes returns from the body of the drone. To compress point clouds, \textsc{ares}\xspace simply removes zero returns. In practice, we have found it to be important to also filter out returns from the drone itself, and also returns further away from the nominal range of the LiDAR, since these are erroneous. So, \textsc{ares}\xspace filters all points closer than 5~m and further than 120~m.

\parab{Octree compression.}
After filtering the point cloud, \textsc{ares}\xspace compresses the retained data using a
standard octree compression algorithm~\cite{octree} designed specifically for point clouds (and hence this is better than data-agnostic compression techniques like gzip). An octree is a three-dimensional tree data structure where each node is a cube that spans a 3D region, and has exactly eight children. The dimensions of the cubes at the leaves of the tree determine the \textit{octree resolution}. The numerical precision used to encode point positions determines the \textit{point resolution}. Octree compression efficiently encodes empty leaves or empty tree-internal nodes (those whose descendant leaves are empty). It also performs inter-frame compression (similar to video encoders), efficiently encoding unchanged leaves or internal nodes between two successive point clouds.
As we show in \secref{sec:eval}, we can parameterize octree compression to achieve point-cloud transmission rates of 1.2-4~Mbps. \textsc{ares}\xspace chooses different values of octree resolution and point resolution, two parameters that govern the compressibility of point clouds, to achieve point-cloud transmission rates of 1.2--4~Mbps (\secref{sec:eval}), well within the range of achievable LTE speeds.

\subsection{Implementation Details.}
\label{s:app_impl}
We have implemented \textsc{ares}\xspace using the Point Cloud Library (PCL~\cite{octree}), the Cartographer~\cite{Cartographer} LiDAR SLAM implementation\footnote{We use Cartographer but it can be replaced by other LiDAR SLAM algorithms like LOAM~\cite{zhang2014loam}}, the Boost C++ libraries~\cite{Boost}, and the Robotic Operating System (ROS~\cite{ros}). For the recon phase, we used functions from the Point Cloud Library (PCL~\cite{octree}) for plane-fitting, outlier removal and clustering. Our compression and extraction modules also use PCL and are implemented as ROS nodes. The drift detection module uses a Python package for the Umeyama alignment~\cite{grupp2017evo}. Not counting libraries and packages it uses, \textsc{ares}\xspace is 15,500 lines of code.

\subsection{Recon Flight}
\label{s:app_recon_flight}
The goal of the recon flight is to survey the area and find the boundary of the structure as fast as possible. \textsc{ares}\xspace uses a flight trajectory as shown in \figref{fig:recon_traj} in which parallel scans of length $d$ are separated by a scan width $s$. In designing the recon flight, \textsc{ares}\xspace can change the height, speed and LiDAR orientation of the drone. To find the right set of parameters, we performed an exhaustive parameter sweep.

\parab{Optimum height for recon.} To find the optimum height for the recon flight, we planned recon trajectories for a 20~m building (within a 300~m x 300~m area) in AirSim at different heights (from 40~m to 90~m). We flew the drone and ran the boundary estimation on the collected \textit{highly compressed} LiDAR point clouds at 10~Hz. For each height, we collected data and ran the boundary detection module five times. Higher flights increase scan width (~\figref{fig:lidar_coverage_area}) at the expense of point density. However, \textsc{ares}\xspace's boundary detection algorithm is robust to lower density point clouds (up till 80~m) and can accurately estimate the boundary of the building from a height of upto 80~m. \figref{fig:recon_height} shows the 2D boundary detection accuracy, completeness (lower is good) and flight duration (as a proxy for battery usage) as a function of the height of the drone. We find that at 80~m (or 60~m from the building), \textsc{ares}\xspace can jointly optimize for battery efficiency and boundary detection accuracy. At 80~m, \textsc{ares}\xspace can complete the recon flight in 150~seconds and estimate the boundary to within 2.5~m accuracy and completeness. Beyond 80~m, the scan width and point density decrease. This results in longer flights and higher boundary detection accuracy and completeness.

\begin{figure}[t]
  \centering
  \includegraphics[width=0.5\columnwidth]{figs/fig_recon_trajectory.pdf}
  \caption{Recon flight trajectory for \textsc{ares}\xspace.}
  \label{fig:recon_traj}
\end{figure}

\begin{figure}[t]
  \centering
  \includegraphics[width=0.5\columnwidth]{figs/fig_boundary_estimation_height_vs_battery.pdf}
  \caption{Finding the right height for boundary detection accuracy and battery efficiency in the recon flight.}
  \label{fig:recon_height}
\end{figure}

\parab{Optimum speed for recon.}
To find the optimum speed for the recon flight, we planned a recon trajectory for the drone to fly over the same 20~m building at a height of 80~m from the ground. We flew the drone in the planned trajectory at speeds from 1~m/s to 8~m/s and ran boundary detection on the \textit{highly compressed} point clouds at 10~Hz. For each speed, we collected data and ran the boundary detection module five times. \figref{fig:recon_speed} illustrates the effect of drone speed on the boundary detection accuracy, completeness and the flight duration. A higher speed results in lower flight duration but at the expense of boundary detection accuracy and completeness. Even then, \textsc{ares}\xspace robustly extracts the boundary up till 6~m/s. At higher speeds, the overlap between consecutive frames is smaller and hence \textsc{ares}\xspace cannot accurately stitch the frames together. As such, \textsc{ares}\xspace flies the drone at the sweet spot \emph{i.e.,}\xspace 4~m/s where the flight duration is approximately 150~seconds and accuracy and completeness are 2.5~m.

\parab{Optimum LiDAR orientation.}
LiDAR orientation controls scan width and point cloud overlap. A parallel orientation means larger overlap but small scan width $s$. On the other hand, a perpendicular orientation means smaller overlap but larger scan width $s$. Larger scan width $s$ means a smaller flight duration (\figref{fig:recon_traj}). A large overlap means better scan matching accuracy. Since \textsc{ares}\xspace uses GPS for stitching in the recon phase, so it is robust to the overlap. Hence, to minimize flight duration, it uses a perpendicular orientation of the LiDAR. We conducted experiments (omitted for brevity) without different orientations of the LiDAR and confirmed that a perpendicular orientation minimzes flight duration without any loss in accuracy/completeness.

\begin{figure}[b]
  \centering
  \includegraphics[width=0.5\columnwidth]{figs/fig_boundary_estimation_speed_vs_battery.pdf}
  \caption{Finding the right speed for boundary detection accuracy and battery efficiency in the recon flight.}
  \label{fig:recon_speed}
\end{figure}

\parab{Boundary extraction for different buildings.}
To show that \textsc{ares}\xspace can accurately extract the 2D boundary of any building, we collected LiDAR traces of a drone flying over five different buildings in Airsim at a height of 80~m and speed of 4~m/s. We collected data over each building five times. Then, we ran boundary detection on the \textit{highly compressed} point clouds at 10~Hz. We summarize the boundary detection accuracy, completeness and the flight duration in \tabref{tab:recon_buildings}. As expected, the flight duration for all buildings is independent of the underlying building. For all building types, \textsc{ares}\xspace can accurately extract the boundary of all buildings within 2.5~m accuracy and completeness. This shows that \textsc{ares}\xspace's boundary detection is scalable to all building shapes.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Structure\\ type\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Flight \\ duration (s)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Comp.\\ (m)\end{tabular} \\ \hline
Star-shaped        & 150 & 1.39 & 1.67 \\ \hline
H-shaped           & 150 & 1.31 & 1.83 \\ \hline
Plus-shaped        & 150 & 1.35 & 1.55 \\ \hline
Pentagon           & 150 & 2.58 & 2.58 \\ \hline
Rectangular        & 150 & 2.50 & 2.53 \\ \hline
\end{tabular}
\caption{\textsc{ares}\xspace boundary estimation accuracy, completeness and flight duration for different building types using high compression.}
\label{tab:recon_buildings}
\end{table}

\parab{Effect of point cloud compression.}
To evaluate the effect of point cloud compression on boundary extraction, we compressed a real-world over the 70~m~x~40~m~x~20~m building with the four different compression profiles described above. Then, we ran our boundary extraction algorithm on the compressed traces. \tabref{tab:recon_compression} shows that \textsc{ares}\xspace's boundary extraction algorithm is robust to compression. While bringing down bandwidth by a factor of 377, for high compression, \textsc{ares}\xspace only trades off 36~cm in accuracy and 24~cm in completeness. With higher bandwidths promised with the emergence of 5G, \textsc{ares}\xspace can achieve the same boundary extraction accuracy as an uncompressed trace.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Compression\\ profile\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Required \\ bandwidth (Mbps)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Comp.\\ (m)\end{tabular} \\ \hline
Uncompressed        & 480.0 & 1.09 & 1.09 \\ \hline
View-point          & 42.7 & 1.09 & 1.09 \\ \hline
Lossless            & 7.86 & 1.09 & 1.09 \\ \hline
Low                 & 3.80 & 1.09 & 1.10 \\ \hline
Medium              & 2.50 & 1.13 & 1.07 \\ \hline
High                & 1.27 & 1.45 & 1.33 \\ \hline
\end{tabular}
\caption{\textsc{ares}\xspace boundary estimation accuracy and completeness for different levels of compression.}
\label{tab:recon_compression}
\end{table}

\parab{Effect of sub-sampling.}
\textsc{ares}\xspace's boundary detection algorithm runs at 10~fps. A Ouster-64 beam LiDAR generates 20 point clouds per second. So, the boundary detection algorithm must be robust to sub-sampling of point clouds. Our evaluations show that, for a drone traveling at 4~m/s, it works well even when using one point cloud every 3 seconds. Because \textsc{ares}\xspace's boundary detection uses GPS for stitching, it does not need overlap between 3D frames.

\subsection{Data Collection}
\label{s:app_data_coll}
In this section, we perform a parameter sensitivity study to find the optimum parameters for running SLAM accurately on \textit{real-world} UAV flights. To do this, we report positioning error generated by SLAM. For the lack of accurate ground truth in the real-world, we compare SLAM positions against a GPS trace. Positioning accuracy is directly related to 3D model RMSE because these poses are used to position 3D point cloud in generating a 3D model. A higher positioning error leads to a higher reconstruction error and vice-versa.

\parab{Effect of drone speed.}
Because GPS is erroneous, we only draw qualitative conclusions. As \tabref{tab:real_speed_vs_rmse}, taken from our drone traces, shows, slower flights have lower SLAM error than faster one, and parallel orientations have lower SLAM error than perpendicular.

\parab{Effect of drone height.}
Similarly, SLAM error increases with height and, in real-world traces, the parallel orientation seems to be significantly better than the perpendicular orientation (\tabref{tab:real_height_vs_rmse}). At a distance of 20~m from the surface of the building, the parallel orientation has the minimum positioning error \emph{i.e.,}\xspace 1.25~m. Beyond 20~m for parallel and 40~m for perpendicular, SLAM loses track completely because of lower point density.

\begin{table}[htbp]
\footnotesize
\centering
\begin{tabular}{|c|c|c|}
\hline
\multirow{2}{*}{LiDAR Orientation} & \multicolumn{2}{c|}{Drone model collection speed (m/s)} \\ \cline{2-3}
              & 1.5 m/s & 3.0 m/s \\ \hline
Parallel      & 1.25    & 3.33    \\ \hline
Perpendicular & 3.12    & 7.64    \\ \hline
\end{tabular}
\caption{Positioning errors for parallel and perpendicular LiDAR orientations at different speeds for \textit{real-world traces} at a vertical height of 20~m from the building.}
\label{tab:real_speed_vs_rmse}
\end{table}

\begin{table}[htbp]
\footnotesize
\centering
\begin{tabularx}{0.48\textwidth} {|c|>{\centering\arraybackslash}X|>{\centering\arraybackslash}X|>{\centering\arraybackslash}X|}
\hline
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}LiDAR\\ orientation\end{tabular}} & \multicolumn{3}{c|}{Drone model collection height from building (m)} \\ \cline{2-4}
 & 20 & 40 & 60 \\ \hline
Parallel & 1.25 & 5.41 & $\infty$ \\ \hline
Perpendicular & 2.18 & $\infty$ & $\infty$ \\ \hline
\end{tabularx}
\caption{Positioning errors different LiDAR orientations at different heights for \textit{real-world traces} at 1~m/s.}
\label{tab:real_height_vs_rmse}
\end{table}

\begin{figure*}[t]
  \centering
  \includegraphics[width=\textwidth]{figs/fig_3D_models.pdf}
  \caption{Reconstructed 3D models at different levels of compression. Top-left: ground truth, top-right: low compression, bottom-left: medium compression, bottom-right: high compression}
  \label{fig:3d_model}
\end{figure*}

\section{Conclusions}
\label{s:conclusions}

In this paper, we have taken a step towards accurate, near-real time 3D reconstruction using drones. Our system, \textsc{ares}\xspace, uses novel techniques for navigating the tension between cellular bandwidths, SLAM positioning errors, and compute constraints on the drone. It contains algorithms for estimating building geometry, for determining excessive SLAM drift, and for recovering from excessive drift. It can achieve reconstruction accuracy to within 10s of centimeters in near real-time, even after compressing LiDAR data enough to fit within achievable LTE speeds. Future work can include using more sophisticated drone battery models, cooperative reconstruction of large campuses using multiple drones, and generalizing further to structures of arbitrary shape.

\section{Abstract}
\begin{abstract}

A 3D model represents an accurate reconstruction of a object. This paper explores the design and implementation of \textsc{ares}\xspace, which provides near real-time, accurate, autonomous reconstruction of 3D models using a drone-mounted LiDAR; such a capability can be useful to document building construction or check aircraft integrity between flights. Accurate reconstruction requires high drone positioning accuracy, and, because GPS can be inaccurate, \textsc{ares}\xspace uses SLAM. However, in doing so it must deal with several competing constraints: drone battery and compute resources, SLAM error accumulation, and LiDAR resolution. \textsc{ares}\xspace uses careful trajectory design to find a sweet spot in this constraint space and a fast reconnaissance flight to narrow the search area for structures, and offloads SLAM to the cloud by streaming compressed LiDAR data over LTE. It reconstructs buildings to within 10s of cms and incurs less than 100~ms compute latency.

\end{abstract}
\section{DART Design}
\label{sec:design}
\begin{figure}[b]
  \centering
  \includegraphics[width=0.6\columnwidth]{figs/overview}
  \caption{\textsc{ares}\xspace architecture\label{fig:overview}}
\end{figure}

Because drone-based 3-D reconstruction is a complex multi-dimensional problem (\tabref{tab:challenges}), we have focused on a geometrically regular, but important, subset of structures for reconstruction: \textit{buildings}. As this section will make clear, even this choice poses significant challenges. It also brings out the computation and communication issues in 3-D reconstruction that are the main focus of this paper. In \secref{s:slam-phase}, we discuss what it would take to generalize to other, more complex, structures.

\parab{Overview.}
To use \textsc{ares}\xspace, a user specifies: a) an \textit{area of interest}, and b) a minimum \textit{target point density}. Point density is the number of points per unit area on the surface of a point cloud; this knob controls the quality of the 3D model. In the area of interest, \textsc{ares}\xspace guides a drone to automatically discover buildings, and constructs a 3D model of the buildings in \textit{near real-time} (\emph{i.e.,}\xspace during the drone flight) while \textit{minimizing flight duration} at that given minimum point density. (To a first approximation, drone battery usage increases with flight duration; we have left it to future work to incorporate drone battery models.) \textsc{ares}\xspace splits its functionality across two components: (a) a lightweight subsystem that runs on the drone, and (b) a cloud-based component that discovers buildings, generates drone trajectories, and reconstructs the 3D models on-the-fly.

\textsc{ares}\xspace's cloud component (\figref{fig:overview}) generates an efficient \textit{reconnaissance} trajectory over the area of interest to discover the \textit{rooftop geometry} of buildings. Extracting the geometry from LiDAR data can be computationally intensive (\secref{sec:intro}), so \textsc{ares}\xspace \textit{streams compressed point clouds} to a cloud service during flight over a cellular (LTE) connection. The cloud service extracts the geometry, then prepares a more careful \textit{model collection} trajectory that designs a minimal duration flight while ensuring high 3D model accuracy. During this second flight, the drone also streams compressed point clouds, and the cloud service runs SLAM to estimate point cloud poses, and  composes the received point clouds into the building's 3D model. Mid-flight, the cloud service may \textit{re-calibrate} the trajectory dynamically to minimize drift accumulation.

Below, we first describe model collection (\secref{s:slam-phase}), since that is the most challenging of \textsc{ares}\xspace's components. We then describe how \textsc{ares}\xspace extracts the rooftop geometry (\secref{s:area-inter-reconn}), then conclude by describing point-cloud compression (\secref{s:compression}).

\subsection{Model Collection}
\label{s:slam-phase}

Given the building geometry (\secref{s:area-inter-reconn}), \textsc{ares}\xspace designs a \textit{model collection trajectory} to capture the 3D model (\secref{sec:intro}) of the building.

\parab{What constitutes a good 3D model?}
Prior work on 3D reconstruction~\cite{reconstructionmetrics} has proposed two metrics, \textit{accuracy} and \textit{completeness}. Consider a 3D model $M$ and a corresponding ground-truth $M_g$. Accuracy is the Root Mean Square Error (RMSE) of the distance from each point in $M$ to the nearest point in $M_g$. Completeness is the RMSE of the distance from each point in $M_g$ to the nearest point in $M$. If both values are zero, $M$ perfectly matches $M_g$. If $M$ captures all points in $M_g$, but the positions of the points are erroneous, then both accuracy and completeness will be non-zero. If $M$ captures only one point in $M_g$, but positions it correctly, its accuracy is perfect, but completeness is poor.

\begin{figure*}[t]
  \begin{minipage}{0.38\linewidth}
     \includegraphics[width=\linewidth]{figs/lidar_orient}
    \caption{Parallel and perpendicular LiDAR orientation\label{fig:lidar_orientation}}
  \end{minipage}
  \begin{minipage}{0.32\linewidth}
  \centering
  \includegraphics[width=0.7\linewidth]{figs/fig_slam_vs_degree_of_overlap}
  \caption{Impact of orientation on SLAM positioning error.}
  \label{fig:overlap_error}
\end{minipage}
\begin{minipage}{0.29\linewidth}
  \centering
  \includegraphics[width=0.6\linewidth]{figs/equidense}
  \caption{Equi-dense trajectory scan width}
  \label{fig:equidense}
\end{minipage}
\end{figure*}

\parab{Trajectories, SLAM and 3D reconstruction error.}
As compared to an autonomous vehicle, a drone-mounted LiDAR can only perceive ($\frac{45\degree}{360\degree}$) of the 3D point cloud. This makes scan matching more difficult for SLAM. Thus, the trajectory of the drone flight can impact 3D model completeness and accuracy, in part because a poorly designed trajectory can increase SLAM error. In designing the drone's trajectory, \textsc{ares}\xspace can control the following parameters: the actual \textit{path} of the drone over the building, its \textit{speed}, its \textit{height}, and the \textit{orientation} of the LiDAR with respect to the ground. We now discuss the qualitative impact of these parameter choices; later (\secref{sec:eval}), we empirically quantify the best parameter choices.

\textit{\textbf{Orientation}} impacts accuracy. At a fixed height and speed, a \textit{parallel} orientation of the LiDAR (\figref{fig:lidar_orientation}) in which its scan plane aligns with the drone's direction of motion, results in higher overlap between two successive point clouds than with a \textit{perpendicular} trajectory, therefore, lower SLAM error and better accuracy. \figref{fig:overlap_error}, obtained using the methodology described in \secref{sec:eval}, quantifies this intuition: different orientations have different degrees of overlap, and as overlap decreases, SLAM's positioning error increases. A parallel orientation (0$\degree$) has the lowest SLAM error because it has the highest visibility lifespan. (\textit{Visibility lifespan}, the time for which a point on the building's surface is visible during flight, is a proxy for overlap; a longer lifespan indicates greater overlap).

\textit{\textbf{Speed}} impacts model accuracy. If the drone flies fast, two successive point clouds will have fewer overlapping points, resulting in errors in the SLAM's pose transformations and (therefore) pose estimates (for a reason similar to \figref{fig:overlap_error}) which leads to poor 3D model accuracy. So, \textsc{ares}\xspace must fly as slow as possible.

\textit{\textbf{Height}} impacts both accuracy and completeness. Because LiDAR beams are radial, the higher a drone flies, the less dense the points on the surface of the building. Lower density results in worse completeness. Accuracy is also worse, because the likelihood of matching the same point on the surface between two scans decreases with point density. For instance, the positioning errors for point densities of 2.2 points per m$^{2}$ and 3.0 points per m$^{2}$ are 2.5~m and 1.0~m respectively (graph omitted for brevity).
So, \textsc{ares}\xspace must fly as low as possible.

The drone's path must ensure \textit{\textbf{coverage}} of the buildings rooftop and sides. Consider a narrow and wide building. The drone must fly several times over the building to capture all its surfaces. If it flies low, slowly, and at a parallel orientation, the flight duration can be significant. Over long durations, SLAM accumulates \textit{drift}, which can worsen model accuracy and completeness.

\textsc{ares}\xspace designs \textit{equi-dense trajectories} to control model completeness, and uses \textit{offline data-driven parameter estimation} to find the choice of speed, height and orientation. To minimize drift accumulation, \textsc{ares}\xspace performs online \textit{drift estimation and re-calibration}. We describe these below.

\parab{Equi-dense Trajectories.} An equi-dense trajectory ensures that the resulting model is (a) complete, and (b) captures the building with a point density that is no less than the specified minimum target point density $d$.

\parae{Point density depends on LiDAR parameters and height.}
The height (more generally, distance for vertical surfaces like the sides of the building) at which a LIDAR flies from a surface governs the average density of
points it obtains from that surface; larger heights result in lower \textit{point density}.

For a given LiDAR configuration, we can compute the point density as a function of height. For instance, for an Ouster LiDAR with 64 beams, horizontal resolution of 1024, and a vertical field of view of 45$\degree$, to a first approximation, two consecutive beams are at an angular separation of 0.7$\degree$ ($\frac{45}{64}$) and lasers from same beam are 0.35$\degree$ ($\frac{360}{1024}$) apart. Using geometry, for a surface at a distance $h$ from the drone, we can compute the projection of the LIDAR on that surface. Using this projection, we can compute the point density throughout the whole point cloud. Central regions of the point cloud have much higher density than regions at the extremities.

\begin{figure*}[t]
  \centering
  \begin{subfigure}{0.18\linewidth}

    \includegraphics{figs/mod_traj1_updown_small.pdf}
    \caption{The lateral flight}
    \label{fig:mod_lateral}
  \end{subfigure}
  \begin{subfigure}{0.18\linewidth}

    \includegraphics{figs/mod_traj2_updown_small.pdf}
    \caption{The longitudinal flight}
    \label{fig:mod_longitudinal}
  \end{subfigure}
  \begin{subfigure}{0.18\linewidth}

    \includegraphics{figs/mod_traj4_updown_small.pdf}
    \caption{An alternative flight plan}
    \label{fig:mod_total}
  \end{subfigure}
  \begin{subfigure}{0.18\linewidth}

    \includegraphics{figs/mod_traj3_updown_small.pdf}
    \caption{Mid-flight re-calibration}
    \label{fig:mod_mid_flight}
  \end{subfigure}
  \caption{Model collection trajectory design}
  \label{fig:model_trajectory}
\end{figure*}

\parae{Coverage depends on height.}
The density of points at which a drone captures a surface depends on its height $h$. Given a height $h$, \figref{fig:equidense} shows the coverage of the LiDAR on a given surface. In general, the drone can only capture a subset of this \textit{full} coverage region with a minimum target point density $d$ (shown by target density region in \figref{fig:equidense}). Now, suppose the drone's trajectory performs a rectilinear scan over the surface, like the one shown in \figref{fig:mod_lateral}. Then, to ensure that \textsc{ares}\xspace captures the entire surface at least at a density $d$, the \textit{scan width} must be equal to or smaller than the width of the target density coverage region (\figref{fig:equidense}).

\parae{\textsc{ares}\xspace estimates scan width from LiDAR parameters.}
To estimate the width of the target-density coverage region, \textsc{ares}\xspace uses the LiDAR parameters, and models LiDAR geometry, to derive a function which returns the scan width for a given target density $d$, and a given height $h$. It models the beams of the LiDAR as having equal angular separation, so it can compute the points at which these beams intersect with the plane at a height (or distance) $h$ away. Given a target point density $d$, \textsc{ares}\xspace can compute the largest width at this height that will ensure minimum point density $d$.

This density guarantee is \textit{nominal}; in practice, LiDARs may drop some reflections if they are noisy~\cite{Lidarsim}. Future work can model this noise for better equi-dense trajectory designs.

\parab{Trajectory path planning, and parameter selection.} \textsc{ares}\xspace uses a fixed orientation, height and speed for the drone flight; it uses an \textit{offline data-driven} approach to determine these.

\parae{Offline simulations to estimate parameters.} As described above, LiDAR orientation, and drone height, and speed determine how well SLAM can estimate positions to ensure accuracy and completeness. We could have tried to analytically model the system to derive the optimal parameter choices. Modeling the LiDAR is feasible (as we discuss above); modeling SLAM's feature matching mathematically is much harder. So, we resort to exploring the space of parameters using simulation. Specifically, we use game engine driven photorealistic simulators like AirSim~\cite{airsim} to sweep the space of parameters. Then, we validate these results using traces that we capture from our real-world prototype. We discuss this methodology in greater detail in \secref{sec:eval}, where we show that there exists a sweet spot in the parameter space that ensures high accuracy and completeness while minimizing flight duration. Specifically, \secref{s:data-collection} shows that a parallel orientation, while flying at a distance of 20~m from the surface (or lower, if necessary, to meet the point density constraint) at 1~m/s gives the best accuracy.

\parae{A fixed trajectory.}
Given these parameters, the building geometry and scan width, \textsc{ares}\xspace designs a drone flight path. \figref{fig:model_trajectory} describes this for a building shaped like a rectangular solid; \textsc{ares}\xspace supports other building shapes (\secref{sec:eval}). \textsc{ares}\xspace's model collection trajectory starts from an \textit{origin}; this point defines the origin of the coordinate system for the resulting 3D model, then laterally traverses the building to capture the two sides of the building (\figref{fig:mod_lateral}). Its flight path extends a distance $\delta$ beyond the building edges to account for errors in building geometry estimation. As it moves to each side laterally, it moves up/down on each face to capture them at the same minimum point density. Then, it returns to the origin, and traverses longitudinally (\figref{fig:mod_longitudinal}).

\parae{Avoiding LiDAR rotations.}
Why return to the origin?\footnote{Drones use fiducials (\emph{e.g.,}\xspace a drone landing pad with a distinctive design) to identify the origin.} This \textit{loop closure} maneuver is an important technique in SLAM to correct for drift~\cite{slam_part1}. If loop closure were not necessary, we could have designed a trajectory as shown in \figref{fig:mod_total}. However, this trajectory requires a \textit{rotation} of the drone at the dashed segment to ensure that the lateral and longitudinal segments have the same drone orientation. Rotation can significantly increase SLAM drift; \figref{fig:slam_rotation} shows an example in which the green dashed line depicts the actual (ground truth) drone trajectory, and the blue line SLAM's estimated pose. At the bottom right corner of the trajectory, when the drone rotates, SLAM is completely thrown off.

\parab{Drift Estimation and Re-calibration.} \textsc{ares}\xspace uses return-to-origin to re-calibrate SLAM drift. The second, longitudinal, flight starts a new SLAM session; to ``stitch'' the two sessions together, \textsc{ares}\xspace needs to compute a transformation matrix that transforms the coordinate system of the first session to that of the second. \textsc{ares}\xspace uses standard techniques for this. More important, \textsc{ares}\xspace designs the longitudinal flight to start close to the origin, which has two benefits: (a) shorter flight time resulting in lower overall energy consumption and (b) less drift accumulation.

\parae{Mid-flight re-calibration for accuracy and flight efficiency.} Return-to-origin re-calibration might also be necessary in the middle of one of the flights (\figref{fig:mod_mid_flight}), if the environment is sparse and SLAM tracking fails.  To combat this, \textsc{ares}\xspace could have added more loop closure maneuvers in the lateral and longitudinal flights. However, returning to origin is an expensive operation in terms of the drone's battery. Instead, \textsc{ares}\xspace actively monitors drift-error and returns to the origin only when needed. In that case, the flight resumes at the point at which it detected excessive drift: the direct path from the origin to that point is always shorter than the initial segment, ensuring that the resumed flight starts with a lower drift.

\parae{Using deviation from GPS trajectory to detect drift.}
 However, detecting excessive drift is non-trivial, since \textsc{ares}\xspace has no way of knowing when SLAM's position estimates are wrong, because it does not have accurate ground truth. \textsc{ares}\xspace leverages the GPS readings associated with SLAM poses: each sequence of readings gives a \textit{GPS trajectory}, and \textsc{ares}\xspace attempts to find the best possible match between the GPS trajectory (\emph{e.g.,}\xspace the green line in \figref{fig:slam_rotation}) and the SLAM-generated trajectory (the blue line in \figref{fig:slam_rotation}). If there is a significant deviation, \textsc{ares}\xspace assumes there is a drift and invokes re-calibration.

This approach is robust to GPS errors, since it matches the \textit{shape} of the two trajectories, not their precise positions (\algoref{algo:imperfection_detection}). Specifically, \textsc{ares}\xspace continuously executes 3D SLAM on the stream of compressed LiDAR frames from the drone, and estimates the pose of each frame. It synchronizes the GPS timestamps with the LiDAR timestamps (line 1), then transforms GPS readings using the Mercator projection (line 2). It then aligns the GPS trajectory and the SLAM-generated trajectory using the Umeyama algorithm~\cite{umeyama} to determine the rigid transformation matrices (\emph{i.e.,}\xspace translation, and rotation) that best align the SLAM and GPS poses (lines 3-4). \textsc{ares}\xspace partitions trajectories into fixed length segments and after alignment, computes the RMSE between the two trajectories in these segments, and uses these RMSE values as an indicator of excessive drift: if the RMSE is greater than a threshold $\rho$ (lines 5-12), \textsc{ares}\xspace invokes return-to-origin.

\begin{figure*}[t]
  \begin{minipage}{0.49\columnwidth}
     \includegraphics[width=0.9\columnwidth]{figs/fig_slam_rotation.pdf}
    \caption{Rotation throws off SLAM}
  \label{fig:slam_rotation}
  \end{minipage}
  \begin{minipage}{0.49\columnwidth}
     \includegraphics[width=0.8\columnwidth]{figs/fig_blimp_generalization.pdf}
    \caption{To reconstruct other structures (\emph{e.g.,}\xspace a blimp), \textsc{ares}\xspace wraps them in a rectangular solid and plans a model collection trajectory for it.}
  \label{fig:bounding_box_blimp}
  \end{minipage}
  \begin{minipage}{0.49\columnwidth}
  \centering
  \includegraphics[width=0.9\columnwidth]{figs/fig_recon_phase_scan_width.pdf}
    \caption{Coverage and scan widths for different orientations and heights.
    \label{fig:lidar_coverage_area}}
\end{minipage}
\begin{minipage}{0.49\columnwidth}
  \centering
  \includegraphics[width=0.6\columnwidth]{figs/fig_boundary_detection.pdf}
  \caption{\textsc{ares}\xspace's building detector on a real building.}
  \label{fig:boundary}
\end{minipage}
\end{figure*}

\begin{algorithm}[t]
  \caption{Detecting Excessive Drift}
  \label{algo:imperfection_detection}
\SetKwFunction{TimeSynchronization}{TimeSynchronization}
\SetKwFunction{GPSToMercator}{GPSToMercator}
\SetKwFunction{UmeyamaAlignment}{UmeyamaAlignment}
\SetKwFunction{TransformTrajectory}{TransformTrajectory}
\SetKwFunction{RMSE}{RMSE}
\SetKwFunction{IsExcessive}{IsExcessive}
\SetKwFunction{Append}{Append}

\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}
\Input{SLAM poses $S$ and GPS tags $G$}
\Output{Imperfect regions $I$}
\BlankLine

$ S^{'}, G^{'} \leftarrow $ \TimeSynchronization {$S$, $G$} \\% time sync poses
$ G_{a}^{'} \leftarrow $ \GPSToMercator { $G^{'}$ } \\% convert GPS to mercator
$ t_{cw} \leftarrow $ \UmeyamaAlignment { $G_{a}^{'}$, $S^{'}$ } \\% find tcw to align SLAM and GPS
$ S_{a}^{'} \leftarrow $ \TransformTrajectory { $S^{'}$, $t_cw$ } \\% transform SLAM trajectory using tcw
\ForEach { $s_{a-i}^{'}$ in $S_{a}^{'}$, $g_{a-i}^{'}$ in $G_{a}^{'}$ }
{
    $r_{i} \leftarrow$ \RMSE { $s_{a-i}^{'}$, $g_{a-i}^{'}$ } \\
    \eIf { \IsExcessive {$r_{i}$} }
    {
        $I$.\Append {$g_{a-i}$}
    }
    {
        pass
    }
}
\end{algorithm}

\parab{Generalizing to other structures.}
Some aspects of model collection trajectory design depend on the geometry of the structure whose model we seek to reconstruct. To simplify the discussion and because geometric manipulations are not the focus of this paper, we have chosen rectangular buildings. We demonstrate in \secref{sec:eval} that our approach generalizes to other regular building geometries. It also generalizes to other structures that can be tightly bounded within rectangular solids, such as aircraft fuselages or blimps (\figref{fig:bounding_box_blimp}, \secref{sec:eval}). To accommodate arbitrary solid geometries, we expect that our techniques for generating equi-dense trajectories, in-flight re-calibration and our conclusions about orientation, height and speed will apply, but the actual trajectory design (\figref{fig:model_trajectory}) will need to match the shape of the solid. We leave these extensions to future work.

\subsection{Estimating Building Geometry}
\label{s:area-inter-reconn}
Given a region of interest, \textsc{ares}\xspace conducts a reconnaissance (``recon'') flight to determine the boundary of the building's roof. It uses this boundary for trajectory design (\secref{s:slam-phase}).

\parab{Goal and requirements.}
Accurate model reconstruction requires a low, slow, flight which can be battery-intensive. The recon flight helps \textsc{ares}\xspace scope its model collection to the part of the region of interest that contains the building to reduce battery usage. For instance, if in a large campus, buildings occupy only a small fraction of the surface area, a fast recon flight can reduce overall drone battery consumption. So, we require that the recon flight should have as short a duration as possible. In addition, (a) boundary estimation must not assume prior existence of a 3D model of the building (prior work in the area makes this assumption~\cite{lidarbuildingdetection, lidarimagebuildingdetection}); (b) boundary estimation must be robust to nearby objects like trees that can introduce error; and (c) buildings come in many shapes (\emph{e.g.,}\xspace rectangles, squares, hexagons \emph{etc.}\xspace), so boundary estimation must generalize to these.

\parab{The Recon Trajectory.}
Similar to model collection, recon uses a rectilinear scan (\figref{fig:mod_lateral}, \figref{fig:recon_traj}). Unlike model collection, however, during recon the drone flies \textit{fast} (4~m/s) and \textit{high} (60~m above the building's roof\footnote{We assume the nominal building heights in an area are known, for example, from zoning restrictions.}), with the LiDAR mounted in a \textit{perpendicular} orientation in order to have the shortest duration flight possible. We justify these parameter choices in \secref{sec:eval}, but \figref{fig:lidar_coverage_area} depicts the intuition for these choices. It shows, for an Ouster-64 LiDAR, the ground coverage area as a function of height. Coverage is highest between 40 and 60~m. At a given height, a perpendicular orientation covers a wider swathe of ground than a parallel orientation; this allows \textsc{ares}\xspace to use a larger scan width $s$ (\figref{fig:lidar_coverage_area}), resulting in a shorter flight. As with model collection, during this flight, \textsc{ares}\xspace streams point clouds to its cloud component, which runs the boundary detection algorithms described below.

\parab{Challenges and Overview.}
This flight design poses two challenges for boundary detection. First, to detect the building's boundary, it is still necessary to align all point clouds to the same coordinate frame of reference. In recon, \textsc{ares}\xspace cannot use SLAM because fast, high flights can cause SLAM to lose tracking frequently. We show below that, because boundary detection can afford to be approximate, we can use GPS. Second, high and fast flights result in low point density, and boundary detection algorithms must be robust to this.

\textsc{ares}\xspace's building boundary detection takes as input the area of interest and outputs the \textit{GPS locations} that constitute the boundary of the building. Model collection uses these outputs (\secref{s:slam-phase}). Boundary detection runs two different algorithms: rooftop \textit{surface extraction}, followed by \textit{boundary estimation}.

\parab{Step 1. Surface Extraction.} The cloud component receives GPS-tagged compressed point clouds from the drone. It first uncompresses them, then computes the \textit{surface normal} of every point in the point cloud. A surface normal for a point determines the direction normal to the surface formed by points within a fixed radius of the point. Then, \textsc{ares}\xspace uses RANSAC~\cite{ransac} (a plane-fitting algorithm) to segment the LiDAR points into groups of points that fall onto planes. RANSAC is fast, but is not robust to outliers: (a) it combines into one surface all LiDAR points that satisfy the same planar equation, including disjoint sets of points (\emph{e.g.,}\xspace from trees) at the same height; (b) point clouds can have multiple detected planes (\emph{e.g.,}\xspace building rooftop, ground surface, vehicles \emph{etc.}\xspace), and RANSAC cannot distinguish between these.

To address the first issue, in each plane, \textsc{ares}\xspace removes outlying points that are further away from neighboring points in the same plane using a statistical outlier filter. Using the statistical outlier on every point cloud can be compute-intensive. To this end, \textsc{ares}\xspace tunes the statistical outlier's parameters to find a sweet spot between filtering accuracy and performance. To find the rooftop among the multiple detected planes, \textsc{ares}\xspace uses surface normals to compute surface statistics for each plane (\emph{e.g.,}\xspace plane height, 3D centroid, normal variance \emph{etc.}\xspace). It uses these statistics to find the rooftop in the extracted planes. (As an aside, surface normal computation is computationally intensive but is parallelizable, so we use a GPU to accelerate this, as discussed in \secref{sec:eval}). Intuitively, the rooftop is a large, uniformly oriented surface (surface normal variance is low) that lies above the ground plane. \textsc{ares}\xspace can eliminate the ground plane as that plane whose points are consistent with the drone's height. So, it discards all planes that do not satisfy this definition (this includes planes with high variances and the ground surface). At the end of this step, \textsc{ares}\xspace classifies a single plane as the roof surface.

To remove the possibility of false positives, \textsc{ares}\xspace uses majority voting to remove erroneous surface detections; it classifies a plane as the rooftop only if it detects it in multiple consecutive frames. Lastly, even though the outlier filter removes small sets of outliers in planes, it is unable to remove large clusters of points belonging to objects like trees found nearby the building. For this, \textsc{ares}\xspace forms clusters of points based on their spatial relationships such that neighboring points belong to the same cluster. This way, points belonging to different objects form clusters. Since the roof is normally a relatively larger surface, \textsc{ares}\xspace simply discards smaller clusters. To do this in near real-time, \textsc{ares}\xspace finds the right set of parameters for the clustering algorithm.

\parab{Step 2. Estimating the boundary of the building.}
In the previous step, \textsc{ares}\xspace obtains parts of the rooftop from each point cloud. In this step, it uses the drone's GPS location to transform each surface to the same coordinate frame of reference, then combines all surfaces into a single point cloud that represents the extracted rooftop of the building.
To extract the boundary of the building, it extracts the \textit{alpha shape}~\cite{alphashape} of the stitched point cloud. A generalization of a convex hull, an alpha shape is a sequence of piecewise linear curves in 2-D encompassing the point cloud representing the rooftop. This allows \textsc{ares}\xspace to generalize to non-convex shapes as well. Finally, to detect the boundary of multiple buildings, \textsc{ares}\xspace clusters the rooftop point clouds.

\figref{fig:boundary} shows results from the building boundary detection algorithm on real data taken from our drone. The green rectangle is the ground truth boundary of the building. The blue points illustrate the drone's recon trajectory and the grey points depict locations where \textsc{ares}\xspace detects a rooftop. Some grey points are beyond the building's boundary because the LiDAR has a wide field of view and can see the rooftop even after it has passed it. The red points show the GPS stitched 3D point cloud of the building's rooftop.

\subsection{Point-Cloud Compression}
\label{s:compression}

LiDARs generate voluminous 3D data. For instance, the Ouster OS1-64 LiDAR (which we use in this paper), generates 20 point clouds per second that add upto 480~Mbps, well beyond the capabilities of even future cellular standards. \textsc{ares}\xspace compresses these point clouds to a few Mbps (1.2 to 4.0), using two techniques: viewpoint filtering, and octree compression. We describe these in \secref{s:app_compression}.

\section{Evaluation}
\label{sec:eval}

We evaluate (a) \textsc{ares}\xspace's ability to reconstruct 3D models in near real-time, and (b) the accuracy of these 3D models (~\secref{s:slam-phase}). We also describe our parameter sensitivity analyses for data collection, and evaluate boundary detection performance.

\parab{Implementation.} Not counting external libraries and packages, \textsc{ares}\xspace is 15,500 lines of code (discussion in \secref{s:app_impl}).

\parab{Simulations.} We evaluate \textsc{ares}\xspace using a photorealistic simulator, AirSim~\cite{airsim}, that models realistic physical environments using a game engine, then simulates drone flights over these environments and records sensor readings taken from the perspective of the drone. AirSim is widely accepted as a leading simulation platform for autonomous vehicles and drones by manufacturers, academia and leading industrial players. AirSim has a parametrizable model for a LiDAR; we used the parameters for the Ouster OS1-64 in our simulation experiments. \textsc{ares}\xspace generates trajectories for the AirSim drone, then records the data generated by the LiDAR, and processes it to obtain the 3D model. For computing the metrics above, we obtain ground truth from AirSim. To build the ground truth 3D model, we flew a drone equipped with a LiDAR several times over the region of interest in AirSim (using exhaustive flights) and then stitched all the resulting point clouds using ground truth positioning information from AirSim.

\parab{Real-world Traces.}
In addition, we have collected data from nearly 30 flights (each of about 25 minutes) on an M600Pro drone with an Ouster OS1-64 LiDAR on a commercial complex. For almost all experiments, we evaluated \textsc{ares}\xspace on both \textit{real-world} and simulation-driven traces. Simulation-driven traces give us the flexibility to explore the parameter space more (as we show below). However, we use \textit{real-world} traces to validate all these parameter choices and estimate reconstruction accuracy in practice.

\parab{Metrics.} In this section, we quantify end-to-end latency, 3D model accuracy and completeness (\secref{s:slam-phase}), and positioning error for a number of experimental scenarios. We also quantify \textsc{ares}\xspace's energy-efficiency (using flight duration as a proxy for drone battery usage) and the computational capabilities of its processing pipeline.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Reconstruction \\ scheme\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Comp.\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Reconstruction \\ time (s)\end{tabular} \\ \hline

\multicolumn{4}{|c|}{ Large building (100~m~x~50~m~x~20~m)} \\ \hline
\begin{tabular}[c]{@{}c@{}}Offline-SDF\end{tabular}    & $\infty$      & $\infty$      & 3500 \\ \hline
\begin{tabular}[c]{@{}c@{}}Offline-TP\end{tabular}    & 0.87          & 0.35          & 3900 \\ \hline
\begin{tabular}[c]{@{}c@{}}\textsc{ares}\xspace\end{tabular}        & \textbf{0.21} & \textbf{0.24} & \textbf{719} \\ \hline
\begin{tabular}[c]{@{}c@{}}\textsc{ares}\xspace-raw \end{tabular}        & 0.14 & 0.17 & 719 \\ \hline

\multicolumn{4}{|c|}{ Small building (50~m~x~50~m~x~20~m)} \\ \hline
\begin{tabular}[c]{@{}c@{}}Offline-SDF\end{tabular}    & 3.36          & 1.30          & 2400  \\ \hline
\begin{tabular}[c]{@{}c@{}}Offline-TP\end{tabular} & 0.62          & 0.43          & 3300 \\ \hline
\begin{tabular}[c]{@{}c@{}}\textsc{ares}\xspace\end{tabular}        & \textbf{0.25} & \textbf{0.14} & \textbf{656}\\ \hline
\begin{tabular}[c]{@{}c@{}}\textsc{ares}\xspace-raw \end{tabular}        & 0.21 & 0.09 & 656\\ \hline

\end{tabular}
\caption{Reconstruction accuracy, completeness (comp.), and time for two large buildings using three schemes: a) offline reconstruction with shortest duration flight (Offline-SDF), b) offline reconstruction with \textsc{ares}\xspace trajectory planning (Offline-TP), c) \textsc{ares}\xspace, and d) \textsc{ares}\xspace with raw traces.}
\label{tab:e2e_reconstruction_quality_airsim}
\end{table}

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Reconstruction \\ scheme\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Bandwidth \\ (Mbps)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Comp.\\ (m)\end{tabular} \\ \hline
\begin{tabular}[c]{@{}c@{}}Offline SLAM\end{tabular}         & 480 & 2.30 & 1.30 \\ \hline
\begin{tabular}[c]{@{}c@{}}Online w GPS.\end{tabular}        & 3.80 & 1.60 & 0.53 \\ \hline
\begin{tabular}[c]{@{}c@{}}\textsc{ares}\xspace\end{tabular}             & \textbf{3.80} & \textbf{0.13} & \textbf{0.09} \\ \hline
\end{tabular}
\caption{Reconstruction quality of a \textit{real-world} 70~x~40~x~20~m building with online and offline approaches, relative to an uncompressed trace.}
\label{tab:e2e_reconstruction_quality_real}
\end{table}

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Structure\\ type\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Flight \\ duration (s)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Comp.\\ (m)\end{tabular} \\ \hline
Blimp       & 586 & 0.23 & 0.03 \\ \hline
Small rect. & 656 & 0.25 & 0.14 \\ \hline
Star-shaped & 686 & 0.37 & 0.12 \\ \hline
Large rect. & 719 & 0.21 & 0.24 \\ \hline
Plus-shaped & 1024 & 0.31 & 0.06 \\ \hline
H-shaped    & 1044 & 0.34 & 0.10 \\ \hline
Pentagonal  & 1361 & 0.31 & 0.12 \\ \hline
\end{tabular}
\caption{\textsc{ares}\xspace 3D reconstruction times (recon and model collection) and quality for different structures at low compression}
\label{tab:e2e_reconstruction_vs_building_types}
\end{table}

\subsection{3D Model Reconstruction}
\label{sec:eval:accuracy}

\parab{Experiment setup.}
To evaluate the end-to-end performance of \textsc{ares}\xspace in building an accurate 3D model in near real-time, we collected and reconstructed the 3D model of two buildings: a) a \textit{large} 50m~x~100m~x~20m (L~x~W~x~H) and, b) a \textit{small} 50m~x~50m~x~20m building in Airsim. We then compared the reconstruction accuracy of these models with two baseline \textit{offline} approaches (\emph{i.e.,}\xspace approaches that reconstruct the 3D model after the drone lands): a) offline reconstruction\footnote{We assume both offline approaches know the exact location of the building. Without this, SLAM accumulates significant drift and reconstructions are very poor.} with the shortest duration flight (Offline-SDF), b) offline reconstruction with \textsc{ares}\xspace's trajectory planning (Offline-TP). We calculated the accuracy and completeness of the models generated by these approaches by comparing them against ground truth models generated from AirSim. Lower is better for accuracy and completeness. For these experiments, \textsc{ares}\xspace uses \textit{compressed point clouds} with bandwidth requirements that are compatible with LTE speeds today (\emph{i.e.,}\xspace upload bandwidth of 3.8~Mbps). \textsc{ares}\xspace-raw shows accuracy and completeness if \textsc{ares}\xspace were to use raw point clouds; we study the effect of compression on \textsc{ares}\xspace model reconstruction more in \secref{s:ablation-study}.

\parab{\textsc{ares}\xspace builds significantly accurate models in less time.}
As \tabref{tab:e2e_reconstruction_quality_airsim} shows, \textsc{ares}\xspace achieves lower than 25~cm accuracy and completeness for both buildings and reconstructs the entire buildings in just 10-12 minutes (the flight duration). For reconstruction quality, \textsc{ares}\xspace does much better than the two baseline approaches for two reasons: a) careful trajectory planning (TP), and b) in-flight re-calibration. Since Offline-SDF does neither, its accuracy and completeness values are very large. To reconstruct the larger building \emph{i.e.,}\xspace 100~x~50~x~20m building, the drone needs to fly more and accumulates significant drift (as compared to the smaller building) and has poor accuracy and completeness (shown by $\infty$). Offline-TP does better because it uses \textsc{ares}\xspace's trajectory planning, but still exhibits worse accuracy and completeness than \textsc{ares}\xspace because it lacks in-flight calibration. This shows the importance of a real-time quality feedback signal for reconstruction and highlights why offline reconstruction is not accurate even with uncompressed traces. Though \textsc{ares}\xspace uses compressed point clouds,
with in-flight re-calibration and trajectory planning, \textsc{ares}\xspace's models are upto 3.5x more accurate and complete. If \textsc{ares}\xspace were to use raw traces (\textsc{ares}\xspace-raw) instead, loss of accuracy and completeness is attributable to SLAM. Relative to a raw trace, compression accounts for 4-7~cm difference in accuracy and completeness. Moreover, \textsc{ares}\xspace reconstructs while the drone is in-flight whereas the other two baseline approaches do reconstruction offline on uncompressed point clouds, incurring up to 4.6$\times$ higher reconstruction time\footnote{In practice, offline reconstruction will have higher reconstruction times because we did not consider time to upload data to the cloud.}.

To get a visual feel for the degradation resulting from lower accuracy and completeness, consider \figref{fig:3d_model}, which shows the ground-truth model, together with the \textsc{ares}\xspace reconstructions. With an accuracy of 0.25~m (using 3.8~Mbps upload bandwidth), the model closely matches the ground truth, but the textured building surface on the right shows some small artifacts. These artifacts arise not because of compression, but because of SLAM imperfections (\secref{s:ablation-study}).

\parab{\textsc{ares}\xspace generalizes to different building shapes.}
Our results so far, and the descriptions in \secref{sec:design}, have focused on rectangular buildings. \textsc{ares}\xspace can accurately reconstruct a variety of building types, as \tabref{tab:e2e_reconstruction_vs_building_types} shows. For these results, we use \textsc{ares}\xspace's default flight parameters and low compression.
Larger buildings (pentagonal, plus, and H-shaped) have larger flight durations partly because of their size and because they require two re-calibration steps. Even then, for all buildings, \textsc{ares}\xspace achieves tens of centimeter accuracy and completeness.

\parab{\textsc{ares}\xspace generalizes to other types of structures.}
To show that \textsc{ares}\xspace can reconstruct other types of 3D structures \emph{e.g.,}\xspace airplanes, helicopters \emph{etc.}\xspace, we modeled a real-world blimp~\cite{blimp} (15~m x 60~m x 15~m) in AirSim. \textsc{ares}\xspace encloses such structures within an enclosing rectangular solid
(\figref{fig:bounding_box_blimp}, \secref{s:data-collection}). In less than 10 minutes (\tabref{tab:e2e_reconstruction_vs_building_types}), \textsc{ares}\xspace reconstructed the blimp with an accuracy of 23~cm and completeness of just 3~cm.

\parab{High accuracy is possible on real-world traces.}
Results from our drone flights validate that real-world data can result in
comparable performance  (\tabref{tab:e2e_reconstruction_quality_real}). In these experiments, we reconstructed the 3D model of a real-world 70~m~x~40~m~x~20~m building. Because we lack a reference ground truth for real-world data, we use the 3D model generated from raw, uncompressed traces. Offline reconstruction using SLAM after the drone lands fails completely for the same reasons mentioned above (\emph{i.e.,}\xspace no trajectory planning, and no re-calibration). With GPS, it is possible to do in-flight reconstruction, however, the accuracy and completeness being 1.60~m and 0.53~m, make such 3D models unusable. With \textsc{ares}\xspace, on the other hand, we can build accurate, and complete 3D models whose completeness and accuracy are 9 and 13~cm respectively (top-down view of 3D model in \figref{fig:3d_real_model}).

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline

\begin{tabular}[c]{@{}c@{}}Compression scheme\end{tabular}
& \begin{tabular}[c]{@{}c@{}}Low \end{tabular}
& \begin{tabular}[c]{@{}c@{}}Medium \end{tabular}
& \begin{tabular}[c]{@{}c@{}}High \end{tabular} \\ \hline \hline

\begin{tabular}[c]{@{}c@{}}Upload bandwidth (Mbps)\end{tabular} & 3.80 & 2.50 & 1.27 \\ \hline
\begin{tabular}[c]{@{}c@{}}Compression time (ms)\end{tabular} & 62.9 & 65.3 & 65.3 \\ \hline
\begin{tabular}[c]{@{}c@{}}Network latency  (ms)\end{tabular} & 15.5 & 14.6 & 13.7 \\ \hline
\begin{tabular}[c]{@{}c@{}}Extraction time  (ms)\end{tabular} & 5.05 & 3.07 & 3.03 \\ \hline
\begin{tabular}[c]{@{}c@{}}SLAM time (ms)\end{tabular} & 34.5 & 30.8 & 23.9 \\ \hline
\begin{tabular}[c]{@{}c@{}}Total processing time (ms)\end{tabular} & 117 & 113 & 106 \\ \hline

\end{tabular}
\caption{\textsc{ares}\xspace enables real-time 3D reconstruction over LTE. Each row shows \textit{per frame} latency for that operation.}
\label{tab:real_time_experiments}
\end{table}

\subsection{Performance}
\label{s:other-meas-sysn}

\parab{Real-time 3D reconstruction over LTE is feasible.} To validate that \textsc{ares}\xspace can collect a 3D model end-to-end in near real-time, we used our implementation to conduct an experiment in which we replayed 15 minutes worth of \textit{real-world data} on the drone compute (a Jetson TX2). It then compressed and streamed point clouds over an LTE connection, to a 16-core AWS VM with 64~GB RAM and a Nvidia T4 GPU. (Our experiment only ran  model collection; recon also runs in real-time as discussed below).

To compress point clouds, we used three different levels of compression (\textit{low}, \textit{medium} and \textit{high}), corresponding to the following combinations of octree resolution and point resolution (\secref{s:compression}): $(0.25, 0.10)$, $(0.25, 0.25)$ and $(0.50, 0.50)$ (effect of compression is studied in \secref{s:ablation-study}). In our experiments with our drone, we have found achievable LTE throughput to range from 1-4~Mbps; we chose these compression levels to correspond to this range. (In \secref{s:ablation-study}, we discuss how 5G deployments would alter these conclusions).

At all three compression modes, \textsc{ares}\xspace was able to stream point clouds in real time (\tabref{tab:real_time_experiments}), and the total end-to-end processing time \textit{per frame} is about 110~ms, of which nearly 65~ms is network latency. Thus, \textsc{ares}\xspace builds the 3D model whilst the drone is in-flight, adds a frame within 100~ms after receiving it and can make available \textit{a complete 3D model of a building in about a 100~ms} after receiving the last frame!

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|}
\hline
\textsc{ares}\xspace component &
Sub-component &
{\begin{tabular}[c]{@{}c@{}}Per-frame\\execution time\\ (ms)\end{tabular}} \\ \hline
\multirow{8}{*}{\begin{tabular}[c]{@{}c@{}}Recon phase\end{tabular}}
& 3D frame compression & 13.0 $\pm 1.3$ \\ \cline {2-3}
& 3D frame extraction & 3.0 $\pm 0.3$ \\ \cline {2-3}
& GPU normal estimation & 76.0 $\pm 82$ \\ \cline {2-3}
& RANSAC plane-fitting & 5.0 $\pm 9.0$ \\ \cline {2-3}
& Outlier removal & 0.2 $\pm 0.3$ \\ \cline {2-3}
& Rooftop detection & 0.005 \\ \cline {2-3}
& Rooftop extraction & 6.0 $\pm 5.0$ \\ \cline {2-3}
& Rooftop stitching & 3.0 $\pm 2.0$ \\ \cline {2-3}
& Total time & 100 $\pm 90.0$ \\ \hline
Model collection & LiDAR SLAM & 37.0 $\pm 3.0$ \\ \hline
\multicolumn{2}{|c|}{3D Reconstruction} & 10.3 $\pm 2.0$ \\ \hline
\end{tabular}
\caption{Processing times for \textsc{ares}\xspace components.}
\label{tab:system_execution_time_eval}
\end{table}

\parab{\textsc{ares}\xspace supports full frame rate processing.} We profiled the execution time of each component of \textsc{ares}\xspace on a 15-minute \textit{real-world} trace.  Point cloud compression executes on the  drone, and other components run on the AWS VM mentioned above.
We use the GPU to offload the computation of surface normals for building detection. During recon, point cloud compression takes 13~ms per frame (\tabref{tab:system_execution_time_eval}). Extracting the building geometry requires 100~ms per frame; with these numbers, we can sustain about 10~fps, so with a 20~fps LiDAR, we process roughly every other frame. Despite this, our building detector is quite accurate (\secref{sec:eval:building}). During model collection, SLAM requires 37~ms per frame, and 3D reconstruction requires about 10~ms (\tabref{tab:system_execution_time_eval}). The former uses 8 cores, so we have been able to run these two components in a pipeline to sustain 20~fps. Thus, a moderately provisioned, cloud VM suffices to run \textsc{ares}\xspace at full frame rate with an end-to-end compute latency of about 100~ms for reconnaissance, and 50~ms for model collection.

\subsection{Ablation Studies}
\label{s:ablation-study}
In this section, we explore how \textsc{ares}\xspace's techniques contribute to 3D reconstruction quality and performance.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|c|c|c|}
\hline
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Structure \\type\end{tabular}} &
\multicolumn{2}{c|}{Flight dur. (s)} &
\multicolumn{2}{c|}{Accuracy (m)} &
\multicolumn{2}{c|}{Comp. (m)} \\
\cline{2-7} &
\begin{tabular}[c]{@{}c@{}}w/o\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w/o\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w/o\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w\end{tabular} \\ \hline
Star-shaped & 613 & 686 & 1.05 & \textbf{0.37} & 0.39 & \textbf{0.12} \\ \hline
Small rect. & 573 & 656 & 0.63 & \textbf{0.25} & 0.40 & \textbf{0.14} \\ \hline
Large rect. & 694 & 719 & 0.96 & \textbf{0.21} & 0.39 & \textbf{0.24}\\ \hline
Plus-shaped & 766 & 1024 & 0.51 & \textbf{0.31} & 0.08 & \textbf{0.06} \\ \hline
H-shaped    & 866 & 1044 & 1.10 & \textbf{0.34} & 0.27 & \textbf{0.10} \\ \hline
Pentagonal  & 1062 & 1361 & 1.47 & \textbf{0.31} & 0.42 & \textbf{0.12} \\ \hline
\end{tabular}
\caption{Flight duration (dur.) and reconstruction quality for buildings at low compression with (w) and without (w/o) re-calibration.}
\label{tab:reconstruction_with_recalibration}
\end{table}

\begin{table}[t]
\centering
\footnotesize
\begin{tabular}{|c|c|c|c|c|c|}
\hline
\multicolumn{2}{|c|}{Velocity (m/s)}    & 0.5  & 1.0 & 2.0 & \begin{tabular}[c]{@{}c@{}}Exhaustive \\ @~1~m/s\end{tabular} \\ \hline
\multicolumn{2}{|c|}{Height (m)}        & 30   & 40  & 50  & 40   \\ \hline
\multirow{4}{*}{\begin{tabular}[c]{@{}c@{}}Flight\\ duration\\ (s)\end{tabular}}
        & Recon                   & 136  & 136 & 136 & -                      \\ \cline{2-6}
        & Model coll.        & 673  & 343 & 72  & 1520 \\ \cline{2-6}
        & Re-calib.          & 476  & 240 & 272 & -    \\ \cline{2-6}
        & Total time       & 1285 & 719 & 480 & 1520 \\ \hline
\multicolumn{2}{|c|}{Accuracy (m)}            & 0.43   & 0.14  & 0.91  & $\infty$ \\ \hline
\multicolumn{2}{|c|}{Completeness (m)}        & 0.34   & 0.17  & 0.45  & $\infty$   \\ \hline
\end{tabular}
\caption{\textsc{ares}\xspace's flight duration for various parameter choices.}
\label{tab:battery_usage}
\end{table}

\parab{Re-calibration helps reduce error.}
To show the effect of in-flight re-calibration, we built online 3D models of the 7 large buildings mentioned above using \textsc{ares}\xspace with (w) and without (w/o) re-calibration in Airsim. In these experiments, we evaluate flight duration and reconstruction quality at low compression (3.8~Mbps upload bandwidth) using accuracy and completeness metrics. \tabref{tab:reconstruction_with_recalibration} shows that, on average, at the expense of only 18\% (150~seconds) longer flights, \textsc{ares}\xspace improves accuracy by 65\% (65~cm) and completeness by 55\% (20~cm) with re-calibration flights. Larger buildings (plus-shaped, H-shaped, and pentagonal) require longer aerial flights which accumulate higher drift. This results in relatively more re-calibration flights and hence higher flight duration. Even so, \textsc{ares}\xspace is able to reconstruct these buildings accurately, demonstrating the importance of re-calibration.

\parab{Short flight durations can produce accurate models.} \textsc{ares}\xspace strives to reduce drone battery depletion in its design by generating short duration flights without sacrificing accuracy and completeness.
To show that \textsc{ares}\xspace's defaults of 1~m/s speed and 40~m height represent the best point in this tradeoff space, we compare it to a lower, slower flight (30~m, 0.5~m/s), and a faster, higher flight (50~m, 2~m/s).
\tabref{tab:battery_usage} shows that, on the large building the lower, slower flight has a longer trajectory, resulting in more re-calibrations. The resulting model has worse accuracy and completeness; re-calibration can limit drift error, but not reverse it. A faster, higher flight has a slightly shorter trajectory, but the resulting model's accuracy is very poor, because there is less overlap between point clouds at higher speeds (\secref{s:slam-phase}). Finally, \tabref{tab:battery_usage} also shows the benefits of a recon flight: an exhaustive flight that uses the model collection parameters and does not perform recon is 3$\times$ longer than \textsc{ares}\xspace's flight (and accumulates significant drift, resulting in poor quality 3D models). Results on the small building are qualitatively similar (omitted for brevity).

\parab{\textsc{ares}\xspace builds accurate models at low bandwidths.}
We explore the impact of compression on accuracy and completeness using (a) a synthetic building in AirSim and (b) \textit{real-world} traces. In addition to the three compression schemes discussed earlier, we compute accuracy and completeness for (a) raw point clouds, (b) viewpoint compression and (c) lossless compression. The first two alternatives provide calibration, while the third alternative explores reconstruction performance under higher bandwidth as would be available, for example, in 5G deployments.

As \tabref{tab:reconstruction_vs_compression} shows, viewpoint filtering achieves a 10$\times$ compression throughout. Low compression is an order of magnitude more efficient beyond this. Despite this, \textsc{ares}\xspace can achieve high quality reconstruction. For the AirSim building, consider accuracy: the raw-point cloud has an accuracy of 0.21~m and 0.09~m, which is attributable entirely to SLAM error. View-point filtering does not degrade accuracy since it only omits zero returns. Low compression, with a bandwidth of 3.8~Mbps (easily achievable over LTE and over 100$\times$ more compact than the raw LiDAR output) only adds 4~cm and 5~cm to accuracy and completeness (respectively). Medium and high compression have significantly poorer accuracy and completeness. Similar results hold true for the other AirSim building, so we omit for brevity.

Results from our drone flights validate that \textit{real-world} data of a large building (dimensions in \tabref{tab:reconstruction_vs_compression}) can result in comparable performance (\tabref{tab:reconstruction_vs_compression}). Since we lack a reference ground truth for real-world data, we use the 3D model generated from raw traces. With real-world traces, we can build accurate, and complete 3D models that are within 9-13~cm completeness and accuracy for low compression, and about 16-23~cm for medium compression, with respect to the uncompressed traces. This suggests that highly compressed point clouds do not significantly impact accuracy and completeness.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Compression\\ profile\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Bandwidth\\ (Mbps)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Completeness\\ (m)\end{tabular} \\ \hline
\multicolumn{4}{|c|}{\textit{Real-world} 70~m~x~40~m~x~20~m large building} \\ \hline
\begin{tabular}[c]{@{}c@{}}Raw\end{tabular}         & 480.0 & 0.00 & 0.00 \\ \hline
\begin{tabular}[c]{@{}c@{}}View-point\end{tabular}  & 42.7  & 0.00 & 0.00 \\ \hline
\begin{tabular}[c]{@{}c@{}}Lossless\end{tabular}    & 7.86  & 0.06 & 0.07 \\ \hline
\begin{tabular}[c]{@{}c@{}}Low\end{tabular}         & 3.80  & 0.13 & 0.09 \\ \hline
\begin{tabular}[c]{@{}c@{}}Medium\end{tabular}      & 2.50  & 0.23 & 0.16 \\ \hline
\begin{tabular}[c]{@{}c@{}}High\end{tabular}        & 1.27  & 0.28 & 0.29 \\ \hline

 \multicolumn{4}{|c|}{\textit{AirSim} 50~m~x~50~m~x~20~m small building} \\ \hline
\begin{tabular}[c]{@{}c@{}}Raw\end{tabular}         & 480.0 & 0.21 & 0.09 \\ \hline
\begin{tabular}[c]{@{}c@{}}View-point\end{tabular}  & 42.7  & 0.21 & 0.09 \\ \hline
\begin{tabular}[c]{@{}c@{}}Lossless\end{tabular}    & 7.86  & 0.22 & 0.10 \\ \hline
\begin{tabular}[c]{@{}c@{}}Low\end{tabular}         & 3.80  & 0.25 & 0.14 \\ \hline
\begin{tabular}[c]{@{}c@{}}Medium\end{tabular}      & 2.50  & 0.66 & 0.21 \\ \hline
\begin{tabular}[c]{@{}c@{}}High\end{tabular}        & 1.27  & 0.73 & 0.24 \\ \hline
\end{tabular}
\caption{The impact of compression on accuracy/completeness.}
\label{tab:reconstruction_vs_compression}
\end{table}

\parab{Higher bandwidths provide centimeter-level improvements.}
The emergence of 5G promises larger upload bandwidths. However, as \tabref{tab:reconstruction_vs_compression} illustrates, room for improvement in accuracy and completeness is small. For the AirSim building, the gap between raw point clouds and low compression accuracy (completeness) is only 4 cm (5cm); for the \textit{real-world} building, it is 7 cm (2cm). \textit{Lossless} point cloud compression, which requires 7.86~Mbps bandwidth comes within 1~cm of the raw point cloud accuracy and completeness for the AirSim building and within 7~cm for the real-world building.

\parab{Lower target density worsens completeness.} To demonstrate that users can use the target density tuning knob to obtain less complete models more quickly, we conducted an experiment with \textsc{ares}\xspace (with re-calibration) at two different densities: 7.5 points per m$^2$ and 1 point per m$^2$. For the former, accuracy and completeness were 0.21~m and 0.14~m, and for the latter 0.68~m, 0.17~m respectively. The lower density flight took 20\% less time. As expected, completeness is worse at lower target densities. At the lower density, accuracy is worse because two adjacent scan lines have smaller overlap. Put another way, a side benefit of specifying higher density is the higher accuracy from scan line overlap.

\subsection{Data Collection}
\label{s:data-collection}

\textsc{ares}\xspace relies on a careful parameter sensitivity analysis (in both simulation and on \textit{real-world traces}) to determine model collection flight parameters: speed, height, and orientation (\secref{s:slam-phase}). We have evaluated SLAM error for every combination of drone speed (ranging from 0.5~m/s to 3~m/s), distance from building (10~m to 40~m) and orientation (parallel to perpendicular). We present a subset of these results for space reasons. For these experiments, we use the trajectory described in \figref{fig:mod_total}. We report the average numbers for each experiment.

\begin{figure}[t]
\begin{minipage}[b]{0.45\columnwidth}
    \centering
 \includegraphics[width=0.99\columnwidth]{figs/fig_parallel_vs_other_orientations.pdf}
    \caption{SLAM errors for LiDAR orientations.}
    \label{fig:parallel_vs_other_orientations}
\end{minipage}
\hspace{0.5cm}
\begin{minipage}[b]{0.45\columnwidth}
\centering
 \includegraphics[width=0.99\columnwidth]{figs/fig_parallel_slam_param_study.pdf}
    \caption{SLAM error for different speeds and building distances.}
    \label{fig:parallel_slam_vs_height_speed}
\end{minipage}
\end{figure}

\parab{Best choice of orientation is parallel.}
\figref{fig:parallel_vs_other_orientations} plots SLAM error as a function of LiDAR orientation (\figref{fig:lidar_orientation}) with respect to the direction of motion. A parallel orientation has lowest SLAM error (in \figref{fig:parallel_vs_other_orientations}, yaw 0$\degree$ corresponds to parallel and yaw 90$\degree$ to perpendicular), because it has highest overlap between successive frames; as yaw increases, overlap decreases, resulting in higher SLAM error (\secref{s:slam-phase}).

\parab{Best choice of distance is 20~m.}
\figref{fig:parallel_slam_vs_height_speed} plots the SLAM error as a function of the drone's distance from the building surface for the \textit{parallel} orientation of the LiDAR. Error increases slowly with height; beyond a 20~m distance from the building, the error is more than 1~m. Point densities decrease with height and affect SLAM's ability to track features/points across frames (\secref{s:slam-phase}). Rather than fly lower, \textsc{ares}\xspace operates at a 20~m distance (or a 40~m height, since in our experiments buildings are 20~m tall) to reduce flight duration.

\parab{Best choice of speed is 1~m/s.}
Speed impacts SLAM positioning error significantly (\figref{fig:parallel_slam_vs_height_speed}). Beyond 1~m/s, SLAM cannot track frames accurately because of lower overlap between frames (\secref{s:slam-phase}). Below 1~m/s \emph{i.e.,}\xspace at 0.5~m/s, the flight duration (in seconds) is twice that of 1~m/s which results in drift error accumulation. To achieve accurate reconstruction, \textsc{ares}\xspace chooses to fly the drone at 1~m/s.

\parab{Real-world traces confirm these observations.}
\textit{Real-world traces} (\secref{s:app_data_coll}) validate these parameter choices (~\tabref{tab:real_speed_vs_rmse}, ~\tabref{tab:real_height_vs_rmse}) \emph{i.e.,}\xspace fly slow, close to the building and in parallel.

\subsection{Boundary Detection}
\label{sec:eval:building}

\parab{Methodology and metrics.}
We use two metrics for building boundary estimation: accuracy, and completeness. Accuracy is the average (2-D) distance between each point (quantized to 0.1~m) on the predicted boundary and the nearest point on the actual building boundary. Completeness, is the average distance between each point on the actual boundary and the nearest point on \textsc{ares}\xspace's predicted boundary. Lower values of accuracy and completeness are better.

We use both real-world traces collected from our \textsc{ares}\xspace prototype and synthetic traces from AirSim. To compute ground truth for real-world traces, we pin-pointed the building's boundary on Google Maps~\cite{google_maps}. For AirSim, we collected the ground truth from the Unreal engine.

\parab{Boundary detection can run at full frame rate.}
\tabref{tab:system_execution_time_eval} shows the time taken for each component of boundary detection, on our \textit{real-world} traces on a single core of a GPU-equipped desktop. The average processing time per point cloud is 100~ms, dominated by GPU-accelerated surface normal estimation (76~ms). This can sustain 10~fps. However, our LiDAR generates 20~fps, so \textsc{ares}\xspace uses every other frame, without sacrificing accuracy.

\parab{Boundary detection is accurate.}
To evaluate the accuracy of \textsc{ares}\xspace's boundary extraction, we experimented with 3 \textit{real-world traces} collected over a 70~m x 60~m x 20~m building. For these traces, \textsc{ares}\xspace's average accuracy is 1.42~m and its completeness is 1.25~m, even at the highest compression and when it samples every other frames.

\parab{Other results.}
We extensively evaluated \textsc{ares}\xspace's boundary detection algorithm's robustness to different building shapes (\tabref{tab:recon_buildings}), point cloud compression (\tabref{tab:recon_compression}), and point cloud sub-sampling. Furthermore, we performed an extensive parameter study to find the right flight parameters \emph{i.e.,}\xspace speed (\figref{fig:recon_speed}), height (\figref{fig:recon_height}) and orientation. For brevity, we have included results and discussions in the appendix (\secref{s:app_recon_flight}). We summarize two results. First, recon flights can be short (boundary detection is insensitive to point density and overlap). So, it can use perpendicular orientation, fly at 60~m from the building at 4~m/s. Second, it tolerates sub-sampling upto one point cloud per second.

\section{Evaluation}
\label{sec:eval}

We evaluate (a) \textsc{ares}\xspace's ability to reconstruct 3D models in near real-time, and (b) the accuracy of these 3D models. We also describe our parameter sensitivity analyses for data collection, and evaluate boundary detection performance.

\parab{Implementation.}
Our evaluations use a complete implementation of \textsc{ares}\xspace, which uses the Point Cloud Library (PCL~\cite{octree}), the Cartographer~\cite{Cartographer} LiDAR SLAM implementation, the Boost C++ libraries~\cite{Boost}, and the Robotic Operating System (ROS~\cite{ros}). For the recon phase, we used functions from the Point Cloud Library (PCL~\cite{octree}) for plane-fitting, outlier removal and clustering. Our compression and extraction modules also use PCL and are implemented as ROS nodes. The drift detection module uses a Python package for the Umeyama alignment~\cite{grupp2017evo}. Not counting libraries and packages it uses, \textsc{ares}\xspace is 15,500 lines of code.

\parab{Simulations.} We evaluate \textsc{ares}\xspace using a photorealistic simulator, AirSim~\cite{airsim}, that models realistic physical environments using a game engine, then simulates drone flights over these environments and records sensor readings taken from the perspective of the drone. AirSim has a parametrizable model for a LiDAR; we used the parameters for the Ouster OS1-64 in our simulation experiments. \textsc{ares}\xspace generates trajectories for the AirSim drone, then records the data generated by the LiDAR, and processes it to obtain the 3D model. For computing the metrics above, we obtain ground truth from AirSim. To build the ground truth 3D model, we flew a drone equipped with a LiDAR several times over the region of interest in AirSim (using a slow and low flight) and then stitched all the resulting point clouds using ground truth positioning information from AirSim.

\parab{Real-world Traces.}
In addition, we have collected data from nearly 30 flights (each of about 25 minutes) on an M600Pro drone with an Ouster OS1-64 LiDAR on a commercial complex. We use this data to validate parameter choices and estimate reconstruction accuracy in practice.

\parab{Metrics.} In this section, we quantify end-to-end latency, 3D model accuracy and completeness (\secref{s:slam-phase}), and positioning error for a variety of experimental scenarios. We also quantify \textsc{ares}\xspace's energy-efficiency (using flight duration as a proxy for drone battery usage) and the computational capabilities of its processing pipeline.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|c|c|}
\hline
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Compression\\ profile\end{tabular}} & \multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Bandwidth\\ (Mbps)\end{tabular}} & \multicolumn{2}{c|}{Accuracy (m)} & \multicolumn{2}{c|}{Comp. (m)} \\ \cline{3-6}
 &  & \begin{tabular}[c]{@{}c@{}}w/o\end{tabular} & \begin{tabular}[c]{@{}c@{}}w\end{tabular} & \begin{tabular}[c]{@{}c@{}}w/o\end{tabular} & \begin{tabular}[c]{@{}c@{}}w\end{tabular} \\ \hline
                                            Raw    & 480.00 & 0.62 & \textbf{0.14} & 0.43 & \textbf{0.17} \\ \hline
\begin{tabular}[c]{@{}c@{}}View-point\end{tabular} & 42.70  & 0.62 & \textbf{0.14} & 0.43 & \textbf{0.17} \\ \hline
\begin{tabular}[c]{@{}c@{}}Low\end{tabular}        & 3.80   & 0.63 & \textbf{0.21} & 0.40 & \textbf{0.24} \\ \hline
\begin{tabular}[c]{@{}c@{}}Medium\end{tabular}     & 2.50   & 0.91 & \textbf{0.37} & 0.40 & \textbf{0.33} \\ \hline
\begin{tabular}[c]{@{}c@{}}High\end{tabular}       & 1.27   & 1.08 & \textbf{0.52} & 0.63 & \textbf{0.38} \\ \hline
\end{tabular}
\caption{Reconstruction accuracy and completeness (comp.) for a 50m~x~100m~x~20m building with (w) and without (w/o) recalibration.}
\label{tab:e2e_reconstruction_errors_vs_compression_building_one}
\end{table}

\begin{table}[htbp]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|c|c|}
\hline
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Compression\\ profile\end{tabular}} & \multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Bandwidth\\ (Mbps)\end{tabular}} & \multicolumn{2}{c|}{Accuracy (m)} & \multicolumn{2}{c|}{Comp. (m)} \\ \cline{3-6}
 &  & \begin{tabular}[c]{@{}c@{}}w/o\end{tabular} & \begin{tabular}[c]{@{}c@{}}w\end{tabular} & \begin{tabular}[c]{@{}c@{}}w/o\end{tabular} & \begin{tabular}[c]{@{}c@{}}w\end{tabular} \\ \hline
                                                Raw & 480.00 & 0.87 & \textbf{0.21} & 0.35 & \textbf{0.09} \\ \hline
\begin{tabular}[c]{@{}c@{}}View-point\end{tabular}  & 42.70  & 0.87 & \textbf{0.21} & 0.35 & \textbf{0.09} \\ \hline
\begin{tabular}[c]{@{}c@{}}Low\end{tabular}         & 3.80   & 0.96 & \textbf{0.25} & 0.40 & \textbf{0.14} \\ \hline
\begin{tabular}[c]{@{}c@{}}Medium\end{tabular}      & 2.50   & 1.12 & \textbf{0.66} & 0.43 & \textbf{0.21} \\ \hline
\begin{tabular}[c]{@{}c@{}}High\end{tabular}        & 1.27   & 1.20 & \textbf{0.73} & 0.59 & \textbf{0.24} \\ \hline
\end{tabular}
\caption{Reconstruction accuracy and completeness (comp.) for a 50m~x~50m~x~20m building with (w) and without (w/o) recalibration.}
\label{tab:e2e_reconstruction_errors_vs_compression_building_two}
\end{table}

\subsection{3D Model Reconstruction}
\label{sec:eval:accuracy}

\begin{figure*}[t]
  \centering
  \includegraphics[width=2.0\columnwidth]{figs/fig_3D_models.pdf}
  \caption{Reconstructed 3D models at different accuracies}
  \label{fig:3d_model}
\end{figure*}

\parab{Experiment setup.}
To evaluate the end-to-end performance of \textsc{ares}\xspace in building an accurate 3D model in near real-time, we collected and reconstructed the 3D model of two buildings: a) a \textit{large} 50m~x~100m~x~20m (L~x~W~x~H) and, b) a \textit{small} 50m~x~50m~x~20m building. We then compared the reconstruction performance of these models with and without re-calibration against ground truth models generated from AirSim. Lower is better for accuracy and completeness.

\tabref{tab:e2e_reconstruction_errors_vs_compression_building_one} and \tabref{tab:e2e_reconstruction_errors_vs_compression_building_two} show accuracy and completeness, \textit{\textbf{with}} and \textit{\textbf{without}} re-calibration. Furthermore, they show results both for the uncompressed point cloud, as well as various compression options. These compression options include: view-point filtering (\secref{s:compression}), and 3 different levels of compression (\textit{low}, \textit{medium} and \textit{high}), corresponding to the following combinations of octree resolution and point resolution (\secref{s:compression}): $(0.25, 0.10)$, $(0.25, 0.25)$ and $(0.50, 0.50)$. In our experiments with our drone, we have found achievable LTE throughput to range from 1-4~Mbps; we chose these compression levels to correspond to this range.

\parab{Sub-meter reconstruction on compressed point clouds.}
As \tabref{tab:e2e_reconstruction_errors_vs_compression_building_one} and \tabref{tab:e2e_reconstruction_errors_vs_compression_building_two} shows, viewpoint filtering achieves a 10$\times$ compression. Low compression is an order of magnitude more efficient beyond this. Despite this, \textsc{ares}\xspace can achieve high quality reconstruction. Consider accuracy with re-calibration: the raw-point cloud has an accuracy of 0.14~m and 0.21~m, which is attributable entirely to SLAM error. View-point filtering does not degrade accuracy since it only omits zero returns. Low compression, with a bandwidth of 3.8~Mbps (easily achievable over LTE and over 100$\times$ more compact than the raw LiDAR output) only adds 4~cm to 7~cm to the accuracy metric. Even at high compression, \textsc{ares}\xspace has sub-meter for both buildings. Similarly, completeness is also remarkably good. Low compression adds 5~cm to 7~cm to the completeness, while high compression (350$\times$ more bandwidth efficient than the raw readings) only adds 15~cm to 21~cm to the completeness.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Compression\\ profile\end{tabular} & \begin{tabular}[c]{@{}c@{}}Bandwidth \\ (Mbps)\end{tabular} & \begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} & \begin{tabular}[c]{@{}c@{}}Completeness\\ (m)\end{tabular} \\ \hline
\begin{tabular}[c]{@{}c@{}}View-point\end{tabular} & 42.70 & 0.00 & 0.00 \\ \hline
\begin{tabular}[c]{@{}c@{}}Low\end{tabular} & 3.80 & 0.13 & 0.09 \\ \hline
\begin{tabular}[c]{@{}c@{}}Medium\end{tabular} & 2.50 & 0.23 & 0.16 \\ \hline
\begin{tabular}[c]{@{}c@{}}High\end{tabular} & 1.27 & 0.28 & 0.29 \\ \hline
\end{tabular}
\caption{Reconstruction from real-world traces, relative to uncompressed trace.}
\label{tab:e2e_reconstruction_errors_vs_compression_real_world}
\end{table}

To understand the degradation resulting from lower accuracy and completeness, consider \figref{fig:3d_model}, which shows the ground-truth model, together with reconstructions at varying fidelity. As accuracy worsens, artifacts in the reconstruction become more visible. With an accuracy of 0.25~m (low compression), the model closely matches the ground truth, but the textured building surface on the right shows some small artifacts. With an accuracy of 0.73~m (high compression), the textures on the right are almost invisible, but textures in other parts of the scene are visible. As we show later, LTE can sustain low compression, so when possible \textsc{ares}\xspace should use this setting.

\parab{Re-calibration helps reduce error.}
With re-calibration, our low compression achieves an accuracy of 0.21~m and 0.25~m for both buildings. Across the board, re-calibration accounts for 42-71~cm (upto 77~\%) improvement in accuracy, and 7-35~cm (upto 74~\%) improvement in completeness, depending on the degree of compression.

\parab{Sub-meter accuracy is possible on real-world traces.}
Results from our drone flights validate that real-world data can result in
comparable performance  (\tabref{tab:e2e_reconstruction_errors_vs_compression_real_world}). Because we lack a reference ground truth for real-world data, we use the 3D model generated from raw, uncompressed traces. With real-world traces, we can build accurate, and complete 3D models that are within 9-13~cm completeness and accuracy for low compression, and about 16-23~cm for medium compression, with respect to the uncompressed traces. These results suggest that highly compressed point clouds do not significantly impact accuracy and completeness.

\parab{\textsc{ares}\xspace generalizes to different building shapes.}
Our results so far, and the descriptions in \secref{sec:design}, have focused on rectangular buildings. \textsc{ares}\xspace can accurately reconstruct a variety of building types, as \tabref{tab:e2e_reconstruction_vs_building_types} shows. For these results, we use \textsc{ares}\xspace's default flight parameters and low compression.
Larger buildings (pentagonal, plus, and H-shaped) have larger flight durations partly because of their size and because they require two re-calibration steps. Even then, in all buildings, \textsc{ares}\xspace achieves sub-meter accuracy and completeness and re-calibration improves accuracy and completeness values by a factor of 3.1 and 2.5 respectively.

\parab{Lower target density worsens completeness.} To demonstrate that the user use the target density tuning knob to obtain less complete models more quickly, we conducted an experiment with \textsc{ares}\xspace (with re-calibration) at two different densities: 7.5 points per m$^2$ and 1 point per m$^2$. For the former, accuracy and completeness were 0.21~m and 0.14~m, and for the latter 0.68~m, 0.17~m respectively. The lower density flight took 20\% less time. As expected, completeness is worse at lower target densities. At the lower density, accuracy is worse because two adjacent scan lines have smaller overlap. Put another way, a side benefit of specifying higher density is the higher accuracy from scan line overlap.

\parab{\textsc{ares}\xspace generalizes to other types of structures.}
To show that \textsc{ares}\xspace can generalise and reconstruct other types of 3D structures \emph{e.g.,}\xspace airplanes, helicopters \emph{etc.}\xspace, we modeled a real-world blimp~\cite{blimp} (15~m x 60~m x 30~m) in AirSim and collected its 3D model using \textsc{ares}\xspace. In less than 7.5 minutes, with re-calibration, \textsc{ares}\xspace reconstructed the blimp with an accuracy of 23~cm and completeness of just 3~cm. This shows that, besides large buildings, \textsc{ares}\xspace can be used to reconstruct other large 3D structures in real-time.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|c|c|}
\hline
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Building \\type\end{tabular}} &
\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Flight \\duration (s)\end{tabular}} &
\multicolumn{2}{c|}{Accuracy (m)} &
\multicolumn{2}{c|}{Comp. (m)} \\
\cline{3-6} &  &
\begin{tabular}[c]{@{}c@{}}w/o\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w/o\end{tabular} &
\begin{tabular}[c]{@{}c@{}}w\end{tabular} \\ \hline
Star-shaped & 493 & 1.05 & \textbf{0.37} & 0.40 & \textbf{0.12} \\ \hline
Rectangular & 719 & 0.63 & \textbf{0.21} & 0.40 & \textbf{0.24} \\ \hline
Pentagonal  & 884 & 1.47 & \textbf{0.31} & 0.42 & \textbf{0.12} \\ \hline
Plus-shaped & 888 & 0.51 & \textbf{0.31} & 0.08 & \textbf{0.06} \\ \hline
H-shaped    & 908 & 1.10 & \textbf{0.34} & 0.27 & \textbf{0.10} \\ \hline
\end{tabular}
\caption{\textsc{ares}\xspace 3D reconstruction for different building types at low compression with (w) and without (w/o) re-calibration.}
\label{tab:e2e_reconstruction_vs_building_types}
\end{table}

\begin{table*}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline

\multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Velocity\\ (m/s)\end{tabular}}
& \multirow{2}{*}{\begin{tabular}[c]{@{}c@{}}Height\\ (m)\end{tabular}}
& \multicolumn{4}{c|}{Flight duration (s)}
& \multicolumn{2}{c|}{3D Model} \\

\cline{3-8}
 &  & Recon &
 \begin{tabular}[c]{@{}c@{}}Model\\ collection\end{tabular}
 & Re-calibration
 & Total
 & \begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular}
 & \begin{tabular}[c]{@{}c@{}}Completeness\\ (m)\end{tabular} \\ \hline

0.5 & 30 & 136 & 673 & 476 & 1285 & 0.43 & 0.34 \\ \hline
1 & 40 & 136 & 343 & 240 & 719 & 0.14 & 0.17 \\ \hline
2 & 50 & 136 & 72 & 272 & 480 & 0.91 & 0.45 \\ \hline
\begin{tabular}[c]{@{}c@{}}Exhaustive \\ SLAM flight (1 m/s)\end{tabular} & 40 & \multicolumn{4}{c|}{1520} & 0.14 & 0.17 \\ \hline
\end{tabular}
\caption{\textsc{ares}\xspace's flight durations for various parameter choices.}
\label{tab:battery_usage}
\end{table*}

\begin{table*}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|}
\hline

\begin{tabular}[c]{@{}c@{}}Compression \\ profile\end{tabular}
& \begin{tabular}[c]{@{}c@{}}Bandwidth\\ requirement (mbps)\end{tabular}
& \begin{tabular}[c]{@{}c@{}}Compression\\ time (ms)\end{tabular}
& \begin{tabular}[c]{@{}c@{}}Network\\ latency (ms)\end{tabular}
& \begin{tabular}[c]{@{}c@{}}Extraction\\ time (ms)\end{tabular}
& \begin{tabular}[c]{@{}c@{}}SLAM\\ time (ms)\end{tabular}
& \begin{tabular}[c]{@{}c@{}}Total\\ time (ms)\end{tabular} \\ \hline

Low      & 3.8                          & 15.49                 & 62.87                & 5.05                & 34.49          & 117.91          \\ \hline
Medium   & 2.5                          & 14.64                 & 65.31                & 3.07                 & 30.78          & 113.78          \\ \hline
High     & 1.3                          & 13.65                 & 65.33                & 3.03                 & 23.86          & 106.88 \\ \hline
\end{tabular}
\caption{Feasibility of transmitting compressed point clouds over LTE network.}
\label{tab:real_time_experiments}
\end{table*}

\subsection{\textsc{ares}\xspace Performance}
\label{s:other-meas-sysn}

\parab{Real-time model collection over LTE is feasible.} To validate that \textsc{ares}\xspace can collect a 3D model end-to-end in near real-time, we used our implementation to conduct an experiment in which a mobile device streamed 10-12 minutes worth of compressed point clouds from a real-world trace, over an LTE connection, to a 16-core GPU-enabled Amazon VM. (Our experiment only ran  model collection; recon also runs in real-time as discussed below).
At all three compression modes, \textsc{ares}\xspace was able to stream point clouds in real time (\tabref{tab:real_time_experiments}), and the total end-to-end processing time per frame is about 110~ms, of which nearly 65~ms is network latency. Thus, \textsc{ares}\xspace can make available \textit{a 3D model of a building in about a 100~ms} after receiving the last frame!

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|}
\hline
\textsc{ares}\xspace component &
Sub-component &
{\begin{tabular}[c]{@{}c@{}}Per-frame\\execution time\\ (ms)\end{tabular}} \\ \hline
\multirow{8}{*}{\begin{tabular}[c]{@{}c@{}}Recon phase\end{tabular}}
& 3D frame compression & 13.0 $\pm 1.3$ \\ \cline {2-3}
& 3D frame extraction & 3.0 $\pm 0.3$ \\ \cline {2-3}
& GPU normal estimation & 76.0 $\pm 82$ \\ \cline {2-3}
& RANSAC plane-fitting & 5.0 $\pm 9.0$ \\ \cline {2-3}
& Outlier removal & 0.2 $\pm 0.3$ \\ \cline {2-3}
& Rooftop detection & 0.005 \\ \cline {2-3}
& Rooftop extraction & 6.0 $\pm 5.0$ \\ \cline {2-3}
& Rooftop stitching & 3.0 $\pm 2.0$ \\ \cline {2-3}
& Total time & 100 $\pm 90.0$ \\ \hline
Model collection & LiDAR SLAM & 37.0 $\pm 3.0$ \\ \hline
\multicolumn{2}{|c|}{3D Reconstruction} & 10.3 $\pm 2.0$ \\ \hline
\end{tabular}
\caption{Processing times for \textsc{ares}\xspace components.}
\label{tab:system_execution_time_eval}
\end{table}

\parab{\textsc{ares}\xspace supports full frame rate processing.} We profiled the execution time of each component of \textsc{ares}\xspace on a 15-minute real-world trace.  Point cloud compression executes on the  drone, and other components run on an AWS VM with 16 cores, 64 GB RAM and a Nvidia T4 GPU. We use the GPU to offload the computation of surface normals for building detection. During recon, point cloud compression takes 13~ms per frame (\tabref{tab:system_execution_time_eval}). Extracting the building geometry requires 100~ms per frame; with these numbers, we can sustain about 10~fps, so with a 20~fps LiDAR, we process roughly every other frame. Despite this, our building detector is quite accurate (\secref{sec:eval:building}). During model collection, SLAM requires 37~ms per frame, and 3D reconstruction requires about 10~ms (\tabref{tab:system_execution_time_eval}). The former uses 8 cores, so we have been able to run these two components in a pipeline to sustain 20~fps. Thus, a moderately provisioned, cloud VM suffices to run \textsc{ares}\xspace at full frame rate with an end-to-end compute latency of about 100~ms for reconnaissance, and 50~ms for model collection.

\parab{Short flight durations can produce accurate models.} \textsc{ares}\xspace strives to reduce drone battery depletion in its design by generating short duration flights without sacrificing accuracy and completeness.
To show that \textsc{ares}\xspace's defaults of 1~m/s speed and 40~m height represent the best point in this tradeoff space, we compare to a lower, slower flight (30~m, 0.5~m/s), and a faster, higher flight (50~m, 2~m/s).
\tabref{tab:battery_usage} shows that, on the large building the lower, slower flight has a longer trajectory, resulting in more re-calibrations. The resulting model has worse accuracy and completeness; re-calibration can limit drift error, but not reverse it. A faster, higher flight has a slightly shorter trajectory, but the resulting model's accuracy is very poor, because there is less overlap between point clouds at higher speeds (\secref{s:slam-phase}). Finally, \tabref{tab:battery_usage} also shows the benefits of a recon flight: an exhaustive flight that uses the model collection parameters and does not perform recon is 3$\times$ longer than \textsc{ares}\xspace's flight. Results on the small building are qualitatively similar, so we omit these for brevity.

\subsection{Data Collection}
\label{s:data-collection}

\textsc{ares}\xspace relies on a careful parameter sensitivity analysis to determine model collection flight parameters: speed, height, and orientation (\secref{s:slam-phase}). We have evaluated SLAM error for every combination of drone speed (ranging from 0.5~m/s to 3~m/s), distance from building (10~m to 40~m) and orientation (parallel to perpendicular). We present a subset of these results for space reasons. For these experiments, we use the trajectory described in \figref{fig:mod_total}. We report the average numbers from three runs of each experiment.

\begin{figure}[t]
    \centering
 \includegraphics[width=0.6\columnwidth]{figs/fig_parallel_slam_param_study.pdf}
    \caption{SLAM error as a function of drone speed and distance from building.}
    \label{fig:parallel_slam_vs_height_speed}
\end{figure}

\begin{figure}[t]
    \centering
 \includegraphics[width=0.6\columnwidth]{figs/fig_parallel_vs_other_orientations.pdf}
    \caption{SLAM error for different orientations}
    \label{fig:parallel_vs_other_orientations}
\end{figure}

\parab{Best choice of distance is 20~m.}
\figref{fig:parallel_slam_vs_height_speed} plots the SLAM error as a function of the drone's distance from the building surface for the \textit{parallel} orientation of the LiDAR. Error increases slowly with height; beyond a 20~m distance from the building, the error is more than 1~m. This is because point densities decrease with height, affecting SLAM's ability to track features or points across frames (\secref{s:slam-phase}). Rather than fly lower, \textsc{ares}\xspace chooses to operate at a 20~m distance (or a 40~m height, since in our experiments buildings are 20~m tall) to reduce flight duration.

\parab{Best choice of speed is 1~m/s.}
Speed impacts SLAM positioning error significantly (\figref{fig:parallel_slam_vs_height_speed}). Beyond 1~m/s, SLAM cannot track frames accurately because of lower overlap between frames (\secref{s:slam-phase}). Below 1~m/s \emph{i.e.,}\xspace at 0.5~m/s, the flight duration (in seconds) is twice that of 1~m/s which results in drift error accumulation. To  achieve sub-meter reconstruction, \textsc{ares}\xspace chooses to fly the drone at 1~m/s.

\parab{Best choice of orientation is parallel.}
When the drone flies at 1~m/s and 20~m from the building, the parallel orientation has lowest SLAM error (\figref{fig:parallel_vs_other_orientations}) plots in which yaw 0\degree corresponds to parallel and yaw 90\degree to perpendicular). The parallel orientation has highest overlap between successive frames; as yaw increases, overlap decreases, resulting in higher SLAM error (\secref{s:slam-phase}).

\parab{Real-world traces confirm these observations.}
From the preceding discussion, \textsc{ares}\xspace chooses to fly slow, low, and at a parallel orientation. Our real-world traces confirm these choices. On these traces, because we cannot assess exact positioning error, we compute error relative to GPS. Our results show that SLAM-to-GPS error at 1.5~m/s with parallel orientation is 1.25~m. A perpendicular orientation at the same speed has 2$\times$ higher error, and this error doubles at twice the speed. Similar results hold as a function of height, which we have omitted for brevity.

\subsection{Boundary Detection Performance}
\label{sec:eval:building}

\parab{Methodology and metrics.}
We use two metrics for building boundary estimation: accuracy, and completeness. Accuracy is the average (2-D) distance between each point (quantized to 0.1~m) on the predicted boundary and the nearest point on the actual building boundary. Completeness, is the average distance between each point on the actual boundary and the nearest point on \textsc{ares}\xspace's predicted boundary. Lower values of accuracy and completeness are better.

We use both real-world traces collected from our \textsc{ares}\xspace prototype and synthetic traces from Airsim. To compute ground truth for real-world traces, we pin-pointed the building's boundary on Google Maps~\cite{google_maps}. For Airsim, we collected the ground truth from the Unreal engine's editor window.

\parab{Boundary detection can run at full frame rate.}
\tabref{tab:system_execution_time_eval} shows the time taken for each component of boundary detection, on our real-world traces on a single core of a GPU-equipped desktop. The average processing time per point cloud is 100~ms, dominated by GPU-accelerated surface normal estimation (76~ms). This can sustain 10~fps. However, our LiDAR generates 20~fps, so \textsc{ares}\xspace uses every other frame, without sacrificing accuracy.

\parab{Boundary detection is accurate.}
To evaluate the accuracy of \textsc{ares}\xspace's building boundary detection, we experimented with five building shapes (\emph{i.e.,}\xspace rectangular,  pentagonal, H-shaped, star-shaped, and plus-shaped). \textsc{ares}\xspace flies the drone at 4~m/s and at a distance of 60~m away from the building's roof using a perpendicular orientation. For all building shapes, \textsc{ares}\xspace accurately (to within 2.5~m) extracted the boundary of the building (\tabref{tab:boundary_estimation_shapes}) even when using high compression at 10~Hz (\emph{i.e.,}\xspace every other point cloud). The average accuracy is then 1.8~m and completeness is less than 2.0~m for all buildings.

On real-world traces collected over a 70~m x 60~m x 20~m building, \textsc{ares}\xspace's accuracy is 1.42~m and its completeness is 1.25~m (\tabref{tab:real_world_building_detector}), even at the highest compression and when it samples every other frames. Relative to view-point filtering, boundary accuracy and completeness drops by only 25~cm, even with two orders of magnitude compression.

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Building type\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Flight time (s)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Acc. (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Comp. (m)\end{tabular} \\ \hline

\begin{tabular}[c]{@{}c@{}}Rectangular\end{tabular} & 136 & 2.55 & 2.58 \\ \hline
\begin{tabular}[c]{@{}c@{}}Pentagon\end{tabular} & 136 & 2.58 & 2.58 \\ \hline
\begin{tabular}[c]{@{}c@{}}H-shaped \end{tabular} & 136 & 1.39 & 1.67 \\ \hline
\begin{tabular}[c]{@{}c@{}}Star-shaped\end{tabular} & 136 & 1.31 & 1.83 \\ \hline
\begin{tabular}[c]{@{}c@{}}Plus-shaped\end{tabular} & 136 & 1.35 & 1.55 \\ \hline
\end{tabular}
\caption{Boundary detection accuracy for different building shapes.}
\label{tab:boundary_estimation_shapes}
\end{table}

\parab{Other results.}
We obtained these recon flight parameters from an exhaustive parameter sweep. We omit a detailed discussion, but summarize two results. First, recon flights can be short, because boundary detection is less sensitive to point density and overlap. So, it can use perpendicular orientation, fly at 60~m from the building at 4~m/s. Second, boundary detection can tolerate point cloud sub-sampling. \textsc{ares}\xspace already uses only every other frame, but our evaluations show that boundary detection works well even when using one point cloud every 12 seconds.

\begin{table}[b]
\footnotesize
\centering
\begin{tabular}{|c|c|c|c|}
\hline
\begin{tabular}[c]{@{}c@{}}Compression\\ profile\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Bandwidth \\ (Mbps)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Accuracy\\ (m)\end{tabular} &
\begin{tabular}[c]{@{}c@{}}Comp.\\ (m)\end{tabular} \\ \hline

\begin{tabular}[c]{@{}c@{}}View-point @20~Hz\end{tabular} & 42.70 & 1.16 & 1.06 \\ \hline
\begin{tabular}[c]{@{}c@{}}Low @20~Hz\end{tabular} & 3.80 & 1.15 & 1.11 \\ \hline
\begin{tabular}[c]{@{}c@{}}Medium @20~Hz\end{tabular} & 2.50 & 1.17 & 1.06 \\ \hline
\begin{tabular}[c]{@{}c@{}}High @20~Hz\end{tabular} & 1.27 & 1.42 & 1.25 \\ \hline
\begin{tabular}[c]{@{}c@{}}High @10 Hz\end{tabular} & 0.63 & 1.40 & 1.30 \\ \hline
\end{tabular}
\caption{Boundary detection performance on real-world traces.}
\label{tab:real_world_building_detector}
\end{table}

\section{Introduction}
\label{sec:intro}

In the coming years, drones will revolutionize several industry segments, ranging from aerial photography to mapping and three-dimensional (3D) reconstruction. Recent estimates put the total market for these drone-based services at 63 billion US dollars by 2025~\cite{drone-market-1}, with 3D reconstruction, the task of generating a digital \textit{3D model} of the environment, accounting for nearly a third of the drone services market~\cite{drone-market-2}. In this paper, we focus on reconstructing the 3D model of large structures specifically buildings\footnote{Though our evaluations mostly focus on buildings, we show (\secref{sec:eval}) that \textsc{ares}\xspace can generalise and be used for other large structures \emph{e.g.,}\xspace blimps as well}.

A 3D model (\secref{sec:motivation}) captures the three-dimensional geometry of a structure such as a building. Different representations of this geometry exist, the finest of which is the 3D \textit{point cloud}. This is a collection of points on the surface of an object or structure, where each point has a 3D position and other attributes.

In general, 3D reconstruction can useful for a plethora of applications. For instance, they can be used for animations in films and video games, preserving historical and tourist sites, for archaeology, city planning, and to capture buildings or rooftops for repair and solar installation \emph{etc.}\xspace. More recently, these 3D models are used as inputs to immersive virtual reality (VR) applications.

The most dominant approach (\secref{sec:related}) to model collection is \textit{photogrammetry}, a technique which infers 3D models from a sequence of 2D camera images (and the positions associated with these images), captured either using drone-mounted cameras~\cite{7139681, 7989530, 8124461, 8628990} or by humans using ordinary mobile devices~\cite{company_hover}. With the increasing commoditization of LiDAR, 3D model reconstruction can use drone-mounted LiDARs; prior work~\cite{uav_lidar_1,uav_lidar_2} uses \textit{remote sensing} technologies,  specialized unmanned aerial vehicles (UAVs) with long-range LiDAR sensors and expensive GPS equipment. Both these approaches are compute-intensive and they compute the 3D model \textit{offline} \emph{i.e.,}\xspace after the drone has landed.

Although most prior research work has explored such \textit{offline} reconstruction~\cite{uav_lidar_1,uav_lidar_2,7139681, 7989530, 8124461, 8628990} (reporting accuracies in the centimeter to sub-meter range), Industry is starting to use the \textit{time} to do reconstruction as an important market discriminator for brand positioning. Startups promise to deliver models in hours or within a day~\cite{company_drone_deploy} after drone data collection. This trend has inspired newer, more time-sensitive applications of 3D modeling like post-disaster reconstruction for search \& rescue missions~\cite{drone_disaster_relief_1, drone_disaster_relief_2, drone_disaster_relief_3, drone_disaster_relief_4, drone_disaster_relief_5}, for post-flight integrity checks of airplanes~\cite{company_austrian_airline}, for construction site monitoring~\cite{company_drone_deploy}, to map mines \& tunnels~\cite{hovermap, Prometheus}, and to document interior office spaces~\cite{company_kaarta}. Only one company we know of so far promises \textit{real-time} reconstruction. However, this involves a human-operator holding a hand-held kit with a stereo camera to reconstruct interior spaces~\cite{company_kaarta}.

\parab{Our focus.} Motivated by these trends, improvements in 3D sensing, and the increasing  availability of off-the-shelf drones, we focus on \textit{upto sub-meter accurate, autonomous, real-time reconstruction of large buildings} using a drone-mounted LiDAR. Unlike the hand-held kit developed by~\cite{company_kaarta} which requires a human-in-the-loop to navigate the interior space, we aim to develop a completely automated solution given only an approximate location of the building(s) to reconstruct. To achieve the three requirements \emph{i.e.,}\xspace accuracy, autonomy and real-time 3D reconstruction, our approach devises algorithms to reconnoiter the specified area, then designs a trajectory to obtain the 3D model. However, these tasks are challenging for the following reasons. To achieve sub-meter accuracy, to position 3D point clouds, we cannot use GPS (because it is inaccurate), so we must use Simultaneous Localization and Mapping (SLAM~\cite{slam_part1})\footnote{Two startups we know of~\cite{hovermap, Prometheus} also use SLAM for drone \textit{offline} reconstruction (\secref{sec:related}).}. technologies. However using SLAM on a drone is particularly challenging because: a) relative to vehicle SLAM, on a drone, SLAM algorithms have access to 10~x sparser point clouds, b) at the same time, model collection must be relatively fast because of a drone's limited battery life, and c) both these can result in significant drift error accumulation, inherent to SLAM algorithms. To achieve autonomous reconstruction, identifying the exact location and boundary of the building can be challenging due to the large variety of building shapes and nearby objects like trees \emph{etc.}\xspace. Lastly, to enable real-time reconstruction, processing point clouds can require compute resources well beyond those on today's drones.

\parab{Contributions.} To address these challenges, this paper develops a system called \textsc{ares}\xspace, architected as follows: Given a rectangular region that encompasses a building, a cloud-based service generates energy-efficient drone trajectories to find the building and collect its 3D model. As the drone flies, it streams compressed LiDAR data over LTE; the cloud service runs SLAM in near real-time, adjusts the trajectory to minimize drift, and reconstructs the 3D model on the fly (\secref{sec:design}). To minimize drone battery usage, \textsc{ares}\xspace conducts a fast, energy-efficient \textit{reconnaissance} flight to determine the boundaries of the building, then runs a slower \textit{model collection} flight within the building's boundaries that carefully balances drone battery usage while ensuring high reconstruction accuracy. To achieve this, \textsc{ares}\xspace makes three contributions.

\textsc{ares}\xspace's first contribution is the design of \textit{model collection trajectories} that navigate the competing constraints \emph{i.e.,}\xspace accuracy and battery life (\secref{s:slam-phase}). Model collection uses SLAM, and SLAM error is sensitive to how the LiDAR is mounted, how fast and at what height the drone flies. Faster, higher flights use less of the drone's battery, but can incur high SLAM error. The converse is true of slower and lower flights. \textsc{ares}\xspace finds a sweet spot in this trade-off space to balance its accuracy goals with drone battery usage. Even so, SLAM can incur \textit{drift} on longer flights. \textsc{ares}\xspace needs to detect excessive drift, and correct for it. It uses a novel algorithm that tracks consistency between GPS traces and SLAM positions to detect excessive drift, then incorporates a \textit{re-calibration} step to correct for it.

Even with \textsc{ares}\xspace's efficient \textit{model collection trajectories}, scans over the entire rectangular area can exhaust a drone's battery. To this end, \textsc{ares}\xspace's second contribution is a robust and efficient \textit{building geometry} extraction algorithm (\secref{s:area-inter-reconn}) that helps focus \textit{model collection} efforts only on the building or region of interest. This algorithm, which runs during the reconnaissance flight, works well even with a fast, high flight that can minimize drone flight duration (and hence battery usage). During the flight, this algorithm extracts the building geometry without constructing the 3D model; doing so using the sparse sensor data from fast flights can introduce significant error. Instead, it relies on detecting planar surfaces by using the consistency of surface normals across points on a plane, then estimates building height and boundary from the plane forming the rooftop. \textsc{ares}\xspace then uses this boundary to plan the \textit{model collection trajectories} described above.

Its third contribution is the design of a software pipeline, leveraging GPU acceleration, that ensures that these algorithms can run on the cloud in near real-time (\emph{i.e.,}\xspace at the frame rate of the LiDAR), and with minimal end-to-end processing delays (\secref{sec:eval}).

Experiments (\secref{sec:eval}) on a photo-realistic drone simulator (Airsim~\cite{airsim}), validated with data from real-world drone flights, demonstrate that \textsc{ares}\xspace can achieve sub-meter reconstruction accuracy even after compressing the raw sensor data by almost two orders of magnitude. An experiment with a complete \textsc{ares}\xspace implementation is able to stream compressed point clouds over LTE and deliver a 3D model about 100~ms after flight completion. \textsc{ares}\xspace's choice of trajectories drains the battery least while achieving the accuracy target, and its pipeline can process frames at 20~fps while incurring a processing latency of less than 100~ms. To our knowledge, \textsc{ares}\xspace is the first to demonstrate on-line, cloud-based, autonomous, sub-meter, 3D model reconstruction in near real-time.

\section{Introduction}
\label{sec:intro}

\parab{Drone-based 3D reconstruction.}
The last few years have seen impressive advances in the commoditization of drones. Today, drones can be equipped with on board compute, a cellular (LTE) radio and sophisticated sensors (\emph{e.g.,}\xspace cameras, stereo cameras and LiDAR). Given this, in the coming years, drones will likely revolutionize aerial photography, mapping and three-dimensional (3D) reconstruction. Recent estimates put the total market for these drone-based services at 63 billion US dollars by 2025~\cite{drone-market-1}, with \textit{3D reconstruction}, the task of generating a digital \textit{3D model} of the environment, accounting for nearly a third of the drone services market~\cite{drone-market-2}.

\parab{What is a 3D model?}
The term \textit{3D model} covers a wide range of geometric representations of the surfaces of objects, from coarse-grained approximations (cylinders, cubes, intersection of planes), to more fine-grain representations such as \textit{meshes} (small-scale surface tesselations that capture structural variations). In this paper, we seek to extract a fine-grain \textit{point-cloud} of a large structure (\emph{e.g.,}\xspace a building, blimp or airplane) which consists of dense \textit{points} on the surface of the structure. Each point has an associated 3D position, together with other attributes (depending on the sensor used to generate the point-cloud). The point-cloud based 3D model can generate all other representations.

\parab{Applications.}
3D models are used in animations in films and video games, for preserving historical and tourist sites, for archaeology, city planning, and to capture buildings or rooftops for repair and solar installation \emph{etc.}\xspace, and as inputs to immersive virtual reality (VR) applications. Most prior research work has explored such \textit{offline} reconstruction~\cite{uav_lidar_1,uav_lidar_2,7139681, 7989530, 8124461, 8628990} and report centimeter to sub-meter accuracy.

\parab{Towards real-time 3D reconstruction.}
The nascent drone-based reconstruction industry is starting to use the \textit{time-to-reconstruction} as an important market discriminator. Startups promise to deliver models in hours or within a day~\cite{company_drone_deploy} after drone data collection. This trend has inspired newer, more time-sensitive applications of 3D modeling like (a) post-disaster reconstruction for search and rescue missions~\cite{drone_disaster_relief_1, drone_disaster_relief_2, drone_disaster_relief_3, drone_disaster_relief_4, drone_disaster_relief_5}, (b) construction site monitoring~\cite{company_drone_deploy}, (c) mapping mines, cell-towers and tunnels~\cite{hovermap, Prometheus}, (d) documenting interior office spaces~\cite{company_kaarta}, and (e) perhaps, most compelling, between-flight modeling of aircraft fuselages or blimps to determine structural integrity~\cite{company_austrian_airline}.

Though startups are starting to promise smaller reconstruction times~\cite{hovermap,Prometheus}, they reconstruct 3D models \textit{offline} \emph{i.e.,}\xspace after the drone has landed. Only one company we know of so far promises \textit{real-time} reconstruction~\cite{company_kaarta}, employing a human-operator holding a hand-held kit with a stereo camera (instead of a drone) to reconstruct interior spaces.

\parab{The Goal. } Given this, we want to \textit{accurately} reconstruct large 3D structures in \textit{near real-time}, with \textit{no human-intervention}.

\parab{Photogrammetry based reconstruction.}
Most existing work (\secref{sec:related}) uses \textit{photogrammetry}, which takes as input a sequence of 2D camera images (and the positions associated with these images), captured either using drone-mounted cameras~\cite{7139681, 7989530, 8124461, 8628990} or by humans using ordinary mobile devices~\cite{company_hover}. Photogrammetry \textit{infers} 3D models using a technique known as multi-view stereo~\cite{furukawa15:_multi_view_stereo}, which combines information from successive images, together with information about local motion, and inferred camera parameters such as the focal length. Photogrammetry approaches estimate depth from 2D images so their reconstruction times are relatively large \emph{i.e.,}\xspace from several hours to days~\cite{company_drone_deploy} for large structures.

\parab{LiDAR-based reconstruction.}
Unlike photogrammetry, LiDAR-based reconstruction can more directly infer 3-D models, because LiDARs directly provide depth information (unlike cameras). A LiDAR sensor measures distances to surfaces using lasers mounted on a mechanical rotating platform. The lasers and mechanical rotator sit within the enclosure of the LiDAR sensor. With each revolution of the lasers, the sensor returns a point cloud, or a \textit{LiDAR frame}. The point cloud is a set of 3D data points, each corresponding to a distance measurement of a particular position of the surrounding environment from the LiDAR. For instance, an Ouster OS1-64 LiDAR has 64 lasers that scan at 20~Hz, with a horizontal and vertical field of view of 360$\degree$ and 45$\degree$ respectively.

To reconstruct a structure \emph{e.g.,}\xspace a building, it suffices, in theory, to \textit{merge} points clouds captured from different locations around the building. To understand what it means to merge point clouds, consider two successive clouds $p$ and $p'$. A point $x$ on the surface of the building may appear both in $p$ and $p'$. However, since the drone has moved, this point $x$ appears at different positions (relative to the LiDAR) in the two point clouds. If we precisely transform both point clouds to the same coordinate frame of reference, then the union of points in $p$ and $p'$ constitutes part of the 3D model of the building.

\parab{Accurate positioning ensures high-quality models.}
This requires the precise position of the LiDAR at the positions where it captured the point clouds. The accuracy of positioning determines model quality. Two metrics define quality. \textit{Accuracy}, the average distance between the position of a point in the model and the corresponding point on the surface (see \secref{sec:design} for a more precise definition), clearly relies on accurate positioning. \textit{Completeness}, the degree to which a model captures the entire structure, also relies on accurate positioning, since positioning errors can lead to gaps in the captured model.

\begin{figure}
    \centering
    \includegraphics[width=0.95\columnwidth]{figs/fig_slam_vs_gps_3d_model.pdf}
    \caption{GPS-derived (left), SLAM-derived (middle), and \textsc{ares}\xspace-derived (right) models of a large complex on our campus.
    \label{fig:slam_vs_gps_3d_model}}
\end{figure}

\parab{Strawman: Using GPS for positioning.}
Because drones can carry GPS receivers, GPS can be used to position point clouds. Unfortunately, GPS errors can be several meters in obstructed settings, resulting in poor accuracy and completeness of the 3D model. The left image of \figref{fig:slam_vs_gps_3d_model} shows a 3D model of a building assembled using a drone flight over a commercial building; the building's outline is fuzzy, as are the contours of the trees surrounding the building. The right image shows a 3D reconstruction using techniques proposed in this paper, which does not use GPS. (All reconstructions in \figref{fig:slam_vs_gps_3d_model} use real data captured from a drone-mounted LiDAR~\secref{sec:eval}).

High-precision GNSS/RTK receivers can provide more accurate positioning but require additional infrastructure, are costly, and can perform poorly in urban environments due to non line-of-sight signals that degrade accuracy (\emph{e.g.,}\xspace prior work~\cite{Carloc} reports that RTK-enabled receivers can exhibit tens of meters error in downtown environments). Prior work~\cite{uav_lidar_1,uav_lidar_2} has used expensive GPS receivers in the context of \textit{remote sensing}, using specialized unmanned aerial vehicles (UAVs) with long-range LiDAR sensors (but for \textit{offline} model reconstruction). In contrast, in this paper we consider solutions that employ off-the-shelf technologies: drones, and commodity GPS and LiDAR.

\parab{An alternative: Using SLAM for positioning.}
LiDAR SLAM (Simultaneous Localization And Mapping~\cite{slam_part1, slam_part2}) algorithms can provide accurate \textit{pose} (position and orientation) estimates, which can be used to position point clouds. These algorithms use scan or feature matching techniques to align consecutive LiDAR frames to determine the pose of the drone throughout its flight. For example, scan matching uses techniques~\cite{icp} to find the closest match for each point in the source point cloud $\mathbf{A}$ to a point in the reference point cloud $\mathbf{B}$. It then estimates the transformation (translation and rotation) that best aligns each point in $\mathbf{A}$ to its corresponding point in $\mathbf{B}$. By repeating this process across consecutive LiDAR frames, SLAM can position each frame in the first frame's coordinate frame of reference.

\parab{
  Challenges.}
However, using SLAM on drone-mounted LiDAR is challenging for the following reasons (\tabref{tab:challenges}).

\paraspace\noindent$\blacktriangleright$
Reconstruction quality is critically dependent on the design of the drone \textit{trajectory}. \figref{fig:slam_vs_gps_3d_model} (middle) shows reconstruction using SLAM from a poorly planned drone flight. This reconstruction is visually worse than GPS-based reconstruction (\figref{fig:slam_vs_gps_3d_model} (left)), because the underlying SLAM algorithm is unable to \textit{track} LiDAR frames (\emph{i.e.,}\xspace it is unable to match points in successive point clouds).

\noindent$\blacktriangleright$
SLAM algorithms \textit{accumulate drift}~\cite{slam_part1,slam_part2}, so position estimates can degrade on longer flights.

\noindent$\blacktriangleright$
Drones have \textit{limited compute} because they can carry limited payloads. For instance, a DJI M600Pro (which we use in this paper) hexacopter has a maximum payload weight of 5~kg. It carries an A3Pro flight controller that contains three IMUs and three GNSS receivers. We have also mounted an LTE radio and an Ouster OS1-64 LiDAR, as well as a Jetson TX2 board. This compute capability is far from sufficient to run LiDAR SLAM\footnote{With a 64-beam LiDAR, SLAM processes upto 480 Mbps of 3D data}. On the TX2, plane-fitting~\cite{ransac}, a primitive used in our reconstruction pipeline, takes 0.5~seconds per LiDAR frame (LiDARs generate 20 frames per second) (\figref{fig:tx2ransac}) and plane-fitting accounts for 5\% of the execution time of our entire pipeline (\secref{sec:eval}).

\noindent$\blacktriangleright$
Drones have \textit{limited flight endurance}. When fully loaded and starting from a full charge, the M600Pro can fly for approximately 25 minutes. This necessitates careful trajectory planning to minimize flight duration.

{\small
\begin{table}[t]
  \centering
  \footnotesize
  \begin{tabular}{|l|l|} \hline
    \textit{\textbf{Challenge}} & \textit{\textbf{Mechanism}} \\ \hline \hline
    Model accuracy & Model collection trajectory planning \\ \hline
    Limited duration & Cheap reconnaissance flight\\ \hline
    Limited compute & Offload compressed point clouds\\ \hline
    Drift accumulation & Flight re-calibration\\ \hline
  \end{tabular}
  \caption{Challenges and contributions}
  \label{tab:challenges}
\end{table}
}

\parab{Contributions.} This paper presents the design and implementation of a system called \textsc{ares}\xspace, which makes the following contributions to address these challenges (\tabref{tab:challenges}).

\textsc{ares}\xspace's first contribution is the design of \textit{model collection trajectories} that navigate the competing constraints \emph{i.e.,}\xspace accuracy and battery life (\secref{s:slam-phase}). Model collection uses SLAM, and SLAM error is sensitive to how the LiDAR is mounted, how fast and at what height the drone flies. Faster, higher flights use less of the drone's battery, but can incur high SLAM error. The converse is true of slower and lower flights. \textsc{ares}\xspace finds a sweet spot in this trade-off space to balance its accuracy goals with drone battery usage. Even so, SLAM can incur \textit{drift} on longer flights.
\textsc{ares}\xspace needs to detect excessive drift, and correct for it in \textit{real-time}. It uses a novel algorithm that tracks consistency between GPS traces and SLAM positions to detect excessive drift, then incorporates a \textit{re-calibration} step to correct for it while the drone is \textit{in-flight}.

Even with \textsc{ares}\xspace's efficient model collection trajectories, scans over large areas can exhaust a drone's battery. To this end, \textsc{ares}\xspace's second contribution is a robust and efficient \textit{geometry extraction} algorithm (\secref{s:area-inter-reconn}) that helps \textit{focus model collection} only on the structure\footnote{In this paper, we focus on extracting the boundary of a building and leave generalization to future work} to reconstruct. This algorithm, which runs during a \textit{reconnaissance flight} before model collection, works well even with a fast, high flight that minimizes drone flight duration (and hence battery usage). During the flight, this algorithm extracts the building geometry without constructing the 3D model; it relies on detecting planar surfaces by using the consistency of surface normals across points on a plane, then estimates building height and boundary from the plane forming the rooftop. \textsc{ares}\xspace uses this boundary to plan the \textit{model collection trajectories} described above.

\textsc{ares}\xspace's third contribution is the design of a processing pipeline that (a) offloads computation to a cloud server by compressing point clouds on the drone to the point where they can be transmitted over LTE (\secref{s:compression}), and (b) that leverages GPU acceleration to process these point clouds in near real time at the LiDAR frame rate and with minimal end to end processing delay (\secref{sec:eval}).

Experiments (\secref{sec:eval}) on real-world drone flights and a photo-realistic drone simulator (AirSim~\cite{airsim}) , demonstrate that \textsc{ares}\xspace can achieve 21-30~cm reconstruction accuracy even after compressing the raw sensor data by almost two orders of magnitude. Not only is \textsc{ares}\xspace \textit{faster} than an \textit{offline} approach, it is also \textit{more accurate}, since the latter cannot re-calibrate mid-flight. An experiment with a complete \textsc{ares}\xspace implementation is able to stream compressed point clouds over LTE, reconstruct the 3D model on-the-fly and deliver an accurate 3D model about 100~ms after flight completion. \textsc{ares}\xspace's choice of trajectories drains the battery least while achieving the accuracy target, and its pipeline can process frames at 20~fps while incurring a processing latency of less than 100~ms. To our knowledge, \textsc{ares}\xspace is the first to demonstrate on-line, cloud-based, autonomous, sub-meter, 3D model reconstruction in near real-time.

\section{Background}
\label{sec:motivation}

\parab{3D Models.} The term \textit{3D model} covers a wide range of geometric representations of the surfaces of objects, from coarse-grained approximations (cylinders, cubes, intersection of planes), to more fine-grain representations such as \textit{meshes} (small-scale surface tesselations that capture structural variations). In this paper, we seek to extract a fine-grain \textit{point-cloud} of a building which consists of dense \textit{points} on the surface of the building. Each point has an associated 3D position, together with other attributes (depending on the sensor used to generate the point-cloud). The point-cloud based 3D model can generate all other representations.

\parab{LiDAR.} A \textbf{Li}ght \textbf{D}etection \textbf{A}nd \textbf{R}anging (LiDAR) sensor measures distances to surfaces using one or more lasers mounted on a mechanical rotating platform. The lasers and mechanical rotator sit within the enclosure of the LiDAR sensor. With each revolution of the lasers, the sensor returns a point cloud, or a \textit{LiDAR frame}. The point cloud is set of 3D data points, each corresponding to a distance measurement of a particular position of the surrounding environment from the LiDAR. For instance, an Ouster OS1-64 LiDAR has 64 lasers that scan at 20~Hz, with a horizontal and vertical field of view of 360$\degree$ and 45$\degree$ respectively.

\parab{Drones.} A drone is a remotely operated multi-rotor platform that carries a flight controller, an embedded compute platform, a cellular (LTE) radio and a LiDAR sensor. In general, drones have limited compute, the ability to carry minimal payload, and a limited flight time. For instance, a DJI M600Pro (which we use in this paper) hexacopter has a maximum payload weight of 5~kg. This drone uses an A3Pro flight controller that contains three IMUs and three GNSS receivers. It has an NVIDIA Jetson TX2 connected both to an LTE radio and the Ouster OS1-64 LiDAR. When fully loaded, the M600Pro has a flight endurance of approximately 25 minutes.

\begin{figure}
  \centering
  \includegraphics[width=0.5\columnwidth]{figs/fig_tx2_ransac.pdf}
  \caption{Plane-fitting on an nVidia TX2 \label{fig:tx2_plane_fitting}}
\end{figure}

\parab{The challenges.}
3D model collection from aerial drones is particularly challenging due to two competing constraints (accuracy and battery-life). Unlike ground robots like vehicles, on an aerial drone, LiDAR SLAM algorithms have access to only 9\% of the entire point cloud (because the drone is in the air). To compensate for this, drones can fly slow and close to the building for better positioning. However, this is not possible because of a drone's limited battery life. \textsc{ares}\xspace tackles this challenge on two fronts: a) it uses an offline parameter study and a real-time recalibration module to reduce positioning error, and b) it collects the 3D model in two steps \emph{i.e.,}\xspace a fast reconnaissance phase to identify the region of interest and a slower data collection flight to collect the actual 3D model.

A strawman solution to our problem of real-time model collection is to execute LiDAR data processing and model construction entirely on the drone. This is not likely to be feasible anytime in the near future. To demonstrate this, we ran a standard plane-fitting algorithm~\cite{ransac} on a single Ouster LiDAR point cloud; plane-fitting is a primitive we use in \secref{sec:design} to detect the rooftop of the building. On our drone's TX2, this operation can take nearly 0.5~s~ (\figref{fig:tx2_plane_fitting}) on a single point cloud, on average. Our LiDAR generates 20 point clouds every second, and plane-fitting is only one of the many expensive operations we require (\secref{sec:eval}). To put this in perspective, the plane-fitting algorithm contributes 5\% to the execution time of the entire pipeline. This means, if we ran \textsc{ares}\xspace on the drone, it would process a single second's data in 200 seconds.

For this reason, \textsc{ares}\xspace \textit{offloads} most computation to the cloud (\figref{fig:overview}). The drone obtains point clouds from the LiDAR, compresses them, and streams the point clouds to a cloud VM that performs near real-time 3D model construction.

\section{Related Work}
\label{sec:related}

\parab{Networked 3D sensing and drone positioning.}
Some recent work has explored, in the context of cooperative perception~\cite{AVR} and real-time 3D map update~\cite{CarMap}, transmitting 3D sensor information over wireless networks. Compared to \textsc{ares}\xspace, they use different techniques to overcome wireless capacity constraints. Robotics literature has studied efficient coverage path-planning for single~\cite{sensorplanning}, and multiple drones~\cite{ubanc}. \textsc{ares}\xspace's trajectory design is influenced by more intricate constraints like SLAM accuracy and equi-density goals. Accurately inferring drone motion is important for SLAM-based positioning~\cite{observability}. Cartographer~\cite{Cartographer}, which \textsc{ares}\xspace uses for positioning, utilizes motion models and on-board IMU's for estimating motion. In future work, \textsc{ares}\xspace can use drone orchestration systems~\cite{beecluster}, for larger campus-scale reconstruction with multiple drones.

\parab{Offline reconstruction using images.}
UAV photogrammetry~\cite{federman2017} reconstructs 3D models offline from 2D photographs. Several pieces of work~\cite{7139681, 7989530, 8124461, 8628990} study the use of cameras (either RGB or RGB-D) on UAVs for 3D reconstruction. Prior work~\cite{7139681} has proposed a real-time, interactive interface into the reconstruction process for a human guide. The most relevant of these~\cite{mostegel2016uav, 7422384} predicts the completeness of 3D reconstruction in-flight, using a quality confidence predictor trained offline, for a better offline 3D reconstruction. However, unlike \textsc{ares}\xspace, this work requires human intervention, computes the 3D model offline, requires close-up flights, cannot ensure equi-dense reconstructions, cannot dynamically re-calibrate for drift and is not an end-to-end system. A body of work has explored factors affecting reconstruction accuracy: sensor error~\cite{6899451}, tracking drift, and the degree of image overlap~\cite{7139681, LIENARD2016264}. Other work~\cite{8793729, 8628990, bylow2019combining} has explored techniques to reduce errors by fusing with depth information, or using image manipulations such as upscaling. Unlike \textsc{ares}\xspace, almost all of this work reconstructs the 3-D model offline.

\parab{Offline reconstruction using LiDAR.}
3D model reconstruction using LiDAR~\cite{uav_lidar_1,uav_lidar_2} relies on additional positioning infrastructure such as base stations for real-time kinematic (RTK) positioning, and long-range specialized LiDAR to achieve tens of centimeters model accuracy. \textsc{ares}\xspace explores a different part of the design space: online reconstruction with sub-meter accuracy using commodity drones, GPS and LiDAR. More recent work has explored drone-mounted LiDAR based offline reconstruction of tunnels and mines, but require specialized LiDARs and a human-in-the-loop~\cite{Prometheus, hovermap} for drone guidance (either manually or by defining a set of waypoints).

\parab{Rooftop boundary detection.}
Prior work has used infrared sensors, RGB-D cameras~\cite{rgbfasterboundarydetection} and a fusion of LiDAR~\cite{lidarbuildingdetection} with monocular cameras~\cite{lidarimagebuildingdetection, lidarorthophotoboundarydetection}. These assume a pre-existing stitched 3D point cloud~\cite{lidarbuildingdetection} or orthophoto~\cite{lidarorthophotoboundarydetection} and are not designed to operate in real-time. \textsc{ares}\xspace's boundary detection accuracy is comparable to these pieces of work, even though it does not rely on these assumptions.