\section{Background on Homomorphic Encryption}
\label{sec:HE}
{\em Homomorphic Encryption} schemes allow computations on encrypted data such that the decrypted results are equal to the result of a mathematical or logical operation applied to the corresponding plaintext data.
Therefore, calculations on encrypted data can be performed without decrypting the data first.
This property makes it possible to outsource not only the storage of encrypted data, but also the computation on sensitive data to untrusted third parties (e.g., the cloud).
The computation on cloud servers has two major advantages.
If only the result and not the whole dataset needs to be transferred to the client, a lower network bandwidth is required. Furthermore, the client can be a thin client like a tablet without much computing and storage resources because the computational expensive rendering is done on the server.
Homomorphic encryption schemes are classified into three categories:
\begin{itemize}
	\itemsep0em
	\item {\em partially homomorphic encryption (PHE)}: are homomorphic with regard to only one type of operation (addition or multiplication)
	\item {\em somewhat homomorphic encryption (SHE)}: Can perform more general calculations than PHE, but only a limited number of them.
	\item {\em fully homomorphic encryption (FHE)}: Can perform any computation on encrypted data.
\end{itemize}
Rivest et al. \cite{pdf:FirstHE.Rivest1978} invented the idea of Homomorphic Encryption in 1978. They showed the demand of a secure Homomorphic Encryption scheme that supports a large set of operations.
However, it took more than 30 years until the first proposal of such a FHE was made by C. Gentry in 2009 \cite{phdthesis:homencGentry}.
While the first FHE schemes were just concepts, due to an ongoing development and optimization process, they are currently at least efficient enough for functional implementations.

However, FHE schemes are still not practical for real applications, because the storage and computational costs are too high \cite{phdthesis:Hu2013ImprovingHE, article:MartinsFheSurvey2018, article:AbbasHeSurvey2018}.
Therefore, in our approach, we propose to take advantage of the Paillier PHE scheme, whose homomorphic properties that are relevant for our encrypted volume rendering will be introduced next.

\subsection{The Paillier Cryptosystem}
\label{sec:PaillierCryptosystem}
Paillier's cryptosystem \cite{pdf:Paillier} is an additive PHE scheme.
The important property of this scheme is that a multiplication of two encrypted numbers is equivalent to an addition in the plaintext domain.
This means that it is possible to calculate the sum of plaintext numbers that have been encrypted by multiplying the encrypted numbers.
This relation is stated in \autoref{equ:paillierAddition}.
For \autoref{equ:paillierAddition} and  \autoref{equ:paillierMultiplication}, we adopt the  notation by Ziad et al. \cite{pdf:CryptoImg-IEEE}.
The $\oplus$ symbol is used for the operation on encrypted numbers that is equivalent to an addition on plaintext numbers.
The encrypted version of the value $m$ is denoted as $\llbracket m \rrbracket$ and $\textrm{Dec}(\llbracket m \rrbracket) = m$ means decrypting $\llbracket m \rrbracket$ to $m$ by the decryption function of Paillier's cryptosystem (see: \autoref{alg:PaillierDecrypt}).
Paillier works on finite fields and thus performs a modulo (mod $N^2$) operation after each multiplication ensures to stay in the field and does not change the decrypted result because the corresponding plaintext numbers need to be less than the modulus $N$ for a correct decryption anyway.
However, the modulo operation makes further calculations more efficient because it prevents unnecessary large numbers ($n$ multiplications will blow up the number length by about $n$ times).
\begin{align}
	\begin{split}
		\textrm{Dec}(\llbracket m_1 \rrbracket \oplus \llbracket m_2 \rrbracket) &= \textrm{Dec}((\llbracket m_1 \rrbracket \times \llbracket m_2 \rrbracket) \mod N^2)\\
		&= (m_1 + m_2) \mod N
	\end{split}
	\label{equ:paillierAddition}
\end{align}
Since it is possible to add encrypted numbers to each other, in a \texttt{for-loop}, an encrypted number can be added to itself $d$ times to simulate a multiplication with $d$.
However, note that the value $d$ is in plaintext.
Since addition is performed by doing multiplication on the encrypted numbers, we can, instead of a \texttt{for-loop}, take the encrypted value to the power of $d$ to get this result, which can be implemented more efficiently than a \texttt{for-loop}.
Furthermore, it has the advantage that it also works for $d < 0$.
The case $d = -1$ is of special interest, because this makes subtraction of two encrypted numbers possible ($\textrm{Dec}(\llbracket m_1 - m_2 \rrbracket) = \textrm{Dec}( \llbracket m_1 \rrbracket \times \llbracket m_2 \rrbracket^{-1} \: \mod \: N^2)$).
The symbol $\otimes$ is used for such an operation on one plaintext number $d$ and one encrypted number $\llbracket m_1 \rrbracket$.
\autoref{equ:paillierMultiplication} shows how to calculate this multiplication with one encrypted number.
\begin{align}
\begin{split}
	\textrm{Dec}(\llbracket m_1 \rrbracket \otimes d) &= \textrm{Dec}(\llbracket m_1 \rrbracket^d \mod N^2)\\
	&= (m_1 \times d) \mod N
\end{split}
\label{equ:paillierMultiplication}
\end{align}
While the Paillier PHE supports an efficient method to multiply an encrypted and a plaintext number, it does not support the multiplication of two encrypted numbers and is, therefore, not a fully homomorphic encryption scheme.

\input{algorithm/paillierCreateKeys-short}
\input{algorithm/paillierEncrypt}
\input{algorithm/paillierDecrypt}

The Paillier HE is a probabilistic asymmetric encryption scheme like the well-known RSA scheme \cite{article:rsa}.
That means that an encryption can be performed by using the public key, which is derived from the secure (or private) key.
However, for the task of decryption, the secure key is required, which cannot (efficiently) be calculated from the public key because this would require factoring a product of two large prime numbers.
It is essential for the security of Paillier's cryptosystem (and also for RSA) that there is no known fast method for integer factorization of a product of two large prime numbers.
The \autoref{alg:PaillierCreateKeys} shows an example of a key generation function for the Paillier cryptosystem, which sets the generator $g$ always to $N+1$, as this allows a more efficient encryption function (see: \cite{inproc:Fazio:2018:paillierOptimizingGenerator}).
Furthermore, the algorithm clearly shows that the secure key, which needs to be kept secret, contains the two large prime numbers $p$ and $q$ (e.g., 1024 bit long) and the public key contains the product $N$ (e.g., 2048 bit long) of these two prime numbers.

\autoref{alg:PaillierEncrypt} and \autoref{alg:PaillierDecrypt} show the pseudocode for the encrypt and decrypt routines for the Paillier HE.
The encryption algorithm also contains the obfuscating of an encrypted number with a random number $r$, which qualifies the Paillier HE as a probabilistic encryption scheme.
This means that a specific plaintext message $m$ can be represented by many possible ciphertexts $ \llbracket m \rrbracket_1,  \llbracket m \rrbracket_2,  \llbracket m \rrbracket_3 \dots,  \llbracket m \rrbracket_r$.
The decryption with the right secure key will return the original message $m$ for all the possible ciphertext representations.
While this is not required for correct homomorphic calculations with Paillier (imagine $r = 1$), it is important for the semantic security against chosen-plaintext attacks (IND-CPA) that Paillier's cryptosystem provides \cite{pdf:Paillier, inbook:homoEncApps:2014:paillier}.
Without this obfuscating, it would be possible to decrypt datasets without knowing the secure key because an attacker would only need to encrypt all possible plaintext values with the public key,
store the plaintext and the corresponding ciphertext values in pairs, and compare the values of the encrypted dataset with the self-encrypted values for which the correct decryption is known.
For datasets with a limited number of possible values like the voxel values of a volume, which usually contains no more than $2^{10} = 1024$ different values, this would be a trivial task.

\section{Related Work}
We have only found two  works that address the topic of privacy-preserving rendering of volumetric data.
The most similar work to ours is that of Mohanty et al.\cite{inproc:3DCrypt}.
They present a cryptosystem for privacy-preserving volume rendering in the cloud.
Unlike our approach, they achieve correct alpha compositing.
However, to attain this goal, they end up with a solution that cannot be considered secure, that has a fixed transfer function, and that requires that the volume is sent from one server to another server for each rendered frame.

Their approach requires two servers for rendering: a Public Cloud Server and a Private Cloud Server.
The first step of their rendering approach is to apply a color and opacity to each voxel before encrypting the volume.
This means that the transfer function is pre-calculated and cannot be changed by a user without performing a time-consuming reencryption and uploading of the volume.
In the next step, the encrypted data is uploaded to the Public Cloud Server, which stores the volume data.
When the Public Cloud Server receives an authorized rendering request from a client, the server calculates all sample positions for the requested ray casting and interpolates the encrypted color and opacity values for each sample position.
All interpolated sample values then need to be individually sent to the Private Cloud Server, which decrypts the opacity value of each sample in order to perform the alpha blending along the viewing rays.
For alpha compositing, the opacity values of samples represent object structures in the volume; therefore, anyone who can gain access to the Private Cloud Server, such as an administrator or a hacker, will be able to observe these structures in the volume dataset.
If an unauthorized person has access to this server,
the whole approach collapses.
For the task of encrypting and decrypting parts of the volume data on the servers, their approach requires a central Key Management Authority (KMA).
While this brings the advantage that an organization can centrally control which users have access to a specific volume, it enlarges the attack surface of their system considerably, because the KMA has all keys required for decrypting all volume data.
Therefore, the confidentiality of the KMA is constitutional for the privacy of all datasets, no matter who they belong to.

Another weakness of their approach is the required network bandwidth between the Public and the Private Cloud Server because all sample values of a ray casting frame need to be transferred from the Public and the Private Cloud Server (more than 1GB).
With our approach, the privacy of the volume data and rendered image depends only on a single secure key. Also, our approach should scale linearly with the computing power of the hardware it is running on.

Chou and Yang \cite{ObfuscatedVolRend} present a volume rendering approach that attempts to make it difficult for an unintended observer to make sense of the volume dataset that resides on a server.
This is done by, on the client's side, subdividing the original data into equally sized blocks.
The blocks are rearranged in a random order and then sent to the server as a volume.
The server then performs volume rendering on each block and sends the result back to the client, which will reorder the individual block renderings and composite them to create a correct rendering.
To obfuscate the data further, on the client's side, the data values in each block are changed using one out of three possible monotonic operations: flipping, scaling, and translating.
Monotonic operations are used as they are invertible and associative under the volume rendering integration.
Therefore, doing the inverse operators on the resulting rendering gives the same result as doing them on the data values before performing the rendering.
This algorithm cannot be considered safe, and the authors acknowledge this as they state that the goal is only to not trivially reveal the volume to unauthorized viewers.
A possible attack would be to consider the gradient magnitude of the obfuscated volume.
This should reveal the block borders.
The gradient magnitude can further be used inside each block to reveal structures in the data that can be used for aligning the blocks correctly.

\deleted{
Our goal is to develop an approach that is open and secure by design
(Kerckhoffs's principle \cite{article:Kerckhoffs83}) and not {\em secure through obscurity} \cite{article:hoepman2008securityThrough}
or relies on assumptions that are outside the algorithm itself, such that the internal memory of a cloud server is not accessible to an intruder.
The former is the case for Chou and Yang's approach \cite{ObfuscatedVolRend} and the latter is the case for the approach by Mohanty et al. \cite{inproc:3DCrypt}.
}

\replaced{
	To attain our goal of developing an approach that is open and secure by design,
	we use the Paillier cryptosystem developed by Paillier in 1999 \cite{pdf:Paillier}.
	This cryptosystem  is an
	asymmetric encryption scheme,
	where the secure key contains two large prime numbers $p$ and $q$%
	, and the public key contains the product $N$ (modulus)
	of $p$ and $q$.
	The cryptosystem supports an additive homomorphic operation ($\oplus$).
	If this operation is applied to two encrypted values $\llbracket m_1 \rrbracket, \llbracket m_2 \rrbracket$ ($\llbracket m \rrbracket$ means encrypted $m$), the decrypted result is the sum of the $m_1$ and $m_2$ ($\textrm{Dec}(\llbracket m_1 \rrbracket \oplus \llbracket m_2 \rrbracket) = (m_1 + m_2) \mod N$).
	Furthermore, a homomorphic multiplication ($\otimes$) between an encrypted value and a plaintext value $d$ is supported
	($\textrm{Dec}(\llbracket m_1 \rrbracket \otimes d) = (m_1 \times d) \mod N$).
	Since Paillier's cryptosystem does not carry over  multiplication of two encrypted values to plaintext, it is classified as a partially homomorphic encryption (PHE) scheme.
	Paillier can securely encrypt many values (e.g., $512^3$ voxels of a volume) from a small number space (e.g., $2^{10}$ possible density values), because it is {\em probabilistic}, which means that during the encryption, the {\em obfuscation} can map a single plaintext value randomly to a large number of possible encrypted values.
	This makes a simple \enquote{probing} for finding out the number correspondence
	impossible.
	Further details about Paillier's cryptosystem such as the encryption and decryption algorithm is provided in the Supplementary material document.
	We are limited to the arithmetic operations supported by Paillier for creating a volume rendering that captures as much structure as possible from the data.
} {
To attain this goal, we use the Paillier cryptosystem, and are limited in the types of arithmetic operations we can use for creating a volume rendering that captures as much structure as possible from the data.
}
This forces us to think unconventionally and creatively when designing the volume renderer.

For homomorphic image processing, the work by Ziad et al. \cite{pdf:CryptoImg-IEEE} makes use of the additive homomorphic property of Paillier's cryptosystem. They demonstrate that they are able to implement many image processing filters using the limited operations allowed with Paillier. They implement filters for negation, brightness adjustment, low pass filtering, Sobel filter, sharpening, erosion, dilation and equalization.
\added{
While most of these filters are computed entirely on the server side, erosion, dilation, and equalization require the client for parts of the computation.
There are various works that make use of such a {\em trusted client protocol} approach to overcome the limitation of a
PHE
scheme
and enable operations such as addition, multiplication, and comparisons on the encrypted data~\cite{inProc:SecureMR, inProc:Crypsis, inProc:JCrypt}.
A {\em trusted client} knows the secure key and can, therefore, perform any computation on the data or {\em convert / re-encrypt} it from one encryption scheme to another (e.g., from an additive to a multiplicative {\em homomorphic encryption}).
These client-side computations introduce latency because the data needs to be transferred back and forth between the server and the client.
Furthermore, the client needs to have enough computational power to avoid becoming the bottleneck of the system.
To mitigate this problem, automated code conversions can be used that minimize the required client side {\em re-encryptions} \cite{inProc:JCrypt,inProc:SecureMR}.
While a {\em trusted client} approach could theoretically solve many of the problems we face with our untrusted server-only approach, it is not practical for volume rendering.
The most demanding problems of volume rendering, such as transferring a voxel value and advanced compositing (alpha blending, maximum intensity projection, ...), need to be done per voxel.
Hence, every voxel that could contribute to the image synthesis (all voxels of a volume for many rendering cases) needs to be transferred to the {\em trusted client} and processed there for every rendered frame.
The encryption and decryption on the client side are more expensive than the operations required for a classical sample compositing due to the size of encrypted values (e.g., 1000 bit per voxel).
If an amount of data in the range of the volume itself needs to be transferred from the server to the client, where the data would need to be encrypted and decrypted, it is pointless to perform any calculations on the server, because the client then has more work to do than in a classical volume rendering on the client.
Moreover, it does not save any network traffic as compared to a simple download, decrypt, and process use case.
Therefore, we argue that {\em trusted client} approaches are not suitable for our work.
Furthermore, a {\em trusted client} approach will not work with thin clients, which contradicts our third requirement.
Our second requirement is also contradicted because, in real-world use cases, the network bandwidth between a client like a tablet computer and a cloud server will not have enough bandwidth (e.g., more than 1Gbit/s) to support interactive frame rates.
}

\section{Encrypted Rendering Overview}
The first step of the introduced privacy preserving rendering system is the encryption of the volume dataset \added{(\autoref{fig:setup} Acquisition Device)}.
During the encryption stage, every single scalar voxel value of a volume dataset needs to be encrypted with Paillier's approach (see \autoref{alg:PaillierEncrypt} in the Supplementary Material document).
Meta data of the volume such as width, height, depth and the storage order of voxels will not be encrypted.
The next step is to upload the encrypted volume dataset to a server \added{(\autoref{fig:setup} arrow from Acquisition Device to Cloud Server)}.
For our approach, the device that encrypts the volume and uploads it to a server does not even need the secure key, because for encryption, only the public key is required.

When a rendered image is requested to be shown on a client, the client sends a rendering request to the server, which has the encrypted volume dataset \added{(\autoref{fig:setup} arrow from Client to Cloud Server)}.
The rendering request contains further information about the settings of the rendering pipeline, such as the camera position, view projection, and (depending on the selected rendering type) also information about the transfer function that should be used.
After the server receives such a rendering request, it uses the included pipeline settings and the already stored encrypted volume dataset to render the requested image \added{(\autoref{fig:setup} the rendering pipeline stages of the Cloud Server)}.
To preserve privacy, the server does not have the secure key and can not, therefore, decrypt the volume data.
The operations that are used for rendering an image from an encrypted volume dataset are limited to the homomorphic operations add ($\oplus$\deleted{, \autoref{equ:paillierAddition}}) and multiply with plaintext ($\otimes$\deleted{, \autoref{equ:paillierMultiplication}}), which are defined for Paillier's encryption scheme.
When the rendering is finished, the server will send the calculated image data to the client \added{(\autoref{fig:setup} arrow from Cloud Server to Client)}.
The resulting image that the client receives is still encrypted.
Decrypting such an image is only possible for a client that knows the correct secure key.
For everyone else, the image will be random noise (shown in Supplementary Video Material).
Since every single pixel value is an encrypted number, every single pixel can be decrypted independently of the other pixels.
For a gray-scale image, that means one number per pixel.
An RGB colored image requires three values that need to be decrypted per pixel.

In \autoref{sec:xray}, we explain how the homomorphic operations of Paillier's HE can be used for X-ray sample integration. Furthermore, we will show how to use Paillier's cryptosystem with floating-point numbers, which allows us to perform trilinear interpolation.
\autoref{sec:transferFunction} explains a more advanced approach that allows the emphasizing of different density ranges in the rendered images.

\section{Encrypted X-Ray Rendering}
\label{sec:xray}
\added{
	Ray-casting \cite{proc:Krueger:2003:ATGV} is the most frequently used approach for volume rendering.
	Furthermore, ray-casting based algorithms can be easily and efficiently parallelized and can be implemented with fewer memory reads than slicing-based algorithms.
	Memory access is time-consuming, especially if every number that needs to be read is thousands of bits long.
	Therefore, we implement our privacy-preserving volume rendering approach with ray-casting.
}
\replaced{
	However, other direct volume rendering approaches developed for unencrypted data, such as slicing, can be used as well.
	Slicing on the server can be built by the same encrypted rendering pipeline components (sampling / interpolation, color mapping, compositing),
	which we will explain anon.
	Slicing could also be used to just perform the sampling on the server, transfer the slices to the client, and perform the compositing there.
	However, this would not fulfill our
	requirements because of the required network bandwith and the high computational requirement on the client.
}{
}

\added{
The ray casting algorithm first calculates a viewing ray for every pixel of the final image (\autoref{fig:setup} Ray Traversal - stage of the Server).
These viewing rays will be calculated based on the camera position, up vector, opening angle, image resolution, and pixel index.
At discrete and equidistant steps along the ray, the data of the volume is sampled (\autoref{fig:setup} Sampling - stage of the Server).
The last step is the compositing, where the final pixel value is calculated based on the sample values of a viewing ray (\autoref{fig:setup} Compositing - stage of the Server).
}

\replaced{
	X-ray rendering is a volume rendering approach where the sample value is mapped to a white color with monotonically increasing opacity, and the compositing is a summation followed by a normalization at the end of the ray traversal.
}{
}
If the sampling of the voxel values is done by nearest-neighbor filtering, the sum along a viewing ray can be calculated by only using the homomorphic add operation ($\oplus$) which is already defined for Paillier's cryptosystem\deleted{ (see \autoref{equ:paillierAddition})}.
\replaced{
The final normalization of all samples along a view ray cannot be done directly by the homomorphic operations of Paillier's encryption scheme because this requires a division that can result in a non-integer value that is not supported.
}{
}
However, the server
\replaced{could}{}
send the encrypted sum together with the sample count to the client, which can perform the division after decrypting the sum.

\begin{figure}[tb]
	\centering
	\subfigure[Nearest Neighbor]{
		\includegraphics[width=0.478\columnwidth,trim=0 3mm 0 3mm,clip]{figures/enc-xray-nearestNeighbor-x300.png}
		\label{fig:xrayTrilinear:nn}
	}
	\subfigure[Trilinear Interpolation]{
		\includegraphics[width=0.478\columnwidth,trim=0 3mm 0 3mm,clip]{figures/enc-xray-trilinearInterpolation-x300.png}
		\label{fig:xrayTrilinear:ti}
	}
	\caption{Results from encrypted X-ray rendering showing nearest neighbor (a) and Trilinear interpolation (b), which we also support.}
	\label{fig:xrayTrilinear}
\end{figure}

\added{

To improve the nearest-neighbor sampling with trilinear interpolation,

	a mechanism that allows the summing and normalization of encrypted values ($\llbracket m_1 \rrbracket$, $\llbracket m_1 \rrbracket$), which are scaled by some plaintext weights ($\alpha_1$, $\alpha_2$), is required.
	For plaintext integers, the interpolation could be implemented

	around the integer arithmetic operations add, multiply, and divide
	(1D example: $(m_1 \cdot \alpha_1 + m_2 \cdot \alpha_2) / (\alpha_1 + \alpha_2)$).
	Since an arbitrary division is not supported by Paillier's cryptosystem, this is not directly feasible on encrypted data.
	A possible solution could be to use fraction types, which has an encrypted denominator and a plaintext numerator for storage and calculations.
	After the image is rendered, which contains such fractions as pixel values,

	 the client can download it, decrypt the denominators, and perform the deferred divisions%
}%
\footnote{
	\added{
		If the rendering pipeline is designed in a very static way, it is theoretically possible to know the final numerator upfront and let the client perform the required division without explicitly specifying the numerator. However, this is very inflexible, error prone, and requires an update for the client whenever a change on the server leads to a change of the final numerator.
	}
}.
\added{
	However, we decided to use a floating-point encoding, which is easier to implement
	and allows a shader code development as is usual for hardware accelerated rendering.
	With a floating-point representation of encrypted values, it is possible to multiply the eight neighboring voxels of a sample position with the distances between the samples and voxel position. These distances, which have a sum of $1.0$, are the weights of the interpolation (1D example: $m_1 \cdot \alpha_1 + m_2 \cdot \alpha_2$).
}
\deleted{
To improve the nearest-neighbor sampling with trilinear interpolation, a floating-point number encoding for encrypted values is required because the distances between the sample position and the eight neighboring voxels, which are used as weights for the interpolation, are fractions.
}
A floating-point encoding will also make the final division of the sample sum for X-ray rendering on the server side possible.
While a floating-point encoding does not directly enable divisions in the encrypted domain, it can be used to approximate a division by a multiplication with the reciprocal of the divisor, as shown in \mbox{\autoref{equ:fpDeviation}}.
\begin{equation}
	\begin{split}
	\frac{\sum}{n}
	\approx \textrm{Dec}\left(
				\left\llbracket\sum\right\rrbracket
				\otimes
				\left\lfloor \frac{1}{n} \cdot 10^\gamma \right\rceil
			\right)
		\cdot 10^{-\gamma}
	\end{split}
	\label{equ:fpDeviation}
\end{equation}
The sum of samples along a viewing ray is denoted as $\sum$, and $n$ is the count of samples.
The precision of the approximation is defined by the count of decimal digits $\gamma$ (e.g., $\gamma = 3$ for thousandth).
Before the reciprocal of $n$ is multiplied with $\sum$, the comma is moved $\gamma$ digits to the right ($\cdot 10^\gamma$) and then rounded ($\lfloor \rceil$).
The multiplication with $10^{-\gamma}$, which moves the comma back to the correct position, can be achieved by subtracting $\gamma$ from the exponent of the floating-point encoded result.
Since the Paillier cryptosystem is defined over $\mathbb{Z}_N$, the result is only correct if no intermediate result is greater than $N-1$.

We will discuss the used floating-point encoding in \autoref{sec:FPNumbers}.
\autoref{fig:xrayTrilinear} shows two images that were rendered from an encrypted floating-point encoded dataset.
For the rendering of the left image, a nearest-neighbor sampling was used, and for the right image, a trilinear interpolation was used.
The used dataset contains three objects with different densities: a solid cube in the center wrapped inside a sphere and another sphere at the top left front corner.
The same dataset is also used for renderings shown in \autoref{fig:densityEmphasising} and \autoref{fig:generic4dimColored}.

\subsection{Encrypted Floating-Point Numbers}
\label{sec:FPNumbers}
A floating-point number is defined as $m \cdot b^e$, where
$m$ is called the mantissa.
The exponent $e$ defines the position of the comma in the final number.
The base $b$ is a constant that is defined upfront (e.g., during the compilation of the application).
We used a decimal system for convenience; therefore, our prototype uses $b = 10$.
However, $b$ can be any positive integer that is greater or equal to $2$.

To calculate with floating-point arithmetic in the encrypted domain, we have chosen to use the approach developed for Google’s Encrypted BigQuery Client \cite{web:GoogleEncryptedBigqueryClientGit}.
The idea is to store the mantissa $m$ and the exponent $e$ of a floating-point number in two different integer variables.
During the encryption of the floating-point number $(m, e)$, only the mantissa $m$ is encrypted using Paillier's cryptosystem.
The exponent $e$ remains unencrypted, which results in the floating-point number $(\llbracket m \rrbracket, e)$.
This floating-point number representation is also used by the {\em python-paillier} library \cite{web:PythonPaillierGit}, the Java library {\em javallier} \cite{web:PythonPaillierGit} and in the work by Ziad et al. \cite{pdf:CryptoImg-IEEE}.

For an addition of two such encrypted floating-point numbers, both need to have the same exponent.
Therefore, the exponents of both numbers must be made equal before the actual addition, if they are not already equal.
Hence, it is not possible to increase the exponent if the mantissa is encrypted because that would require a homomorphic division of the encrypted mantissa, which is not possible.
Therefore, the floating-point number with the greater exponent needs to be changed.
On the other hand, decreasing the exponent of a floating-point number is not a problem because it requires a homomorphic multiplication of the encrypted mantissa with a plaintext number, which is possible with Paillier.
\autoref{equ:decreaseExponentTo} shows how to calculate the new mantissa $\llbracket m_n \rrbracket$ that is required for decreasing the exponent of the floating-point number ($\llbracket m_o \rrbracket$, $e_o$) to the lower exponent $e_n$.
The new floating-point number is defined as ($\llbracket m_n \rrbracket$, $e_n$), which represents exactly the same number as ($\llbracket m_o \rrbracket$, $e_o$).
It is just another way to store it.
\begin{align}
\begin{split}
	\llbracket m_n \rrbracket = \llbracket m_o \rrbracket \otimes b^{e_o - e_n}
\end{split}
\label{equ:decreaseExponentTo}
\end{align}
When both floating-point numbers ($\llbracket m_1 \rrbracket$, $e_1$) and ($\llbracket m_2 \rrbracket$, $e_2$) have the same exponent $e_1 = e_2 = e_n$, the homomorphic sum $\llbracket m_s \rrbracket$ of both mantissas can be calculated by the add operation defined for Paillier \deleted{(see \autoref{equ:paillierAddition}) }, which results in the final floating-point number ($\llbracket m_s \rrbracket$, $e_n$).
The \autoref{alg:PaillierFpAdd} shows this approach for summing two floating-point numbers with encrypted mantissas.
The lines from 2 to 10 bring the exponents of both floating-point numbers to the same value ($e_n$), and line number 11 contains the addition of the encrypted mantissas.
\input{algorithm/paillierFpAdd}
A multiplication with a floating-point number that contains an encrypted mantissa ($\llbracket m_1 \rrbracket$, $e_1$) and a floating-point number with a plaintext mantissa ($m_2$, $e_2$) can be achieved by multiplying the mantissas with the multiplication operation defined for Paillier ($\llbracket m_n \rrbracket = \llbracket m_1 \rrbracket \otimes m_2 \;\;$\deleted{~ , \autoref{equ:paillierMultiplication}}) and a plaintext addition of the exponents ($e_n = e_1 + e_2 $).
This is also stated in line 10 and 11 of the \autoref{alg:PaillierFpMultiply}, which is sufficient for a correct result.
The lines from 2 to 9 contain a performance optimization, which prevents the intermediate result of $\llbracket m_e \rrbracket^{m_d}$, which is computed before $\mathrm{mod}\, N^2$ is applied in line 10, from being unnecessarily large\deleted{ (see \autoref{equ:paillierMultiplication})}.
This optimization is also used by the python library {\em python-paillier} \cite{web:PythonPaillierGit} in \texttt{paillier.py}
and the java library {\em javallier} \cite{web:JavallierGit} in \texttt{PaillierContext.java}.

\input{algorithm/paillierFpMultiply}

Signed numbers can be represented by using a two's complement representation for the mantissa $m$.
The exponent $e$ does not change.
If $v$ is a negative integer, the two's complement in the integer modulo $N$ can be calculated by: $m = v + N$.
In the encrypted domain, the additive inverse $-m$ of $m$ is defined by the multiplicitive inverse $\llbracket m \rrbracket ^{-1} = \llbracket i \rrbracket$ of $\llbracket m \rrbracket$ in the integers, modulo $N^2$
($\llbracket i \rrbracket$ is defined by: $\llbracket m \rrbracket \cdot \llbracket i \rrbracket = 1 \mod N^2$ and can be computed from $\llbracket m \rrbracket$ and $N^2$ by the {\em extended Euclidian algorithm} \cite{book:KnuthArtV2}).
This
complement representation for encrypted numbers can also be used for a subtraction of two encrypted numbers\deleted{, as already stated in \autoref{sec:PaillierCryptosystem}}.
Since, the first operand of a subtraction can be added to the additive inverse of the second operand ($\textrm{Dec}(\llbracket m_1 - m_2 \rrbracket) = \textrm{Dec}( \llbracket m_1 \rrbracket \times \llbracket m_2 \rrbracket^{-1} \: \mod \: N^2)$).

With the floating-point encoding explained in this section, it is possible to perform a trilinear interpolation of voxel values because the encrypted voxel values can be multiplied by the fractional distances between a sample position on a viewing ray and the actual voxel position.
Furthermore, divisions of an encrypted number $(\llbracket m \rrbracket, e)$ by a plaintext number $d$ can be approximated by a multiplication of the encrypted number $(\llbracket m \rrbracket, e)$ with the reciprocal $(\left\lfloor 1/d \cdot 10^\gamma \right\rceil , -\gamma)$ of $d$ ($\gamma$ defines the precision~ -~compare with \autoref{equ:fpDeviation}).

\section{Transfer Function}
\label{sec:transferFunction}
In this section, we discuss the challenges of building a transfer function approach that works for a probabilistic PHE scheme, and we show a novel and practical solution for a simplified transfer function.
It is not possible to use the transferred values for an alpha blending sample compositing because this would require a multiplication of two encrypted values, which is not possible with Paillier's cryptosystem.
However, the transfer function can be used to highlight specific density ranges at X-ray rendering, which helps an observer to distinguish between different objects inside a volume.

A transfer function for non-encrypted voxel values can be implemented as an array with the possible voxel values as indices and the assigned color as values of the array.
The evaluation of such a transfer function is as simple as reading the value from the array at the index, which is equal to the voxel value that should be mapped.
However, this cannot be efficiently implemented for encrypted data.
For non-encrypted voxel values, such a transfer function array will have a length that is equal to the amount of possible voxel values, which is only $2^8 = 256$ for $8$-bit voxels or $2^{10} = 1024$ for $10$-bit voxels.
An encrypted volume dataset will probably not contain two equal voxel values, because of the obfuscation during the encryption.
That means an encrypted dataset will probably have as many different voxel values as it has voxels.
Therefore,
an array as transfer function will not work because it would be at least as big as the volume itself.

Another approach for non-encrypted data is to store just some supporting points that contain the density and color.
The evaluation for this transfer function approach is achieved by interpolating the color between the value of the next lower and next greater supporting point.
To find the neighboring supporting points of the voxel value that should be transferred, comparison operators such as lower than ($<$) or greater than ($>$) are required.
However, comparison operators cannot exist for probabilistic PHE schemes like Paillier because that would break its security (see \autoref{sec:ComparisonOperators}).
Therefore, the question is how to implement a function $f:X \to Y$ that can map finite sets of numbers $X$ to another set of numbers $Y$ by just using the operations {\em add} ($\oplus$) and {\em multiply with constant} ($\otimes$).
The result of this function is again an encrypted number.
A promising approach that can achieve this was presented by Wamser et al. \cite{pdf:ObliviousLookupTables} in their work on \enquote{oblivious lookup-tables}.

\subsection{Oblivious Lookup Tables}
\label{sec:olut}
Let $X = \{x_1, x_2,...,x_n\}$ be an enumeration of values that should be mapped to $Y = \{y_1, y_2,...,y_n\}$ by the lookup function $f(x_i) = y_1$.
The idea is to create a vector $\vec{v_i}$ for every $x_i \in X$ with the same cardinality as $X$ ($|\vec{v_i}|=|X|$) and define the evaluation of a lookup by the dot product
$\vec{v_i} \cdot \vec{l} = y_i$.
The scalar value $y_i$ is the result of the lookup. For a transfer function, this would be the value of one color channel.
The vector $\vec{l}$ can be calculated form the linear
equation $V \cdot \vec{l} = \vec{y}$.
$V$ is a square matrix of full rank with $n = |X|$, that uses all vectors $\vec{v_i}$ as rows.
However, this linear equation needs to be solved only once.
Therefore, the client can calculate $\vec{l}$ upfront based on unencrypted numbers.
The
equation $V \cdot \vec{l} = \vec{y}$
has a unique solution, if all vectors  $\vec{v_i}$ are linearly independent.
Hence, the crucial part is to find an approach to extrapolate every vector $\vec{v_i}$ only from one single $x_i$ so that the $\vec{v_i}$ are linearly independent from each other.
Wamser et al. \cite{pdf:ObliviousLookupTables} suggest to use a Vandermonde-Matrix as $V$ (\autoref{equ:olutVandermonde}), because it fulfills these requirements.
\begin{align}
\begin{split}
	V = \begin{pmatrix}
	  1 & x_1^1 & x_1^2 & \cdots & x_1^{n-1} \\
	  1 & x_2^1 & x_2^2 & \cdots & x_2^{n-1} \\
	  \vdots  & \vdots & \vdots  & \ddots & \vdots  \\
	  1 & x_n^1 & x_n^2 & \cdots & x_n^{n-1}
	\end{pmatrix}
\end{split}
\label{equ:olutVandermonde}
\end{align}
From the creation rule of the Vandermonde-Matrix, it follows that a $\vec{v_i}$, which is equal to the $i$-th row of the matrix $V$, is defined as  $\vec{v_i} = (1, x_i^1, x_i^2, \cdots, x_i^{n-1})$.
The lookup function $f(x_i)$ can, therefore, be stated as:
\begin{align}
\begin{split}
	f(x_i) = (1, x_i^1, x_i^2, \cdots, x_i^{n-1}) \cdot \vec{l} = y_i
\end{split}
\label{equ:olutEvaluationVandermonde}
\end{align}
The dot product in \autoref{equ:olutEvaluationVandermonde}
can be calculated even if
$\vec{v_i} = (1, x_i^1, x_i^2, \cdots, x_i^{n-1})$
is encrypted because only the operations {\em add} ($\oplus$) and {\em multiply} ($\otimes$) that are defined for the Paillier HE are required for calculating a dot product.
However, it is not possible to calculate the vector $\vec{v_i}$ from an encrypted $\llbracket x_i \rrbracket$, because this would involve multiplications of two encrypted numbers, which is not possible with Paillier.
A theoretical solution for this could be to store the vector $\vec{v_i}$ instead of scalar $x_i$ as the value of a voxel.
For a volume dataset, where the voxel values have only a resolution of $8$ bits, this would lead to a vector length of $n = 2^8 = 256$.
Therefore, the required storage size for the volume will increase $256$ times.

A volume with $512 \times 512 \times 512$ voxels and a resolution of $8$ bits per voxel requires $512^3 \cdot 8~\mathrm{bits} / 8~\mathrm{bits} = 134,217,728~\mathrm{Bytes} = 128~\mathrm{MB}$.
The same volume encrypted by Paillier HE with a public key length that can be considered as secure ($2048$ bits) requires $512^3 \cdot 2 \cdot 2048 \mathrm{bits} / 8 \mathrm{bits}
= 64~\mathrm{GB}$.
If the scalar voxel values $x_i$ are replaced by the vectors $\vec{v_i}$ with a length of $256$, the volume will require $64~\mathrm{GB} * 256
= 16 \mathrm{TB}$.
While a volume dataset with $16$ Terabyte is probably better than a transfer function that is at least as big as the encrypted volume, the overhead in terms of storage and computation is still too big to be practical.
Therefore, we develop a simplified and novel transfer function approach with a considerably lower storage overhead, which we discuss in the next two sections.

\subsection{Density Range Emphasizing}
Our simplified transfer function approach is based on the observation that it is possible to compute the dot product of a vector with encrypted values and a vector with plaintext values.
Furthermore, the dot product can be used to calculate an encrypted scalar value indicating  the similarity of an encrypted vector and a plaintext vector. This will work if both vectors have length $1$.
Therefore, our approach is to encode the density values of each voxel as a vector and encrypt each component of this vector by the Paillier encryption algorithm (Supplementary Material \autoref{alg:PaillierEncrypt}).
In order to highlight a user-defined density range, the density value at the center of this range needs to be encoded as a vector.
Note that this vector is not encrypted.
The encrypted volume rendering engine can now compute the dot product between this vector and the encrypted vector of a sample position.
Then the ray-casting algorithm needs to sum up the results of the dot products along a ray instead of the density values.
This approach allows a user to emphasize a selectable density range in the rendered image.
\autoref{fig:densityEmphasising} contains images that were created using this approach.
The top left subfigure shows a result of an X-ray rendering for comparison.
All other subfigures show results for different density ranges that are emphasized.
The density that is encoded as vector that was used for the dot-product calculation is specified in the caption of each sub figure.

\begin{figure}[tb]
	\centering
	\subfigure[X-ray]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-01-x-ray-smaller.png}
	}
	\subfigure[emphasized density: 0.653]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-02-colored-1-1-smaller.png}
	}
	\\
	\vspace*{-1mm}
	\subfigure[emphasized density: 0.331]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-04-colored-1-3-smaller.png}
	}
	\subfigure[emphasized density: 0.781]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-05-colored-1-4-smaller.png}
	}
	\caption{First image shows an X-ray rendering result for comparison with the other three images that are created by our encrypted density emphasizing approach. The volume density values are encoded with 4-dimensional vectors.}
	\label{fig:densityEmphasising}
\end{figure}

\begin{figure}[tb]
	\centering
	\subfigure{
		\includegraphics[width=0.478\columnwidth]{figures/voxelEncoding3dNormalized-dot_045and085-c.pdf}
	}
	\subfigure{
		\includegraphics[width=0.478\columnwidth]{figures/voxelEncoding6dNormalized-dot_045and085-c.pdf}
	}
	\caption{
		Visualization of density encoded as 3 dimensional (left) and 6 dimensional (right) vectors.
		The scalar value (density) of the voxel is represented on the x-axes.
		The magnitude of each vector component at a specific density is represented by the curves.
		The first component is drawn in red, the second in green, red, purple, olive and light blue.
		The dashed curves shows the result of the dot product between the encoded voxel value and a {\em TF-Node} vector for a density of 0.45 in cyan and a density of 0.85 in orange.
	}
	\label{fig:densityEncodingPlots}
\end{figure}

\input{algorithm/extendedHsvMapping-short}

The density-to-vector encoding scheme we used is based on an HSV-to-RGB color conversion.
The exact encoding scheme is stated in \autoref{alg:encodeDensity}.
\autoref{fig:densityEncodingPlots} illustrates the magnitude of the vector components for all possible density values.
Furthermore, the response intensities for user-defined emphasizing densities at 0.45 and 0.85 are shown.
At the last line of \autoref{alg:encodeDensity}, the calculated vector is normalized.
This is important to make sure that the result of the dot product is always between $0$ and $1$ and to ensure that the highest possible dot product result ($1$) is at the user-defined emphasizing density.

There are other and possibly better density-to-vector encoding schemes.
However, the HSV-based encoding leads to results that feel natural, especially while smoothly increasing or decreasing the emphasizing density.
The encoding scheme should in any case be chosen in such a way that the curve created by the dot product is steep and narrow (see dashed lines in \autoref{fig:densityEncodingPlots}), so that the density selected by the user can be seen as clearly as possible in the resulting image.
The \autoref{alg:encodeDensity} takes not only the density that should be encoded as parameter, but also the count of dimensions of the returned vector.
Increasing the count of dimension not only makes the dot product response curve more steep
(See \autoref{fig:densityEncodingPlots} and compare the dashed lines in the left and right plot.),
but also increases the required storage size of the encoded and encrypted volume dataset.
Note that the count of dimensions must be the same during the encryption of the volume and for the encoding of the user-defined emphasizing density.
This also means that the amount of computations required for the volume rendering depends on the number of dimensions used for encoding the volume.

\subsection{Simplified Transfer Function}
It is possible to add RGB colors to the rendered images based on the density range emphasizing described in the last section.
This is useful because RGB colors allow a user to emphasize different densities in the same image while keeping the densities distinguishable (see \autoref{fig:generic4dimColored}).
Since the dot product between an encoded and encrypted voxel value and a user-defined encoded density is an encrypted scalar value, a multiplication with another plaintext number is possible.
For our simplified transfer function approach, the dot product result needs to be multiplied with a user-defined RGB color vector.
As the dot product expresses the similarity between the voxel value and the user-defined density, the intensity of the resulting RGB color will be high if the densities are similar, and low otherwise.
Since the RGB color vector is not encrypted,
the multiplication between the encrypted dot product result and the RGB color vector can be archived by three separate homomorphic multiplications ($\otimes$) of one encrypted and one plaintext number\deleted{ (see \autoref{equ:paillierMultiplication})}.
The result of such a multiplication is an encrypted RGB color.
This calculation can be performed not only for one density-RGB-color-pair, but also for multiple such pairs.
For a better understanding, we will call such a pair consisting of a density and an RGB color a {\em transfer function node (TF-Node)}.

\autoref{equ:encVoxel2Rgb} shows the transformation for one encoded and encrypted voxel value $\llbracket \vec{v} \rrbracket$ to an encrypted RGB color $\llbracket \vec{c_v} \rrbracket$.
The symbol $\bigoplus$ is used instead of $\sum$, because the sum of encrypted vectors needs to be calculated.
The variable $n$ denotes the count of user defined {\em TF-Nodes}.
The vectors $\vec{d_i}$ and $\vec{c_i}$ are the encoded density and RGB color of the {\em TF-Node} with index $i$.
The symbol $\odot$ is used as operator for a dot product between one encrypted vector and one plaintext vector.

\begin{align}
\begin{split}
	\llbracket \vec{c_v} \rrbracket = \bigoplus_{i=0}^{n}\left( \llbracket \vec{v} \rrbracket \odot \vec{d_i} \right) \otimes \vec{c_i}

\end{split}
\label{equ:encVoxel2Rgb}
\end{align}
To obtain the final encrypted RGB color of a pixel, the sum of all encrypted RGB sample values $\llbracket \vec{c_v} \rrbracket$ along a viewing ray needs to be calculated.
The total RGB vector needs to be divided by the sample count as usual for averaging and, furthermore,
by the count of {\em TF-Nodes}.
This can be achieved by dividing each component of the total RGB vector by the product of the sample count and the count of {\em TF-Nodes}.
The method to approximate a division of an encrypted number is stated in \autoref{equ:fpDeviation}.
After calculating this for every image pixel, the entire encrypted image is sent to the client.
A client that knows the right secure key can now decrypt each RGB component of each pixel and display the colored image.
Example images rendered with this approach are shown in \autoref{fig:teaser} and \autoref{fig:generic4dimColored}.

\begin{figure}[tb]
	\centering
	\subfigure[blue at 0.279, red at 0.797]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-06-colored-2-1-smaller.png}
		\label{fig:generic4dimColored_xray}
	}
	\subfigure[blue at 0.000, red at 1.000]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-07-colored-2-2-smaller.png}
	}
	\\
	\vspace*{-1mm}
	\subfigure[green at 0.076, blue at 0.651, red at 1.000]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-08-colored-3-1-smaller.png}
	}
	\subfigure[blue at 0.000, yellow at 0.293, green at 0.664, purple at 1.000]{
		\includegraphics[width=0.478\columnwidth,trim=0 1mm 0 1mm,clip]{figures/c-09-colored-4-1-smaller.png}
	}
	\caption{Images are created by our simplified transfer function approach. The volume data voxel values are encoded by four-dimensional vectors. The subfigures shows results of different transfer functions applied to the same encrypted dataset.}
	\label{fig:generic4dimColored}
\end{figure}

\section{Results}
\label{sec:Results}
All performance tests are executed on a Mac Book Pro (15-inch, 2016) with an 2.9 GHz Intel Core i7.
All algorithms are implemented in Java \added{and are only single-threaded}.
The purpose of the implementation is to prove the concept and, in its current form, is not performance-optimized.
All runtimes shown in \autoref{table:performanceXray} and \autoref{table:performanceColor} are measured with volume size of $100 \times 100 \times 100$ voxels.
The rendered image always has a size of $150 \times 150$ pixels.

\autoref{table:performanceXray} shows the runtime performance required for encrypting a volume with scalar voxel values, X-ray rendering, and image decryption with different public key modulus lengths. The table is divided into four groups of rows.
The first two groups show the required time for rendering with nearest-neighbor sampling.
Group three and four show the resulting performance for trilinear interpolation.
The numbers in group one and three of the \autoref{table:performanceXray} are measured without obfuscation during the encryption; therefore, the encrypted volume is not secure.
While this type of \enquote{encryption} does not have any practical relevance, it is interesting to compare these runtime numbers with those in the group two and four, which are measured from a secure encryption with obfuscation.
It can be seen that the obfuscation takes a significant amount of time.
Therefore, the random number generation ($r$) that is required for the obfuscation and the calculation of $r^N$ (see Supplementary Material \autoref{alg:PaillierEncrypt}) has a substantial impact on the time required for encrypting the volume dataset.
We use the \texttt{java.security.SecureRandom} class from the java standard runtime framework as random number generator for the obfuscation.

\input{tables/javaPerformanseXrayTables}
\input{tables/volumeStorageSize}
\autoref{table:volumeStorageSize} shows the required memory size for this volume with a single scalar value per voxel and also for encodings in multiple dimensions at different modulus lengths.
\autoref{table:performanceColor} shows the runtime required for encrypting a volume with different voxel encodings (two, three, and four dimensions), rendering with our simplified transfer function approach at different counts of {\em TF-Nodes} (one, two, ... colors) and image decryption. The resulting performance for all these operations is provided for different public key modulus lengths.
\input{tables/javaPerformanseColorTable}

\added{
	The rendering results of \autoref{fig:teaser} show what can be done with our simplified transfer function.
	The right image demonstrates the  utilization in nuclear medicine.
	During the diagnosis, these datasets are usually investigated either by showing single slices or by X-ray renderings, where the depth cues are provided through rotating the dataset around an axis. This  is possible with our homomorphic-encrypted volume rendering with the added  privacy, which is useful for diagnosing from such a highly sensitive type of modality and associated pathologies.
}

\section{Discussion}
\added{
	First, we discuss possible performance improvements of our prototype and how the approach could scale to interactive frame rates for larger real-world datasets.
	Later, starting with general noteworthy considerations, we discuss security-related aspects of our volume rendering approach. Finally, we follow with an explanation for the invisibility of comparisons.
	In \autoref{sec:FPNumbersSecurity}, we will show that the used floating-point encoding with an encrypted mantissa and a plaintext exponent does not weaken the privacy of the encrypted volume data.
}

\subsection{\added{Performance}}%
\label{sec:PerformanceImprovements}%
Our prototype is implemented as a single-threaded application; however, a major strength of our approach is that it is highly parallelizable and should scale linearly with the processing power.
There are obvious opportunities to improve the performance to a multi-threaded implementation, and multiple memory allocations (\texttt{new} statements) during the rendering could be avoided.
During the encryption, every voxel can be processed independently.
Therefore, it should be relatively easy to use as many processing units (e.g., CPU cores or shader hardware on GPU) as voxels in the volume for the encryption.
In the rendering and decryption stage, every pixel of the image can be processed independently.
Therefore, the number of processing units that can be used efficiently in parallel is equal to the number of pixels in the final image.
Furthermore, a better storage order of voxel values, such as Morton order \cite{tech:mortonOder} (recursive Z curve) extended to three dimensions, could lead to a better cache usage, which will further improve the performance.
The implementation used for all shown results is based only on a naive three-dimensional \texttt{BigInteger} array as volume storage.

\added{
If we consider a real-world dataset with a resolution of $512 \times 512 \times 512$ voxels encrypted with a perfectly secure 2048bit long key for the purpose of X-ray rendering
with a single value per voxel, the encrypted dataset will have a size of 64GB ($=(512^3 \cdot 2048 \cdot 2) / (8 \cdot 1024^3)$).
While this is a considerable data amplification compared to the 16bit plaintext representation of the dataset with $256$MB ($=(512^3 \cdot 16) / (8 \cdot 1024^2)$), it will nevertheless perfectly fit in the video-memory of two NVIDIA Quadro RTX 8000 that have $48$~GB of
memory each.
An encrypted volume with a four-dimensional encoding for our simplified transfer function approach will be four times bigger and will, therefore, have a size of $256$GB.
Consequently, at least six GPUs with $48$GB memory each will be required.
While six GPUs in one server is absolutely possible, our privacy-preserving volume rendering approach should scale much further.
It should be possible to use our proposed encrypted voxel compositing scheme as mapper for the MapReduce implementation proposed by Stuart et al. \cite{inProc:GPUMapReduce}, which can make use of a GPU-accelerated distributed memory system for volume rendering.
}

\subsection{\added{Security Considerations}}
The data privacy of our approach depends entirely on the security of Paillier's cryptosystem.
Our approach does not store any voxel value or any information that is computed from a voxel value without an encryption by Paillier's cryptosystem.
The Paillier cryptosystem is semantically secure against chosen-plaintext attacks (IND-CPA) \cite{inbook:homoEncApps:2014:paillier}.
Therefore, we conclude that the data that our approach provides to the storage and rendering server are protected in a semantically secure way.
The computational complexity required for breaking a secure key of Paillier's cryptosystem depends on the length of the modulus $N$.
The larger the modulus $N$ is, the harder it is to be factorized, which would be required for data decryption.
For the required length of the modulus, the same conditions as for the RSA cryptosystem \cite{article:rsa} should hold.
From 2018 until 2022, a modulus $N$ with a length of at least 2048 bits is considered to be secure \cite{web:cryptographicKeyLength, NIST:800_56B_Rev.2}.

\subsection{Encrypted Comparison Operators}
\label{sec:ComparisonOperators}
It is not possible to compare encrypted numbers with each other.
During the encryption of a number, the obfuscation is performed,
\deleted{(see \autoref{sec:PaillierCryptosystem})}
which randomly distributes the encrypted values between $0$ an $N^2-1$.
Therefore, the order of the encrypted values $\llbracket M \rrbracket$ has nothing to do with the order of the underlying numbers $M$ that were encrypted.
Consequently, operators such as lower than ($<$) or greater than ($>$) cannot provide a result that is meaningful for the numbers $M$, if they are applied to encrypted values $\llbracket M \rrbracket$.

We can also argue that comparison operators cannot exist if the Paillier cryptosystem is secure, since the existence of a comparison operator would break the security of the cryptosystem.
Consider a less-than comparison for example: if such a comparator could be implemented, every value could be decrypted within $\mathrm{log_2}(N)$ comparisons by a binary search.
For a modulus $N$ with a length of $2048$ bit, an attacker would need to encrypt and then compare only  $\mathrm{log_2}(2^{2048}) = 2048$ numbers with the encrypted value $\llbracket m \rrbracket$ in order to find the decrypted number $m$.
This would effectively break the security of the encryption scheme.

\subsection{Plaintext Exponent Does Not Leak Private Data}
\label{sec:FPNumbersSecurity}
At first glance, it may look like the floating-point representation (encrypted mantissa, plaintext exponent) we used will allow an attacker to obtain more important information than within an encoding where all number components are encrypted.
However, if it is implemented correctly, an attacker cannot take any advantage from this number representation.
\added{
	First, we will discuss this for the data in the server memory and, in the last paragraph, we will show how the exponent can be protected during the data transfer from the server to the client.
}

For the following, we will suppose a secure system with an at least 2048-bit long modulus $N$ and, therefore, a mantissa $\llbracket m \rrbracket$ with at least $600$ decimal digits usable in the plaintext domain.
Voxel values that are stored as 10 bit values are probably precise enough for most volume-rendering use cases.
To store numbers between $0$ and $2^{10} = 1024$, the exponent $e$ is not required at all, because the voxel information can be stored only in the mantissa $m$.
Therefore, the exponent $e$ can be $1$ for all voxels.
This means that the exponent does not even have to be transferred to the server, because the server can implicitly assume that the exponents of all numbers is $1$.
An addition of any of these numbers that have an exponent of $1$ does not change the exponent, because for an addition, the exponent needs to be taken into account only if the summands have different exponents (see \autoref{alg:PaillierFpAdd}).
Therefore, only a multiplication (e.g., an interpolation between voxel values) can change the exponent to anything other than $1$.
However, the Paillier cryptosystem only supports the multiplication of an encrypted number with an unencrypted number.
Consequently, the number $d$ that changes an exponent has to be unencrypted.
Furthermore, this number $d$ can only depend on unencrypted data, because Paillier does not support comparison operators (see \autoref{sec:ComparisonOperators}), which are required for flow control statements like \texttt{if} or \texttt{for-loops}, and arithmetic operations with an encrypted number will result in useless random noise, except those add ($\oplus$) and multiply ($\otimes$) that are defined for the Paillier cryptosystem%
.
Therefore, the number $d$ can only be the result of some computation with other unencrypted variables.
This implies that $d$ does not need to be encrypted, because everyone can calculate $d$ itself.
In other words, if the variable $d$ can be computed from some variables that need to be considered as publicly available, because they are unencrypted, it is pointless to encrypt $d$.
If $d$, which is unencrypted and can only depend on unencrypted data, influences an exponent, the exponent exposes only the information that is already publicly available.

The important observation here is that an unencrypted value (e.g., an exponent) can influence an encrypted value (e.g., a mantissa), but an encrypted value (e.g., a mantissa) cannot influence an unencrypted value (e.g., an exponent).
This means that no information that is only available as encrypted data can ever be exposed in unencrypted values like the exponent.

In our rendering system, a number $d$ that changes an exponent can either be the result of a computation with a constant or with an unencrypted number that is provided in unencrypted form to the rendering system, such as the camera properties (position of eye point, opening angle, view direction...).
Therefore, an attacker could possibly learn the constants used in our program code and data, such as the camera properties that are provided in the unencrypted form, from the exponents of the rendering result (the image).
However, we want to develop an approach that is open and semantically secure
\added{by design and not {\em secure through obscurity}}
(compare: \cite{article:hoepman2008securityThrough, NIST:800_123, web:BruceSchneierTheInsecurity2, web:ChadPerrinSecurity101}).
Therefore, we have to treat the source code of the application as publicly available, which means that a constant cannot be considered to be private.
Furthermore, for our approach, the camera properties need to be provided in an unencrypted form to the rendering system.
Therefore, we cannot consider it as private anyway.

It should be noted that the camera properties could possibly provide interesting information to an attacker, because it could be possible to learn something about the volume data by tracking the camera properties over time.
For instance, if a user rotates the camera around a specific region for a considerable amount of time, an attacker could guess that the region contains some interesting data.
During the transfer of the camera properties from the client to the server over the network, the camera properties could be secured by using an encrypted tunnel, such as {\em IPsec} \cite{tech:RFC4301.IPsec} or {\em TLS} \cite{tech:RFC8446.TLS}.
However, our basic assumption is that we cannot trust the server that hosts our rendering program.
This means that an attacker has access to the entire memory of the server and, therefore, can read the camera properties directly from the memory of the server, regardless of the used network transfer method.
While the unencrypted camera properties could
\replaced{indirectly expose some information}{be a security problem}, we will not discuss this further because it is beyond the scope of this work.

Based on the arguments stated in this section, we can conclude that using plaintext exponents for the rendering process on an untrusted computer system does not provide more information to a third party than using encryption for all components of a floating-point number.

The only remaining part that needs to be considered is the transfer of the final image from the server back to the client across a network.
Operations like trilinear interpolation will change the exponents during the rendering.
Therefore, the final image will contain floating-point numbers with exponents unequal to $1$ and, because the interpolation weights that change the exponents depend on the camera properties, the exponents of the final image will provide some information about the camera properties.
The privacy of the information that is stored in the exponents is only important if it can be assumed that the server is trustworthy, which contradicts the basic assumption of this work.
Therefore, this is somewhat beyond the scope of this work, but we nonetheless discuss it here for the sake of completeness.
In order to encrypt as much information as possible during the image transfer from the server to the client, ideally all information should be stored in the encrypted mantissa.
While it is not possible to divide an encrypted number, it is possible to multiply an encrypted number.
Furthermore, the encrypted mantissa can store numbers in the range from $0$ to $2^{2047}$.
Therefore, it is possible to bring all exponents to the value of the smallest exponent of any pixel of the final image.
This can be achieved by the calculation shown in \autoref{equ:decreaseExponentTo}.
For the new exponent $e_n$, the value of the smallest exponent of any pixel must be used.
If this exponent-decrease operation is applied to all image values on the server before transferring the image to the client, the exponent should not contain any important information during the transfer, because all exponents then contain the same value.
However, if there is concern that even this might contain something useful, it is possible to encrypt this exponent with the public key because the client that has the secure key can decrypt it anyway.
Since it is the same value for every number that is sent back to the client, this exponent needs to be sent and decrypted only once.

\section{Conclusions}
While the expressiveness of our renderings is far from what is possible with state-of-the-art algorithms for non-encrypted data, we have presented a highly parallelizable direct volume rendering approach that allows not only the outsourcing of the storage of the volume data, but also the outsourcing of the whole rendering pipeline, without compromising the privacy of the data.
The approach we propose does not leak any voxel values or any information computed from a voxel value after the volume encryption.
Since we encrypt every single bit of voxel data with Paillier's cryptosystem, which is provably semantically secure (see: \cite{pdf:Paillier, inbook:homoEncApps:2014:paillier}), it is rather obvious that with our approach, the confidentiality of the volume data (densities, shapes, structures,..) and the colors of the rendered image only depends on the privacy of the secure key.
If we trust all devices that have seen the volume data before encryption (e.g.,: MRI-/CT-scanner, the computer that performs the encryption) to safely delete the data after encryption, only the owner of the secure key is able to obtain any useful information of the encrypted volume or rendered images.
This is a significant advantage compared to all previous works to date.

This security naturally comes with associated costs. The storage overhead costs for computation are between four and five orders of magnitude compared to plaintext data.
Compared to our prototype, an optimized implementation of our approach can reduce the computational complexity by an order of magnitude.

While we hope that further improvements of our approach would lead to rendering results with better expressiveness, it will be a non-trivial task because the security aspect needs to be considered for even the slightest change.
Many of the ideas we considered in the algorithmic design eventually led to a leak of sensitive information, which is, in our opinion, intolerable, no matter how small it may be.
\added{Future work definitively needs to improve the rendering performance. We see that the performance can be tremendously accelerated, as ray-casting is an \emph{embarrassingly parallel} workload.
For practical utilization of our privacy-preserving volume rendering, an efficient GPU-based implementation would be necessary.
	A single server full with GPUs should be able to provide five orders of magnitude more computational power than a single CPU core can.
	Based on the measured performance with a non-optimized single threaded implementation, such a server could be able to achieve interactive frame rates for datasets that are small enough to fit into the memory of the graphics cards.
Therefore, we see, as a next step, to port the rendering onto GPUs, where the necessary technological piece will be to design efficient big-integer arithmetic. Another possible improvement within the scope of Paillier HE will be the visual quality of compositing. This can be done with gradient-magnitude opacity modulation, where the gradient magnitude will be pre-calculated and encrypted along with the data values. Such representation can already lead to substantial visual quality improvement, although it will still not reach the outcome of compositing using Porter/Duffs's over operator~\cite{Porter1984}. For the Paillier HE scheme, we do not see a way to implement the over operator compositing, as it requires a multiplication of encrypted numbers. To support alpha blending, new research should be oriented on investigating other homomorphic encryption schemes or a combination of those that, unlike Paillier, would support desired secure alpha blending functionality.}

\acknowledgments{
The authors wish to thank Michal Hojsík for his fruitful discussions on cryptography.
The authors would like to thank Michael Cusack from Publication Services at KAUST for proofreading.
The research was supported by King Abdullah University of Science and Technology (KAUST) under award number BAS/1/1680-01-01.}
\bibliographystyle{abbrv-doi-hyperref}


\section{Introduction}

The Ekert91 protocol~\cite{Ekert1991} exploits pairs of entangled states to exchange keys, and uses Bell inequalities to detect eavesdropping.  Some of the measurement results obtained by the two parties Alice and Bob are perfectly correlated, providing key bits.  Other measurement results must exhibit quantum behavior if there is no alteration of the quantum channel, and this permits to detect evesdropping by testing a Bell inequality violation.

The amount of quantum violation is an important characteristic in key distribution protocols because larger violations leads to a better noise resistance~\cite{ViolAndNoise}.   Some progress has been made to increase this amount of violation with the use of parties with higher dimension (\cite{ViolStrongThan} and~\cite{Q3}, for qutrits).  The choice of the Bell inequality used to detect evesdropping is another parameter which can be considered.

In their article introducing the 3DEB protocol~\cite{3DEB}, Durt, Cerf, Gisin and \^{Z}ukowski choosed to use three-dimensional quantum systems (qutrits), and the Bell inequality for qutrits named CHSH-3.   This way, they obtained better noise resistance than for the Ekert'91 protocol.

Our work makes one step further by using a recent discovered Bell inequality (here called hCHSH-3), which belongs to the family of homogeneous Bell inequalities introduced in~\cite{Arnault2012}.  The amount of violation which can be achieved with entangled states is even better than for CHSH-3. Consequently, the protocol we derive is more tolerant to noise, with a threshold of noise $F\simeq0.409$, instead of $F\simeq0.304$ for 3DEB.

Devices called multiport beam splitters~\cite{Tritters} (or tritters), are mentioned in~\cite{3DEB} as one way to handle measurements of qutrits.  Tritters are analyzed in~\cite{tritter1} and experimentally tested in~\cite{tritter2}.  Our new protocol h3DEB described in this article is analysed in view of the use of tritters to implement measurements.  A crucial point here will be that some products of observables, each implemented with tritters, can also be implemented by another single tritter.  This is needed for our protocol as the inequality hCHSH-3 involves such products.

The paper is organized as follows.
It begins with some reminders and precisions about measurements with tritters in Section~2.  Then Section~3 recalls the 3DEB protocol and the CHSH-3 Bell inequality used by it.   After that, Section~4 introduces the Bell inequality hCHSH-3 we use, then considers the use of tritters for implementing the product of observables, and defines our new protocol h3DEB.  Finally, the paper concludes about the advantage of h3DEB providing better resistance to noise.
\newpage

\section{Prerequisites}

The 3DEB protocol and our protocol use qutrits and trichotomic observables.  For readability, we
assume that the outcomes of these observables are $1,\omega,\omega^2$ where $\omega$ is the third
root of unity ${\omega=e^\frac{2i\pi}{3}}$.  The observables used by the two parties Alice and Bob
will be denoted respectively by $A_i$ and~$B_j$ for some indexes $i$ and~$j$.  We will also use the
correlation functions introduced in~\cite{tritter2}~:
$$
  E(A_iB_j) = \sum_{a,b=1,\omega,\omega^2}  P(A_i=a,B_j=b) \, ab.
$$

\subsection{Measurements with tritters}

A tritter is parameterized by a triplet $(\varphi_0,\varphi_1,\varphi_2)$ of phase shifts.  For
readability we put $\theta_j=\exp(i\varphi_j)$ (for $j=0,1,2$) and
$\Theta=(\theta_0,\theta_1,\theta_2)$.  The tritter performs over a qutrit the following unitary
transformation :
$$
  U_{\Theta} \mathrel{:=} HD_\Theta
  =
  \frac{1}{\sqrt3}\sum\limits_{k,l=0}^{2}\omega^{kl}\theta_{l}\ket{k}\bra{l}
$$
where the matrices $H$ and $D_\Theta$ are $H=(\omega^{kl})_{0\leq k,l\leq2}$ and
$D_\Theta=\mathop{\rm diag}(\theta_0,\theta_1,\theta_2)$.

In the specific and usual case where the phase shifts obey to the relation $\theta_{j}=\theta^{j}$, we have :
$$
 U_{\Theta}
 =\frac{1}{\sqrt3}\sum\limits_{k,l=0}^{2}\omega^{kl}\theta^{l}\ket{k}\bra{l}=\frac{1}{\sqrt3}\begin{pmatrix}
   1 & \theta & \theta^{2} \\
   1 & \omega\theta & \omega^{2}\theta^{2} \\
   1 & \omega^{2}\theta & \omega\theta^{2}
\end{pmatrix}
$$

  After the transformation performed by the tritter, a measurement is made using three detectors.
This measurement is represented by the observable
$$
  Z = \sum_{k=0}^{2} \omega^{k}\ket{k}\bra{k}.
$$
(Note that, as we assumed the three possible outcomes to be labeled by complex roots of unity, we
use unitary observables).  Thus, the measurement obtained by the combination of the tritter and
the detectors corresponds to the following observable

\begin{equation}
\begin{split}
Z_{\Theta} \mathrel{:=} D_{\Theta^{*}}H^{\dagger}ZHD_{\Theta}=\begin{pmatrix}
   0 & 0 & \theta_{2}\theta_{0}^{*} \\
   \theta_{0}\theta_{1}^{*} & 0 & 0 \\
   0 & \theta_{1}\theta_{2}^{*} & 0
\end{pmatrix}.
\end{split}
\label{eq:ZTheta}
\end{equation}

which gives us, in the particular case where $\theta_{j}=\theta^{j}$:

\begin{equation}
\begin{split}
Z_{\Theta}=\begin{pmatrix}
   0 & 0 & \theta^{2} \\
   \theta^{*} & 0 & 0 \\
   0 & \theta^{*} & 0
\end{pmatrix}.
\end{split}
\label{ref1}
\end{equation}

\section{The 3DEB protocol}

We will recall the 3DEB protocol introduced in~\cite{3DEB}.  We begin with the CHSH-3 inequality
(or CHSH for qutrits) as defined in~\cite{CHSH3, ViolCHSH2Q3}, which is used for 3DEB.

\subsection{The inequality CHSH-3}

The CHSH-3 inequality can be written
\begin{equation*}
S \leqslant 2
\end{equation*}
where
\begin{small}
$S=\text{Re}\big(E(A_{1}B_{1})+E(A_{1}B_{2})-E(A_{2}B_{1})+E(A_{2}B_{2})\big)\\
\phantom{espacegrand}+\frac{1}{\sqrt{3}}\text{Im}\big(E(A_{1}B_{1})-E(A_{1}B_{2})-E(A_{2}B_{1})+E(A_{2}B_{2})\big).$
\end{small}
\bigskip

 Some entangled states are known to violate this inequality.  The GHZ state
\begin{equation}
\begin{split}
\ket{\psi}=\frac{1}{\sqrt3}(\ket{00}+\ket{11}+\ket{22})
\label{GHZstate}
\end{split}
\end{equation}
is known to violate CHSH-3 with a violation factor (the quotient of the quantum value with the classical bound) ${v=(6+4\sqrt3)/9\simeq 1.436}$.

 This violation factor is considered very important for the security of the key distribution protocol.  The presence of noise is usually modelized by the replacement of the initial entangled state by
a mixture
$$
  F \frac Id + (1-F) \ket\psi\bra\psi
$$
where $F$ is the proportion of noise.  The point is that the presence of noise decreases the experienced violation to $(1-F)v$ and that the protocol is considered useless when the initial state entanglement cannot be detected anymore.  With this criterion, it has been shown that the protocol 3DEB is resistant to the presence of noise up to a threshold $F=1-1/v=(11-6\sqrt3)/2\simeq 0.304$~\cite{Q3, UptoN16, ViolCHSH3GHZ}.

  The bases considered in most papers~\cite{ADGL, Q3, UptoN16} to obtain these best violations are the following four ``optimal bases'' (two for each party) corresponding to tritter measurements using the following phase shift triples $\Theta=(1,\theta,\theta^2)$ were $\theta$ is a suitable
power of $\zeta=\e^{\frac{2i\pi}{12}}$:
\input ADGLbases
\begin{figure}[h]
\begin{minipage}[c]{0.5\linewidth}
\begin{align*}
  A_1: &\ (1, 1, 1) \\
  A_2: &\ (1, \zeta^2, \zeta^4) \\
  B_1: &\ (1, \zeta, \zeta^2) \\
  B_2: &\ (1, \zeta^{-1}, \zeta^{-2})
\end{align*}
\end{minipage}
\begin{minipage}[c]{0.5\linewidth}
\qquad\box\graph
\end{minipage}
\caption{\label{pic:hresult1} Representation of the four optimal bases in a plane parametrized by~$\theta$}
\end{figure}

  In~\cite{3DEB}, it is remarked that the phases used for $B_2$ can
be $(1,\zeta^3,\zeta^6)$ instead, as this change merely consists in reorder the three vectors of this base.

It was also remarked in~\cite{ADGL} that the non maximally entangled state
\begin{equation}
\frac{1}{\sqrt{2+\gamma^{2}}}(\ket{00}+\gamma\ket{11}+\ket{22})
\qquad\hbox{with $\gamma=\frac{\sqrt{11}-\sqrt{3}}{2}$}
\label{NMEstate}
\end{equation}
achieves an even better violation of CHSH-3, with a factor equal to ${\frac{1+\sqrt{\frac{11}{3}}}{2}\simeq 1.457}$.  This allows to reach a noise resistance up to the threshold $F\simeq 0.314$.

The expression of $S$ was rewritten in~\cite{Arnault2012} in the following form
\begin{equation}
S=-\frac{2}{9}\text{Re}(T)
\label{InegS}
\end{equation}
with
\begin{center}
$T=3\big((\omega^{2}-1)E(A_{1}^{2}B_{1}^{2}) + (\omega-1)E(A_{1}^{2}B_{2}^{2}) + (1-\omega^{2})E(A_{2}^{2}B_{1}^{2}) + (\omega^{2}-1)E(A_{2}^{2}B_{2}^{2})\big)$
\end{center}
Note that, the measurements outcomes being $1,\omega,\omega^2$, it is equivalent to consider the square of an observable and its complex conjugate.  For completeness, a derivation of this formula
is given in Appendix~A.   Now, the CHSH-3 inequality can be rewritten as
\begin{equation}
\text{Re}(-T) \leqslant 9.
\label{inegT}
\end{equation}
This last formulation will be useful to compare with the inequality we will use for our new
protocol.  Note that it was also shown in~\cite{Arnault2012} that the inequality obtained for qudits
($d\geq3$) in~\cite{ViolCHSH3GHZ} is the same as CHSH3 for the special case $d=3$.

\subsection{The 3DEB procedure}

Alice uses four observables $A_a$ with $a=0$ to 3, corresponding to tritter measurements
with phase shift triples $(1, \zeta^a, \zeta^{2a})$.  Bob use four observables $b_b$ with $b=0$ to
3, corresponding to tritter measurements with phase shift triples $(1,\zeta^{-b},\zeta^{-2b})$.
The following steps are repeated until Alice and Bob obtained a shared key of desired length.

\begin{enumerate}
\item Alice and Bob obtain an entangled pair of states in the GHZ state defined in~(\ref{GHZstate}).

\item Alice draws randomly a value for $a \in \{0,1,2,3\}$ and makes the measurement corresponding to the observable $A_{a}$ whereas Bob draws randomly a value for $b\in \{0,1,2,3\}$ and makes the measurement corresponding to the observable $B_{b}$.

\item When $a=b$, the results obtained by Alice and Bob are perfectly correlated. Indeed, the two
tritters used by Alice and Bob perform on the shared GHZ state the transformation ${(H \otimes H)(D_{\Theta} \otimes D_{\Theta^{*}})}$, with $\Theta=(1,\zeta^a,\zeta^{2a})$.  But it is easy to check that:
\begin{equation}
{(H \otimes H)(D_{\Theta} \otimes D_{\Theta^{*}})(\ket{00}+\ket{11}+\ket{22})=(\ket{00}+\ket{12}+\ket{21})}
\label{CollectTrits}
\end{equation}
Consequently, in this case where $a=b$, Alice and Bob obtain a new trit for the shared key.

\item When $(a,b) \in \{(0,1), (0,3), (2,1), (2,3)\}$, Alice and Bob can use their joint measurements to detect eavesdropping, because the four observables $A_{0}, A_{2}, B_{1}, B_{2}$ correspond to a configuration of maximal violation of CHSH-3.  The same is true when $(a,b)\in\{(1,0),(1,2),(3,0),(3,2)\}$.
\end{enumerate}

These different cases can be summarized in Table~1.
\begin{table}[h]
\begin{minipage}[h]{0.5\linewidth}
\centering
\begin{tabular}{c | c c c c}
\hline\hline
    & $B_{0}$ & $B_{1}$ & $B_{2}$ & $B_{3}$\\
\hline
$A_{0}$ & $k$ & $c_{1}$ & $ $ & $c_{1}$\\
$A_{1}$ & $c_{2}$ & $k$ & $c_{2}$ & $ $\\
$A_{2}$ & $ $ & $c_{1}$ & $k$ & $c_{1}$\\
$A_{3}$ & $c_{2}$ & $ $ & $c_{2}$ & $k$\\
\end{tabular}
\label{tab:hresult1}
\caption{Cases for the 3DEB protocol}
\end{minipage}
\hfil
\begin{minipage}[h]{0.45\linewidth}
$k$ : Alice and Bob obtain key trits.\par
$c_{i}$ : Alice and Bob obtain values for two sets of data which can be used to check CHSH-3 violation.
\end{minipage}
\end{table}

When used with the maximally entangled GHZ state $\ket{\psi}$, the violation factor of CHSH-3 observed using this protocol (in the absence of noise) is equal to ${v=\frac{6+4\sqrt3}{2}\simeq1.436}$.  This corresponds to a noise resistance up to a threshold $F \simeq 0.304$.  Our aim was to create a new protocol more tolerant to noise, with a threshold $F$ greater than $0.304$.

\section{The new h3DEB protocol}

We will now describe our protocol.  It achieves better noise resistance because it will use an
homogeneous Bell inequality, which has a larger violation factor than CHSH-3.

\subsection{The homogeneous inequality}

  We will use the inequality :
\begin{equation}
 -2\text{Re}(T_{1}) \leqslant 9
\label{HBI}
\end{equation}
with
$
 T_{1}
  = (4\omega+2)E(A_1^2B_1^2) + (\omega-1)E(A_1^2B_1B_2) + (4\omega-1)E(A_1^2B_2^2) \\
\phantom{espace}-(2\omega+1)E(A_1A_2B_1^2) + (\omega-1)E(A_1A_2B_1B_2) + (\omega+2)E(A_1A_2B_2^2) \\
\phantom{espace}+(\omega+5)E(A_2^2B_1^2) + (\omega-1)E(A_2^2B_1B_2) - (2\omega+4)E(A_2^2B_2^2). \\
$
\smallskip

This inequality belongs to the set of homogeneous Bell inequalities described in~\cite{Arnault2012}.
It has been shown in this paper that these inequalities are satisfied under the hypothesis of
local realism, and that they form a complete set.

A feature of the homogeneous Bell inequalities is that they involve some products (namely $A_1A_2$
and $B_1B_2$) of observables which become incompatible when considered as quantum observables.  The
outcomes of such a product of course cannot be meant to be the products of outcomes of incompatible
observables.  But if we use the unitary observables $Z_\Theta$ defined above for the $A_i$ and~$B_j$, the product of them is also a unitary observable which outcomes can be obtained with a single measurement.  Moreover, we argue here that this single measurement can be implemented by a slightly modified tritter.

\subsection{Products of incompatible observables}

 Suppose that we have two tritters, which implement the observables $Z_\Theta$ and~$Z_\Lambda$
described by Equation~(\ref{eq:ZTheta}), with $\Theta=(\theta_0,\theta_1,\theta_2)$ and
$\Lambda=(\lambda_0,\lambda_1,\lambda_2)$.  Then we need to implement the product observable
$Z_\Theta Z_\Lambda$.  But
\begin{small}
\begin{equation*}
\begin{split}
Z_\Theta Z_\Lambda=\begin{pmatrix}
   0 & 0 & \theta_{2}\theta_{0}^{*} \\
   \theta_{0}\theta_{1}^{*} & 0 & 0 \\
   0 & \theta_{1}\theta_{2}^{*} & 0
\end{pmatrix}
\begin{pmatrix}
   0 & 0 & \lambda_{2}\lambda_{0}^{*} \\
   \lambda_{0}\lambda_{1}^{*} & 0 & 0 \\
   0 & \lambda_{1}\lambda_{2}^{*} & 0
\end{pmatrix}
=
\begin{pmatrix}
   0 & \gamma_{0}^{*}\gamma_{1} & 0 \\
   0 & 0 & \gamma_{1}^{*}\gamma_{2} \\
   \gamma_{2}^{*}\gamma_{0} & 0 & 0
\end{pmatrix}
\end{split}
\end{equation*}
\end{small}
where
$$
  (\gamma_0,\gamma_1,\gamma_2)
  =
  (\theta_{2}^{*}\lambda_{1}^{*}, \theta_{0}^{*}\lambda_{2}^{*},  \theta_{1}^{*}\lambda_{0}^{*}).
$$
Hence, $Z_\Theta Z_\Lambda=Z_\Gamma^\dagger$ where $\Gamma$ has the components $(\gamma_0,\gamma_1,\gamma_2)$ just given.  From $Z_{\Gamma}=D_{\Gamma}^{*}H^{\dagger}ZHD_{\Gamma}$, we obtain ${Z_{\Theta}Z_{\Lambda}=Z_{\Gamma}^{\dagger}=D_{\Gamma}^{*}H^{\dagger}Z^{\dagger}HD_{\Gamma}}$. The product observable $Z_\Theta Z_\Lambda$ can consequently also be implemented by a tritter and a detector, but with the detector performing a measurement corresponding to the observable $Z^{\dagger}$ instead of $Z$.

  Violations of Inequality~(\ref{HBI}) by quantum states have been computed in~\cite{Arnault2012},
using observables (among them, product observables) obtained from the ones of
Figure~\ref{pic:hresult1}.  More precisely, the local realistic elements $A_1^2$, $A_1A_2$ and
$A_2^2$ have to be replaced by the three observables
$$
  Z_{\Theta_A}^*, \quad Z^\dagger_{\Gamma_A},\quad  Z_{\Lambda_A}^*
$$
where the $Z_{\Gamma_A}^\dagger$ is a product observable as just described (and $\Theta_A$, $\Lambda_A$ are the parameters corresponding to the configuration of Figure~\ref{pic:hresult1}).  Similarily,
the party Bob has to use three  observables $Z_{\Theta_B}^*$, $Z^\dagger_{\Gamma_B}$, $Z_{\Lambda_B}^*$.

  As mentionned in~\cite{Arnault2012}, the violation factor obtained with the bases obtained from
Figure~\ref{pic:hresult1} and the GHZ state is $v\simeq1.693$.  Hence, this amount of violation can be
observed using six tritters with detectors.

\subsection{The h3DEB procedure}

As for 3DEB, we denote $A_i$ with $i=0,1,2,3$ the observable parameterized by phase shift triple $(1,\zeta^i,\zeta^{2i})$, and $B_j$ with $j=0,1,2,3$ the observable parameterized by $(1,\zeta^{-j},\zeta^{-2j})$.
\medskip

For each pair $ij$ in the set $\mathcal{C}=\{00,02,22,11,13,33\}$, we note now $A_{ij}$ the product observable $A_{i}A_{j}$ (which is expected to be implemented with a single tritter).

\begin{enumerate}
\item Alice and Bob obtain an entangled pair of states in the GHZ state.
\item Alice draws randomly a value of $ij$ in $\mathcal{C}$ and performs her measurement in the basis associated to the observable $A_{ij}$ whereas Bob draws randomly a value of $kl$ in $\mathcal{C}$ and performs his measurement in the basis associated to $B_{kl}$.
\item When the pairs $ij$ and $kl$ are equal, the triplets of phase shifts $(\theta_{0}, \theta_{1}, \theta_{2})$ and $(\theta_{0}^{*}, \theta_{1}^{*}, \theta_{2}^{*})$ corresponding to the observables $A_{ij}$ and $B_{kl}$ are related each other by complex conjugation.   Equation~(\ref{CollectTrits}), which has yet been used in the special case where $\theta_j=\theta^j$ for some $\theta$, remains true in the present slightly more general case.  Thus Alice and Bob obtain a new trit for the shared key.
\item For some choices of pairs $ij$ and $kl$ (see the table below) Alice and Bob collect the issues
of their measurements, in order to detect eavesdropping.  Indeed, these pairs correspond to two
configurations of maximal violation of the homogeneous Bell inequality hCHSH-3 given by
Equation~(\ref{HBI}).
\end{enumerate}

These different cases can be summarized in the table :

\begin{table}[h]
\begin{minipage}[h]{0.57\linewidth}
\centering
\begin{tabular}{c | c c c c c c}
\hline\hline
    & $B_{00}$ & $B_{02}$ & $B_{22}$ & $B_{11}$ & $B_{13}$ & $B_{33}$\\
\hline
$A_{00}$ & $k$ &   &    &  $c_{1}$  & $c_{1}$ & $c_{1}$  \\
$A_{02}$ &  &  $k$ &    & $c_{1}$   & $c_{1}$ & $c_{1}$  \\
$A_{22}$ &  &   &   $k$ &   $c_{1}$ & $c_{1}$ & $c_{1}$  \\
$A_{11}$ & $c_{2}$ &  $c_{2}$ &  $c_{2}$ &   $k$ &  &   \\
$A_{13}$ & $c_{2}$ &  $c_{2}$ &  $c_{2}$  &    & $k$ &   \\
$A_{33}$ & $c_{2}$ &  $c_{2}$ &   $c_{2}$ &    &  & $k$  \\
\end{tabular}
\label{tab:hresult2}
\caption{Cases for the h3DEB protocol}
\end{minipage}
\hfill
\begin{minipage}[h]{0.4\linewidth}
$k$ : Alice and Bob obtain a key trit.\par
$c_{i}$ : Alice and Bob collect values for two sets of data which will be used to check hCHSH-3 violation.
\end{minipage}
\end{table}
\vskip-1cm\hbox{}

\subsection{Resistance to noise}

Without the presence of noise, the violation of the inequality hCHSH-3 observed with this protocol
is~$v\simeq1.693$.  By the same argument as the one used for the resistance of 3DEB, we obtain
that our protocol is resistant to noise up to a threshold $F=1-\frac{1}{v} \simeq 0.409$.  This
is better than the resistance of the 3DEB protocol, even when the latter uses the non maximally
entangled state~(\ref{NMEstate}).

\section{Conclusion}

Our goal was to improve the noise resistance of the qutrits key distribution protocols.  By using the homogeneous Bell inequality hCHSH-3 which reaches a violation factor $v\simeq1.693$ with the GHZ state, better than for CHSH-3, we obtain a threshold of noise resistance $F\simeq0.409$,  better than the threshold $\simeq0.304$ obtained for 3DEB using the GHZ state~\cite{UptoN16, ViolCHSH3GHZ} and even to the threshold ${\simeq0.341}$ resulting of the use of the non maximally entangled state~(\ref{NMEstate}).

As our inequality hCHSH-3 involves products of observables which become incompatibles for quantum
states, an important fact is the
possibility to implement with slightly modified tritters the single observable corresponding to
these products.  This can be done by replacing the final measurement with observable
$Z=\mathop{\rm diag}(1,\omega,\omega^2)$ by a measurement with observable
$Z^\dagger=\mathop{\rm diag}(1,\omega^2,\omega)$.  Physically, this replacement corresponds just to a permutation of the detectors.

  The gain in noise resistance of our protocol over 3DEB is due to the use of the inequality hCHSH3.
This inequality detects violations of local realism when some measurements are multiplicatively
related.  By using tritters measurements which respect this multiplicative constraints, the parties
running the protocol are able to exploit its larger violation capabilities.

\bigskip
\noindent{\bf Acknowledgement}.  One of the author (Z.A.) was partially supported by Thales Alenia
Space during this work.
\newpage

\bibliographystyle{unsrt}