text:
# 1 Intro

The viral quasispecies is currently defined as a collection of closely related viral genomes undergoing a continuous process of genetic variation, competition between the variants generated, and selection of the fittest genomes in a given environment. (Domingo, Sheldon, and Perales 2012)

The high replication error rate that generates this quasispecies is due to a lack of genetic proofreading mechanisms, and it is estimated that for viruses with typically high replicative rates, every possible point mutation and many double mutations are generated with each viral replication cycle, and these may be present within the population at any time.

Quasispecies complexity can explain or predict the behavior of a virus; hence, it has an obvious interest for clinical reasons. We are often interested in comparing the viral diversity indices between sequential samples from a single patient or between samples from different groups of patients. These comparisons can provide information on the patient’s clinical progression or the appropriateness of a given treatment. (Gregori et al. 2016) (Gregori et al. 2014)

QSUtils is a package intended for use with quasispecies amplicon data obtained by NGS, but it could also be useful for analyzing 16S/18S ribosomal-based metagenomics or tumor genetic diversity by amplicons.

In this tutorial, we illustrate use of the functions provided in the package to explore and manipulate sequence alignments, convert reads to haplotypes and frequencies, repair reads, intersect strand haplotypes, and visualize haplotype alignments.

# 2 Install package

library(Biostrings)
library(ape)
library(ggplot2)

BiocManager::install("QSutils")
library(QSutils)

# 3 Data

The package contains functions that work on quasispecies data, defined by an alignment of haplotypes and their frequencies. Data are loaded from fasta formatted files, where the header of each sequence describes the ID of a haplotype and its corresponding frequency in the quasispecies population. These two pieces of information are separated by a vertical bar ‘|’. When frequency information is missing, each sequence is considered as a single read.

filepath<-system.file("extdata","ToyData_10_50_1000.fna", package="QSutils")
cat(readLines(filepath) , sep = "\n")
## >Hpl_0_0001|464|46.4
## CACCCTGTGACCAGTGTGTGCGAGTTTGGTGCCCCGTTGGCATTGACTAT
## >Hpl_1_0001|62|6.2
## CACTCTGTGACCAGTGTGTGCGAGTTTGGTGCCCCGTTGGCATTGACTAT
## >Hpl_1_0002|39|3.9
## CACCCTGTGACCAGCGTGTGCGAGTTTGGTGCCCCGTTGGCATTGACTAT
## >Hpl_1_0003|27|2.7
## CACCCTGTGACCAGTGTGTGCGAGTTTGGTGCCCCGTTGGCATTGACTAC
## >Hpl_2_0001|37|3.7
## CACTCTGTGACCAGTGTGTGCGAGTTTGGTGCCCCGTTAGCATTGACTAT
## >Hpl_4_0001|16|1.6
## CACTCGGTGACCAGTGTGCGTGAGTTTGGTGCCCCGTTGGCATTGACTAT
## >Hpl_5_0001|33|3.3
## CACTCTGTGACCAGTGTGCGCGAGTTTAGTGCCCTGTCGGCATTGACTAT
## >Hpl_8_0001|54|5.4
## CACTCTGTGATCAGTGTGCGCGAGTTTAGTGCCCTGCCGGCATCGACTAT
## >Hpl_9_0001|248|24.8
## CACTCTGTGATCAGTGTGCGCGAGTTTAGTGCCCTGCCGGCATCGACTAC
## >Hpl_10_0001|20|2
## CACTCTGTGATCAGTGTGCGCGAGTTTAGTGCCCTGCCGGCACCGACTAC

The ReadAmplSeqs function loads the data from the fasta file and returns a list with two elements: the DNAStringSet hseqs with haplotype sequences, and the vector of counts, nr.

filepath<-system.file("extdata","ToyData_10_50_1000.fna", package="QSutils")
lst
## $nr ## [1] 464 62 39 27 37 16 33 54 248 20 ## ##$hseqs
## DNAStringSet object of length 10:
##      width seq                                              names
##  [1]    50 CACCCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_0_0001|464|46.4
##  [2]    50 CACTCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_1_0001|62|6.2
##  [3]    50 CACCCTGTGACCAGCGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_1_0002|39|3.9
##  [4]    50 CACCCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAC Hpl_1_0003|27|2.7
##  [5]    50 CACTCTGTGACCAGTGTGTGCGA...GTGCCCCGTTAGCATTGACTAT Hpl_2_0001|37|3.7
##  [6]    50 CACTCGGTGACCAGTGTGCGTGA...GTGCCCCGTTGGCATTGACTAT Hpl_4_0001|16|1.6
##  [7]    50 CACTCTGTGACCAGTGTGCGCGA...GTGCCCTGTCGGCATTGACTAT Hpl_5_0001|33|3.3
##  [8]    50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCATCGACTAT Hpl_8_0001|54|5.4
##  [9]    50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCATCGACTAC Hpl_9_0001|248|24.8
## [10]    50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCACCGACTAC Hpl_10_0001|20|2

The GetQSData function loads the data from the fasta file, removes haplotypes with relative abundances below a given threshold, and sorts the remaining haplotypes, first by an increasing number of mutations with respect to the dominant haplotype and then by decreasing frequencies within the number of mutations.

filepath<-system.file("extdata","ToyData_10_50_1000.fna", package="QSutils")
lstG <- GetQSData(filepath,min.pct= 2,type="DNA")
lstG
## $seqs ## DNAStringSet object of length 9: ## width seq names ## [1] 50 CACCCTGTGACCAGTGTGTGCGA...GGTGCCCCGTTGGCATTGACTAT Hpl_0_0001 ## [2] 50 CACTCTGTGACCAGTGTGTGCGA...GGTGCCCCGTTGGCATTGACTAT Hpl_1_0001 ## [3] 50 CACCCTGTGACCAGCGTGTGCGA...GGTGCCCCGTTGGCATTGACTAT Hpl_1_0002 ## [4] 50 CACCCTGTGACCAGTGTGTGCGA...GGTGCCCCGTTGGCATTGACTAC Hpl_1_0003 ## [5] 50 CACTCTGTGACCAGTGTGTGCGA...GGTGCCCCGTTAGCATTGACTAT Hpl_2_0001 ## [6] 50 CACTCTGTGACCAGTGTGCGCGA...AGTGCCCTGTCGGCATTGACTAT Hpl_5_0001 ## [7] 50 CACTCTGTGATCAGTGTGCGCGA...AGTGCCCTGCCGGCATCGACTAT Hpl_8_0001 ## [8] 50 CACTCTGTGATCAGTGTGCGCGA...AGTGCCCTGCCGGCATCGACTAC Hpl_9_0001 ## [9] 50 CACTCTGTGATCAGTGTGCGCGA...AGTGCCCTGCCGGCACCGACTAC Hpl_10_0001 ## ##$nr
## [1] 464  62  39  27  37  33  54 248  20
##
## $nm ## [1] 0 1 1 1 2 5 8 9 10 Note that the haplotype present in an amount below 2% of the total population has now been removed. ### 3.0.1 Collapsing reads to haplotypes Although ReadAmplSeqs can read fasta files that have no frequency information, it is more efficient to use Biostrings::readDNAStringSet directly. Then reads can be converted to haplotypes and frequencies with the help of function Collapse. filepath<-system.file("extdata","Toy.GapsAndNs.fna", package="QSutils") reads <- readDNAStringSet(filepath) reads ## DNAStringSet object of length 100: ## width seq names ## [1] 50 TGACGCGCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGNCGC ## [2] 50 TGACGCGCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGTCGC ## [3] 50 TGACGCGCACAGAGTGCTGCTA...TGGGNTACCCCGTCGTGGTCGC ## [4] 50 TGACGCGCACAGAGTGCTGCT-...TGGGTTACCCCGTCGTGGTCGC ## [5] 50 TGACGCGCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGTCGC ## ... ... ... ## [96] 50 TGACGCNCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGTCGC ## [97] 50 TGACGCGCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGTCGC ## [98] 50 TGACGCGCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGTCGC ## [99] 50 TGACGCGCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGTCGC ## [100] 50 TGACGCGCACAGAGTGCTGCTA...TGGGTTACCCCGTCGTGGTCGC lstCollapsed <- Collapse(reads) str <- DottedAlignment(lstCollapsed$hseqs)
data.frame(Hpl=str,nr=lstCollapsednr) ## Hpl nr ## 1 TGACGCGCACAGAGTGCTGCTAAATGACTGGGTTACCCCGTCGTGGTCGC 61 ## 2 ......-........................................... 2 ## 3 .....................-............................ 2 ## 4 ........................-......................... 2 ## 5 ........................................-......... 2 ## 6 ..........................................-....... 2 ## 7 ......N........................................... 2 ## 8 -.......................................-......... 1 ## 9 .-................................................ 1 ## 10 ..-...........................-................... 1 ## 11 ....-.......-................-.................... 1 ## 12 ..........-....................................... 1 ## 13 ...............-.................................. 1 ## 14 ......................-.........N................. 1 ## 15 ........................N...........-.....N....... 1 ## 16 ..........................-....................... 1 ## 17 ...........................-..-................... 1 ## 18 ...............................-.................. 1 ## 19 ................................N................. 1 ## 20 ..............................................N... 1 ## 21 ................................................-. 1 ## 22 ................................................N. 1 ## 23 ..........................................N....... 1 ## 24 ..........................................N......N 1 ## 25 .........................................N........ 1 ## 26 ..................................N.N............. 1 ## 27 ...............................N.-................ 1 ## 28 ...........................N...................... 1 ## 29 .........................N........................ 1 ## 30 ......................N........................... 1 ## 31 ................N................................. 1 ## 32 .............N......................N............. 1 ## 33 ........N......................................... 1 ## 34 .....N............................................ 1 Aligned raw reads may contain missing information in the form of gaps, noted as ‘-’, or indeterminates, noted as ‘N’. CorrectGapsAndNs returns the alignment with these positions corrected based on the reference sequence, in this case the dominant haplotype. lstCorrected<-CorrectGapsAndNs(lstCollapsedhseqs[2:length(lstCollapsed$hseqs)], lstCollapsed$hseqs[[1]])
#Add again the most abundant haplotype.
lstCorrected<- c(lstCollapsedhseqs[1],lstCorrected) lstCorrected ## DNAStringSet object of length 34: ## width seq names ## [1] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 1 ## [2] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 2 ## [3] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 3 ## [4] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 4 ## [5] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 5 ## ... ... ... ## [30] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 30 ## [31] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 31 ## [32] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 32 ## [33] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 33 ## [34] 50 TGACGCGCACAGAGTGCTGCTAA...TGGGTTACCCCGTCGTGGTCGC 34 After these corrections, some sequences may be duplicated, so it is useful to recollapse the alignment to obtain corrected haplotypes with updated frequencies. lstRecollapsed<-Recollapse(lstCorrected,lstCollapsednr)
lstRecollapsed
## $nr ## [1] 100 ## ##$seqs
## DNAStringSet object of length 1:
##     width seq                                               names
## [1]    50 TGACGCGCACAGAGTGCTGCTAA...CTGGGTTACCCCGTCGTGGTCGC 1

### 3.0.2 Forward and reverse strand haplotype intersection

A key step in error correction with amplicon NGS is selecting haplotypes above a minimum frequency represented in both strands. In the next example we load forward and reverse haplotypes from two separate fasta files.

cat("Reads: ",sum(lstFW$nr),", Haplotypes: ",length(lstFW$nr),"\n",sep="")
## Reads: 63736, Haplotypes: 1153
cat("Reads: ",sum(lstRV$nr),", Haplotypes: ",length(lstRV$nr),"\n",sep="")
## Reads: 65520, Haplotypes: 1162

Haplotypes in each strand that do not reach a minimum frequency of 0.1% are then removed, and haplotypes above this frequency and common to both strands are then selected and their frequencies updated by the function IntersectStrandHpls.

lstI <- IntersectStrandHpls(lstFW$nr,lstFW$hseqs,lstRV$nr,lstRV$hseqs)

cat("FW and Rv total reads:",sum(lstFW$nr)+sum(lstRV$nr),"\n")
cat("FW and Rv reads above thr:",sum(lstI$pFW)+sum(lstI$pRV),"\n")
cat("FW haplotypes above thr:",sum(lstFW$nr/sum(lstFW$nr)>0.001),"\n")
cat("RV haplotypes above thr:",sum(lstRV$nr/sum(lstRV$nr)>0.001),"\n")
cat("\n")
cat("Reads in FW unique haplotypes:",sum(lstI$pFW[lstI$pRV==0]),"\n")
cat("Reads in RV unique haplotypes:",sum(lstI$pRV[lstI$pFW==0]),"\n")
cat("\n")
cat("Reads in common:",sum(lstI$nr),"\n") cat("Haplotypes in common:",length(lstI$nr),"\n")
## FW and Rv total reads: 129256
## FW and Rv reads above thr: 99982
## FW haplotypes above thr: 35
## RV haplotypes above thr: 35
##
## Reads in FW unique haplotypes: 874
## Reads in RV unique haplotypes: 1047
##
## Reads in common: 98061
## Haplotypes in common: 1

## 3.1 Simulate quasispecies data

Several functions in this package enable simulation of quasispecies data. This is useful for various proposes, such as comparing data and testing diversity indices. The vignette Simulating Quasispecies Composition provides examples of such data simulation.

# 4 Quasispecies data exploration

Let’s load a toy data on which to exemplify different tasks and functions.

filepath<-system.file("extdata","ToyData_10_50_1000.fna", package="QSutils")
lst
## $nr ## [1] 464 62 39 27 37 16 33 54 248 20 ## ##$hseqs
## DNAStringSet object of length 10:
##      width seq                                              names
##  [1]    50 CACCCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_0_0001|464|46.4
##  [2]    50 CACTCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_1_0001|62|6.2
##  [3]    50 CACCCTGTGACCAGCGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_1_0002|39|3.9
##  [4]    50 CACCCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAC Hpl_1_0003|27|2.7
##  [5]    50 CACTCTGTGACCAGTGTGTGCGA...GTGCCCCGTTAGCATTGACTAT Hpl_2_0001|37|3.7
##  [6]    50 CACTCGGTGACCAGTGTGCGTGA...GTGCCCCGTTGGCATTGACTAT Hpl_4_0001|16|1.6
##  [7]    50 CACTCTGTGACCAGTGTGCGCGA...GTGCCCTGTCGGCATTGACTAT Hpl_5_0001|33|3.3
##  [8]    50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCATCGACTAT Hpl_8_0001|54|5.4
##  [9]    50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCATCGACTAC Hpl_9_0001|248|24.8
## [10]    50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCACCGACTAC Hpl_10_0001|20|2

The ConsSeq function returns the consensus sequence resulting from an alignment. This function does not consider IUPAC ambiguity codes, and when there is a tie, the consensus nucleotide is decided randomly.

ConsSeq(lst$hseqs) ## [1] "CACTCTGTGACCAGTGTGCGCGAGTTTGGTGCCCCGTTGGCATTGACTAT" To visualize the differences between haplotypes that comprise the quasispecies, the DottedAlignment function returns a vector of character strings, one for each haplotype, where a dot is shown to represent a conserved site with respect to the dominant haplotype. DottedAlignment(lst$hseqs)
##                                  Hpl_0_0001|464|46.4
## "CACCCTGTGACCAGTGTGTGCGAGTTTGGTGCCCCGTTGGCATTGACTAT"
##                                    Hpl_1_0001|62|6.2
## "...T.............................................."
##                                    Hpl_1_0002|39|3.9
## "..............C..................................."
##                                    Hpl_1_0003|27|2.7
## ".................................................C"
##                                    Hpl_2_0001|37|3.7
## "...T..................................A..........."
##                                    Hpl_4_0001|16|1.6
## "...T.G............C.T............................."
##                                    Hpl_5_0001|33|3.3
## "...T..............C........A......T..C............"
##                                    Hpl_8_0001|54|5.4
## "...T......T.......C........A......T.CC.....C......"
##                                  Hpl_9_0001|248|24.8
## "...T......T.......C........A......T.CC.....C.....C"
##                                     Hpl_10_0001|20|2
## "...T......T.......C........A......T.CC....CC.....C"

The output of ReadAmplSeqs can be sorted by the number of mutations with regard to the most abundant haplotype using the function SortByMutations, in which the first haplotype is the most similar one and the last haplotype is the one with the largest number of mutations. This function also returns a vector with the number of mutations in each haplotype.

lstSorted<-SortByMutations(lst$hseqs,lst$nr)
lstSorted
## $bseqs ## DNAStringSet object of length 10: ## width seq names ## [1] 50 CACCCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_0_0001 ## [2] 50 CACTCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_1_0001 ## [3] 50 CACCCTGTGACCAGCGTGTGCGA...GTGCCCCGTTGGCATTGACTAT Hpl_1_0002 ## [4] 50 CACCCTGTGACCAGTGTGTGCGA...GTGCCCCGTTGGCATTGACTAC Hpl_1_0003 ## [5] 50 CACTCTGTGACCAGTGTGTGCGA...GTGCCCCGTTAGCATTGACTAT Hpl_2_0001 ## [6] 50 CACTCGGTGACCAGTGTGCGTGA...GTGCCCCGTTGGCATTGACTAT Hpl_4_0001 ## [7] 50 CACTCTGTGACCAGTGTGCGCGA...GTGCCCTGTCGGCATTGACTAT Hpl_5_0001 ## [8] 50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCATCGACTAT Hpl_8_0001 ## [9] 50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCATCGACTAC Hpl_9_0001 ## [10] 50 CACTCTGTGATCAGTGTGCGCGA...GTGCCCTGCCGGCACCGACTAC Hpl_10_0001 ## ##$nr
##  [1] 464  62  39  27  37  16  33  54 248  20
##
## $nm ## [1] 0 1 1 1 2 4 5 8 9 10 The frequencies of nucleotides or amino acids at each position can be computed with the function FreqMat. FreqMat(lst$hseqs)
##    1  2  3 4  5 6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
## A  0 10  0 0  0 0  0  0  0 10  0  0 10  0  0  0  0  0  0  0  0  0 10  0  0  0
## C 10  0 10 3 10 0  0  0  0  0  7 10  0  0  1  0  0  0  5  0  9  0  0  0  0  0
## G  0  0  0 0  0 1 10  0 10  0  0  0  0 10  0 10  0 10  0 10  0 10  0 10  0  0
## T  0  0  0 7  0 9  0 10  0  0  3  0  0  0  9  0 10  0  5  0  1  0  0  0 10 10
##   27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
## A  0  4  0  0  0  0  0  0  0  0  0  0  1  0  0 10  0  0  0 10  0  0 10  0
## C  0  0  0  0  0 10 10 10  6  0  3  4  0  0 10  0  1  3  0  0 10  0  0  3
## G  0  6 10  0 10  0  0  0  0 10  0  0  9 10  0  0  0  0 10  0  0  0  0  0
## T 10  0  0 10  0  0  0  0  4  0  7  6  0  0  0  0  9  7  0  0  0 10  0  7

To take into account the abundance of each haplotype when computing the mutation frequency, the haplotype abundances are passed to the function that computes the same matrix, but with the abundances.

FreqMat(lst$hseqs,lst$nr)
##      1    2    3   4    5   6    7    8    9   10  11   12   13   14  15   16
## A    0 1000    0   0    0   0    0    0    0 1000   0    0 1000    0   0    0
## C 1000    0 1000 530 1000   0    0    0    0    0 678 1000    0    0  39    0
## G    0    0    0   0    0  16 1000    0 1000    0   0    0    0 1000   0 1000
## T    0    0    0 470    0 984    0 1000    0    0 322    0    0    0 961    0
##     17   18  19   20  21   22   23   24   25   26   27  28   29   30   31   32
## A    0    0   0    0   0    0 1000    0    0    0    0 355    0    0    0    0
## C    0    0 371    0 984    0    0    0    0    0    0   0    0    0    0 1000
## G    0 1000   0 1000   0 1000    0 1000    0    0    0 645 1000    0 1000    0
## T 1000    0 629    0  16    0    0    0 1000 1000 1000   0    0 1000    0    0
##     33   34  35   36  37  38  39   40   41   42  43  44   45   46   47   48
## A    0    0   0    0   0   0  37    0    0 1000   0   0    0 1000    0    0
## C 1000 1000 645    0 322 355   0    0 1000    0  20 322    0    0 1000    0
## G    0    0   0 1000   0   0 963 1000    0    0   0   0 1000    0    0    0
## T    0    0 355    0 678 645   0    0    0    0 980 678    0    0    0 1000
##     49  50
## A 1000   0
## C    0 295
## G    0   0
## T    0 705

We may be interested only in mutated positions, so with MutsTbl the matrix obtained reports only the frequency of the mutated nucleotide or amino acid per position.

MutsTbl(lst$hseqs) ## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 ## A 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 ## C 0 0 0 3 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ## G 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ## T 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 5 0 1 0 0 0 0 0 0 0 0 ## 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 ## A 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 ## C 0 0 0 0 0 0 0 3 4 0 0 0 0 1 3 0 0 0 0 0 3 ## G 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ## T 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 As can be done with FreqMat, with MutsTbl the abundances of the haplotypes can be passed to the function to obtain a mutation table with abundance information. MutsTbl(lst$hseqs, lstnr) ## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 ## A 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ## C 0 0 0 0 0 0 0 0 0 0 0 0 0 0 39 0 0 0 371 0 0 0 0 0 0 0 0 ## G 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ## T 0 0 0 470 0 0 0 0 0 0 322 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 ## 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 ## A 355 0 0 0 0 0 0 0 0 0 0 37 0 0 0 0 0 0 0 0 0 0 0 ## C 0 0 0 0 0 0 0 0 0 322 355 0 0 0 0 20 322 0 0 0 0 0 295 ## G 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ## T 0 0 0 0 0 0 0 355 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 When the sequences are particularly large, the function SummaryMuts computes a table showing the polymorphic positions in the alignment and the frequency of each nucleotide or amino acid observed. SummaryMuts(lsthseqs,lst$nr,off=0) ## pos A C G T ## 4 4 0 530 0 470 ## 6 6 0 0 16 984 ## 11 11 0 678 0 322 ## 15 15 0 39 0 961 ## 19 19 0 371 0 629 ## 21 21 0 984 0 16 ## 28 28 355 0 645 0 ## 35 35 0 645 0 355 ## 37 37 0 322 0 678 ## 38 38 0 355 0 645 ## 39 39 37 0 963 0 ## 43 43 0 20 0 980 ## 44 44 0 322 0 678 ## 50 50 0 295 0 705 Then, the PolyDist function can be used to obtain the fraction of substitutions by polymorphic site in a simpler manner. This function can be used either with or without the vector of abundances. PolyDist(lst$hseqs,lst$nr) ## 4 6 11 15 19 21 28 35 37 38 39 43 44 ## 0.470 0.016 0.322 0.039 0.371 0.016 0.355 0.355 0.322 0.355 0.037 0.020 0.322 ## 50 ## 0.295 PolyDist(lst$hseqs)
##   4   6  11  15  19  21  28  35  37  38  39  43  44  50
## 0.3 0.1 0.3 0.1 0.5 0.1 0.4 0.4 0.3 0.4 0.1 0.1 0.3 0.3

To summarize the mutation information and compute the coverage of each mutation, the ReportVariants function is used. This function requires a reference sequence, which in some cases could be the dominant haplotype.

ReportVariants(lst$hseqs[2:length(lst$hseqs)],lst$hseqs[[1]],lst$nr)
##    WT Pos Var Cov
## 1   C   4   T 879
## 2   T   6   G  37
## 3   C  11   T 335
## 4   T  15   C  62
## 5   T  19   C 388
## 6   C  21   T  37
## 7   G  28   A 351
## 8   C  35   T 351
## 9   T  37   C 335
## 10  T  38   C 351
## 11  G  39   A  27
## 12  T  43   C 248
## 13  T  44   C 335
## 14  T  50   C 341

Another way to explore the positions with mutations is by computing a matrix with the information content (IC) of each position using GetInfProfile. If the sample is DNA, the maximum IC is 2, whereas when working with amino acids, the maximum is 4.32.

GetInfProfile(lst$hseqs,lst$nr)
##        1        2        3        4        5        6        7        8
## 2.000000 2.000000 2.000000 1.002598 2.000000 1.881650 2.000000 2.000000
##        9       10       11       12       13       14       15       16
## 2.000000 2.000000 1.093457 2.000000 2.000000 2.000000 1.762312 2.000000
##       17       18       19       20       21       22       23       24
## 2.000000 2.000000 1.048563 2.000000 1.881650 2.000000 2.000000 2.000000
##       25       26       27       28       29       30       31       32
## 2.000000 2.000000 2.000000 1.061546 2.000000 2.000000 2.000000 2.000000
##       33       34       35       36       37       38       39       40
## 2.000000 2.000000 1.061546 2.000000 1.093457 1.061546 1.771636 2.000000
##       41       42       43       44       45       46       47       48
## 2.000000 2.000000 1.858559 1.093457 2.000000 2.000000 2.000000 2.000000
##       49       50
## 2.000000 1.124907

And this can be plotted:

dplot <- data.frame(IC=GetInfProfile(lst$hseqs,lst$nr),
pos=1:width(lst$hseqs)[1]) ggplot(dplot, aes(x=pos, y=IC)) + geom_point() + scale_x_continuous(minor_breaks = 1:nrow(dplot), breaks = 1:nrow(dplot)) + theme(axis.text.x = element_text(angle=45)) # 5 Quasispecies complexity by biodiversity indices There is a vignette that deepens with the diversity indices of the quasispecies: Characterizing viral quasispecies is also available in this package. # 6 Genotyping Another interesting procedure is to genotype an unknown sample. To that end, a set of reference sequences is needed for each genotype. A minimum of five well characterized sequences by genotype is suggested. The sets are supposed to be representative of each genotype and will provide an estimate of within genotype variance. Function DBrule, used in genotyping, takes into account the distance between the target haplotype and each genotype and the within genotype variability. The first step is to load the target haplotype to be genotyped with ReadAmplSeqs, as is shown: filepath<-system.file("extdata","Unknown-Genotype.fna", package="QSutils") lst2Geno <- ReadAmplSeqs(filepath,type="DNA") hseq <- lst2Geno$hseq[1]
hseq
## DNAStringSet object of length 1:
##     width seq                                               names
## [1]   285 CACGTCGCATGGAGACCACCGTG...GTTCAAGCCTCCAAGCTGTGCCT Hpl.0.0001|18|100

The reference genotype sequences are then loaded using the same procedure.

filepath<-system.file("extdata","GenotypeStandards_A-H.fas", package="QSutils")
RefSeqs <- lstRefs$hseq { cat("Number of reference sequences by genotype:\n") print(table(substr(names(RefSeqs),1,1))) } ## Number of reference sequences by genotype: ## ## A B C D E F G H ## 9 15 18 15 6 7 5 7 Next, the distances between the target haplotype and the reference haplotypes are computed. The matrix of distances between reference haplotypes is stored, in the next code sniped, in dgrp, whereas the distances of the target haplotype to the reference sequences are stored in vector d. The DBrule function computes then the most likely genotype based on both, the distances from the target haplotype to the references, and the distances between references of the same genotype. dm <- as.matrix(DNA.dist(c(hseq,RefSeqs),model="K80")) dgrp <- dm[-1,-1] d <- dm[1,-1] grp <- factor(substr(rownames(dgrp),1,1)) hr <- as.integer(grp) dsc <- DBrule(dgrp,hr,d,levels(grp)) print(dsc) ##$Phi2
##       Phi2.A       Phi2.B       Phi2.C       Phi2.D       Phi2.E       Phi2.F
## 0.0063243414 0.0057484677 0.0027527054 0.0006078818 0.0007949051 0.0188773955
##       Phi2.G       Phi2.H
## 0.0448899802 0.0249913247
##
## $DB.rule ## [1] 4 ## ##$Type
## [1] "D"

The target sequence has been classified as genotype D, giving the lowest Phi square value.

## R version 4.0.0 (2020-04-24)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 18.04.4 LTS
##
## Matrix products: default
## BLAS:   /home/biocbuild/bbs-3.11-bioc/R/lib/libRblas.so
## LAPACK: /home/biocbuild/bbs-3.11-bioc/R/lib/libRlapack.so
##
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=C
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## attached base packages:
## [1] stats4    parallel  stats     graphics  grDevices utils     datasets
## [8] methods   base
##
## other attached packages:
## [1] QSutils_1.5.0       ggplot2_3.3.0       ape_5.3
## [4] Biostrings_2.56.0   XVector_0.28.0      IRanges_2.22.0
## [7] S4Vectors_0.26.0    BiocGenerics_0.34.0 BiocStyle_2.16.0
##
## loaded via a namespace (and not attached):
##  [1] Rcpp_1.0.4.6        highr_0.8           compiler_4.0.0
##  [4] pillar_1.4.3        BiocManager_1.30.10 tools_4.0.0
##  [7] zlibbioc_1.34.0     digest_0.6.25       evaluate_0.14
## [10] lifecycle_0.2.0     tibble_3.0.1        nlme_3.1-147
## [13] gtable_0.3.0        lattice_0.20-41     pkgconfig_2.0.3
## [16] rlang_0.4.5         psych_1.9.12.31     magick_2.3
## [19] yaml_2.2.1          xfun_0.13           withr_2.2.0
## [22] dplyr_0.8.5         stringr_1.4.0       knitr_1.28
## [25] vctrs_0.2.4         tidyselect_1.0.0    grid_4.0.0
## [28] glue_1.4.0          R6_2.4.1            rmarkdown_2.1
## [31] bookdown_0.18       farver_2.0.3        purrr_0.3.4
## [34] magrittr_1.5        scales_1.1.0        htmltools_0.4.0
## [37] ellipsis_0.3.0      mnormt_1.5-6        assertthat_0.2.1
## [40] colorspace_1.4-1    labeling_0.3        stringi_1.4.6
## [43] munsell_0.5.0       crayon_1.3.4

#References

Domingo, E., J. Sheldon, and C. Perales. 2012. “Viral Quasispecies Evolution.” Microbiology and Molecular Biology Reviews 76 (2):159–216. https://doi.org/10.1128/MMBR.05023-11.

Gregori, Josep, Celia Perales, Francisco Rodriguez-Frias, Juan I. Esteban, Josep Quer, and Esteban Domingo. 2016. “Viral quasispecies complexity measures.” https://doi.org/10.1016/j.virol.2016.03.017.

Gregori, Josep, Miquel Salicrú, Esteban Domingo, Alex Sanchez, Juan I. Esteban, Francisco Rodríguez-Frías, and Josep Quer. 2014. “Inference with viral quasispecies diversity indices: Clonal and NGS approaches.” Bioinformatics 30 (8):1104–11. https://doi.org/10.1093/bioinformatics/btt768.


text:
## LATEX-L@LISTSERV.UNI-HEIDELBERG.DE

 Options: Use Forum View Use Monospaced Font Show HTML Part by Default Show All Mail Headers Message: [<< First] [< Prev] [Next >] [Last >>] Topic: [<< First] [< Prev] [Next >] [Last >>] Author: [<< First] [< Prev] [Next >] [Last >>]

J%ORG KNAPPEN writes:

> Long time ago, I wrote my own inputenc files making *all* high characters
> text character. The definition files (latin1jk.def, latin2jk.def,
> and latin3jk.def) are available from CTAN
> (latex/contrib/supported/jknappen).
>

something like this yes (perhaps even exactly this)

> If the standard inputenc files are changed, I strongly plea for "dual use"
> characters. The standard ASCII characters with a few exceptions can be used
> in text and math as well. A user expects the high character not to be
> different
> in this respect.

depends on what dual means. if you mean that you want the result of a key
mapping (eg \"a) be available text or math then

i'm not. i strongly plea for math only accepts the small set of real
ascii, ie

0-9  a-z A-Z  !"/()=?'+*<>|,;.:-
plus commands

by default (repeat: default).

the set of exceptions is a) not well-defined an b) potentially unlimited and
thus even more difficult to explain than anything else.

because what you are really have to do then is to make all font encoding
specific commands available in math and that doesn't work, a lot have to
equivalent there.

> However, the whole setup using active characters makes such kind of change
> rather difficult.

it is not so much that but the question what \c{c} should mean in math, should
it map to \mbox{\c{c}}? probably not; should \"a map to \ddot\a ?
perhaps. what about \\^{a} as supported by T5 Vietnamese?

right now there are roughly 900+ font-encoding-specific commands, i want to
keep those text (only).

so i don't think "math" should be regarded as another "fontencoding" though
that would be possible in theory.

instead i would think something like \DeclareInputTextAndMath should be
offered for those languages/keyboards where a dual nature for input "keys"
really makes sense, greek and cyrillic comes to mind.

but i would consider this a dual mapping from input "key" to either a text
command (if in text) or a math command if in math, eg

format: plain


text:
# Sum of products over k-permutations

Let $$A$$ be a matrix of size $$K \times N$$, $$K \leq N$$. Let $$n_1...n_K$$ denote a $$K$$-permutation of integers $$1...N$$ (understood as a unique assignment of a column to every row in the matrix). How to most efficiently calculate the sum, over all $$K$$-permutations, of products $$A[1,n_1] * A[2,n_2] * ... * A[k,n_K]$$?

For example, let $$K=3$$ and $$N=5$$. One possible K-permutation is (5, 1, 3), and the corresponding product is $$A[1,5] * A[2,1] * A[3,3]$$. But what must be calculated is the sum of such products over all K-permutations, e.g.:

$$A[1,1] * A[2,2] * A[3,3] + \\ A[1,1] * A[2,2] * A[3,4] + \\ A[1,1] * A[2,2] * A[3,5] + \\ A[1,1] * A[2,3] * A[3,4] + \\ A[1,1] * A[2,3] * A[3,5] + \\ A[1,1] * A[2,4] * A[3,5] + \\ ...$$

I am not even sure there is an efficient (polynomial time) solution, the most I have been able to come up with has about $$2^M$$ complexity which is unacceptable.

When $$K = N$$, the polynomial you are trying to compute is known as the permanent. The best algorithms for computing the permanent use $$\tilde{O}(2^N)$$ arithmetic operations, see Wikipedia. The VP≠VNP conjecture implies that the permanent cannot be computed using a polynomial number of operations.
For general $$K$$, your function is a known extension of the permanent. Ryser's algorithm, one of the $$\tilde{O}(2^N)$$ algorithms for the square case, extends to the rectangular case as well.


text:
# Quantum information

Quantum information is the information of the state of a quantum system. It is the basic entity of study in quantum information theory,[2] and can be manipulated using quantum information processing techniques. Quantum information refers to both the technical definition in terms of Von Neumann entropy and the general computational term.

 Optical lattices use lasers to separate rubidium atoms (red) for use as information bits in neutral-atom quantum processors—prototype devices which designers are trying to develop into full-fledged quantum computers.[1] Credit: NIST

It is an interdisciplinary field that involves quantum mechanics, computer science, information theory, philosophy and cryptography among other fields.[3][4] Its study is also relevant to disciplines such as cognitive science, psychology and neuroscience.[5] Its main focus is in extracting information from matter at the microscopic scale. Observation in science is one of the most important ways of acquiring information and measurement is required in order to quantify the observation, making this crucial to the scientific method. In quantum mechanics, due to the uncertainty principle, non-commuting observables cannot be precisely measured simultaneously, as an eigenstate in one basis is not an eigenstate in the other basis. As both variables are not simultaneously well defined, a quantum state can never contain definitive information about both variables.[5]

Information is something that is encoded in the state of a quantum system;[6] it is physical.[6] While quantum mechanics deals with examining properties of matter at the microscopic level,[7][5] quantum information science focuses on extracting information from those properties,[5] and quantum computation manipulates and processes information – performs logical operations – using quantum information processing techniques.[8]

Quantum information, like classical information, can be processed using digital computers, transmitted from one location to another, manipulated with algorithms, and analyzed with computer science and mathematics. Just like the basic unit of classical information is the bit, quantum information deals with qubits. Quantum information can be measured using Von Neumann entropy.

Recently, the field of quantum computing has become an active research area because of the possibility to disrupt modern computation, communication, and cryptography.[8]

## History and development

### Development from fundamental quantum mechanics

The history of quantum information theory began at the turn of the 20th century when classical physics was revolutionized into quantum physics. The theories of classical physics were predicting absurdities such as the ultraviolet catastrophe, or electrons spiraling into the nucleus. At first these problems were brushed aside by adding ad hoc hypotheses to classical physics. Soon, it became apparent that a new theory must be created in order to make sense of these absurdities, and the theory of quantum mechanics was born.[2]

Quantum mechanics was formulated by Schrödinger using wave mechanics and Heisenberg using matrix mechanics.[9] The equivalence of these methods was proven later.[10] Their formulations described the dynamics of microscopic systems but had several unsatisfactory aspects in describing measurement processes. Von Neumann formulated quantum theory using operator algebra in a way that it described measurement as well as dynamics.[11] These studies emphasized the philosophical aspects of measurement rather than a quantitative approach to extracting information via measurements.

 Evolution Picture (.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}) of: Heisenberg Interaction Schrödinger Ket state constant ${\displaystyle |\psi _{\rm {I}}(t)\rangle =e^{iH_{0,\mathrm {S} }~t/\hbar }|\psi _{\rm {S}}(t)\rangle }$ ${\displaystyle |\psi _{\rm {S}}(t)\rangle =e^{-iH_{\rm {S}}~t/\hbar }|\psi _{\rm {S}}(0)\rangle }$ Observable ${\displaystyle A_{\rm {H}}(t)=e^{iH_{\rm {S}}~t/\hbar }A_{\rm {S}}e^{-iH_{\rm {S}}~t/\hbar }}$ ${\displaystyle A_{\rm {I}}(t)=e^{iH_{0,\mathrm {S} }~t/\hbar }A_{\rm {S}}e^{-iH_{0,\mathrm {S} }~t/\hbar }}$ constant Density matrix constant ${\displaystyle \rho _{\rm {I}}(t)=e^{iH_{0,\mathrm {S} }~t/\hbar }\rho _{\rm {S}}(t)e^{-iH_{0,\mathrm {S} }~t/\hbar }}$ ${\displaystyle \rho _{\rm {S}}(t)=e^{-iH_{\rm {S}}~t/\hbar }\rho _{\rm {S}}(0)e^{iH_{\rm {S}}~t/\hbar }}$

#### Development from communication

In 1960s, Stratonovich, Helstrom and Gordon[12] proposed a formulation of optical communications using quantum mechanics. This was the first historical appearance of quantum information theory. They mainly studied error probabilities and channel capacities for communication.[12][13] Later, Holevo obtained an upper bound of communication speed in the transmission of a classical message via a quantum channel.[14][15]

#### Development from atomic physics and relativity

In the 1970s, techniques for manipulating single-atom quantum states, such as the atom trap and the scanning tunneling microscope, began to be developed, making it possible to isolate single atoms and arrange them in arrays. Prior to these developments, precise control over single quantum systems was not possible, and experiments utilized coarser, simultaneous control over a large number of quantum systems.[2] The development of viable single-state manipulation techniques led to increased interest in the field of quantum information and computation.

In the 1980s, interest arose in whether it might be possible to use quantum effects to disprove Einstein's theory of relativity. If it were possible to clone an unknown quantum state, it would be possible to use entangled quantum states to transmit information faster than the speed of light, disproving Einstein's theory. However, the no-cloning theorem showed that such cloning is impossible. The theorem was one of the earliest results of quantum information theory.[2]

#### Development from cryptography

Despite all the excitement and interest over studying isolated quantum systems and trying to find a way to circumvent the theory of relativity, research in quantum information theory became stagnant in the 1980s. However, around the same time another avenue started dabbling into quantum information and computation: Cryptography. In a general sense, cryptography is the problem of doing communication or computation involving two or more parties who may not trust one another.[2]

Bennett and Brassard developed a communication channel on which it is impossible eavesdrop without being detected, a way of communicating secretly at long distances using the BB84 quantum cryptographic protocol.[16] The key idea was the use of the fundamental principle of quantum mechanics that observation disturbs the observed, and the introduction of a eavesdropper in a secure communication line will immediately let the two parties trying to communicate would know of the presence of the eavesdropper.

#### Development from computer science and mathematics

With the advent of Alan Turing's revolutionary ideas of a programmable computer, or Turing machine, he showed that any real-world computation can be translated into an equivalent computation involving a Turing machine.[17][18] This is known as the Church–Turing thesis.

Soon enough, the first computers were made and computer hardware grew at such a fast pace that the growth, through experience in production, was codified into an empirical relationship called Moore's law. This 'law' is a projective trend that states that the number of transistors in an integrated circuit doubles every two years.[19] As transistors began to become smaller and smaller in order to pack more power per surface area, quantum effects started to show up in the electronics resulting in inadvertent interference. This led to the advent of quantum computing, which used quantum mechanics to design algorithms.

At this point, quantum computers showed promise of being much faster than classical computers for certain specific problems. One such example problem was developed by David Deutsch and Richard Jozsa, known as the Deutsch–Jozsa algorithm. This problem however held little to no practical applications.[2] Peter Shor in 1994 came up with a very important and practical problem, one of finding the prime factors of an integer. The discrete logarithm problem as it was called, could be solved efficiently on a quantum computer but not on a classical computer hence showing that quantum computers are more powerful than Turing machines.

### Development from information theory

Around the time computer science was making a revolution, so was information theory and communication, through Claude Shannon.[20][21][22] Shannon developed two fundamental theorems of information theory: noiseless channel coding theorem and noisy channel coding theorem. He also showed that error correcting codes could be used to protect information being sent.

Quantum information theory also followed a similar trajectory, Ben Schumacher in 1995 made an analogue to Shannon's noiseless coding theorem using the qubit. A theory of error-correction also developed, which allows quantum computers to make efficient computations regardless of noise, and make reliable communication over noisy quantum channels.[2]

## Qubits and information theory

Quantum information differs strongly from classical information, epitomized by the bit, in many striking and unfamiliar ways. While the fundamental unit of classical information is the bit, the most basic unit of quantum information is the qubit. Classical information is measured using Shannon entropy, while the quantum mechanical analogue is Von Neumann entropy. Given a statistical ensemble of quantum mechanical systems with the density matrix ${\displaystyle \rho }$ , it is given by ${\displaystyle S(\rho )=-\operatorname {Tr} (\rho \ln \rho ).}$ [2] Many of the same entropy measures in classical information theory can also be generalized to the quantum case, such as Holevo entropy[23] and the conditional quantum entropy.

Unlike classical digital states (which are discrete), a qubit is continuous-valued, describable by a direction on the Bloch sphere. Despite being continuously valued in this way, a qubit is the smallest possible unit of quantum information, and despite the qubit state being continuous-valued, it is impossible to measure the value precisely. Five famous theorems describe the limits on manipulation of quantum information.[2]

1. no-teleportation theorem, which states that a qubit cannot be (wholly) converted into classical bits; that is, it cannot be fully "read".
2. no-cloning theorem, which prevents an arbitrary qubit from being copied.
3. no-deleting theorem, which prevents an arbitrary qubit from being deleted.
4. no-broadcast theorem, which prevents an arbitrary qubit from being delivered to multiple recipients, although it can be transported from place to place (e.g. via quantum teleportation).
5. no-hiding theorem, which demonstrates the conservation of quantum information.

These theorems prove that quantum information within the universe is conserved. They open up possibilities in quantum information processing.

## Quantum information processing

The state of a qubit contains all of its information. This state is frequently expressed as a vector on the Bloch sphere. This state can be changed by applying linear transformations or quantum gates to them. These unitary transformations are described as rotations on the Bloch Sphere. While classical gates correspond to the familiar operations of Boolean logic, quantum gates are physical unitary operators.

• Due to the volatility of quantum systems and the impossibility of copying states, the storing of quantum information is much more difficult than storing classical information. Nevertheless, with the use of quantum error correction quantum information can still be reliably stored in principle. The existence of quantum error correcting codes has also led to the possibility of fault-tolerant quantum computation.
• Classical bits can be encoded into and subsequently retrieved from configurations of qubits, through the use of quantum gates. By itself, a single qubit can convey no more than one bit of accessible classical information about its preparation. This is Holevo's theorem. However, in superdense coding a sender, by acting on one of two entangled qubits, can convey two bits of accessible information about their joint state to a receiver.
• Quantum information can be moved about, in a quantum channel, analogous to the concept of a classical communications channel. Quantum messages have a finite size, measured in qubits; quantum channels have a finite channel capacity, measured in qubits per second.
• Quantum information, and changes in quantum information, can be quantitatively measured by using an analogue of Shannon entropy, called the von Neumann entropy.
• In some cases quantum algorithms can be used to perform computations faster than in any known classical algorithm. The most famous example of this is Shor's algorithm that can factor numbers in polynomial time, compared to the best classical algorithms that take sub-exponential time. As factorization is an important part of the safety of RSA encryption, Shor's algorithm sparked the new field of post-quantum cryptography that tries to find encryption schemes that remain safe even when quantum computers are in play. Other examples of algorithms that demonstrate quantum supremacy include Grover's search algorithm, where the quantum algorithm gives a quadratic speed-up over the best possible classical algorithm. The complexity class of problems efficiently solvable by a quantum computer is known as BQP.
• Quantum key distribution (QKD) allows unconditionally secure transmission of classical information, unlike classical encryption, which can always be broken in principle, if not in practice. Do note that certain subtle points regarding the safety of QKD are still hotly debated.

The study of all of the above topics and differences comprises quantum information theory.

## Relation to quantum mechanics

Quantum mechanics is the study of how microscopic physical systems change dynamically in nature. In the field of quantum information theory, the quantum systems studied are abstracted away from any real world counterpart. A qubit might for instance physically be a photon in a linear optical quantum computer, an ion in a trapped ion quantum computer, or it might be a large collection of atoms as in a superconducting quantum computer. Regardless of the physical implementation, the limits and features of qubits implied by quantum information theory hold as all these systems are mathematically described by the same apparatus of density matrices over the complex numbers. Another important difference with quantum mechanics is that, while quantum mechanics often studies infinite-dimensional systems such as a harmonic oscillator, quantum information theory concerns both with continuous-variable systems[24] and finite-dimensional systems.[25][26][27]

## Entropy and information

Entropy measures the uncertainty in the state of a physical system.[2] Entropy can be studied from the point of view of both the classical and quantum information theories.

### Classical information theory

Classical information is based on the concepts of information laid out by Claude Shannon. Classical information, in principle, can be stored in a bit of binary strings. Any system having two states is a capable bit.[28]

#### Shannon entropy

Shannon entropy is the quantification of the information gained by measuring the value of a random variable. Another way of thinking about it is by looking at the uncertainty of a system prior to measurement. As a result, entropy, as pictured by Shannon, can be seen either as a measure of the uncertainty prior to making a measurement or as a measure of information gained after making said measurement.[2]

Shannon entropy, written as a functional of a discrete probability distribution, ${\displaystyle P(x_{1}),P(x_{2}),...,P(x_{n})}$  associated with events ${\displaystyle x_{1},...,x_{n}}$ , can be seen as the average information associated with this set of events, in units of bits:

${\displaystyle H(X)=H[P(x_{1}),P(x_{2}),...,P(x_{n})]=-\sum _{i=1}^{n}P(x_{i})\log _{2}P(x_{i})}$

This definition of entropy can be used to quantify the physical resources required to store the output of an information source. The ways of interpreting Shannon entropy discussed above are usually only meaningful when the number of samples of an experiment is large.[29]

#### Rényi entropy

The Rényi entropy is a generalization of Shannon entropy defined above. The Rényi entropy of order r, written as a function of a discrete probability distribution, ${\displaystyle P(a_{1}),P(a_{2}),...,P(a_{n})}$ , associated with events ${\displaystyle a_{1},...,a_{n}}$ , is defined as:[28]

${\displaystyle H_{r}(A)={1 \over 1-r}\log _{2}\sum _{i=1}^{n}P^{r}(a_{i})}$

for ${\displaystyle 0  and ${\displaystyle r\neq 1}$ .

We arrive at the definition of Shannon entropy from Rényi when ${\displaystyle r\rightarrow 1}$ , of Hartley entropy (or max-entropy) when ${\displaystyle r\rightarrow 0}$ , and min-entropy when ${\displaystyle r\rightarrow \infty }$ .

### Quantum information theory

Quantum information theory is largely an extension of classical information theory to quantum systems. Classical information is produced when measurements of quantum systems are made.[28]

#### Von Neumann entropy

One interpretation of Shannon entropy was the uncertainty associated with a probability distribution. When we want to describe the information or the uncertainty of a quantum state, the probability distributions are simply swapped out by density operators ${\displaystyle \rho }$ .

${\displaystyle S(\rho )\equiv -\mathrm {tr} (\rho \ \log _{2}\ \rho )=-\sum _{i}\lambda _{i}\ \log _{2}\ \lambda _{i}}$

${\displaystyle \lambda _{i}}$ s are the eigenvalues of ${\displaystyle \rho }$ .

Von Neumann plays a similar role in quantum information that Shannon entropy does in classical information

## Applications

Quantum communication

Quantum communication is one of the applications of quantum physics and quantum information. There are some famous theorems such as the no-cloning theorem that illustrate some important properties in quantum communication. Dense coding and quantum teleportation are also applications of quantum communication. They are two opposite ways to communicate using qubits. While teleportation transfers one qubit from Alice and Bob by communicating two classical bits under the assumption that Alice and Bob have a pre-shared Bell state, dense coding transfers two classical bits from Alice to Bob by using one qubit, again under the same assumption, that Alice and Bob have a pre-shared Bell state.

Quantum key distribution

One of the best known applications of quantum cryptography is quantum key distribution which provide a theoretical solution to the security issue of a classical key. The advantage of quantum key distribution is that it is impossible to copy a quantum key because of the no-cloning theorem. If someone tries to read encoded data, the quantum state being transmitted will change. This could be used to detect eavesdropping.

• BB84

The first quantum key distribution scheme BB84, developed by Charles Bennett and Gilles Brassard in 1984. It is usually explained as a method of securely communicating a private key from a third party to another for use in one-time pad encryption.[2]

• E91

E91 was made by Artur Ekert in 1991. His scheme uses entangled pairs of photons. These two photons can be created by Alice, Bob, or by a third party including eavesdropper Eve. One of the photons is distributed to Alice and the other to Bob so that each one ends up with one photon from the pair.

This scheme relies on two properties of quantum entanglement:

1. The entangled states are perfectly correlated which means that if Alice and Bob both measure their particles having either a vertical or horizontal polarization, they always get the same answer with 100% probability. The same is true if they both measure any other pair of complementary (orthogonal) polarizations. This necessitates that the two distant parties have exact directionality synchronization. However, from quantum mechanics theory the quantum state is completely random so that it is impossible for Alice to predict if she will get vertical polarization or horizontal polarization results.
2. Any attempt at eavesdropping by Eve destroys this quantum entanglement such that Alice and Bob can detect.
• B92

B92 is a simpler version of BB84.[30]

The main difference between B92 and BB84:

• B92 only needs two states
• BB84 needs 4 polarization states

Like the BB84, Alice transmits to Bob a string of photons encoded with randomly chosen bits but this time the bits Alice chooses the bases she must use. Bob still randomly chooses a basis by which to measure but if he chooses the wrong basis, he will not measure anything which is guaranteed by quantum mechanics theories. Bob can simply tell Alice after each bit she sends whether or not he measured it correctly.[31]

Quantum computation

The most widely used model in quantum computation is the quantum circuit, which are based on the quantum bit "qubit". Qubit is somewhat analogous to the bit in classical computation. Qubits can be in a 1 or 0 quantum state, or they can be in a superposition of the 1 and 0 states. However, when qubits are measured the result of the measurement is always either a 0 or a 1; the probabilities of these two outcomes depend on the quantum state that the qubits were in immediately prior to the measurement.

Any quantum computation algorithm can be represented as a network of quantum logic gates.

Quantum decoherence

If a quantum system were perfectly isolated, it would maintain coherence perfectly, but it would be impossible to test the entire system. If it is not perfectly isolated, for example during a measurement, coherence is shared with the environment and appears to be lost with time; this process is called quantum decoherence. As a result of this process, quantum behavior is apparently lost, just as energy appears to be lost by friction in classical mechanics.

Quantum error correction

QEC is used in quantum computing to protect quantum information from errors due to decoherence and other quantum noise. Quantum error correction is essential if one is to achieve fault-tolerant quantum computation that can deal not only with noise on stored quantum information, but also with faulty quantum gates, faulty quantum preparation, and faulty measurements.

Peter Shor first discovered this method of formulating a quantum error correcting code by storing the information of one qubit onto a highly entangled state of ancilla qubits. A quantum error correcting code protects quantum information against errors.

## Journals

Many journals publish research in quantum information science, although only a few are dedicated to this area. Among these are:

## Notes

1. ^ "Free-Images.com – Free Public Domain Images". free-images.com. Retrieved 2020-11-13.
2. Nielsen, Michael A. (2010). Quantum computation and quantum information. Chuang, Isaac L. (10th anniversary ed.). Cambridge: Cambridge University Press. ISBN 978-1107002173. OCLC 665137861.
3. ^ Bokulich, Alisa; Jaeger, Gregg (2010). Philosophy of Quantum Information and Entanglement. Cambridge University Press. ISBN 978-1-139-48766-5.
4. ^ Benatti, Fabio; Fannes, Mark; Floreanini, Roberto; Petritis, Dimitri (2010). Quantum Information, Computation and Cryptography: An Introductory Survey of Theory, Technology and Experiments. Springer Science & Business Media. ISBN 978-3-642-11913-2.
5. ^ a b c d Hayashi, Masahito (2017). Quantum Information Theory. Graduate Texts in Physics. doi:10.1007/978-3-662-49725-8. ISBN 978-3-662-49723-4.[page needed]
6. ^ a b Preskill, John. "Lecture notes for physics 229." Quantum information and computation 16 (1998).
7. ^ "The Feynman Lectures on Physics Vol. III Ch. 1: Quantum Behavior". wayback.archive-it.org. Retrieved 2020-11-11.
8. ^ a b Lo, Hoi-Kwong; Spiller, Tim; Popescu, Sandu (1998). Introduction to Quantum Computation and Information. World Scientific. ISBN 978-981-02-4410-1.
9. ^ Mahan, Gerald D. (2008-12-29). Quantum Mechanics in a Nutshell. Princeton University Press. doi:10.2307/j.ctt7s8nw. ISBN 978-1-4008-3338-2. JSTOR j.ctt7s8nw.
10. ^ Perlman, H. S. (November 1964). "Equivalence of the Schroedinger and Heisenberg Pictures". Nature. 204 (4960): 771–772. Bibcode:1964Natur.204..771P. doi:10.1038/204771b0. S2CID 4194913.
11. ^ Neumann, John von (2018-02-27). Mathematical Foundations of Quantum Mechanics: New Edition. Princeton University Press. ISBN 978-0-691-17856-1.
12. ^ a b Gordon, J. (September 1962). "Quantum Effects in Communications Systems". Proceedings of the IRE. 50 (9): 1898–1908. doi:10.1109/JRPROC.1962.288169. S2CID 51631629.
13. ^ Helstrom, Carl Wilhelm (1976). Quantum Detection and Estimation Theory. Academic Press. ISBN 978-0-08-095632-9.[page needed]
14. ^ A. S. Holevo, “Bounds for the Quantity of Information Transmitted by a Quantum Communication Channel”, Probl. Peredachi Inf., 9:3 (1973); Problems Inform. Transmission, 9:3 (1973), http://www.mathnet.ru/php/archive.phtml?wshow=paper&jrnid=ppi&paperid=903&option_lang=eng
15. ^ A. S. Holevo, “On Capacity of a Quantum Communications Channel”, Probl. Peredachi Inf., 15:4 (1979); Problems Inform. Transmission, 15:4 (1979), http://www.mathnet.ru/php/archive.phtml?wshow=paper&jrnid=ppi&paperid=1507&option_lang=eng
16. ^ Bennett, Charles H.; Brassard, Gilles (December 2014). "Quantum cryptography: Public key distribution and coin tossing". Theoretical Computer Science. 560: 7–11. arXiv:2003.06557. doi:10.1016/j.tcs.2014.05.025. S2CID 27022972.
17. ^ Weisstein, Eric W. "Church–Turing Thesis". mathworld.wolfram.com. Retrieved 2020-11-13.
18. ^ Deutsch, D. (8 July 1985). "Quantum theory, the Church–Turing principle and the universal quantum computer". Proceedings of the Royal Society of London A: Mathematical and Physical Sciences. 400 (1818): 97–117. Bibcode:1985RSPSA.400...97D. doi:10.1098/rspa.1985.0070. S2CID 1438116.
19. ^ Moore, G.E. (January 1998). "Cramming More Components Onto Integrated Circuits". Proceedings of the IEEE. 86 (1): 82–85. doi:10.1109/JPROC.1998.658762. S2CID 6519532.
20. ^ Shannon, C. E. (October 1948). "A Mathematical Theory of Communication". Bell System Technical Journal. 27 (4): 623–656. doi:10.1002/j.1538-7305.1948.tb00917.x. hdl:11858/00-001M-0000-002C-4314-2.
21. ^ "A Mathematical Theory of Communication" (PDF). 1998-07-15. Archived (PDF) from the original on 1998-07-15. Retrieved 2020-11-13.
22. ^ Shannon, C. E.; Weaver, W. (1949). "The mathematical theory of communication". Cite journal requires |journal= (help)
23. ^ "Alexandr S. Holevo". Mi.ras.ru. Retrieved 4 December 2018.
24. ^ Weedbrook, Christian; Pirandola, Stefano; García-Patrón, Raúl; Cerf, Nicolas J.; Ralph, Timothy C.; Shapiro, Jeffrey H.; Lloyd, Seth (1 May 2012). "Gaussian quantum information". Reviews of Modern Physics. 84 (2): 621–669. arXiv:1110.3234. Bibcode:2012RvMP...84..621W. doi:10.1103/RevModPhys.84.621. S2CID 119250535.
25. ^ Masahito Hayashi, "Quantum Information Theory: Mathematical Foundation"[page needed]
26. ^ J. Watrous, The Theory of Quantum Information (Cambridge Univ. Press, 2018). Freely available at [1]
27. ^ Wilde, Mark M. (2013). "Concepts in Quantum Shannon Theory". Quantum Information Theory. pp. 3–25. doi:10.1017/cbo9781139525343.002. ISBN 9781139525343.
28. ^ a b c Jaeger, Gregg (2007-04-03). Quantum Information: An Overview. Springer Science & Business Media. ISBN 978-0-387-36944-0.
29. ^ Watrous, John. "CS 766/QIC 820 Theory of Quantum Information (Fall 2011)." (2011).
30. ^ Bennett, Charles H. (1992-05-25). "Quantum cryptography using any two nonorthogonal states". Physical Review Letters. 68 (21): 3121–3124. Bibcode:1992PhRvL..68.3121B. doi:10.1103/PhysRevLett.68.3121. PMID 10045619.
31. ^ "Quantum Key Distribution - QKD". www.cse.wustl.edu. Retrieved 2020-11-21.
32. ^ "npj Quantum Information". Nature.com. Retrieved 4 December 2018.
33. ^ "Quantum Homepage". Quantum-journal.org. Retrieved 4 December 2018.
34. ^ "Quantum Science and Technology". IOP Publishing. Retrieved 12 January 2019.


text:
# Unpaired Image to Image Translations with Cycle GANs

In this tutorial, we show how to implement cycle GANs from scratch for image to image translation in TensorFlow using a clever combination of a ResNet generator and Patch GAN discriminator.

a minute ago   •   16 min read

Table of contents

One of the primary reasons that deep learning didn't take off when it was introduced a couple of decades ago was the lack of data. Neural networks by their very nature need huge processing power and big data to truly make use of in practice. In the current era, with more modernized equipment, storing large amounts of data is not an issue. It is also possible to create and refine the available data with modern tools for computer vision or natural language processing. Most deep learning models utilize paired examples where a source image is directly linked to its target image. However, there an important question arises. What type of deep learning would be optimal to solve tasks with little to no link between each other containing data?

In our previous articles, we have been exploring conditional GANs and their ability to perform complex tasks of image-to-image translations with relative ease after successful training. We have covered both Conditional GANs (CGANs) and a variation of these conditional GANs in pix2pix GANs. To quickly recap our previous GAN article on pix2pix, we focused on performing image-to-image translations of satellite images to maps. In this article, we will focus on another variation of conditional GANs in Cycle-Consistent Adversarial Networks (Cycle GANs) and develop an unpaired image-to-image translation project.

The Paperspace Gradient platform can be utilized to run the following project by creating a Paperspace Gradient Notebook using the following linked repo as the "Workspace URL." This field can be found by toggling the advanced options button on the Notebook creation page.

## Introduction:

When we have specific conditions and proper pairing of data elements linking one image to its respective counterpart, it is easier to train such conditional GANs. Pix2Pix GANs are one such example where we work with paired images. Paired images are ones where the source image is linked to its corresponding target image. In paired image datasets, we have a clear mapping of both the source and target images, which are provided accordingly. However, such data is difficult to obtain in the real world.

Unpaired data, on the other hand, is relatively easier to obtain and can be found in abundance. In unpaired datasets, the source images and their corresponding target images are not directly provided. One of the primary approaches to solving projects or tasks related to unpaired datasets is to utilize Cycle-Consistent Adversarial Networks (Cycle GANs). Cycle GANs provide the user with an efficient method of achieving compelling results in most cases. In this research paper, most of the primary goals and accomplishments of this network are described in detail.

The Cycle GAN networks are capable of learning a mapping from image source $X$ to $Y$ from unpaired examples such that the generated images produced are of high quality. An additional cycle loss is also introduced in this research paper to confirm that the inverse mapping from the generated image reproduces the source image once again. Let us understand the working procedure and implementation details further in the upcoming section of this article.

## Understanding Cycle GANs:

In this section, we will understand the detailed working explanation of Cycle GANs. In the introduction, we gently acknowledge that the application of Cycle GANs is primarily for unpaired image-to-image translations in opposition to other conditional GANs, such as pix2pix GANs. Let us try to understand the working procedure and the basic concepts of these networks, as well as gain a clear understanding of the generator and discriminator architecture.

In the above image, let us consider the first sub-section with the generators $G$ and $F$ and the discriminators $Dx$ and $Dy$. The $X$ represents the input image, while the $Y$ represents the generated image. Unlike most GAN networks, which utilize a single generator and discriminator, the Cycle GAN network makes use of two generators and discriminators. The generator $G$ works on the input $X$ to generate an image $Y$. The discriminator $Dy$ distinguishes if the generated image is real or fake. Similarly, the generator $F$ generates an image $X$ considering an input $Y$. The discriminator $Dx$ distinguishes if this generated image is real or fake.

Apart from having a double generator and discriminator setup, we also have a cycle consistency loss to help the model to capture a greater intuitive understanding. If the model is trained on one source image to produce a generated image, then the cycle consistency loss ensures that when the generated image is translated back again into the original domain, we should be able to approximately retrieve the original source image once again.

In the above image, we can notice both the forward cycle-consistency loss and backward cycle-consistency loss in their respective sub-figures. Below is the final equation of all the loss functions combined for the Cycle GAN network. In this project, we also use an additional identity loss, which may not be required for most cases.

$$L(G, F, DX, DY ) = LGAN(G, DY , X, Y ) + LGAN(F, DX, Y, X) + λLcyc(G, F)$$

Now that we have understood the working procedure of these Cycle GAN networks, we can also briefly discuss the details behind the implementation of the generator and discriminator networks. The generator architecture comprises of three convolutional layers with varying filters, kernel sizes, and strides. After the three convolutional layers, we have the residual blocks. The network can either contain six or nine residual blocks.

At the end of the residual blocks, we make use of a couple of upsampling layers (or convolutional transpose layers) with a final convolutional layer resulting in the required image size for the generator. In the discriminator architecture, we have a simple 70 x 70 Patch GAN network, which contains four convolutional layers of increasing filter size. In these networks, instance normalization is preferred over batch normalization alongside Leaky ReLU activation functions. We will discuss more on the architectural details further when we construct these models from scratch.

## Developing Unpaired Image To Image Translation using Cycle GANS:

In this section of the article, we will focus on developing the unpaired image-to-image translation project with the help of Cycle GANs. Now that we have a brief understanding of the basic working procedure of these Cycle-Consistent Adversarial Networks, we can build the overall architecture to process and compute the required task. We will utilize the TensorFlow and Keras deep learning frameworks for constructing these networks. If the viewers are not familiar with these libraries, I would recommend checking out a couple of my previous articles to gain more familiarity with these topics. You can check out the following article to learn more about TensorFlow and the Keras article here. Once done, we can start with the importing of the essential libraries.

Bring this project to life

### Importing the essential libraries:

As discussed previously, one of the major changes from the previous pix2pix GAN architecture is the use of instance normalization layers over the batch normalization layers. Since there is no way to directly call the instance normalization layer through the existing Keras API, we will proceed to install an additional requirement. We will install the Keras-Contrib repository that will allow us to directly utilize these necessary layers without going through too much hassle.

pip install git+https://www.github.com/keras-team/keras-contrib.git

Once we have finished installing the pre-requisite requirement for the project, we can proceed to import all the essential libraries that we will utilize for constructing the Cycle GAN architecture and train the model accordingly to obtain the desired results. We will use the TensorFlow and Keras deep learning frameworks, as discussed previously, for building the network. We will use the functional API model type instead of the Sequential model to have overall higher control of the network design. We will also use a dataset that is available in the TensorFlow datasets library. Other necessary imports include matplotlib for visualization and the OS for handling tasks related to the local operating system. Below is the list of all the required imports.

import tensorflow as tf
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.initializers import RandomNormal
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Conv2D, LeakyReLU, Conv2DTranspose
from tensorflow.keras.layers import Activation, Concatenate, BatchNormalization
from keras_contrib.layers.normalization.instancenormalization import InstanceNormalization
from tensorflow.keras.utils import plot_model
import tensorflow_datasets as tfds

import matplotlib.pyplot as plt
from IPython.display import clear_output
import time
import os

AUTOTUNE = tf.data.AUTOTUNE

### Obtaining and preparing the dataset:

In this step, we will obtain the dataset for which we will perform the translation of images. We make use of the TensorFlow datasets library, through which we can retrieve all the relevant information. The dataset contains images of horses and zebras. For obtaining similar datasets for Cycle GAN projects, I would recommend checking out the following link. Below is the code snippet to load the datasets into their respective train and test variables.

dataset, metadata = tfds.load('cycle_gan/horse2zebra',
with_info=True, as_supervised=True)

train_horses, train_zebras = dataset['trainA'], dataset['trainB']
test_horses, test_zebras = dataset['testA'], dataset['testB']

Once we have retrieved the dataset, we can proceed to define some basic parameters that we will utilize for preparing the dataset.

BUFFER_SIZE = 1000
BATCH_SIZE = 1
IMG_WIDTH = 256
IMG_HEIGHT = 256

In the next code block, we will define some basic functions for the dataset preparation. We will perform the normalization of the dataset to avoid extra memory usage and solve the task with relatively lesser resources. We will also apply jittering and mirroring to the existing data available, as suggested in the research paper, to avoid overfitting. Such augmentation techniques are usually quite useful. In this step, we are resizing the image to 286 x 286 and then cropping it back to the required 256 x 256 size, as well as flipping the images horizontally from left to right. Below is the code block for performing the following actions.

def random_crop(image):
cropped_image = tf.image.random_crop(
image, size=[IMG_HEIGHT, IMG_WIDTH, 3])

return cropped_image

# normalizing the images to [-1, 1]
def normalize(image):
image = tf.cast(image, tf.float32)
image = (image / 127.5) - 1
return image

def random_jitter(image):
# resizing to 286 x 286 x 3
image = tf.image.resize(image, [286, 286],
method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)

# randomly cropping to 256 x 256 x 3
image = random_crop(image)

# random mirroring
image = tf.image.random_flip_left_right(image)

return image

def preprocess_image_train(image, label):
image = random_jitter(image)
image = normalize(image)
return image

def preprocess_image_test(image, label):
image = normalize(image)
return image

Finally, we will compute all these data elements into a final dataset. We will map the data with a random shuffle and pre-defined batch size through which all the components are accessible. We will define the dataset for all the training elements and testing ones for both the horse and zebra images, as shown in the below code snippet.

train_horses = train_horses.cache().map(
preprocess_image_train, num_parallel_calls=AUTOTUNE).shuffle(
BUFFER_SIZE).batch(BATCH_SIZE)

train_zebras = train_zebras.cache().map(
preprocess_image_train, num_parallel_calls=AUTOTUNE).shuffle(
BUFFER_SIZE).batch(BATCH_SIZE)

test_horses = test_horses.map(
preprocess_image_test, num_parallel_calls=AUTOTUNE).cache().shuffle(
BUFFER_SIZE).batch(BATCH_SIZE)

test_zebras = test_zebras.map(
preprocess_image_test, num_parallel_calls=AUTOTUNE).cache().shuffle(
BUFFER_SIZE).batch(BATCH_SIZE)

sample_horse = next(iter(train_horses))
sample_zebra = next(iter(train_zebras))

Below are some sample images with the datasets containing the original image and its corresponding image with some random jitter.

Once we have completed all the required steps for obtaining and preparing the dataset, we can proceed to create the discriminator and generator networks to build the overall Cycle GAN architecture.

### Define the discriminator architecture:

For the discriminator architecture, we have four convolutional blocks defined as follows - $C64-C128-C256-C512$. The Leaky ReLU activation function with an alpha (slope) value of 0.2 is used. Except for the first convolutional block, all other blocks make use of an instance normalization right after the convolutional layers. The strides and kernel sizes are as shown in the code snippet below. We will finally compile the model with a mean square error loss function and an Adam optimizer to complete the Patch GAN discriminator type network.

# define the discriminator model
def define_discriminator(image_shape):
# weight initialization
init = RandomNormal(stddev=0.02)
# source image input
in_image = Input(shape=image_shape)
# C64
d = Conv2D(64, (4,4), strides=(2,2), padding='same', kernel_initializer=init)(in_image)
d = LeakyReLU(alpha=0.2)(d)
# C128
d = Conv2D(128, (4,4), strides=(2,2), padding='same', kernel_initializer=init)(d)
d = InstanceNormalization(axis=-1)(d)
d = LeakyReLU(alpha=0.2)(d)
# C256
d = Conv2D(256, (4,4), strides=(2,2), padding='same', kernel_initializer=init)(d)
d = InstanceNormalization(axis=-1)(d)
d = LeakyReLU(alpha=0.2)(d)
# C512
d = Conv2D(512, (4,4), strides=(2,2), padding='same', kernel_initializer=init)(d)
d = InstanceNormalization(axis=-1)(d)
d = LeakyReLU(alpha=0.2)(d)
# second last output layer
d = Conv2D(512, (4,4), padding='same', kernel_initializer=init)(d)
d = InstanceNormalization(axis=-1)(d)
d = LeakyReLU(alpha=0.2)(d)
# patch output
patch_out = Conv2D(1, (4,4), padding='same', kernel_initializer=init)(d)
# define model
model = Model(in_image, patch_out)
# compile model
model.compile(loss='mse', optimizer=Adam(learning_rate=0.0002, beta_1=0.5), loss_weights=[0.5])
return model

# define image shape
image_shape = (256,256,3)

# create the model
model = define_discriminator(image_shape)

# summarize the model
model.summary()

### Define the Generator architecture:

The generator architecture consists of one convolutional block with 64 filters and a 7 x 7 kernel size with a stride of 2, followed by two convolutional layers of 3 x 3 kernel size with a stride of 1, respectively, with 128 and 256 filters. We will then define the residual block, which is nothing but a bunch of convolutional layers. This residual block is followed by a couple of upsampling layers defined by the Convolutional 2D Transpose layers. The final convolutional layer contains a 7 x 7 kernel size with a stride of 1 and 3 filters.

They generator architecture for 9 residual blocks is defined as follows: $c7s1-64, d128, d256, R256, R256, R256, R256, R256, R256, R256, R256, R256, u128, u64, c7s1-3$

# generator a resnet block
def resnet_block(n_filters, input_layer):
# weight initialization
init = RandomNormal(stddev=0.02)
# first layer convolutional layer
g = Conv2D(n_filters, (3,3), padding='same', kernel_initializer=init)(input_layer)
g = InstanceNormalization(axis=-1)(g)
g = Activation('relu')(g)
# second convolutional layer
g = Conv2D(n_filters, (3,3), padding='same', kernel_initializer=init)(g)
g = InstanceNormalization(axis=-1)(g)
# concatenate merge channel-wise with input layer
g = Concatenate()([g, input_layer])
return g

# define the standalone generator model
def define_generator(image_shape=(256, 256, 3), n_resnet=9):
# weight initialization
init = RandomNormal(stddev=0.02)
# image input
in_image = Input(shape=image_shape)
# c7s1-64
g = Conv2D(64, (7,7), padding='same', kernel_initializer=init)(in_image)
g = InstanceNormalization(axis=-1)(g)
g = Activation('relu')(g)
# d128
g = Conv2D(128, (3,3), strides=(2,2), padding='same', kernel_initializer=init)(g)
g = InstanceNormalization(axis=-1)(g)
g = Activation('relu')(g)
# d256
g = Conv2D(256, (3,3), strides=(2,2), padding='same', kernel_initializer=init)(g)
g = InstanceNormalization(axis=-1)(g)
g = Activation('relu')(g)
# R256
for _ in range(n_resnet):
g = resnet_block(256, g)
# u128
g = Conv2DTranspose(128, (3,3), strides=(2,2), padding='same', kernel_initializer=init)(g)
g = InstanceNormalization(axis=-1)(g)
g = Activation('relu')(g)
# u64
g = Conv2DTranspose(64, (3,3), strides=(2,2), padding='same', kernel_initializer=init)(g)
g = InstanceNormalization(axis=-1)(g)
g = Activation('relu')(g)
# c7s1-3
g = Conv2D(3, (7,7), padding='same', kernel_initializer=init)(g)
g = InstanceNormalization(axis=-1)(g)
out_image = Activation('tanh')(g)
# define model
model = Model(in_image, out_image)
return model

# create the model
model = define_generator()
# summarize the model
model.summary()

### Defining loss function and checkpoints:

In the next code snippet, we will explore the different types of loss functions that we will utilize for the Cycle GAN architecture. We will define generator loss, discriminator loss, cycle consistency loss, and identity loss. The reason for the cycle consistency loss, as discussed previously, is to maintain an approximate relationship between the source image and the reproduced image. Finally, we will define the Adam optimizers for both the generator and discriminator networks, as shown in the below code block.

LAMBDA = 10

loss_obj = tf.keras.losses.BinaryCrossentropy(from_logits=True)

def discriminator_loss(real, generated):
real_loss = loss_obj(tf.ones_like(real), real)

generated_loss = loss_obj(tf.zeros_like(generated), generated)

total_disc_loss = real_loss + generated_loss

return total_disc_loss * 0.5

def generator_loss(generated):
return loss_obj(tf.ones_like(generated), generated)

def calc_cycle_loss(real_image, cycled_image):
loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))

return LAMBDA * loss1

def identity_loss(real_image, same_image):
loss = tf.reduce_mean(tf.abs(real_image - same_image))
return LAMBDA * 0.5 * loss

generator_g_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)
generator_f_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)

discriminator_x_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)
discriminator_y_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)

Once we have finished defining the loss functions and optimizers, we will also define a checkpoint system where we will store the desired checkpoint with the most recent results being saved. We can do this to allow us to reload and retrain the weights if needed.

checkpoint_path = "./checkpoints/train"

ckpt = tf.train.Checkpoint(generator_g=generator_g,
generator_f=generator_f,
discriminator_x=discriminator_x,
discriminator_y=discriminator_y,
generator_g_optimizer=generator_g_optimizer,
generator_f_optimizer=generator_f_optimizer,
discriminator_x_optimizer=discriminator_x_optimizer,
discriminator_y_optimizer=discriminator_y_optimizer)

ckpt_manager = tf.train.CheckpointManager(ckpt, checkpoint_path, max_to_keep=5)

# if a checkpoint exists, restore the latest checkpoint.
if ckpt_manager.latest_checkpoint:
ckpt.restore(ckpt_manager.latest_checkpoint)
print ('Latest checkpoint restored!!')

### Define the final training function:

In the final step, we will define the training function to train our model and generate the desired images as required. Firstly, let us set the number of epochs that we plan to train the model for and create a function for creating the respective plots for visualizing the input and predicted image. Note that for this training, I used a keyboard interrupt after twenty epochs, but the viewers can train for longer periods to achieve better results.

EPOCHS = 50

def generate_images(model, test_input):
prediction = model(test_input)

plt.figure(figsize=(12, 12))

display_list = [test_input[0], prediction[0]]
title = ['Input Image', 'Predicted Image']

for i in range(2):
plt.subplot(1, 2, i+1)
plt.title(title[i])
# getting the pixel values between [0, 1] to plot it.
plt.imshow(display_list[i] * 0.5 + 0.5)
plt.axis('off')
plt.show()

In the training step function, we will call the tf.function and Gradient tape for faster training computation of the backpropagation weights. The training method is similar to the previous GAN architectures that we have previously built with the exception that we will train two generators and discriminators in this method as well as evaluate the cycle consistency loss as required. Below is the code block for the complete training procedure of the Cycle GAN architecture.

@tf.function
def train_step(real_x, real_y):
# persistent is set to True because the tape is used more than
# once to calculate the gradients.
with tf.GradientTape(persistent=True) as tape:
# Generator G translates X -> Y
# Generator F translates Y -> X.

fake_y = generator_g(real_x, training=True)
cycled_x = generator_f(fake_y, training=True)

fake_x = generator_f(real_y, training=True)
cycled_y = generator_g(fake_x, training=True)

# same_x and same_y are used for identity loss.
same_x = generator_f(real_x, training=True)
same_y = generator_g(real_y, training=True)

disc_real_x = discriminator_x(real_x, training=True)
disc_real_y = discriminator_y(real_y, training=True)

disc_fake_x = discriminator_x(fake_x, training=True)
disc_fake_y = discriminator_y(fake_y, training=True)

# calculate the loss
gen_g_loss = generator_loss(disc_fake_y)
gen_f_loss = generator_loss(disc_fake_x)

total_cycle_loss = calc_cycle_loss(real_x, cycled_x) + calc_cycle_loss(real_y, cycled_y)

# Total generator loss = adversarial loss + cycle loss
total_gen_g_loss = gen_g_loss + total_cycle_loss + identity_loss(real_y, same_y)
total_gen_f_loss = gen_f_loss + total_cycle_loss + identity_loss(real_x, same_x)

disc_x_loss = discriminator_loss(disc_real_x, disc_fake_x)
disc_y_loss = discriminator_loss(disc_real_y, disc_fake_y)

# Calculate the gradients for generator and discriminator
generator_g_gradients = tape.gradient(total_gen_g_loss,
generator_g.trainable_variables)
generator_f_gradients = tape.gradient(total_gen_f_loss,
generator_f.trainable_variables)

discriminator_x_gradients = tape.gradient(disc_x_loss,
discriminator_x.trainable_variables)
discriminator_y_gradients = tape.gradient(disc_y_loss,
discriminator_y.trainable_variables)

# Apply the gradients to the optimizer
generator_g_optimizer.apply_gradients(zip(generator_g_gradients,
generator_g.trainable_variables))

generator_f_optimizer.apply_gradients(zip(generator_f_gradients,
generator_f.trainable_variables))

discriminator_x_optimizer.apply_gradients(zip(discriminator_x_gradients,
discriminator_x.trainable_variables))

discriminator_y_optimizer.apply_gradients(zip(discriminator_y_gradients,
discriminator_y.trainable_variables))

Finally, we can start the training process for the defined number of epochs. We will loop through our training data and train the model accordingly. At the end of each epoch, we can generate the resulting image to notice the progression made by the model. We can also save the checkpoints, as shown in the code snippet below.

for epoch in range(EPOCHS):
start = time.time()

n = 0
for image_x, image_y in tf.data.Dataset.zip((train_horses, train_zebras)):
train_step(image_x, image_y)
if n % 10 == 0:
print ('.', end='')
n += 1

clear_output(wait=True)
# Using a consistent image (sample_horse) so that the progress of the model
# is clearly visible.
generate_images(generator_g, sample_horse)

if (epoch + 1) % 5 == 0:
ckpt_save_path = ckpt_manager.save()
print ('Saving checkpoint for epoch {} at {}'.format(epoch+1,
ckpt_save_path))

print ('Time taken for epoch {} is {} sec\n'.format(epoch + 1,
time.time()-start))
Saving checkpoint for epoch 20 at ./checkpoints/train/ckpt-4

Below are some of the results that I was able to obtain after twenty epochs of training. Training for a higher number of epochs and varying a few parameters could yield even better results.

With the help of Cycle GANs, we can notice the overall results obtained with the trained architecture are quite compelling. We are able to produce the desired results for the unpaired image translation task to an obviously interpretable degree. While there are a few limitations of this network, including the loss of background color and texture details, we are still able to retrieve approximately most of the necessary information for the particular task. I would recommend trying out numerous parameters, variations, and minor architectural changes to achieve more optimal results.

The major portions of this code are considered from the official TensorFlow website, which can be viewed from this link. However, they utilize a variation of the pix-2-pix model for training the Cycle GAN network for simplicity making use of a U-Net-like generator network and the corresponding discriminator network. In this article, we solely focused on reconstructing the research paper from scratch and used the ResNet architecture for the generator and the slightly modified Patch GAN architecture for the discriminator. I would recommend checking out the following reference link for a more detailed guide on only the Cycle GAN generator and discriminator implementation.

## Conclusion:

In the natural world, it is difficult to obtain a large dataset of paired examples for complex problems. Creating such paired datasets linking source image to their respective target images are often expensive and time-consuming. However, a large number of unpaired examples of image-to-image translations are available over the internet. The horse and zebra dataset discussed in this article is one such example where the desired output is not clearly well defined. Most modern GANs can solve tasks and projects related to paired examples on image-to-image translations. But the Cycle GAN conditional GAN has the ability to achieve desirable outputs on unpaired datasets.

In this article, we explored another variation of conditional GANs in Cycle-Consistent Adversarial Networks (Cycle GANs). We understood how powerful these GANs are and their ability to learn image-to-image translations even in the absence of paired examples. We had a brief introduction of some of the basic concepts of these Cycle GANs, along with a detailed breakdown of some of the core concepts of their network architecture. Finally, we concluded our conceptual understanding with the development of the "Unpaired Image-To-Image Translation using Cycle GANS" project from scratch.

In future articles, we will cover more types of GANs, such as ProGAN, StyleGAN, and so much more. We will also experiment with some audio tasks with deep learning and learn about constructing neural networks from scratch. Until then, keep learning and exploring!

Add speed and simplicity to your Machine Learning workflow today


text:
Getting the hypervisor installed is the next step.

# ESXI

I’ve selected ESXi because it’s free, and it allows me to manage multiple VMs from a headless machine. I’ll be learning about it in future posts. But now it’s time to get it running on my 2006 MacPro.

# Challenges with old MacPro

Were this some shiny new hardware, the post would be short. Download the iso, burn it to disk, pop that disk and a USB drive into the machine, turn it on, and install.

But while the MacPro 1,1 is a 64-bit machine, it runs off a 32-bit EFI architecture, and unfortunately, ESXi stopped supporting 32-bit EFI a few releases ago. There are several posts out there about how to do this that I leaned on (1, 2 primarily).

# Installing ESXi on a 2006 MacPro

The general approach:

1. Install ESXi on the USB using a VM
2. Modify some of the boot files to work in the 32-bit EFI
3. Move the USB over to the Mac Pro to boot

## Installing ESXi on the USB

While I typically use VirtualBox for almost everything, I actually had much better luck getting the USB drive recognized in VMWare Player, so that’s what I’ll show here.

In VMWare Player, Player -> New Virtual Machine. In the wizard, we’ll select the ESXi 6.5 ISO.

Continuing through the rest of the dialogs, making sure to give the machine at least 4GB of ram. Hard disk size is irrelevant. When you’re done, boot the machine.

While the machien is booting, we need to get our USB drive to be recognized by the VM, by going to Player -> Removable Devices -> [USB Drive] -> Connect (Disconnect from host).

Then, when the ESXi installer boots, the USB drive should be one of the options:

Select the USB drive, and continue with the installation, selecting a keyboard layout and setting a good password. When you get to the reboot screen, hit enter to let the VM shutdown. Once it restarts, you can shutdown (and delete) the VM.

## Modifying the Boot Files

We’re going to mount the USB so we can interact with the files. First we’ll use fdisk to find the device:

$sudo fdisk -l [snip...] Disk /dev/sdf: 14.3 GiB, 15376000000 bytes, 30031250 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 016CA5E2-FC26-49B9-A82B-306F199BD187 Device Start End Sectors Size Type /dev/sdf1 64 8191 8128 4M EFI System /dev/sdf5 8224 520191 511968 250M Microsoft basic data /dev/sdf6 520224 1032191 511968 250M Microsoft basic data /dev/sdf7 1032224 1257471 225248 110M unknown /dev/sdf8 1257504 1843199 585696 286M Microsoft basic data /dev/sdf9 1843200 7086079 5242880 2.5G unknown  We’ll grab the modified boot files from here: https://communities.vmware.com/servlet/JiveServlet/download/2183936-108290/macpro%20efi%20boot%20files.zip Mount the partition labeled EFI System. We’ve got 4 files to replace, from the unzipped file: $ find . -type f
./EFI-BOOT/BOOTx64.EFI
./EFI-BOOT/BOOTIA32.EFI
./EFI-VMWARE/mboot64.efi
./EFI-VMWARE/mboot32.efi

$sudo mount /dev/sdf1 /mnt/$ ls /mnt/EFI/
BOOT VMware

$sudo cp EFI-BOOT/* /mnt/EFI/BOOT/$ sudo cp EFI-VMWARE/* /mnt/EFI/VMware/

Make sure the files copied correctly:

$find . -type f -exec md5sum {} \; 40e5ca413f3a9f9a0d70acc004d9c21b ./EFI-BOOT/BOOTx64.EFI c1c2a7593bf0a81210f04d0bba5b9678 ./EFI-BOOT/BOOTIA32.EFI e8ab21b08a119b14598e6ecd53698cf9 ./EFI-VMWARE/mboot64.efi e8d519aa27495ad8d483c25e23b878d3 ./EFI-VMWARE/mboot32.efi$ find /mnt/EFI/ -type f -exec md5sum {} \;
c1c2a7593bf0a81210f04d0bba5b9678 /mnt/EFI/BOOT/BOOTIA32.EFI
40e5ca413f3a9f9a0d70acc004d9c21b /mnt/EFI/BOOT/BOOTx64.EFI
e8ab21b08a119b14598e6ecd53698cf9 /mnt/EFI/VMware/mboot64.efi

\$ sudo umount /mnt

## Boot Mac Pro from USB in ESXi

I’ve removed the MacOS hard drive from the tower, and replaced it with an old 500GB drive I have laying around. Plug in the USB drive, and boot. It’s pretty slow to start up. But just let it go. Success means you’ve reached this page:

Now we can visit that ip given and log in to manage. But that’s a later post.

# Notes

• “24GB? I thought you said 32?”. When you go to power on a MacPro and it doesn’t boot but instead the light above the power button blinks, it means bad ram. One of my 4GB DIMMs went bad, and they need to be installed in pairs.
• In troubleshooting this, I had some very strange results. Turns out the MacPro is picked about how you seat the ram. Use this guide: https://manuals.info.apple.com/MANUALS/0/MA430/en_US/MacPro_Early2008_MemoryDIMM_DIY.pdf
• ESXi 6.5 shouldn’t work on this machine. According to the ESXi 6.5 release notes, vSphere 6.5 no longer supports the Intel Xeon 53XX series processor. This may cause me problems down the road. I could always roll back to 6.0.
• Now is a good time to configure the ESXi server. I like to use DHCP, but with a static mapping at my EdgeRouterX gateway/DHCP server. The server is also VLAN aware, so I’ll have to figure that out at a later date.


text:
## Arg max

Does everybody except me know the notation “arg max?”  I learned it in an applied math talk today.  I had never seen this before.  It means “the value at which the maximum is achieved” — e.g.

$arg max (-x^2 + x - 1) = 1/2$.

Slick.

Tagged

## 10 thoughts on “Arg max”

1. Scott McKuen says:

It’s more of a CS idiom, I think. Very common in machine learning applications, but I’ve never seen it in a straight math lecture.

2. Anon. IV says:

Seen it, yes, but … what’s argmax(x^4-x^2+1)?

3. Anon says:

Presumably \emptyset .

4. Anon says:

This is used in the dynamic programming literature; probably also in the optimization literature more generally.

5. I think I saw that once in a number theory paper, though I don’t remember when…

6. I just learned it yesterday too. At our departmental coffee hour, someone used it to describe the “Chebychev center” of a set, the better to discuss the collaboration graph of the department.

7. Michael Lugo says:

I associate it with CS. There are definitely times where I’ve wanted to use it in straight math contexts, but I tend to refrain from doing so.

8. Lior Silberman says:

I thought it was a well-known notation. Here’s one useful instance: take a bounded set A in Euclidean space (more generally, a probably measure of compact support), and let $d(z)$ be the average over $A$ of the squared distance to $z$: $d_A(z) = \int_A dx \norm{z-x}^2$). Then the center of mass of $A$ is precisely $\argmin(d_A)$.

9. Anon. IV says:

I meant something like argmax(x^2-x^4) [or argmin(x^4-x^2+1)], for which the answer would still need to be a set of cardinality other than 1, albeit for a different reason: as it happens x^4-x^2+1 illustrates another difficulty with this notation (and could just have easily been asked about x^2, or indeed just x).

10. Florian says:

I didn’t know it, but I would have learnt it today. Did we see the same talk?? (Spielman just started his special lecture series at U of T.)


text:
# Thinking About Fractional Ideals Geometrically

So algebraic geometry gives one a way of thinking about about rings geometrically. Like prime ideals correspond to points in the spectrum of a ring, maximal ideals are closed points and so on. This works well for rings like $\mathbb{C}[x_1,..,x_n]$ and the formalism of an affine scheme extends to other rings such as $\mathbb{Z}$ as well.

My question then is how should I think of fractional ideals geometrically?

If an ideal of some integral domain $R$ corresponds to some closed set in $\operatorname{Spec}R$ with smaller ideals corresponding to bigger closed sets, then what should a really big fractional ideal like $\frac{1}{p} \mathbb{Z} \subset \mathbb{Q}$ correspond to? Maybe there is a scheme here that one can define that is bigger than $\operatorname{Spec}Z$ in this case?

• Fractional ideals of $\mathbb Z$ correspond to line bundles (or invertible sheaves, or locally free sheaves of rank $1$) on $\mathrm{Spec} \mathbb Z$. – Fredrik Meyer Aug 28 '14 at 17:20

Locally principal ideals turn into locally principal ideal sheaves, which are same thing as invertible sheaves equipped with an embedding $\mathcal L \hookrightarrow \mathcal O_X.$
Locally principal fractional ideals turn into locally principal fractional ideal sheaves, which are the same things as invertible sheaves equipped with an embedding $\mathcal L \hookrightarrow \mathcal K_X$, where $\mathcal K_X$ is the sheaf of rational functions on $X$ (an integral scheme, say).
General ideal sheaves (say on a Noetherian scheme $X$) are just coherent sheaves equipped with an embedding $\mathcal F \hookrightarrow \mathcal O_X$, and general fractional ideal sheaves are just coherent sheaves equipped with an embedding $\mathcal F \hookrightarrow \mathcal K_X$.
So when you work geometrically, the single piece of data (an ideal or fractional ideal) converts into two pieces of data: an abstract sheaf of $\mathcal O_X$-modules (invertible, or, more generally, coherent) together with an embedding into some standard sheaf, either $\mathcal O_X$ or $\mathcal K_X$. Forgetting the embedding corresponds to looking at equivalence classes of fractional ideals (in the sense of the class group).