text:
## Post #1

Estimation of the mean of a log-normally distributed variable

## Summary

I present below how to estimate the mean (and the standard deviation) of a log-normally distributed variable. We first start by simulating such a variable, we then define the log-likelihood function, and estimate the parameters. At the end, we compute the standard errors of our estimates.

## Simulating data

Let us consider the case of a log-normally distributed variable for which we want to estimate the (unobserved) mean µ. We start by simulating data by taking the exponential of a normally distributed variable with mean 0.5 and standard deviation 1.2.

mu=0.5
sigma=1.2
y_lognorm=exp(rnorm(n=2000,mean=mu,sd=sigma))

## Likelihood function

We now determine the log-likelihood function. This function takes as inputs the data $$y$$ and the vector of parameters theta_funct.

The function proceeds as follows:
(1) We retrieve the number of observations in n_funct.
(2) We set the mean mu_funct equal to the first element of the vector theta_funct.
(3) We set the standard deviation sigma_funct equal to the exponential of the second element of the vector theta_funct. (Note: we take the exponential to ensure positive values.)
(4) We compute the log-likehood of each observation.
(5) We return the sum of the log-likelihood of the dataset.

llfunction=function(y_funct,theta_funct){
n_funct=length(y_funct)
mu_funct=theta_funct[1]
sigma_funct=exp(theta_funct[2])
ll_funct=log(1/(y_funct*sigma_funct*sqrt(2*pi))*exp(-(log(y_funct)-mu_funct)^2/(2*sigma_funct^2)))
return(sum(ll_funct))
}

An alternative for to compute the contribution to the log-likelihood is to use the dedicated function in R. One would need to replace the definition of ll_funct in the previous code by:

  ll_funct=dlnorm(y_funct,meanlog=mu_funct,sdlog=sd_funct,log=TRUE)

## Estimation

We now can estimate the mean and the standard deviation of our variable of interest y. To do so, we need to prepare a function for the maximization procedure that has only one argument, i.e., the parameters to, be estimated.

my_mle=function(theta_funct){
return(llfunction(y_lognorm,theta_funct))
}

We can test the MLE function for the true values of the parameters to check that the function works:

my_mle(c(mu,sigma))
## [1] -5359.956

We can now launch the maximization process. We first need to define initial values (here we set both parameters equal to one). We also add fnscale=-1 to tell optim() to maximize my_mle() (the default is minimization).

theta_init=c(1,1) #Initial values: Mean 1 and st. dev. exp(1)
optimResults=optim(theta_init,my_mle,method="BFGS",control=list(fnscale=-1))

We first must check whether the algorithm achieved convergence. It is equal to 0 in case of successful convergence.

optimResults$convergence ## [1] 0 We can then retrieve the estimated parameters: estimated_mu=optimResults$par[1]
estimated_mu
## [1] 0.4949707
estimated_sd=exp(optimResults$par[2]) estimated_sd ## [1] 1.206872 Our estimates are very close to the true values $$0.5$$ and $$1.2$$. ## Standard errors Last, we can estimate the standard errors for our estimates. To do so, we compute the variance-covariance matrix, i.e., the inverse of Fisher information matrix. We need the package numDeriv to compute the hessian matrix. library("numDeriv") H=hessian(my_mle,optimResults$par)
CovarianceMatrix=solve(-H)
CovarianceMatrix
##               [,1]          [,2]
## [1,]  7.282706e-04 -8.886646e-11
## [2,] -8.886646e-11  2.500002e-04

The standard error of our estimate for the mean is given by the square-root elements of the first element of the diagonal:

se_mu=sqrt(CovarianceMatrix[1,1])
se_mu
## [1] 0.02698649


text:
# Single Spacing LaTeX TOC

• LaTeX
Gold Member
I'm running in the report document class and I was wondering how does one make the table of contents single spaced? Or possibly even 1 1/2 spaced. I tried some suggestions involving the tocloft package and everything pretty much went to hell. Due to some annoying issues with how the theses need to be done (I hate my university), the table of contents is created by

\chapter*{Chapter name}

Does anyone know a simple way to single space or 1 1/2 space the entries in the TOC?

Code:
 \def\tableofcontents{\@restonecolfalse\[email protected]\@restonecoltrue\onecolumn
\@starttoc{toc}\[email protected]\twocolumn\fi}

but I'm a bit clueless as to what's going on there :)

Simon Bridge
Homework Helper
\setlength in the preamble with tocloft is the usual way to customize TOC without editing the style.

The $\LeTeX$ default is single space though - oh but you have a TOC with only chapter headings?

Code:
\documentclass{book}
\usepackage{tocloft}

\setlength\cftparskip{-2pt}
\setlength\cftbeforechapskip{0pt}

\begin{document}
\tableofcontents
\clearpage

\chapter{Test chapter one}
\chapter{Test chapter two}

\end{document}

Gold Member
I have chapters and sections :) The thesis is mainly double spaced so I declared double spacing at the very beginning of everything.

I really want to avoid using tocloft because the style file I'm using does it's own stuff to the table of contents. Tocloft makes my headings for the TOC, LOF, and LOT go a bit crazy.

D H
Staff Emeritus

Code:
\def\tableofcontents{\@restonecolfalse\[email protected]\@restonecoltrue\onecolumn
\@starttoc{toc}\[email protected]\twocolumn\fi}

but I'm a bit clueless as to what's going on there :)
It's forcing the table of contents to be single column even if the document proper is two column.

Code:
\let\@[email protected]\tableofcontents
\def\tableofcontents{\begingroup\singlespace\@[email protected]\endgroup}

Gold Member
I assume I put that after the first part that I posted? I'm pretty clueless about latex :P

I put it after and I noticed the TOC shifted but was still double spaced.

Could it be because the TOC isn't being generated automatically as expected and instead is being compiled with \addcontents{toc}{chapter}{blahblahblah}?

Simon Bridge
Homework Helper
Oh now I get you - you want to start out single space for the toc and switch to double-space for the body. That would be quite unusual as a standard, which is probably why the style doesn't support it.

How about set the document to single-space and :

\usepackage{setspace}

Then switch between different spacing options with:

\doublespacing
\singlespacing
\onehalfspacing

For other sizes use the \setstretch command like this:

\setstretch{1.8}

Double-spacing is usually only used in drafts - which is often what is requested for marking. The actual paper would use a different format.

Gold Member
Thanks for the help guys. I just realized some code after that going as

Code:
 \def\[email protected]{\par
\baselineskip 0.15in  % added for smaller spacing
\@dottedtocline{1}{0.0em}{2.6em}}
\def\[email protected]{\baselineskip 0..15in  % added for smaller spacing
\@dottedtocline{1}{2.6em}{2.3em}}
\def\[email protected]{\baselineskip 0..15in  % added for smaller spacing
\@dottedtocline{2}{4.9em}{3.2em}}
\def\[email protected]{\baselineskip 0.20in  % added for smaller spacing
\@dottedtocline{3}{8.1em}{4.1em}}
\def\[email protected]{\baselineskip 0.20in  % added for smaller spacing
\@dottedtocline{4}{12.2em}{5em}}
\def\[email protected]{\baselineskip 0.20in  % added for smaller spacing
\@dottedtocline{5}{17.2em}{6em}}

was responsible for changing the spacing upon adding things to the TOC. I didn't really see anything that made me think it was doing anything to the TOC, but apparently it is :)

AlephZero
Homework Helper
I just realized some code after that going as...

Yup, that's the right place to be. If you do
Code:
\addcontents{toc}{foo}{...}
the TOC entry is printed by the command
Code:
\[email protected]

There's nothing special about using format identifiers like "chapter", "section" etc, except those are what the \chapter{...} and \section{...} commands generate for free. If you want to do something really fancy (like including short chapter summaries in the contents or whatever) you might want to define some additional formats.

The spacing of the TOC is (almost) completely independent of the spacing of the rest of the document, so \singlespace etc won't change anything.

I didn't really see anything that made me think it was doing anything to the TOC
Look at your .toc file, then read the code of \contentsline (assuming your packages haven't renamed it as something else). \contentsline is what makes the parameter "foo" execute the command \[email protected]

Last edited:

format: plain


text:
# Intuition on parameter gradient for the backward() method?

Let’s have this code:

import torch
y = 0.5*x+1
z = 3*y

It is written that backward() method parameter gradient is:

tensor. If it is a tensor, it will be automatically converted
to a Tensor that does not require grad unless create_graph is True.
None values can be specified for scalar Tensors or ones that
don’t require grad. If a None value would be acceptable then
this argument is optional.

What is the intuition to call this parameter gradient?
Usually we compute the gradient on scalar loss. loss.backward().

But what happens if we call the backward on a tensor that is not a scalar value?
Feedback is greatly appreciated.


text:
Can one be prosecuted in the US for buying stolen goods if one was unaware that it was stolen?

I have a practical example question that this applies to, so I'd like to know the answer in general, but specifically how it applies here.

Let's say I find something I'd like to buy on Craigslist, being sold on the order of $1000. Let's say I text with the guy, we meet up in public, and I buy the item with cash. Then let's say, it turns out it was stolen, the guy gets caught, and confesses to the police/victim and tells them my name/phone number/transaction details. I have several similar/related questions about this scenario: 1. How would this proceed? It seems like it would be very difficult to prove (short of getting public surveillance footage) that I even bought the item. What if I just said "sorry, I never ended up buying it." Surely they couldn't/wouldn't get a warrant to search my house? 2. Let's say they somehow do prove that I bought it (very contrived, but the guy was recording our conversation or something). Could I be prosecuted if I didn't know it was stolen? That's what this section seems to be saying, but that sounds crazy to me, because it would mean basically buying anything puts you at risk. I ask this because I was getting a little paranoid and was thinking about the implications of this: I'm imagining a scam where someone sells me a "stolen" item and then the "victim" (really their accomplice) says "I found out that you bought an item stolen from me, but the thief got away" and presents me with irrefutable evidence (e.g., a receipt for the item with its serial number) that the item was his. He could potentially threaten me to return it to him, and then I've lost my money (since the "thief" is in the wind) but they still have their item. It's unlikely, I know, but I'm wondering what the legal implications of this would be. Thank you! • If you were being smart then in #1 you would simply refuse to answer questions by any investigators. Lying to investigators about the substance of an investigation could constitute a crime like obstruction of justice. – feetwet Aug 17 '15 at 17:46 • Also, if soneobe says "you bought this but it was stolen from me," you should probably talk to a lawyer before handing anything over. – cpast Aug 17 '15 at 18:35 • Also, what state is your hypo in? – cpast Aug 17 '15 at 18:40 1 Answer You are conflating the crime against the state of possession stolen goods with the common law tort against the owner for conversion. To your questions: How would this proceed? It seems like it would be very difficult to prove (short of getting public surveillance footage) that I even bought the item. If you read the second paragraph of the page you linked it says: In many jurisdictions, if an individual has accepted possession of goods or property and knew they were stolen, then the individual is typically charged ... If the individual did not know the goods were stolen, then the goods are returned to the owner and the individual is not prosecuted. Proof of the crime involves a "beyond reasonable doubt" standard of evidence of both the fact that you have the goods and that you knew they were stolen. If you become aware that they were stolen (e.g. the police tell you) and try to keep them then you have just committed the crime. Proof of the tort requires a "balance of probabilities" standard of evidence that you have the goods and that they belong to someone else; your knowledge that they were stolen is immaterial. In the first instance, the police would probably knock on your door, tell you why they were there and ask if the version of the story they have from the thief is essentially true. What happens next depends on your response: 1. "Yes, I knew it was stolen; you better arrest me and I will plead guilty." This will play out as you expect. 2. "Yes, I didn't know it was stolen, I will go and get it for you." You return the goods, give a statement and may have to act as a witness in the prosecution of the thief. You are down$1,000 but are now older and wiser.
3. "No, I have no idea what you are talking about." Well, you have now committed the crime of hindering a police investigation and have also committed the crime of possessing stolen goods - you can no longer claim that you didn't know the goods were stolen; the police have told you they are. What happens next depends on if the police believe you or the thief.

Surely they couldn't/wouldn't get a warrant to search my house?

Want to bet? They certainly have enough to get a search warrant if they want one (probably). Whether they seek one probably depends on the value of the goods, how busy they are and how much you pissed them off.

Could I be prosecuted if I didn't know it was stolen?

Not if you return it as soon as practicable after being made aware that they were.

The scam

This seems like a lot of work for a very small return - spend your time worrying about things that are more likely to happen.

Good Title

All of this is tied up with the concept of good title. Basically, you cannot gain good title to property from someone who does not have good title themselves; if you buy goods from a thief you do not own them.

For example, if A has good title to the goods, B steals them and sells them to C who sells them to D then A still owns them and can demand their return from D, D could demand the return of their money from C and C could do likewise with B but as far as A is concerned it doesn't matter that C & B have lost money; that is simply too bad for them.

• Why not #4, "Oh, I didn't know it was stolen. I'll be glad to return it to the rightful owner as soon as the balance of probabilities proves it in civil court or as soon as I'm convinced of it, whichever comes first"? (But +1.) – msh210 Aug 18 '15 at 6:06
• This question law.stackexchange.com/questions/4554/… seems to contain usegul information... – DJohnM Dec 25 '17 at 23:11


text:
# Proof of additivity of the positive linear functional $\phi^+$ on a vector/Banach lattice that will be $\phi\vee0$.

For context, this is used in defining $$\phi\vee0$$ in a proof that the dual of a vector lattice is a vector lattice.

Given a linear functional $$\phi$$ on a vector lattice $$V$$, define $$\phi^+$$ on $$V^+$$ as $$\phi^+: v \mapsto \sup\{\phi(u) : 0.

How can I show $$\phi^+(v+w) = \phi^+(v) + \phi^+(w)\ \forall v,w \in V^+$$ ?

Thanks!


text:
# Gaussian Processes¶

See the Gaussian Processes tutorial for an introduction.

class Parameterized[source]

A wrapper of PyroModule whose parameters can be set constraints, set priors.

By default, when we set a prior to a parameter, an auto Delta guide will be created. We can use the method autoguide() to setup other auto guides.

Example:

>>> class Linear(Parameterized):
...     def __init__(self, a, b):
...         super().__init__()
...         self.a = Parameter(a)
...         self.b = Parameter(b)
...
...     def forward(self, x):
...         return self.a * x + self.b
...
>>> linear = Linear(torch.tensor(1.), torch.tensor(0.))
>>> linear.a = PyroParam(torch.tensor(1.), constraints.positive)
>>> linear.b = PyroSample(dist.Normal(0, 1))
>>> linear.autoguide("b", dist.Normal)
>>> assert "a_unconstrained" in dict(linear.named_parameters())
>>> assert "b_loc" in dict(linear.named_parameters())
>>> assert "b_scale_unconstrained" in dict(linear.named_parameters())

Note that by default, data of a parameter is a float torch.Tensor (unless we use torch.set_default_tensor_type() to change default tensor type). To cast these parameters to a correct data type or GPU device, we can call methods such as double() or cuda(). See torch.nn.Module for more information.

set_prior(name, prior)[source]

Sets prior for a parameter.

Parameters: name (str) – Name of the parameter. prior (Distribution) – A Pyro prior distribution.
autoguide(name, dist_constructor)[source]

Sets an autoguide for an existing parameter with name name (mimic the behavior of module pyro.infer.autoguide).

Note

dist_constructor should be one of Delta, Normal, and MultivariateNormal. More distribution constructor will be supported in the future if needed.

Parameters: name (str) – Name of the parameter. dist_constructor – A Distribution constructor.
set_mode(mode)[source]

Sets mode of this object to be able to use its parameters in stochastic functions. If mode="model", a parameter will get its value from its prior. If mode="guide", the value will be drawn from its guide.

Note

This method automatically sets mode for submodules which belong to Parameterized class.

Parameters: mode (str) – Either “model” or “guide”.
mode

## Models¶

### GPModel¶

class GPModel(X, y, kernel, mean_function=None, jitter=1e-06)[source]

Base class for Gaussian Process models.

The core of a Gaussian Process is a covariance function $$k$$ which governs the similarity between input points. Given $$k$$, we can establish a distribution over functions $$f$$ by a multivarite normal distribution

$p(f(X)) = \mathcal{N}(0, k(X, X)),$

where $$X$$ is any set of input points and $$k(X, X)$$ is a covariance matrix whose entries are outputs $$k(x, z)$$ of $$k$$ over input pairs $$(x, z)$$. This distribution is usually denoted by

$f \sim \mathcal{GP}(0, k).$

Note

Generally, beside a covariance matrix $$k$$, a Gaussian Process can also be specified by a mean function $$m$$ (which is a zero-value function by default). In that case, its distribution will be

$p(f(X)) = \mathcal{N}(m(X), k(X, X)).$

Gaussian Process models are Parameterized subclasses. So its parameters can be learned, set priors, or fixed by using corresponding methods from Parameterized. A typical way to define a Gaussian Process model is

>>> X = torch.tensor([[1., 5, 3], [4, 3, 7]])
>>> y = torch.tensor([2., 1])
>>> kernel = gp.kernels.RBF(input_dim=3)
>>> kernel.variance = pyro.nn.PyroSample(dist.Uniform(torch.tensor(0.5), torch.tensor(1.5)))
>>> kernel.lengthscale = pyro.nn.PyroSample(dist.Uniform(torch.tensor(1.0), torch.tensor(3.0)))
>>> gpr = gp.models.GPRegression(X, y, kernel)

There are two ways to train a Gaussian Process model:

• Using an MCMC algorithm (in module pyro.infer.mcmc) on model() to get posterior samples for the Gaussian Process’s parameters. For example:

>>> hmc_kernel = HMC(gpr.model)
>>> mcmc = MCMC(hmc_kernel, num_samples=10)
>>> mcmc.run()
>>> ls_name = "kernel.lengthscale"
>>> posterior_ls = mcmc.get_samples()[ls_name]

• Using a variational inference on the pair model(), guide():

>>> optimizer = torch.optim.Adam(gpr.parameters(), lr=0.01)
>>> loss_fn = pyro.infer.TraceMeanField_ELBO().differentiable_loss
>>>
>>> for i in range(1000):
...     svi.step()  # doctest: +SKIP
...     loss = loss_fn(gpr.model, gpr.guide)  # doctest: +SKIP
...     loss.backward()  # doctest: +SKIP
...     optimizer.step()

To give a prediction on new dataset, simply use forward() like any PyTorch torch.nn.Module:

>>> Xnew = torch.tensor([[2., 3, 1]])
>>> f_loc, f_cov = gpr(Xnew, full_cov=True)

Reference:

[1] Gaussian Processes for Machine Learning, Carl E. Rasmussen, Christopher K. I. Williams

Parameters: X (torch.Tensor) – A input data for training. Its first dimension is the number of data points. y (torch.Tensor) – An output data for training. Its last dimension is the number of data points. kernel (Kernel) – A Pyro kernel object, which is the covariance function $$k$$. mean_function (callable) – An optional mean function $$m$$ of this Gaussian process. By default, we use zero mean. jitter (float) – A small positive term which is added into the diagonal part of a covariance matrix to help stablize its Cholesky decomposition.
model()[source]

A “model” stochastic function. If self.y is None, this method returns mean and variance of the Gaussian Process prior.

guide()[source]

A “guide” stochastic function to be used in variational inference methods. It also gives posterior information to the method forward() for prediction.

forward(Xnew, full_cov=False)[source]

Computes the mean and covariance matrix (or variance) of Gaussian Process posterior on a test input data $$X_{new}$$:

$p(f^* \mid X_{new}, X, y, k, \theta),$

where $$\theta$$ are parameters of this model.

Note

Model’s parameters $$\theta$$ together with kernel’s parameters have been learned from a training procedure (MCMC or SVI).

Parameters: Xnew (torch.Tensor) – A input data for testing. Note that Xnew.shape[1:] must be the same as X.shape[1:]. full_cov (bool) – A flag to decide if we want to predict full covariance matrix or just variance. loc and covariance matrix (or variance) of $$p(f^*(X_{new}))$$
set_data(X, y=None)[source]

Sets data for Gaussian Process models.

Some examples to utilize this method are:

• Batch training on a sparse variational model:

>>> Xu = torch.tensor([[1., 0, 2]])  # inducing input
>>> likelihood = gp.likelihoods.Gaussian()
>>> vsgp = gp.models.VariationalSparseGP(X, y, kernel, Xu, likelihood)
>>> loss_fn = pyro.infer.TraceMeanField_ELBO().differentiable_loss
>>> batched_X, batched_y = X.split(split_size=10), y.split(split_size=10)
>>> for Xi, yi in zip(batched_X, batched_y):
...     vsgp.set_data(Xi, yi)
...     svi.step()  # doctest: +SKIP
...     loss = loss_fn(vsgp.model, vsgp.guide)  # doctest: +SKIP
...     loss.backward()  # doctest: +SKIP
...     optimizer.step()

• Making a two-layer Gaussian Process stochastic function:

>>> gpr1 = gp.models.GPRegression(X, None, kernel)
>>> Z, _ = gpr1.model()
>>> gpr2 = gp.models.GPRegression(Z, y, kernel)
>>> def two_layer_model():
...     Z, _ = gpr1.model()
...     gpr2.set_data(Z, y)
...     return gpr2.model()

References:

[1] Scalable Variational Gaussian Process Classification, James Hensman, Alexander G. de G. Matthews, Zoubin Ghahramani

[2] Deep Gaussian Processes, Andreas C. Damianou, Neil D. Lawrence

Parameters: X (torch.Tensor) – A input data for training. Its first dimension is the number of data points. y (torch.Tensor) – An output data for training. Its last dimension is the number of data points.

### GPRegression¶

class GPRegression(X, y, kernel, noise=None, mean_function=None, jitter=1e-06)[source]

Gaussian Process Regression model.

The core of a Gaussian Process is a covariance function $$k$$ which governs the similarity between input points. Given $$k$$, we can establish a distribution over functions $$f$$ by a multivarite normal distribution

$p(f(X)) = \mathcal{N}(0, k(X, X)),$

where $$X$$ is any set of input points and $$k(X, X)$$ is a covariance matrix whose entries are outputs $$k(x, z)$$ of $$k$$ over input pairs $$(x, z)$$. This distribution is usually denoted by

$f \sim \mathcal{GP}(0, k).$

Note

Generally, beside a covariance matrix $$k$$, a Gaussian Process can also be specified by a mean function $$m$$ (which is a zero-value function by default). In that case, its distribution will be

$p(f(X)) = \mathcal{N}(m(X), k(X, X)).$

Given inputs $$X$$ and their noisy observations $$y$$, the Gaussian Process Regression model takes the form

$\begin{split}f &\sim \mathcal{GP}(0, k(X, X)),\\ y & \sim f + \epsilon,\end{split}$

where $$\epsilon$$ is Gaussian noise.

Note

This model has $$\mathcal{O}(N^3)$$ complexity for training, $$\mathcal{O}(N^3)$$ complexity for testing. Here, $$N$$ is the number of train inputs.

Reference:

[1] Gaussian Processes for Machine Learning, Carl E. Rasmussen, Christopher K. I. Williams

Parameters: X (torch.Tensor) – A input data for training. Its first dimension is the number of data points. y (torch.Tensor) – An output data for training. Its last dimension is the number of data points. kernel (Kernel) – A Pyro kernel object, which is the covariance function $$k$$. noise (torch.Tensor) – Variance of Gaussian noise of this model. mean_function (callable) – An optional mean function $$m$$ of this Gaussian process. By default, we use zero mean. jitter (float) – A small positive term which is added into the diagonal part of a covariance matrix to help stablize its Cholesky decomposition.
model()[source]
guide()[source]
forward(Xnew, full_cov=False, noiseless=True)[source]

Computes the mean and covariance matrix (or variance) of Gaussian Process posterior on a test input data $$X_{new}$$:

$p(f^* \mid X_{new}, X, y, k, \epsilon) = \mathcal{N}(loc, cov).$

Note

The noise parameter noise ($$\epsilon$$) together with kernel’s parameters have been learned from a training procedure (MCMC or SVI).

Parameters: Xnew (torch.Tensor) – A input data for testing. Note that Xnew.shape[1:] must be the same as self.X.shape[1:]. full_cov (bool) – A flag to decide if we want to predict full covariance matrix or just variance. noiseless (bool) – A flag to decide if we want to include noise in the prediction output or not. loc and covariance matrix (or variance) of $$p(f^*(X_{new}))$$
iter_sample(noiseless=True)[source]

Iteratively constructs a sample from the Gaussian Process posterior.

Recall that at test input points $$X_{new}$$, the posterior is multivariate Gaussian distributed with mean and covariance matrix given by forward().

This method samples lazily from this multivariate Gaussian. The advantage of this approach is that later query points can depend upon earlier ones. Particularly useful when the querying is to be done by an optimisation routine.

Note

The noise parameter noise ($$\epsilon$$) together with kernel’s parameters have been learned from a training procedure (MCMC or SVI).

Parameters: noiseless (bool) – A flag to decide if we want to add sampling noise to the samples beyond the noise inherent in the GP posterior. sampler function

### SparseGPRegression¶

class SparseGPRegression(X, y, kernel, Xu, noise=None, mean_function=None, approx=None, jitter=1e-06)[source]

Sparse Gaussian Process Regression model.

In GPRegression model, when the number of input data $$X$$ is large, the covariance matrix $$k(X, X)$$ will require a lot of computational steps to compute its inverse (for log likelihood and for prediction). By introducing an additional inducing-input parameter $$X_u$$, we can reduce computational cost by approximate $$k(X, X)$$ by a low-rank Nymström approximation $$Q$$ (see reference [1]), where

$Q = k(X, X_u) k(X,X)^{-1} k(X_u, X).$

Given inputs $$X$$, their noisy observations $$y$$, and the inducing-input parameters $$X_u$$, the model takes the form:

$\begin{split}u & \sim \mathcal{GP}(0, k(X_u, X_u)),\\ f & \sim q(f \mid X, X_u) = \mathbb{E}_{p(u)}q(f\mid X, X_u, u),\\ y & \sim f + \epsilon,\end{split}$

where $$\epsilon$$ is Gaussian noise and the conditional distribution $$q(f\mid X, X_u, u)$$ is an approximation of

$p(f\mid X, X_u, u) = \mathcal{N}(m, k(X, X) - Q),$

whose terms $$m$$ and $$k(X, X) - Q$$ is derived from the joint multivariate normal distribution:

$[f, u] \sim \mathcal{GP}(0, k([X, X_u], [X, X_u])).$

This class implements three approximation methods:

• Deterministic Training Conditional (DTC):

$q(f\mid X, X_u, u) = \mathcal{N}(m, 0),$

which in turns will imply

$f \sim \mathcal{N}(0, Q).$
• Fully Independent Training Conditional (FITC):

$q(f\mid X, X_u, u) = \mathcal{N}(m, diag(k(X, X) - Q)),$

which in turns will correct the diagonal part of the approximation in DTC:

$f \sim \mathcal{N}(0, Q + diag(k(X, X) - Q)).$
• Variational Free Energy (VFE), which is similar to DTC but has an additional trace_term in the model’s log likelihood. This additional term makes “VFE” equivalent to the variational approach in SparseVariationalGP (see reference [2]).

Note

This model has $$\mathcal{O}(NM^2)$$ complexity for training, $$\mathcal{O}(NM^2)$$ complexity for testing. Here, $$N$$ is the number of train inputs, $$M$$ is the number of inducing inputs.

References:

[1] A Unifying View of Sparse Approximate Gaussian Process Regression, Joaquin Quiñonero-Candela, Carl E. Rasmussen

[2] Variational learning of inducing variables in sparse Gaussian processes, Michalis Titsias

Parameters: X (torch.Tensor) – A input data for training. Its first dimension is the number of data points. y (torch.Tensor) – An output data for training. Its last dimension is the number of data points. kernel (Kernel) – A Pyro kernel object, which is the covariance function $$k$$. Xu (torch.Tensor) – Initial values for inducing points, which are parameters of our model. noise (torch.Tensor) – Variance of Gaussian noise of this model. mean_function (callable) – An optional mean function $$m$$ of this Gaussian process. By default, we use zero mean. approx (str) – One of approximation methods: “DTC”, “FITC”, and “VFE” (default). jitter (float) – A small positive term which is added into the diagonal part of a covariance matrix to help stablize its Cholesky decomposition. name (str) – Name of this model.
model()[source]
guide()[source]
forward(Xnew, full_cov=False, noiseless=True)[source]

Computes the mean and covariance matrix (or variance) of Gaussian Process posterior on a test input data $$X_{new}$$:

$p(f^* \mid X_{new}, X, y, k, X_u, \epsilon) = \mathcal{N}(loc, cov).$

Note

The noise parameter noise ($$\epsilon$$), the inducing-point parameter Xu, together with kernel’s parameters have been learned from a training procedure (MCMC or SVI).

Parameters: Xnew (torch.Tensor) – A input data for testing. Note that Xnew.shape[1:] must be the same as self.X.shape[1:]. full_cov (bool) – A flag to decide if we want to predict full covariance matrix or just variance. noiseless (bool) – A flag to decide if we want to include noise in the prediction output or not. loc and covariance matrix (or variance) of $$p(f^*(X_{new}))$$

### VariationalGP¶

class VariationalGP(X, y, kernel, likelihood, mean_function=None, latent_shape=None, whiten=False, jitter=1e-06)[source]

Variational Gaussian Process model.

This model deals with both Gaussian and non-Gaussian likelihoods. Given inputs$$X$$ and their noisy observations $$y$$, the model takes the form

$\begin{split}f &\sim \mathcal{GP}(0, k(X, X)),\\ y & \sim p(y) = p(y \mid f) p(f),\end{split}$

where $$p(y \mid f)$$ is the likelihood.

We will use a variational approach in this model by approximating $$q(f)$$ to the posterior $$p(f\mid y)$$. Precisely, $$q(f)$$ will be a multivariate normal distribution with two parameters f_loc and f_scale_tril, which will be learned during a variational inference process.

Note

This model can be seen as a special version of SparseVariationalGP model with $$X_u = X$$.

Note

This model has $$\mathcal{O}(N^3)$$ complexity for training, $$\mathcal{O}(N^3)$$ complexity for testing. Here, $$N$$ is the number of train inputs. Size of variational parameters is $$\mathcal{O}(N^2)$$.

Parameters: X (torch.Tensor) – A input data for training. Its first dimension is the number of data points. y (torch.Tensor) – An output data for training. Its last dimension is the number of data points. kernel (Kernel) – A Pyro kernel object, which is the covariance function $$k$$. Likelihood likelihood (likelihood) – A likelihood object. mean_function (callable) – An optional mean function $$m$$ of this Gaussian process. By default, we use zero mean. latent_shape (torch.Size) – Shape for latent processes (batch_shape of $$q(f)$$). By default, it equals to output batch shape y.shape[:-1]. For the multi-class classification problems, latent_shape[-1] should corresponse to the number of classes. whiten (bool) – A flag to tell if variational parameters f_loc and f_scale_tril are transformed by the inverse of Lff, where Lff is the lower triangular decomposition of $$kernel(X, X)$$. Enable this flag will help optimization. jitter (float) – A small positive term which is added into the diagonal part of a covariance matrix to help stablize its Cholesky decomposition.
model()[source]
guide()[source]
forward(Xnew, full_cov=False)[source]

Computes the mean and covariance matrix (or variance) of Gaussian Process posterior on a test input data $$X_{new}$$:

$p(f^* \mid X_{new}, X, y, k, f_{loc}, f_{scale\_tril}) = \mathcal{N}(loc, cov).$

Note

Variational parameters f_loc, f_scale_tril, together with kernel’s parameters have been learned from a training procedure (MCMC or SVI).

Parameters: Xnew (torch.Tensor) – A input data for testing. Note that Xnew.shape[1:] must be the same as self.X.shape[1:]. full_cov (bool) – A flag to decide if we want to predict full covariance matrix or just variance. loc and covariance matrix (or variance) of $$p(f^*(X_{new}))$$

### VariationalSparseGP¶

class VariationalSparseGP(X, y, kernel, Xu, likelihood, mean_function=None, latent_shape=None, num_data=None, whiten=False, jitter=1e-06)[source]

Variational Sparse Gaussian Process model.

In VariationalGP model, when the number of input data $$X$$ is large, the covariance matrix $$k(X, X)$$ will require a lot of computational steps to compute its inverse (for log likelihood and for prediction). This model introduces an additional inducing-input parameter $$X_u$$ to solve that problem. Given inputs $$X$$, their noisy observations $$y$$, and the inducing-input parameters $$X_u$$, the model takes the form:

$\begin{split}[f, u] &\sim \mathcal{GP}(0, k([X, X_u], [X, X_u])),\\ y & \sim p(y) = p(y \mid f) p(f),\end{split}$

where $$p(y \mid f)$$ is the likelihood.

We will use a variational approach in this model by approximating $$q(f,u)$$ to the posterior $$p(f,u \mid y)$$. Precisely, $$q(f) = p(f\mid u)q(u)$$, where $$q(u)$$ is a multivariate normal distribution with two parameters u_loc and u_scale_tril, which will be learned during a variational inference process.

Note

This model can be learned using MCMC method as in reference [2]. See also GPModel.

Note

This model has $$\mathcal{O}(NM^2)$$ complexity for training, $$\mathcal{O}(M^3)$$ complexity for testing. Here, $$N$$ is the number of train inputs, $$M$$ is the number of inducing inputs. Size of variational parameters is $$\mathcal{O}(M^2)$$.

References:

[1] Scalable variational Gaussian process classification, James Hensman, Alexander G. de G. Matthews, Zoubin Ghahramani

[2] MCMC for Variationally Sparse Gaussian Processes, James Hensman, Alexander G. de G. Matthews, Maurizio Filippone, Zoubin Ghahramani

Parameters: X (torch.Tensor) – A input data for training. Its first dimension is the number of data points. y (torch.Tensor) – An output data for training. Its last dimension is the number of data points. kernel (Kernel) – A Pyro kernel object, which is the covariance function $$k$$. Xu (torch.Tensor) – Initial values for inducing points, which are parameters of our model. Likelihood likelihood (likelihood) – A likelihood object. mean_function (callable) – An optional mean function $$m$$ of this Gaussian process. By default, we use zero mean. latent_shape (torch.Size) – Shape for latent processes (batch_shape of $$q(u)$$). By default, it equals to output batch shape y.shape[:-1]. For the multi-class classification problems, latent_shape[-1] should corresponse to the number of classes. num_data (int) – The size of full training dataset. It is useful for training this model with mini-batch. whiten (bool) – A flag to tell if variational parameters u_loc and u_scale_tril are transformed by the inverse of Luu, where Luu is the lower triangular decomposition of $$kernel(X_u, X_u)$$. Enable this flag will help optimization. jitter (float) – A small positive term which is added into the diagonal part of a covariance matrix to help stablize its Cholesky decomposition.
model()[source]
guide()[source]
forward(Xnew, full_cov=False)[source]

Computes the mean and covariance matrix (or variance) of Gaussian Process posterior on a test input data $$X_{new}$$:

$p(f^* \mid X_{new}, X, y, k, X_u, u_{loc}, u_{scale\_tril}) = \mathcal{N}(loc, cov).$

Note

Variational parameters u_loc, u_scale_tril, the inducing-point parameter Xu, together with kernel’s parameters have been learned from a training procedure (MCMC or SVI).

Parameters: Xnew (torch.Tensor) – A input data for testing. Note that Xnew.shape[1:] must be the same as self.X.shape[1:]. full_cov (bool) – A flag to decide if we want to predict full covariance matrix or just variance. loc and covariance matrix (or variance) of $$p(f^*(X_{new}))$$

### GPLVM¶

class GPLVM(base_model)[source]

Gaussian Process Latent Variable Model (GPLVM) model.

GPLVM is a Gaussian Process model with its train input data is a latent variable. This model is useful for dimensional reduction of high dimensional data. Assume the mapping from low dimensional latent variable to is a Gaussian Process instance. Then the high dimensional data will play the role of train output y and our target is to learn latent inputs which best explain y. For the purpose of dimensional reduction, latent inputs should have lower dimensions than y.

We follows reference [1] to put a unit Gaussian prior to the input and approximate its posterior by a multivariate normal distribution with two variational parameters: X_loc and X_scale_tril.

For example, we can do dimensional reduction on Iris dataset as follows:

>>> # With y as the 2D Iris data of shape 150x4 and we want to reduce its dimension
>>> # to a tensor X of shape 150x2, we will use GPLVM.

>>> # First, define the initial values for X parameter:
>>> X_init = torch.zeros(150, 2)
>>> # Then, define a Gaussian Process model with input X_init and output y:
>>> kernel = gp.kernels.RBF(input_dim=2, lengthscale=torch.ones(2))
>>> Xu = torch.zeros(20, 2)  # initial inducing inputs of sparse model
>>> gpmodule = gp.models.SparseGPRegression(X_init, y, kernel, Xu)
>>> # Finally, wrap gpmodule by GPLVM, optimize, and get the "learned" mean of X:
>>> gplvm = gp.models.GPLVM(gpmodule)
>>> gp.util.train(gplvm)  # doctest: +SKIP
>>> X = gplvm.X

Reference:

[1] Bayesian Gaussian Process Latent Variable Model Michalis K. Titsias, Neil D. Lawrence

Parameters: base_model (GPModel) – A Pyro Gaussian Process model object. Note that base_model.X will be the initial value for the variational parameter X_loc.
model()[source]
guide()[source]
forward(**kwargs)[source]

Forward method has the same signal as its base_model. Note that the train input data of base_model is sampled from GPLVM.

## Kernels¶

### Kernel¶

class Kernel(input_dim, active_dims=None)[source]

Base class for kernels used in this Gaussian Process module.

Every inherited class should implement a forward() pass which takes inputs $$X$$, $$Z$$ and returns their covariance matrix.

To construct a new kernel from the old ones, we can use methods add(), mul(), exp(), warp(), vertical_scale().

References:

[1] Gaussian Processes for Machine Learning, Carl E. Rasmussen, Christopher K. I. Williams

Parameters: input_dim (int) – Number of feature dimensions of inputs. variance (torch.Tensor) – Variance parameter of this kernel. active_dims (list) – List of feature dimensions of the input which the kernel acts on.
forward(X, Z=None, diag=False)[source]

Calculates covariance matrix of inputs on active dimensionals.

Parameters: X (torch.Tensor) – A 2D tensor with shape $$N \times input\_dim$$. Z (torch.Tensor) – An (optional) 2D tensor with shape $$M \times input\_dim$$. diag (bool) – A flag to decide if we want to return full covariance matrix or just its diagonal part. covariance matrix of $$X$$ and $$Z$$ with shape $$N \times M$$ torch.Tensor

### Brownian¶

class Brownian(input_dim, variance=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

This kernel correponds to a two-sided Brownion motion (Wiener process):

$$k(x,z)=\begin{cases}\sigma^2\min(|x|,|z|),& \text{if } x\cdot z\ge 0\\ 0, & \text{otherwise}. \end{cases}$$

Note that the input dimension of this kernel must be 1.

Reference:

[1] Theory and Statistical Applications of Stochastic Processes, Yuliya Mishura, Georgiy Shevchenko

forward(X, Z=None, diag=False)[source]

### Combination¶

class Combination(kern0, kern1)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

Base class for kernels derived from a combination of kernels.

Parameters: kern0 (Kernel) – First kernel to combine. kern1 (Kernel or numbers.Number) – Second kernel to combine.

### Constant¶

class Constant(input_dim, variance=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

Implementation of Constant kernel:

$$k(x, z) = \sigma^2.$$
forward(X, Z=None, diag=False)[source]

### Coregionalize¶

class Coregionalize(input_dim, rank=None, components=None, diagonal=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

A kernel for the linear model of coregionalization $$k(x,z) = x^T (W W^T + D) z$$ where $$W$$ is an input_dim-by-rank matrix and typically rank < input_dim, and D is a diagonal matrix.

This generalizes the Linear kernel to multiple features with a low-rank-plus-diagonal weight matrix. The typical use case is for modeling correlations among outputs of a multi-output GP, where outputs are coded as distinct data points with one-hot coded features denoting which output each datapoint represents.

If only rank is specified, the kernel (W W^T + D) will be randomly initialized to a matrix with expected value the identity matrix.

References:

[1] Mauricio A. Alvarez, Lorenzo Rosasco, Neil D. Lawrence (2012)
Kernels for Vector-Valued Functions: a Review
Parameters: input_dim (int) – Number of feature dimensions of inputs. rank (int) – Optional rank. This is only used if components is unspecified. If neigher rank nor components is specified, then rank defaults to input_dim. components (torch.Tensor) – An optional (input_dim, rank) shaped matrix that maps features to rank-many components. If unspecified, this will be randomly initialized. diagonal (torch.Tensor) – An optional vector of length input_dim. If unspecified, this will be set to constant 0.5. active_dims (list) – List of feature dimensions of the input which the kernel acts on. name (str) – Name of the kernel.
forward(X, Z=None, diag=False)[source]

### Cosine¶

class Cosine(input_dim, variance=None, lengthscale=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.isotropic.Isotropy

Implementation of Cosine kernel:

$$k(x,z) = \sigma^2 \cos\left(\frac{|x-z|}{l}\right).$$
Parameters: lengthscale (torch.Tensor) – Length-scale parameter of this kernel.
forward(X, Z=None, diag=False)[source]

### DotProduct¶

class DotProduct(input_dim, variance=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

Base class for kernels which are functions of $$x \cdot z$$.

### Exponent¶

class Exponent(kern)[source]

Bases: pyro.contrib.gp.kernels.kernel.Transforming

Creates a new kernel according to

$$k_{new}(x, z) = \exp(k(x, z)).$$
forward(X, Z=None, diag=False)[source]

### Exponential¶

class Exponential(input_dim, variance=None, lengthscale=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.isotropic.Isotropy

Implementation of Exponential kernel:

$$k(x, z) = \sigma^2\exp\left(-\frac{|x-z|}{l}\right).$$
forward(X, Z=None, diag=False)[source]

### Isotropy¶

class Isotropy(input_dim, variance=None, lengthscale=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

Base class for a family of isotropic covariance kernels which are functions of the distance $$|x-z|/l$$, where $$l$$ is the length-scale parameter.

By default, the parameter lengthscale has size 1. To use the isotropic version (different lengthscale for each dimension), make sure that lengthscale has size equal to input_dim.

Parameters: lengthscale (torch.Tensor) – Length-scale parameter of this kernel.

### Linear¶

class Linear(input_dim, variance=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.dot_product.DotProduct

Implementation of Linear kernel:

$$k(x, z) = \sigma^2 x \cdot z.$$

Doing Gaussian Process regression with linear kernel is equivalent to doing a linear regression.

Note

Here we implement the homogeneous version. To use the inhomogeneous version, consider using Polynomial kernel with degree=1 or making a Sum with a Constant kernel.

forward(X, Z=None, diag=False)[source]

### Matern32¶

class Matern32(input_dim, variance=None, lengthscale=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.isotropic.Isotropy

Implementation of Matern32 kernel:

$$k(x, z) = \sigma^2\left(1 + \sqrt{3} \times \frac{|x-z|}{l}\right) \exp\left(-\sqrt{3} \times \frac{|x-z|}{l}\right).$$
forward(X, Z=None, diag=False)[source]

### Matern52¶

class Matern52(input_dim, variance=None, lengthscale=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.isotropic.Isotropy

Implementation of Matern52 kernel:

$$k(x,z)=\sigma^2\left(1+\sqrt{5}\times\frac{|x-z|}{l}+\frac{5}{3}\times \frac{|x-z|^2}{l^2}\right)\exp\left(-\sqrt{5} \times \frac{|x-z|}{l}\right).$$
forward(X, Z=None, diag=False)[source]

### Periodic¶

class Periodic(input_dim, variance=None, lengthscale=None, period=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

Implementation of Periodic kernel:

$$k(x,z)=\sigma^2\exp\left(-2\times\frac{\sin^2(\pi(x-z)/p)}{l^2}\right),$$

where $$p$$ is the period parameter.

References:

[1] Introduction to Gaussian processes, David J.C. MacKay

Parameters: lengthscale (torch.Tensor) – Length scale parameter of this kernel. period (torch.Tensor) – Period parameter of this kernel.
forward(X, Z=None, diag=False)[source]

### Polynomial¶

class Polynomial(input_dim, variance=None, bias=None, degree=1, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.dot_product.DotProduct

Implementation of Polynomial kernel:

$$k(x, z) = \sigma^2(\text{bias} + x \cdot z)^d.$$
Parameters: bias (torch.Tensor) – Bias parameter of this kernel. Should be positive. degree (int) – Degree $$d$$ of the polynomial.
forward(X, Z=None, diag=False)[source]

### Product¶

class Product(kern0, kern1)[source]

Bases: pyro.contrib.gp.kernels.kernel.Combination

Returns a new kernel which acts like a product/tensor product of two kernels. The second kernel can be a constant.

forward(X, Z=None, diag=False)[source]

### RBF¶

class RBF(input_dim, variance=None, lengthscale=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.isotropic.Isotropy

Implementation of Radial Basis Function kernel:

$$k(x,z) = \sigma^2\exp\left(-0.5 \times \frac{|x-z|^2}{l^2}\right).$$

Note

This kernel also has name Squared Exponential in literature.

forward(X, Z=None, diag=False)[source]

class RationalQuadratic(input_dim, variance=None, lengthscale=None, scale_mixture=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.isotropic.Isotropy

$$k(x, z) = \sigma^2 \left(1 + 0.5 \times \frac{|x-z|^2}{\alpha l^2} \right)^{-\alpha}.$$
Parameters: scale_mixture (torch.Tensor) – Scale mixture ($$\alpha$$) parameter of this kernel. Should have size 1.
forward(X, Z=None, diag=False)[source]

### Sum¶

class Sum(kern0, kern1)[source]

Bases: pyro.contrib.gp.kernels.kernel.Combination

Returns a new kernel which acts like a sum/direct sum of two kernels. The second kernel can be a constant.

forward(X, Z=None, diag=False)[source]

### Transforming¶

class Transforming(kern)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

Base class for kernels derived from a kernel by some transforms such as warping, exponent, vertical scaling.

Parameters: kern (Kernel) – The original kernel.

### VerticalScaling¶

class VerticalScaling(kern, vscaling_fn)[source]

Bases: pyro.contrib.gp.kernels.kernel.Transforming

Creates a new kernel according to

$$k_{new}(x, z) = f(x)k(x, z)f(z),$$

where $$f$$ is a function.

Parameters: vscaling_fn (callable) – A vertical scaling function $$f$$.
forward(X, Z=None, diag=False)[source]

### Warping¶

class Warping(kern, iwarping_fn=None, owarping_coef=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Transforming

Creates a new kernel according to

$$k_{new}(x, z) = q(k(f(x), f(z))),$$

where $$f$$ is an function and $$q$$ is a polynomial with non-negative coefficients owarping_coef.

We can take advantage of $$f$$ to combine a Gaussian Process kernel with a deep learning architecture. For example:

>>> linear = torch.nn.Linear(10, 3)
>>> # register its parameters to Pyro's ParamStore and wrap it by lambda
>>> # to call the primitive pyro.module each time we use the linear function
>>> pyro_linear_fn = lambda x: pyro.module("linear", linear)(x)
>>> kernel = gp.kernels.Matern52(input_dim=3, lengthscale=torch.ones(3))
>>> warped_kernel = gp.kernels.Warping(kernel, pyro_linear_fn)

Reference:

[1] Deep Kernel Learning, Andrew G. Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing

Parameters: iwarping_fn (callable) – An input warping function $$f$$. owarping_coef (list) – A list of coefficients of the output warping polynomial. These coefficients must be non-negative.
forward(X, Z=None, diag=False)[source]

### WhiteNoise¶

class WhiteNoise(input_dim, variance=None, active_dims=None)[source]

Bases: pyro.contrib.gp.kernels.kernel.Kernel

Implementation of WhiteNoise kernel:

$$k(x, z) = \sigma^2 \delta(x, z),$$

where $$\delta$$ is a Dirac delta function.

forward(X, Z=None, diag=False)[source]

## Likelihoods¶

### Likelihood¶

class Likelihood[source]

Base class for likelihoods used in Gaussian Process.

Every inherited class should implement a forward pass which takes an input $$f$$ and returns a sample $$y$$.

forward(f_loc, f_var, y=None)[source]

Samples $$y$$ given $$f_{loc}$$, $$f_{var}$$.

Parameters: f_loc (torch.Tensor) – Mean of latent function output. f_var (torch.Tensor) – Variance of latent function output. y (torch.Tensor) – Training output tensor. a tensor sampled from likelihood torch.Tensor

### Binary¶

class Binary(response_function=None)[source]

Bases: pyro.contrib.gp.likelihoods.likelihood.Likelihood

Implementation of Binary likelihood, which is used for binary classification problems.

Binary likelihood uses Bernoulli distribution, so the output of response_function should be in range $$(0,1)$$. By default, we use sigmoid function.

Parameters: response_function (callable) – A mapping to correct domain for Binary likelihood.
forward(f_loc, f_var, y=None)[source]

Samples $$y$$ given $$f_{loc}$$, $$f_{var}$$ according to

$\begin{split}f & \sim \mathbb{Normal}(f_{loc}, f_{var}),\\ y & \sim \mathbb{Bernoulli}(f).\end{split}$

Note

The log likelihood is estimated using Monte Carlo with 1 sample of $$f$$.

Parameters: f_loc (torch.Tensor) – Mean of latent function output. f_var (torch.Tensor) – Variance of latent function output. y (torch.Tensor) – Training output tensor. a tensor sampled from likelihood torch.Tensor

### Gaussian¶

class Gaussian(variance=None)[source]

Bases: pyro.contrib.gp.likelihoods.likelihood.Likelihood

Implementation of Gaussian likelihood, which is used for regression problems.

Gaussian likelihood uses Normal distribution.

Parameters: variance (torch.Tensor) – A variance parameter, which plays the role of noise in regression problems.
forward(f_loc, f_var, y=None)[source]

Samples $$y$$ given $$f_{loc}$$, $$f_{var}$$ according to

$y \sim \mathbb{Normal}(f_{loc}, f_{var} + \epsilon),$

where $$\epsilon$$ is the variance parameter of this likelihood.

Parameters: f_loc (torch.Tensor) – Mean of latent function output. f_var (torch.Tensor) – Variance of latent function output. y (torch.Tensor) – Training output tensor. a tensor sampled from likelihood torch.Tensor

### MultiClass¶

class MultiClass(num_classes, response_function=None)[source]

Bases: pyro.contrib.gp.likelihoods.likelihood.Likelihood

Implementation of MultiClass likelihood, which is used for multi-class classification problems.

MultiClass likelihood uses Categorical distribution, so response_function should normalize its input’s rightmost axis. By default, we use softmax function.

Parameters: num_classes (int) – Number of classes for prediction. response_function (callable) – A mapping to correct domain for MultiClass likelihood.
forward(f_loc, f_var, y=None)[source]

Samples $$y$$ given $$f_{loc}$$, $$f_{var}$$ according to

$\begin{split}f & \sim \mathbb{Normal}(f_{loc}, f_{var}),\\ y & \sim \mathbb{Categorical}(f).\end{split}$

Note

The log likelihood is estimated using Monte Carlo with 1 sample of $$f$$.

Parameters: f_loc (torch.Tensor) – Mean of latent function output. f_var (torch.Tensor) – Variance of latent function output. y (torch.Tensor) – Training output tensor. a tensor sampled from likelihood torch.Tensor

### Poisson¶

class Poisson(response_function=None)[source]

Bases: pyro.contrib.gp.likelihoods.likelihood.Likelihood

Implementation of Poisson likelihood, which is used for count data.

Poisson likelihood uses the Poisson distribution, so the output of response_function should be positive. By default, we use torch.exp() as response function, corresponding to a log-Gaussian Cox process.

Parameters: response_function (callable) – A mapping to positive real numbers.
forward(f_loc, f_var, y=None)[source]

Samples $$y$$ given $$f_{loc}$$, $$f_{var}$$ according to

$\begin{split}f & \sim \mathbb{Normal}(f_{loc}, f_{var}),\\ y & \sim \mathbb{Poisson}(\exp(f)).\end{split}$

Note

The log likelihood is estimated using Monte Carlo with 1 sample of $$f$$.

Parameters: f_loc (torch.Tensor) – Mean of latent function output. f_var (torch.Tensor) – Variance of latent function output. y (torch.Tensor) – Training output tensor. a tensor sampled from likelihood torch.Tensor

## Parameterized¶

class Parameterized[source]

A wrapper of PyroModule whose parameters can be set constraints, set priors.

By default, when we set a prior to a parameter, an auto Delta guide will be created. We can use the method autoguide() to setup other auto guides.

Example:

>>> class Linear(Parameterized):
...     def __init__(self, a, b):
...         super().__init__()
...         self.a = Parameter(a)
...         self.b = Parameter(b)
...
...     def forward(self, x):
...         return self.a * x + self.b
...
>>> linear = Linear(torch.tensor(1.), torch.tensor(0.))
>>> linear.a = PyroParam(torch.tensor(1.), constraints.positive)
>>> linear.b = PyroSample(dist.Normal(0, 1))
>>> linear.autoguide("b", dist.Normal)
>>> assert "a_unconstrained" in dict(linear.named_parameters())
>>> assert "b_loc" in dict(linear.named_parameters())
>>> assert "b_scale_unconstrained" in dict(linear.named_parameters())

Note that by default, data of a parameter is a float torch.Tensor (unless we use torch.set_default_tensor_type() to change default tensor type). To cast these parameters to a correct data type or GPU device, we can call methods such as double() or cuda(). See torch.nn.Module for more information.

set_prior(name, prior)[source]

Sets prior for a parameter.

Parameters: name (str) – Name of the parameter. prior (Distribution) – A Pyro prior distribution.
autoguide(name, dist_constructor)[source]

Sets an autoguide for an existing parameter with name name (mimic the behavior of module pyro.infer.autoguide).

Note

dist_constructor should be one of Delta, Normal, and MultivariateNormal. More distribution constructor will be supported in the future if needed.

Parameters: name (str) – Name of the parameter. dist_constructor – A Distribution constructor.
set_mode(mode)[source]

Sets mode of this object to be able to use its parameters in stochastic functions. If mode="model", a parameter will get its value from its prior. If mode="guide", the value will be drawn from its guide.

Note

This method automatically sets mode for submodules which belong to Parameterized class.

Parameters: mode (str) – Either “model” or “guide”.
mode

## Util¶

conditional(Xnew, X, kernel, f_loc, f_scale_tril=None, Lff=None, full_cov=False, whiten=False, jitter=1e-06)[source]

Given $$X_{new}$$, predicts loc and covariance matrix of the conditional multivariate normal distribution

$p(f^*(X_{new}) \mid X, k, f_{loc}, f_{scale\_tril}).$

Here f_loc and f_scale_tril are variation parameters of the variational distribution

$q(f \mid f_{loc}, f_{scale\_tril}) \sim p(f | X, y),$

where $$f$$ is the function value of the Gaussian Process given input $$X$$

$p(f(X)) \sim \mathcal{N}(0, k(X, X))$

and $$y$$ is computed from $$f$$ by some likelihood function $$p(y|f)$$.

In case f_scale_tril=None, we consider $$f = f_{loc}$$ and computes

$p(f^*(X_{new}) \mid X, k, f).$

In case f_scale_tril is not None, we follow the derivation from reference [1]. For the case f_scale_tril=None, we follow the popular reference [2].

References:

[2] Gaussian Processes for Machine Learning, Carl E. Rasmussen, Christopher K. I. Williams

Parameters: Xnew (torch.Tensor) – A new input data. X (torch.Tensor) – An input data to be conditioned on. kernel (Kernel) – A Pyro kernel object. f_loc (torch.Tensor) – Mean of $$q(f)$$. In case f_scale_tril=None, $$f_{loc} = f$$. f_scale_tril (torch.Tensor) – Lower triangular decomposition of covariance matrix of $$q(f)$$’s . Lff (torch.Tensor) – Lower triangular decomposition of $$kernel(X, X)$$ (optional). full_cov (bool) – A flag to decide if we want to return full covariance matrix or just variance. whiten (bool) – A flag to tell if f_loc and f_scale_tril are already transformed by the inverse of Lff. jitter (float) – A small positive term which is added into the diagonal part of a covariance matrix to help stablize its Cholesky decomposition. loc and covariance matrix (or variance) of $$p(f^*(X_{new}))$$
train(gpmodule, optimizer=None, loss_fn=None, retain_graph=None, num_steps=1000)[source]

A helper to optimize parameters for a GP module.

Parameters: gpmodule (GPModel) – A GP module. optimizer (Optimizer) – A PyTorch optimizer instance. By default, we use Adam with lr=0.01. loss_fn (callable) – A loss function which takes inputs are gpmodule.model, gpmodule.guide, and returns ELBO loss. By default, loss_fn=TraceMeanField_ELBO().differentiable_loss. retain_graph (bool) – An optional flag of torch.autograd.backward. num_steps (int) – Number of steps to run SVI. a list of losses during the training procedure list


text:
# A Brief Introduction to Skyline Problem (Pareto-optimal Tuples) (1)

The skyline problem is to compute the best tuples from a set of ordered d-tuples. The name is originated from what the solution represented on 2d plane resembles the scene that urban buildings comprise. Skyline is one of the recommendation queries, and it is considering multi criteria. It is very interesting problem as well as very useful query. This problem has been being intensively studied for recent years. Today, I’m going to present the problem definition of skyline. Next time, I’ll describe several algorithms for the skyline problem.

First of all, let us know the input data. The input data $D^{d}$ of skyline is a set of n ordered d-tuples, each of which consists of ordered d scalar values. They are shown in below formulas:

$D^{d} = {tp_{1},tp_{2},...tp_{n}}$

$tp_{i} = (v_{1},v_{2},...,v_{d})$

$tp_{i}$ denotes a d-tuple. And, we need to understand the definition of the dominance relation. In addition, because the skyline problem is to find the better tuples, we need an assumption about ‘better’. In most literature, it is assumed that the less value is better, so we follow this assumption.

Definition 1 (Dominance). Let tp and tp’ be tuples in $D^{d}$ where $v_{i}$ is an element of tp and $u_{i}$ is an element of tp’ for $1 < i \leq d$. Then, tp dominates tp’ if and only if  $forall{i}, v_{i} leq u_{i} land exists{j}, v_{j} < u_{j}$.

In other words, it is said that one tuple $tp$ dominates another tuple $tp'$ if $tp$ is not worse (not greater) than $tp'$ in all dimensions and $tp$ is better (less) than $tp'$ in at least one dimension.

Definition 2 (Skyline) Given a data set $D^{d}$, a skyline contains tuples that is not dominated any other tuples in $D^{d}$.

As I described above definition, a skyline is a set of tuples and the tuples are not dominated by any other tuples in $D^{d}$. In literature, a d-dimensional data set and above two definitions are usually represented for comprehensive description to d-points on d-axies.

Without loss of generality, we assume that $D^{d}$ is a 2d data set (i.e., d=2). A data set is given as follows:

• a = (3,2)
• b = (8,1)
• c = (1,10)
• d = (4,3)
• e = (8,6)

Each element of a tuple in $D^{d}$ can be represented to one axis. In other words, the first element and the second element of tuples are represented to X and Y axies respectively. Then, tuples of above list are represented to 2d points as shown in Fig. 1.

Fig. 1. An example of a skyline

In Fig. 1, let us look into a dominance relation. The point a dominates the points {d,e} since elements of the point a less than those of {d,e} in X and Y. The point b dominates only e since X values of {b,e} are same (i.e., X=8) but Y of b (i.e., 1) is less than that (i.e., 6) of e. The points {d,e} cannot belong to the skyline because they are dominated by other tuples. Consequently, the points a,b, and c belong to the skyline since they are not dominated by any other tuples.

Initially, the skyline problem was known as the maxima vector problem (H. T. Kung et. al 1975) for traditional processing system. However, this problem was revisited by the Skyline Operator (Stephan Börzsönyi et. al 2001). Since then, this problem has been intensively studied in database area.

Next time, I’ll describe several algorithms including above algorithms in detail.


text:
# GraphicsGale Goes Freeware

## Recommended Posts

GraphicsGale recently become free software. The raster graphics editor for spriting, pixel art, and animations supports many graphics formats and allows artists to preview their animations in real time while editing sprites.

Previously offered for $25 per license, it's not clear on why GraphicsGale has gone freeware, but it's a bonus for artists making sprite-based games! View full story #### Share this post ##### Link to post ##### Share on other sites Advertisement I had just bought that and Pro Motion a few months back. I like them, but haven't had the time I wanted to learn pixel art so now they are just sitting and patiently waiting on me to get back to them. #### Share this post ##### Link to post ##### Share on other sites I'm pretty excited about this. I've been using GIMP for everything 2d graphics related, but for animation it is a "pain in the a$\$" even after multiple plugins added for easing work with sprite-sheets.
Giving it a try right now !

## Create an account

Register a new account

• 45
• 11
• 17
• 11
• 13
• ### Similar Content

• I'm working on a space game, and I suck at art. I would love to get some help from someone who is more skilled than me. Things I need include modular space ship parts and GUI elements. Nothing too fancy, just functional so I can get a prototype put together. This could potentially become a serious project, but for now this is just a hobby project.

In this video, you can see a few things I already completed
:2018-02-24 20-08-13.flv2018-02-24 20-08-13.flv
• By owenjr
Hi there.
I'm pretty new to this and I don't know if it has been asked before, but here I go.
I'm developing a game using SFML and C++.
I would like to use the "Tiled" tool to load maps into my game but I don't actually find any tutorial or guide on how to exaclty use it (I know that I have to read an XML file and stuff). I just step into diverse projects that make all a mess.
Anyone knows where can I find good information to make my map loader by myself?

• Today again I painted the next drawing. I look at him and I understand that something is wrong in him. Can problems with the prospect or with black and white or with the composition or is everything terrible? help me figure it out, help.. My english is not very well, i'm from russia and i want to draw for game

• Hello guys,
I've released my game for the first time. I'm very excited about it and I hope you'll enjoy the game - Beer Ranger. It's a retro-like puzzle-platfromer which makes you think a lot or die trying. You have a squad of skilled dwarfs with special powers and your goal is tasty beer. There is a lot of traps as well as many solutions how to endure them - it is up to your choice how to complete the level!
Link to the project: Project site
Link to the Steam site with video: Beer Ranger
Have fun and please write feedback if you feel up to.
Some screens:

• Hello fellow devs!
Once again I started working on an 2D adventure game and right now I'm doing the character-movement/animation. I'm not a big math guy and I was happy about my solution, but soon I realized that it's flawed.
My player has 5 walking-animations, mirrored for the left side: up, upright, right, downright, down. With the atan2 function I get the angle between player and destination. To get an index from 0 to 4, I divide PI by 5 and see how many times it goes into the player-destination angle.

In Pseudo-Code:
angle = atan2(destination.x - player.x, destination.y - player.y) //swapped y and x to get mirrored angle around the y axis
index = (int) (angle / (PI / 5));
PlayAnimation(index); //0 = up, 1 = up_right, 2 = right, 3 = down_right, 4 = down

Besides the fact that when angle is equal to PI it produces an index of 5, this works like a charm. Or at least I thought so at first. When I tested it, I realized that the up and down animation is playing more often than the others, which is pretty logical, since they have double the angle.

What I'm trying to achieve is something like this, but with equal angles, so that up and down has the same range as all other directions.

I can't get my head around it. Any suggestions? Is the whole approach doomed?

Thank you in advance for any input!


text:
# Kicked-Harper model versus on-resonance double-kicked rotor model: from spectral difference to topological equivalence.

@article{Wang2013KickedHarperMV,
title={Kicked-Harper model versus on-resonance double-kicked rotor model: from spectral difference to topological equivalence.},
author={Hailong Wang and Derek Y H Ho and Wayne M. Lawton and Jiaozi Wang and Jiangbin Gong},
journal={Physical review. E, Statistical, nonlinear, and soft matter physics},
year={2013},
volume={88 5},
pages={
052920
}
}
• Published 26 June 2013
• Physics
• Physical review. E, Statistical, nonlinear, and soft matter physics
Recent studies have established that, in addition to the well-known kicked-Harper model (KHM), an on-resonance double-kicked rotor (ORDKR) model also has Hofstadter's butterfly Floquet spectrum, with strong resemblance to the standard Hofstadter spectrum that is a paradigm in studies of the integer quantum Hall effect. Earlier it was shown that the quasienergy spectra of these two dynamical models (i) can exactly overlap with each other if an effective Planck constant takes irrational multiples…
23 Citations

## Figures from this paper

### Floquet topological phases in a spin- 1/2 double kicked rotor

• Physics
Physical Review A
• 2018
The double kicked rotor model is a physically realizable extension of the paradigmatic kicked rotor model in the study of quantum chaos. Even before the concept of Floquet topological phases became

### Non-Hermitian Floquet topological phases in the double-kicked rotor

• Physics
Physical Review A
• 2019
Dynamical kicking systems possess rich topological structures. In this work, we study Floquet states of matter in a non-Hermitian extension of double kicked rotor model. Under the on-resonance

### Topological phases of the kicked Harper–Kitaev model with ultracold atoms

• Physics
Journal of physics. Condensed matter : an Institute of Physics journal
• 2017
It is found that both Floquet Majorana modes and Hall chiral edge modes could appear at the sample boundary in the gaps between the quasienergy bands, and the competition of topological superconductor and Chern insulator states in the model is studied.

### Floquet topological phase transitions in a kicked Haldane-Chern insulator

We consider a periodically $\delta$-kicked Haldane type Chern insulator with the kicking applied in the $\hat{z}$ direction. This is known to behave as an inversion symmetry breaking perturbation,

### Floquet systems with Hall effect: Topological properties and phase transitions

• Physics
Physical Review B
• 2019
We study the quantum topological properties of Floquet (time-periodic) systems exhibiting Hall effects due to perpendicular magnetic and electric fields. The systems are charged particles

### Floquet Second-Order Topological Phases in Momentum Space

A momentum-space counterpart of HOTPs is revealed in time-periodic driven systems, which is demonstrated in a two-dimensional extension of the quantum double-kicked rotor and protected by chiral symmetry and characterized by a pair of topological invariants.

### Floquet topological phases with high Chern numbers in a periodically driven extended Su–Schrieffer–Heeger model

• Physics
Journal of physics. Condensed matter : an Institute of Physics journal
• 2022
The high Chern number phases with the Chern number |C| > 1 are observed in this study of a periodically driven extended Su–Schrieffer–Heeger (E-SSH) model with a cyclic parameter. Besides the

### Localization and delocalization properties in quasi-periodically perturbed Kicked Harper and Harper models

• Physics
• 2021
We numerically study the single particle localization and delocalization phenomena of an initially localized wave packet in the kicked Harper model (KHM) and Harper model subjected to quasiperiodic

### Dynamical characterization of Weyl nodes in Floquet Weyl semimetal phases

• Physics
• 2020
Due to studies in nonequilibrium (periodically-driven) topological matter, it is now understood that some topological invariants used to classify equilibrium states of matter do not suffice to

### Aspects of Floquet bands and topological phase transitions in a continuously driven superlattice

• Physics
• 2014
The recent creation of novel topological states of matter via periodic driving fields has attracted much attention. To contribute to the growing knowledge on this subject, we study the well-known

## References

SHOWING 1-10 OF 52 REFERENCES

### Spectral relationships between kicked Harper and on-resonance double kicked rotor operators

• Mathematics
• 2009
Kicked Harper operators and on-resonance double kicked rotor operators model quantum systems whose semiclassical limits exhibit chaotic dynamics. Recent computational studies indicate a striking

### Exponential quantum spreading in a class of kicked rotor systems near high-order resonances.

• Physics
Physical review. E, Statistical, nonlinear, and soft matter physics
• 2013
It is found that the existence of a flat band is one important feature behind why and how the exponential dynamics emerges and a quantitative prediction of the exponential spreading rate based on an interesting and simple pseudoclassical map may be inaccurate.

### Extended and localized states of generalized kicked Harper models.

• Dana
• Mathematics
Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics
• 1995
Basic properties of quantum states for generalized Harper models are studied using the phase-space translational symmetry of the problem, and it is shown that \ensuremath{\sigma} is always different from zero for qg1.

### General approach to the quantum kicked particle in a magnetic field: quantum-antiresonance transition.

• Physics
Physical review. E, Statistical, nonlinear, and soft matter physics
• 2005
The quantum kicked particle in a magnetic field is studied in a weak-chaos regime under realistic conditions, i.e., for general values of the conserved coordinate x(c) of the cyclotron orbit center.

### Antiresonance and localization in quantum dynamics.

• Physics
Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics
• 1996
By considering exactly-solvable cases, numerical evidence is given that QAR-localization is an excellent approximation to DL sufficiently close to QAR, as the level of chaos is increased.

### General quantum resonances of the kicked particle.

• Physics
Physical review. E, Statistical, nonlinear, and soft matter physics
• 2006
It is shown that QR can arise, in general, for any rational value of the Bloch quasimomentum, and this is illustrated in the case of the main QRs for arbitrary potentials.

### Topological singularities and transport in kicked Harper model.

• I. Satija
• Physics
Physical review. E, Statistical, nonlinear, and soft matter physics
• 2005
Quasienergy spectrum of kicked Harper model is found to exhibit a series of diabolic crossings, which suggest a correlation between the transport and the topological characteristics of the system.

### Dimer decimation and intricately nested localized-ballistic phases of a kicked Harper model.

• Computer Science, Physics
Physical review letters
• 2001
A new decimation scheme is introduced to study localization transitions in tight binding models with long range interaction and the renormalization tools provide a new method to compute quasienergy spectrum.

### Sub-Fourier characteristics of a δ-kicked-rotor resonance.

• Physics
Physical review letters
• 2010
A theoretical analysis shows that for an accelerating potential the width of the resonance in acceleration space depends on 1/N3, a property which is verified experimentally and could be useful for high precision gravity measurements.


text:
# Scale Standalone tikzpicture but retain text font size

I have a standalone standalone_tikzpicture.tex file,

\documentclass{standalone}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,shapes,arrows,positioning,calc}
\begin{document}
\begin{tikzpicture}[
sum/.style= {draw, circle},
input/.style = {coordinate},
output/.style= {coordinate},
>=Stealth,
]
\node [input, name=input1] (input1) {};
\node [sum, right of=input1, name=sum1] (sum1) {$+$};
\node [output, right of=sum1, name=output1] (output1) {};
\draw [->] node [left] {$\textbf{A}$} (input1) -- (sum1);
\draw [->] (sum1) -- (output1) node [right] {B};
\end{tikzpicture}
\end{document}

which I want to display in another file main.tex with different scale. My main.tex file is:

\documentclass{article}
\usepackage{tikz}
\usepackage{standalone}
\usetikzlibrary{arrows.meta,shapes,arrows,positioning,calc}
\begin{document}
\foreach \i in {0.5,1,...,4}{
\begin{figure}\label{fig:\i}
\tikzset{every picture/.style={scale=\i,every picture/.style={}}}
\centering
\includestandalone{standalone_tikzpicture}
\caption{Figure size \i.}
\end{figure}
}
\end{document}

Problem is, the standalone picture isn't scaled in the output of main.tex. Figure with same size is obtained. The only way I could scale is by replacing line

\includestandalone{standalone_tikzpicture}

with

\includestandalone[scale = \i]{standalone_tikzpicture}

but this scales texts as well which is definitely not desired. I have read Best way to include a standalone tikz file and scale it without scaling the nodes, solutions in it is not working for me. Is there a way to keep font size same and scale only picture?

\foreach is used here only to simulate multiple instances of same picture. Requirement is to use the same picture in different places. Hence standalone_file is required.

• Admittedly I don't get either \includestandalone or externalize, but [class=minimal]? There is very little defined in the minimal class, and certainly nothing which is used here. – John Kormylo Oct 19 '17 at 4:27
• @JohnKormylo I have updated the code as per your suggestion and removed [class=minimal] from \documentclass[class=minimal]{standalone} in file standalone_tikzpicture.tex. Nevertheless, it has no visible effect on scaling than before. – ItzMe Oct 19 '17 at 8:55
• While tikzpicture itself scales only the drawing and not the text, you would have to pass the option to standalone somehow. Perhaps create a file in the main document and use \input in the standalone. Or you could just not use \includestandalone at all. – John Kormylo Oct 19 '17 at 15:55
• Can you please give examples. I could not get it work. – ItzMe Oct 19 '17 at 18:59
• The \includestandalone[scale = \i] scales the content as a whole box. It is not related to TikZs scale which scales coordinates. standalone can't change TikZ code afterwards. One solution is to define your scale setting as macro in the main document which is then used in the subfiles after beeing set to the wanted value before every inclusion. – Martin Scharrer Nov 24 '17 at 7:54

The following scaled inner sep to make the circle bigger, and specified the separation distance directly. I'm not sure what separation distance left of uses, but it isn't either inner sep or outer sep.

I made no attempt to use standalone. That is a separate problem.

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{arrows.meta}
\begin{document}
\foreach \i in {0.5,1,...,4}{%
\begin{figure}
\centering
\begin{tikzpicture}[
every node/.style={inner sep={\i*.333em}},
sum/.style= {draw, circle},
>=Stealth,
]
\node [sum] (sum1) {$+$};
\node [right={\i*1em}] (output1) at (sum1.east) {B};
\node [left={\i*1em}] (input1) at (sum1.west) {$\textbf{A}$};
\draw [->] (input1) -- (sum1);
\draw [->] (sum1) -- (output1);
\end{tikzpicture}
\caption{Inner sep scale = \i, separation distance = \i em.}
\end{figure}
}
\end{document}

• Even though your code works for my initial question, I had a different intension. I added \foreach to simulate multiple instances of same picture. But the picture is reused in different places. So I kept standalone file to be able to reuse the code and to adjust pictures in all instances by changing code only in standalone file. I will post my solution. I have 3 methods now. First 2 methods inspired by your answer. I did not realize the dependence of node size on the text inside it, thanks for pointing it out. Third solution in my post is the ideal one I was looking for. – ItzMe Oct 21 '17 at 14:47

I have found three ways to solve the problem. Requirement of having a separate standalone file is maintained in these solutions.

How to use files for each solution:

• Main file: Read all comments in main.tex to turn on and off appropriate lines for each solution. This file is required file for all solutions.

• For solution 1 and solution 2: Read all comments in standalone_tikzpicture12.tex to turn on and off appropriate lines for solution 1 and solution 2.

• For solution 3: Please use standalone_tikzpicture3.tex.

Each solution in detail:

1. Solution 1: \tikzset{every picture/.style={scale=\i, every picture/.style={}}} can be used in main.tex to scale only shapes as per argument \i and by not sending the variable explicitly to the standalone file standalone_tikzpicture12.tex.

2. Solution 2: From main.tex send argument \i explicitly to standalone file standalone_tikzpicture12.tex and scale each shape with \draw[scale=\i] option. In standalone_tikzpicture12.tex, all shapes are drawn with absolute scale. Nodes are created only to add text and without any shapes around it. This is common to solution 1 and 2. This becomes challenging for more complex figure. These two methods evolved after I learned from @JohnKormylo that size of nodes fit to contents inside.

3. In Solution 3, a standalone file standalone_tikzpicture3.tex is created not much different that the one posted in the Question, but with additional changes. Argument \i is sent explicitly to the standalone file, and also the whole tikzpicture is scaled in the main.tex file using \includestandalone[scale=\i]{standalone_picture3}. In standalone_tikzpicture3.tex file all nodes are created with shapes with parameters such as node distance, minimum size, circle, etc as per required sizes without any text inside. For each text, nodes are created with parameter [scale=1/\i] and this makes the difference. This nullifies scaling effect made by [scale=\i] in the main.tex. Scale everything by \i and again scale only texts by 1/\i to nullify the scale. Also explicitly using any other variable to adjust scale of only texts is possible here.

Code (by default solution 3 is enabled):

main.tex (Required for all solutions):

    \documentclass{article}
\usepackage{tikz}
\usepackage{standalone}
\usepackage{tikzscale}
\usetikzlibrary{arrows.meta,shapes,arrows,positioning,calc}
\begin{document}
\foreach \i in {1,2,...,4}{
\begin{figure}\label{fig:\i}
\centering
%        \def\i{<MyScaleValue>} % assign a value to \i in General cases when not using \foreach to set \i

%        Enable below 1 line for method 1, Disable for method 2 and 3
%        \tikzset{every picture/.style={scale=\i, every picture/.style={}}} % Method 1
%        Enable below 1 line for method 1 and 2, Disable for method 3
%        \includestandalone{standalone_picture12} % This is for method 1 and 2

%        Enable below line for method 3, Disable for method 1 and 2
\includestandalone[scale=\i]{standalone_picture3} % This is for method 3

\caption{Figure size \i.}
\end{figure}
}
\end{document}

standalone_picture12.tex (Standalone file for Solution 1 and 2):

\documentclass{standalone}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,}
\usetikzlibrary{positioning}
\begin{document}
\begin{tikzpicture}[
>=Stealth,
]
%    Enable below 1 line for method 1, Disable for method 2
\def\i{1} % Method 1
\def\ii{0.25*\i}
\def\ij{3}
\draw [domain=0:360, scale = \ii] plot ({cos(\x)}, {sin(\x)});
\draw [->] (-\ii*\ij,0) -- (-\ii,0);
\draw [->] (\ii,0) -- (\ii*\ij,0);
\draw [line width = \ii] (-\ii*.75,0) -- (\ii*.75,0);
\draw [line width = \ii] (0,-\ii*.75) -- (0,\ii*.75);
\node [anchor=east] at (-\ii*\ij,0) {$\textbf{A}$};
\node [anchor=west] at (\ii*\ij,0) {$\textbf{B}$};
\end{tikzpicture}
\end{document}

standalone_picture3.tex (Standalone file for Solution 3):

\documentclass{standalone}
\usepackage{tikz}
\usetikzlibrary{arrows.meta,}
\usetikzlibrary{positioning}
\begin{document}
\begin{tikzpicture}[
node distance = 10mm and 10mm,
sum/.style= {draw, circle, minimum size=6mm, inner sep=0pt, outer sep=0pt,},
input/.style = {coordinate},
output/.style= {coordinate},
>=Stealth,
]
%    \def\i{1} % to test standalone
\def\j{1} % to test standalone
\node [input,] (input1) {};
\node [sum, right=of input1,scale=\j] (sum1) {$+$};
\node [output, right=of sum1,scale=\j] (output1) {};
\draw [->,scale=\j] (input1) -- (sum1);
\node [anchor=east,scale=1/\i] at (input1) (A) {$\textbf{A}$};
\draw [->,scale=\j] (sum1) -- (output1);
\node [anchor = west,scale=1/\i] at (output1)  (B) {$\textbf{B}$};
\end{tikzpicture}
\end{document}


text:
How to calculate Thermal Noise Power?

There are 3types of noise in electrical circuit- thermal noise, shot noise and flicker noise(see Types of Noise that you should know about). Here the thermal noise is discussed and the thermal noise power equation derived.

Consider a electrical or electronics wire which has some finite resistance R. Let the room temperature be T Kelvin. The electrons in the wire are in constant random motion manifested due to their kinetic energy which is due to the temperature T. These motion will produce fluctuating voltage across the wire. The average value of this voltage is zero but the mean square value is not zero. The mean square value of this voltage is given by Planck's Black Body radiation law as follows:

where h is Planck’s constant (6.546 × 10−34 J · s), k the Boltzmann constant (1.38 × 10−23 J/K), T the temperature in kelvin, B the system bandwidth in hertz, and f the center frequency of the bandwidth in hertz.

In the denominator,  we have two quantities in the exponential term- hf and kT. At temperature T > 10K and frequency f < 100GHz, kT will be larger than hf. Hence the term exp(hf/kT) can be approximated as follows-

and the mean square value of voltage becomes,

This approximation is also called Rayleigh–Jeans approximation.

By using Thevenin theorem we may replace the noisy resistor R with an equivalent circuit which has the mean square voltage and a noiseless resistor R.

The power delivered to a load resistor R is then,

where we have used the Rayleigh–Jeans approximation for mean square voltage. This equation is the equation for Thermal Noise Power.


text:
# Solve System of Equations including XOR and Modular Addition

A Dummy cipher is shown in figure below. Plaintext is 16 bits ($X_1$ and $X_2$ are 8 bit each). Ciphertext is computed after two rounds using two round keys $K_1$ and $K_2$. Can the round keys $K_1$ and $K_2$ be found by solving the system of equations, which involves XOR and Modular Addition?

$\boxplus$ = addition modulo $2^8$

$\oplus$ = XOR

$X_1$, $X_2$, $Y_1$, $Y_2$ are 8-bit. $i=1,2$.

I am confused on how to solve such system of equation which involves XOR and modular addition. How many PT-CT pairs are required to solve such system of equations? How many linear, non-linear/quadratic terms will there be in the system of equations?

• Note of the editor: I kept the original notation, but find it disastrous. I suggest to first change $X_1$ to $L_i$, $X_2$ to $R_i$, $Y_1$ to $L_{i+1}$, $Y_2$ to $L_{i+1}$. Then define $P=L_0\mathbin\|R_0$, $C=L_k\mathbin\|R_k$ with $k=2$ the number of rounds. Then number individual bits and start finding boolean equations; some enlightening may ensue. – fgrieu Apr 6 '18 at 12:28
• if i write the complete system in equations, then how can i solve it because + and Xor seems confusing. – crypt Apr 6 '18 at 12:36
• Addition modulo $2^n$ of two $n$-bit quantities can be expressed as $2n-1$ XOR, $2n-3$ AND, $n-2$ OR of individual Boolean variables, using the well-known ripple carry adder construction. In the end, the whole block cipher boils down to Boolean equations. Hint: start with the equations involving the low-order bit of each 8-bit quantity. Prove that you can find the low-order bit of $K_1$ and $K_2$ from the the low-order bit of each halves of a single $P/C$ pair. Then move on to higher-order bits. – fgrieu Apr 6 '18 at 13:06
• @fgrieu: actually, the problem can be solved a lot easier than that; you can keep the modular add (and xor) as group operations, and find a fairly simple solution that way. Yes, your approach would work better if you extended the cipher past two rounds; for just two rounds, it's overkill – poncho Apr 6 '18 at 14:11
• @fgrieu What about multiplication? how many Xor, AND & OR boolean variables it involve? – crypt Apr 7 '18 at 6:46

Ok, this would appear to be homework, so I won't give the full answer; I'll get you started.

First thing to do (with this attack approach) is to write out the full equations of the system, as in:

$$T_0 = P_0 \oplus K_0$$

$$T_1 = P_1 \boxplus T_0$$

$$T_2 = T_0 \oplus T_1$$

$$T_3 = T_2 \oplus K_1$$

$$C_1 = T_3 \boxplus T_1$$

$$C_0 = T_3 \oplus C_1$$

where you know $P_0, P_1$ (the plaintext), $C_0, C_1$ (the ciphertext), and you don't know $K_0, K_1$ (the keys) or $T_0, T_1, T_2, T_3$ (the internal states of the cipher at various points). And, we'll assume that we have a single plaintext/ciphertext pair (that will turn out to be sufficient against this cipher).

Where to do go from here? Well, the first obvious step is to notice the last relation $C_0 = T_3 \oplus C_1$; we know $C_0, C_1$, and so we can deduce the value $T_3 = C_0 \oplus C_1$. Then, we notice the relation $C_1 = T_3 \boxplus T_1$, we know $C_1$ and $T_3$, and so we can deduce the value $T_1 = C_1 \boxminus T_3 = C_1 \boxminus (C_0 \oplus C_1)$. Where do we go next?

Now, this approach takes apart this two round cipher quite nicely; however it doesn't scale very well; adding a third round to this toy cipher will break the relations this depends on. On the other hand, I suppose it might be a start in learning how to think about how to attack a cipher, although you will need to find other ways to attack less trivial ciphers. fgrieu's suggestion about looking at the lower-order bits first is a more general approach that'll work in cases where this won't.

• How to proceed for more than 2 rounds, like 5 or 10? – crypt Apr 7 '18 at 6:46


text:
# The volume of a sphere as an integral

• We can use definite integrals to calculate volumes of solid.
• Let's first look at the example of a sphere of radius $$r$$.
• Note that the equation of the sphere of radius $$r$$ and centre $$(0,0,0)$$ is $S:x^2+y^2+z^2=r^2.$
• The method is to integrate over $$-r\le x\le r$$ the function $$A(x)$$, which assigns to $$x=x_0$$ the area of the cross section of the solid $$S$$ and the plane $$x=x_0$$.
• Click here for a demonstration. You can see that this means that we approximate the solid $$S$$ with cylinders the base the cross sections.
• In this particular case, if we fix $$x=x_0$$, then from the equation $$x^2+y^2+z^2=r^2$$, we get $$y^2+z^2=r^2-x_0^2$$.
• That is, the cross-section at $$x=x_0$$ is the circular disk with radius $$\sqrt{r^2-x_0^2}$$. That has area $$(r^2-x_0^2)\pi$$.
• This implies $$A(x)=(r^2-x^2)\pi$$.
• Using this, we can calculate the volume: $V=\int_{-r}^r(r^2-x^2)\pi\,\mathrm dx=\pi(r^2x-\frac13x^3)\Big|_{-r}^r=\pi(r^2(r+r)-\frac13(r^3+r^3))=\frac43r^3\pi.$

# Solids of revolution, part 1

• More generally, we can consider solids of revolution, that is solids, which we obtain by rotating a region around an axis.
• If $$f(x)\ge0$$ for $$a\le x\le b$$, then let $$S$$ be the solid obtained by rotating the region under the graph $$y=f(x)$$ around the $$x$$-axis.
• The cross-sections are disks with centre the origin and radius $$f(x)$$.
• Therefore, $$A(x)=f(x)^2\pi$$, so that the volume is $$V=\int_a^bf(x)^2\pi\,\mathrm dx$$.
• Warning. In these problems, you have to be careful to make sure you know which axis is the region rotated around. For a given region, you get different solids if you rotate it around the $$x$$ and the $$y$$-axes.
• Let's find the volume of the solid obtained by rotating the region $$R$$ under the graph $$y=\sqrt x$$ for $$0\le x\le1$$.
• Since we have $$f(x)=\sqrt x$$, the volume will be $V=\int_a^bf(x)^2\pi\,\mathrm dx=\int_0^1\pi x\,\mathrm dx=\frac{\pi}{2}.$
• If $$0\le f(x)\le g(x)$$ for $$a\le x\le b$$, then the cross-section is the ring with outer radius $$g(x)$$, and inner radius $$f(x)$$.
• Therefore, we have $$A(x)=\pi(g(x)^2-f(x)^2)$$, and thus $$V=\int_a^b\pi(g(x)^2-f(x)^2)\,\mathrm dx$$.
• Exercise. Let $$R$$ be the region enclosed by the curves $$y=x$$ and $$y=x^2$$.
1. Find the volume of the solid of revolution obtained by rotating $$R$$ around the $$x$$-axis.
2. Find the volume of the solid of revolution obtained by rotating $$R$$ around the $$y$$-axis.