text:
# We will now add support for register-memory ALU operations to the classic five-stage RISC pipeline. To offset this increase in complexity, all memory addressing will be restricted to register indirect (i.e., all addresses are simply a value held in a register; no offset or displacement may be added to the register value). For example, the register-memory instruction add x4, x5, (x1) means add the contents of register x5 to the contents of the memory location with address equal to the value in register x1 and put the sum in register x4. Register-register ALU operations are unchanged. The following items apply to the integer RISC pipeline: a. List a rearranged order of the five traditional stages of the RISC pipeline that will support register-memory operations implemented exclusively by re

Question
Other
We will now add support for register-memory ALU operations to the classic five-stage RISC pipeline. To offset this increase in complexity, all memory addressing will be restricted to register indirect (i.e., all addresses are simply a value held in a register; no offset or displacement may be added to the register value). For example, the register-memory instruction add x4, x5, (x1) means add the contents of register x5 to the contents of the memory location with address equal to the value in register x1 and put the sum in register x4. Register-register ALU operations are unchanged. The following items apply to the integer RISC pipeline:
a. List a rearranged order of the five traditional stages of the RISC pipeline that will support register-memory operations implemented exclusively by register indirect addressing.
b. Describe what new forwarding paths are needed for the rearranged pipeline by stating the source, destination, and information transferred on each needed new path.
c. For the reordered stages of the RISC pipeline, what new data hazards are created by this addressing mode? Give an instruction sequence illustrating each new hazard.
d. List all of the ways that the RISC pipeline with register-memory ALU operations can have a different instruction count for a given program than the original RISC pipeline. Give a pair of specific instruction sequences, one for the original pipeline and one for the rearranged pipeline, to illustrate each way.
Hint for (d): Give a pair of instruction sequences where the RISC pipeline has “more” instructions than the reg-mem architecture. Also give a pair of instruction sequences where the RISC pipeline has “fewer” instructions than the reg-mem architecture.

2021-02-27

### Relevant Questions

The table below shows the number of people for three different race groups who were shot by police that were either armed or unarmed. These values are very close to the exact numbers. They have been changed slightly for each student to get a unique problem.
Suspect was Armed:
Black - 543
White - 1176
Hispanic - 378
Total - 2097
Suspect was unarmed:
Black - 60
White - 67
Hispanic - 38
Total - 165
Total:
Black - 603
White - 1243
Hispanic - 416
Total - 2262
Give your answer as a decimal to at least three decimal places.
a) What percent are Black?
b) What percent are Unarmed?
c) In order for two variables to be Independent of each other, the P $$(A and B) = P(A) \cdot P(B) P(A and B) = P(A) \cdot P(B).$$
This just means that the percentage of times that both things happen equals the individual percentages multiplied together (Only if they are Independent of each other).
Therefore, if a person's race is independent of whether they were killed being unarmed then the percentage of black people that are killed while being unarmed should equal the percentage of blacks times the percentage of Unarmed. Let's check this. Multiply your answer to part a (percentage of blacks) by your answer to part b (percentage of unarmed).
Remember, the previous answer is only correct if the variables are Independent.
d) Now let's get the real percent that are Black and Unarmed by using the table?
If answer c is "significantly different" than answer d, then that means that there could be a different percentage of unarmed people being shot based on race. We will check this out later in the course.
Let's compare the percentage of unarmed shot for each race.
e) What percent are White and Unarmed?
f) What percent are Hispanic and Unarmed?
If you compare answers d, e and f it shows the highest percentage of unarmed people being shot is most likely white.
Why is that?
This is because there are more white people in the United States than any other race and therefore there are likely to be more white people in the table. Since there are more white people in the table, there most likely would be more white and unarmed people shot by police than any other race. This pulls the percentage of white and unarmed up. In addition, there most likely would be more white and armed shot by police. All the percentages for white people would be higher, because there are more white people. For example, the table contains very few Hispanic people, and the percentage of people in the table that were Hispanic and unarmed is the lowest percentage.
Think of it this way. If you went to a college that was 90% female and 10% male, then females would most likely have the highest percentage of A grades. They would also most likely have the highest percentage of B, C, D and F grades
The correct way to compare is "conditional probability". Conditional probability is getting the probability of something happening, given we are dealing with just the people in a particular group.
g) What percent of blacks shot and killed by police were unarmed?
h) What percent of whites shot and killed by police were unarmed?
i) What percent of Hispanics shot and killed by police were unarmed?
You can see by the answers to part g and h, that the percentage of blacks that were unarmed and killed by police is approximately twice that of whites that were unarmed and killed by police.
j) Why do you believe this is happening?
Do a search on the internet for reasons why blacks are more likely to be killed by police. Read a few articles on the topic. Write your response using the articles as references. Give the websites used in your response. Your answer should be several sentences long with at least one website listed. This part of this problem will be graded after the due date.
The dominant form of drag experienced by vehicles (bikes, cars,planes, etc.) at operating speeds is called form drag. Itincreases quadratically with velocity (essentially because theamount of air you run into increase with v and so does the amount of force you must exert on each small volume of air). Thus
$$\displaystyle{F}_{{{d}{r}{u}{g}}}={C}_{{d}}{A}{v}^{{2}}$$
where A is the cross-sectional area of the vehicle and $$\displaystyle{C}_{{d}}$$ is called the coefficient of drag.
Part A:
Consider a vehicle moving with constant velocity $$\displaystyle\vec{{{v}}}$$. Find the power dissipated by form drag.
Express your answer in terms of $$\displaystyle{C}_{{d}},{A},$$ and speed v.
Part B:
A certain car has an engine that provides a maximum power $$\displaystyle{P}_{{0}}$$. Suppose that the maximum speed of thee car, $$\displaystyle{v}_{{0}}$$, is limited by a drag force proportional to the square of the speed (as in the previous part). The car engine is now modified, so that the new power $$\displaystyle{P}_{{1}}$$ is 10 percent greater than the original power ($$\displaystyle{P}_{{1}}={110}\%{P}_{{0}}$$).
Assume the following:
The top speed is limited by air drag.
The magnitude of the force of air drag at these speeds is proportional to the square of the speed.
By what percentage, $$\displaystyle{\frac{{{v}_{{1}}-{v}_{{0}}}}{{{v}_{{0}}}}}$$, is the top speed of the car increased?
Express the percent increase in top speed numerically to two significant figures.
The unstable nucleus uranium-236 can be regarded as auniformly charged sphere of charge Q=+92e and radius $$\displaystyle{R}={7.4}\times{10}^{{-{15}}}$$ m. In nuclear fission, this can divide into twosmaller nuclei, each of 1/2 the charge and 1/2 the voume of theoriginal uranium-236 nucleus. This is one of the reactionsthat occurred n the nuclear weapon that exploded over Hiroshima, Japan in August 1945.
A. Find the radii of the two "daughter" nuclei of charge+46e.
B. In a simple model for the fission process, immediatelyafter the uranium-236 nucleus has undergone fission the "daughter"nuclei are at rest and just touching. Calculate the kineticenergy that each of the "daughter" nuclei will have when they arevery far apart.
C. In this model the sum of the kinetic energies of the two"daughter" nuclei is the energy released by the fission of oneuranium-236 nucleus. Calculate the energy released by thefission of 10.0 kg of uranium-236. The atomic mass ofuranium-236 is 236 u, where 1 u = 1 atomic mass unit $$\displaystyle={1.66}\times{10}^{{-{27}}}$$ kg. Express your answer both in joules and in kilotonsof TNT (1 kiloton of TNT releases 4.18 x 10^12 J when itexplodes).
The top string of a guitar has a fundamental frequency of 33O Hz when it is allowed to vibrate as a whole, along all its 64.0-cm length from the neck to the bridge. A fret is provided for limiting vibration to just the lower two thirds of the string, If the string is pressed down at this fret and plucked, what is the new fundamental frequency? The guitarist can play a "natural harmonic" by gently touching the string at the location of this fret and plucking the string at about one sixth of the way along its length from the bridge. What frequency will be heard then?
The bulk density of soil is defined as the mass of dry solidsper unit bulk volume. A high bulk density implies a compact soilwith few pores. Bulk density is an important factor in influencing root development, seedling emergence, and aeration. Let X denotethe bulk density of Pima clay loam. Studies show that X is normally distributed with $$\displaystyle\mu={1.5}$$ and $$\displaystyle\sigma={0.2}\frac{{g}}{{c}}{m}^{{3}}$$.
(a) What is thedensity for X? Sketch a graph of the density function. Indicate onthis graph the probability that X lies between 1.1 and 1.9. Findthis probability.
(b) Find the probability that arandomly selected sample of Pima clay loam will have bulk densityless than $$\displaystyle{0.9}\frac{{g}}{{c}}{m}^{{3}}$$.
(c) Would you be surprised if a randomly selected sample of this type of soil has a bulkdensity in excess of $$\displaystyle{2.0}\frac{{g}}{{c}}{m}^{{3}}$$? Explain, based on theprobability of this occurring.
(d) What point has the property that only 10% of the soil samples have bulk density this high orhigher?
(e) What is the moment generating function for X?
4.7 A multiprocessor with eight processors has 20attached tape drives. There is a large number of jobs submitted tothe system that each require a maximum of four tape drives tocomplete execution. Assume that each job starts running with onlythree tape drives for a long period before requiring the fourthtape drive for a short period toward the end of its operation. Alsoassume an endless supply of such jobs.
a) Assume the scheduler in the OS will not start a job unlessthere are four tape drives available. When a job is started, fourdrives are assigned immediately and are not released until the jobfinishes. What is the maximum number of jobs that can be inprogress at once? What is the maximum and minimum number of tapedrives that may be left idle as a result of this policy?
b) Suggest an alternative policy to improve tape driveutilization and at the same time avoid system deadlock. What is themaximum number of jobs that can be in progress at once? What arethe bounds on the number of idling tape drives?
The equation F=−vex(dm/dt) for the thrust on a rocket, can also be applied to an airplane propeller. In fact, there are two contributions to the thrust: one positive and one negative. The positive contribution comes from air pushed backward, away from the propeller (so dm/dt<0), at a speed vex relative to the propeller. The negative contribution comes from this same quantity of air flowing into the front of the propeller (so dm/dt>0) at speed v, equal to the speed of the airplane through the air.
For a Cessna 182 (a single-engine airplane) flying at 130 km/h, 150 kg of air flows through the propeller each second and the propeller develops a net thrust of 1300 N. Determine the speed increase (in km/h) that the propeller imparts to the air.
A pair of forces with equal magnitudes, opposite directions,and different lines of action is called a "couple". When acouple acts on a rigid object, the couple produces a torque thatdoes not depend on the location of the axis. The drawing shows acouple acting on a tire wrench, each force being perpendicular tothe wrench. Determine an expression for the torque produced by thecouple when the axis is perpendicular to the tired and passesthrough (a) point A, (b) point B, and (c) point C. Express youranswers in terms of the magnitude F of the force and the length Lof the wrench
Case: Dr. Jung’s Diamonds Selection
With Christmas coming, Dr. Jung became interested in buying diamonds for his wife. After perusing the Web, he learned about the “4Cs” of diamonds: cut, color, clarity, and carat. He knew his wife wanted round-cut earrings mounted in white gold settings, so he immediately narrowed his focus to evaluating color, clarity, and carat for that style earring.
After a bit of searching, Dr. Jung located a number of earring sets that he would consider purchasing. But he knew the pricing of diamonds varied considerably. To assist in his decision making, Dr. Jung decided to use regression analysis to develop a model to predict the retail price of different sets of round-cut earrings based on their color, clarity, and carat scores. He assembled the data in the file Diamonds.xls for this purpose. Use this data to answer the following questions for Dr. Jung.
1) Prepare scatter plots showing the relationship between the earring prices (Y) and each of the potential independent variables. What sort of relationship does each plot suggest?
2) Let X1, X2, and X3 represent diamond color, clarity, and carats, respectively. If Dr. Jung wanted to build a linear regression model to estimate earring prices using these variables, which variables would you recommend that he use? Why?
3) Suppose Dr. Jung decides to use clarity (X2) and carats (X3) as independent variables in a regression model to predict earring prices. What is the estimated regression equation? What is the value of the R2 and adjusted-R2 statistics?
4) Use the regression equation identified in the previous question to create estimated prices for each of the earring sets in Dr. Jung’s sample. Which sets of earrings appear to be overpriced and which appear to be bargains? Based on this analysis, which set of earrings would you suggest that Dr. Jung purchase?
5) Dr. Jung now remembers that it sometimes helps to perform a square root transformation on the dependent variable in a regression problem. Modify your spreadsheet to include a new dependent variable that is the square root on the earring prices (use Excel’s SQRT( ) function). If Dr. Jung wanted to build a linear regression model to estimate the square root of earring prices using the same independent variables as before, which variables would you recommend that he use? Why?
1
6) Suppose Dr. Jung decides to use clarity (X2) and carats (X3) as independent variables in a regression model to predict the square root of the earring prices. What is the estimated regression equation? What is the value of the R2 and adjusted-R2 statistics?
7) Use the regression equation identified in the previous question to create estimated prices for each of the earring sets in Dr. Jung’s sample. (Remember, your model estimates the square root of the earring prices. So you must actually square the model’s estimates to convert them to price estimates.) Which sets of earring appears to be overpriced and which appear to be bargains? Based on this analysis, which set of earrings would you suggest that Dr. Jung purchase?
8) Dr. Jung now also remembers that it sometimes helps to include interaction terms in a regression model—where you create a new independent variable as the product of two of the original variables. Modify your spreadsheet to include three new independent variables, X4, X5, and X6, representing interaction terms where: X4 = X1 × X2, X5 = X1 × X3, and X6 = X2 × X3. There are now six potential independent variables. If Dr. Jung wanted to build a linear regression model to estimate the square root of earring prices using the same independent variables as before, which variables would you recommend that he use? Why?
9) Suppose Dr. Jung decides to use color (X1), carats (X3) and the interaction terms X4 (color * clarity) and X5 (color * carats) as independent variables in a regression model to predict the square root of the earring prices. What is the estimated regression equation? What is the value of the R2 and adjusted-R2 statistics?
10) Use the regression equation identified in the previous question to create estimated prices for each of the earring sets in Dr. Jung’s sample. (Remember, your model estimates the square root of the earring prices. So you must square the model’s estimates to convert them to actual price estimates.) Which sets of earrings appear to be overpriced and which appear to be bargains? Based on this analysis, which set of earrings would you suggest that Dr. Jung purchase?
1. Find each of the requested values for a population with a mean of $$? = 40$$, and a standard deviation of $$? = 8$$ A. What is the z-score corresponding to $$X = 52?$$ B. What is the X value corresponding to $$z = - 0.50?$$ C. If all of the scores in the population are transformed into z-scores, what will be the values for the mean and standard deviation for the complete set of z-scores? D. What is the z-score corresponding to a sample mean of $$M=42$$ for a sample of $$n = 4$$ scores? E. What is the z-scores corresponding to a sample mean of $$M= 42$$ for a sample of $$n = 6$$ scores? 2. True or false: a. All normal distributions are symmetrical b. All normal distributions have a mean of 1.0 c. All normal distributions have a standard deviation of 1.0 d. The total area under the curve of all normal distributions is equal to 1 3. Interpret the location, direction, and distance (near or far) of the following zscores: $$a. -2.00 b. 1.25 c. 3.50 d. -0.34$$ 4. You are part of a trivia team and have tracked your team’s performance since you started playing, so you know that your scores are normally distributed with $$\mu = 78$$ and $$\sigma = 12$$. Recently, a new person joined the team, and you think the scores have gotten better. Use hypothesis testing to see if the average score has improved based on the following 8 weeks’ worth of score data: $$82, 74, 62, 68, 79, 94, 90, 81, 80$$. 5. You get hired as a server at a local restaurant, and the manager tells you that servers’ tips are $42 on average but vary about $$12 (\mu = 42, \sigma = 12)$$. You decide to track your tips to see if you make a different amount, but because this is your first job as a server, you don’t know if you will make more or less in tips. After working 16 shifts, you find that your average nightly amount is$44.50 from tips. Test for a difference between this value and the population mean at the $$\alpha = 0.05$$ level of significance.


text:
# Factorial moment measure

In probability and statistics, a factorial moment measure is a mathematical quantity, function or, more precisely, measure that is defined in relation to mathematical objects known as point processes, which are types of stochastic processes often used as mathematical models of physical phenomena representable as randomly positioned points in time,space or both. Moment measures generalize the idea of factorial moments, which are useful for studying non-negative integer-valued random variables.[1]

The first factorial moment measure of a point process coincides with its first moment measure or intensity measure,[2] which gives the expected or average number of points of the point process located in some region of space. In general, if the number of points in some region is considered as a random variable, then the moment factorial measure of this region is the factorial moment of this random variable.[3] Factorial moment measures completely characterize a wide class of point processes, which means they can be used to uniquely identify a point process.

If a factorial moment measure is absolutely continuous, then with respect to the Lebesgue measure it is said to have a density (which is a generalized form of a derivative), and this density is known by a number of names including factorial moment density, product density, coincidence density,[1] joint intensity[4] , correlation function or multivariate frequency spectrum [5] The first and second factorial moment densities of a point process are used in the definition of the pair correlation function, which gives a way to statistically quantify the strength of interaction or correlation between points of a point process.[6]

Factorial moment measures serve as useful tools in the study of point processes[1][6][7] as well as the related fields of stochastic geometry[3] and spatial statistics,[6][8] which are applied in various scientific and engineering disciplines such as biology, geology, physics, and telecommunications.[1][3][9]

## Point process notation

Point processes are mathematical objects that are defined on some underlying mathematical space. Since these processes are often used to represent collections of points randomly scattered in space, time or both, the underlying space is usually d-dimensional Euclidean space denoted here by Rd, but they can be defined on more abstract mathematical spaces.[7]

Point processes have a number of interpretations, which is reflected by the various types of point process notation.[3][9] For example, if a point $\textstyle x$ belongs to or is a member of a point process, denoted by N, then this can be written as:[3]

$\textstyle x\in {N},$

and represents the point process being interpreted as a random set. Alternatively, the number of points of N located in some Borel set B is often written as:[2][3][8]

$\textstyle {N}(B),$

which reflects a random measure interpretation for point processes. These two notations are often used in parallel or interchangeably.[3][8][2]

## Definitions

### n th factorial power of a point process

For some positive integer $\textstyle n=1,2,\dots$, the $\textstyle n$-th factorial power of a point process $\textstyle {N}$ on $\textstyle \textbf{R}^d$ is defined as:[2]

${N}^{(n)}(B_1\times,\dots,\times B_n)= \sum_{(x_1\neq,\dots,\neq x_n)\in {N} } \prod_{i=1}^n \mathbf{1}_{B_i}(x_i)$

where $\textstyle B_1,...,B_n$ is a collection of not necessarily disjoint Borel sets in $\textstyle \textbf{R}^d$, which form a $\textstyle n$-fold Cartesian product of sets denoted by:

$B_1\times,\dots,\times B_n.$

The symbol $\textstyle \mathbf{1}$ denotes an indicator function such that $\textstyle \mathbf{1}_{B_1}$is a Dirac measure for the set $\textstyle B_n$. The summation in the above expression is performed over all $\textstyle n$-tuples of distinct points, including permutations, which can be contrasted with the definition of the n-th power of a point process. The symbol $\textstyle \Pi$ denotes multiplication while the existence of various point process notation means that the n-th factorial power of a point process is sometimes defined using other notation.[2]

### n th factorial moment measure

The n th factorial moment measure or n th order factorial moment measure is defined as:

$M^{(n)}(B_1\times,\dots,\times B_n)=E [{N}^{(n)}(B_1\times,\dots,\times B_n)],$

where the E denotes the expectation (operator) of the point process N. In other words, the n-th factorial moment measure is the expectation of the n th factorial power of some point process.

The n th factorial moment measure of a point process N is equivalently defined[3] by:

$\int_{\textbf{R}^{n d}}f(x_1,\dots,x_n) M^{(n)}(dx_1,\dots,dx_n)=E \left[ \sum_{(x_1\neq,\dots,\neq x_n)\in {N} } f(x_1,\dots,x_n) \right],$

where f is any non-negative measurable function on Rnd, and the above summation is performed over all n tuples of distinct points, including permutations. Consequently, the factorial moment measure is defined such that there are no points repeating in the product set, as opposed to the moment measure.[7]

### First factorial moment measure

The first factorial moment measure $\textstyle M^1$ coincides with the first moment measure:[2]

$M^{(1)}(B)=M^1(B)=E [{N}(B)],$

where $\textstyle M^1$ is known, among other terms, as the intensity measure[3] or mean measure,[10] and is interpreted as the expected number of points of $\textstyle {N}$ found or located in the set $\textstyle B$

### Second factorial moment measure

The second factorial moment measure for two Borel sets $\textstyle A$ and $\textstyle B$ is:

$M^{(2)}(A\times B)=M^1(A\times B)-M^1(A\cap B).$

## Name explanation

For some Borel set $\textstyle B$, the namesake of this measure is revealed when the $\textstyle n\,$th factorial moment measure reduces to:

$M^{(n)}(B\times,\dots,\times B)=E [{N}(B)({N}(B)-1)\dots ({N}(B)-n+1)],$

which is the $\textstyle n\,$-th factorial moment of the random variable $\textstyle {N}(B)$.[3]

## Factorial moment density

If a factorial moment measure is absolutely continuous, then it has a density (or more precisely, a Radon–Nikodym derivative or density) with respect to the Lebesgue measure and this density is known as the factorial moment density or product density, joint intensity, correlation function, or multivariate frequency spectrum. Denoting the $\textstyle n$-th factorial moment density by $\textstyle \mu^{(n)}(x_1,\dots,x_n)$, it is defined in respect to the equation:[3]

$M^{(n)}(B_1\times,\dots,\times B_n)=\int_{B_1}\dots\int_{B_n}\mu^{(n)}(x_1,\dots,x_n) dx_1\dots dx_n.$

Furthermore, this means the following expression

$E \left[ \sum_{(x_1\neq,\dots,\neq x_n)\in {N} } f(x_1,\dots,x_n) \right]= \int_{\textbf{R}^{n d}} f(x_1,\dots,x_n) \mu^{(n)}(x_1,\dots,x_n) dx_1\dots dx_n,$

where $\textstyle f$ is any non-negative bounded measurable function defined on $\textstyle \textbf{R}^{ n}$.

## Pair correlation function

In spatial statistics and stochastic geometry, to measure the statistical correlation relationship between points of a point process, the pair correlation function of a point process ${N}$ is defined as:[3][6]

$\rho(x_1,x_2)=\frac{\mu^{(2)}(x_1,x_2)}{\mu^{(1)}(x_1) \mu^{(1)}(x_2) },$

where the points $x_1,x_2\in R^d$. In general, $\rho(x_1,x_2)\geq 0$ whereas $\rho(x_1,x_2)=1$ corresponds to no correlation (between points) in the typical statistical sense.[6]

## Examples

### Poisson point process

For a general Poisson point process with intensity measure $\textstyle \Lambda$ the $\textstyle n$-th factorial moment measure is given by the expression:[3]

$M^{(n)}(B_1\times,\dots,\times B_n)=\prod_{i=1}^n[\Lambda(B_i)],$

where $\textstyle \Lambda$ is the intensity measure or first moment measure of $\textstyle {N}$, which for some Borel set $\textstyle B$ is given by:

$\Lambda(B)=M^1(B)=E[{N}(B)].$

For a homogeneous Poisson point process the $\textstyle n$-th factorial moment measure is simply:[2]

$M^{(n)}(B_1\times,\dots,\times B_n)=\lambda^n \prod_{i=1}^n |B_i|,$

where $\textstyle |B_i|$ is the length, area, or volume (or more generally, the Lebesgue measure) of $\textstyle B_i$. Furthermore, the $\textstyle n$-th factorial moment density is:[3]

$\mu^{(n)}(x_1,\dots,x_n)=\lambda^n.$

The pair-correlation function of the homogeneous Poisson point process is simply

$\rho(x_1,x_2)=1,$

which reflects the lack of interaction between points of this point process.

## Factorial moment expansion

It has been shown that expectations of general functionals of simple point processes, provided some convergence condition, have (possibly infinite) expansions or series consisting of the corresponding factorial moment measures.[11][12] In comparison to the Taylor series, which consists of a series of derivatives of some function, the nth factorial moment measure plays the roll as that of the n th derivative the Taylor series. In other words, given a general functional f of some simple point process, then this Taylor-like theorem for non-Poisson point processes means an expansion exists for the expectation of the function E, provided some mathematical condition is satisfied, which ensures convergence of the expansion.

## References

1. ^ a b c d D. J. Daley and D. Vere-Jones. An introduction to the theory of point processes. Vol. I. Probability and its Applications (New York). Springer, New York, second edition, 2003.
2. F. Baccelli and B. Błaszczyszyn. Stochastic Geometry and Wireless Networks, Volume I – Theory, volume 3, No 3-4 of Foundations and Trends in Networking. NoW Publishers, 2009.
3. D. Stoyan, W. S. Kendall, J. Mecke, and L. Ruschendorf. Stochastic geometry and its applications, volume 2. Wiley Chichester, 1995.
4. ^ Hough, J Ben, Krishnapur, Manjunath, Peres, Yuval, Vir{\'a}g, B{\'a}lint (2006). "Determinantal processes and independence". Probability Surveys 3: 206–229.
5. ^ K. Handa. The two-parameter {Poisson-Dirichlet} point process. Bernoulli, 15(4):1082–1116, 2009.
6. A. Baddeley, I. B{\'a}r{\'a}ny, and R. Schneider. Spatial point processes and their applications. Stochastic Geometry: Lectures given at the CIME Summer School held in Martina Franca, Italy, September 13–18, 2004, pages 1–75, 2007.
7. ^ a b c D. J. Daley and D. Vere-Jones. An introduction to the theory of point processes. Vol. {II}. Probability and its Applications (New York). Springer, New York, second edition, 2008
8. ^ a b c J. Moller and R. P. Waagepetersen. Statistical inference and simulation for spatial point processes. CRC Press, 2003.
9. ^ a b F. Baccelli and B. Błaszczyszyn. Stochastic Geometry and Wireless Networks, Volume II – Applications, volume 4, No 1-2 of Foundations and Trends in Networking. NoW Publishers, 2009.
10. ^ J. F. C. Kingman. Poisson processes, volume 3. Oxford university press, 1992.
11. ^ B. Blaszczyszyn. Factorial-moment expansion for stochastic systems. Stoch. Proc. Appl., 56:321–335, 1995.
12. ^ D. P. Kroese and V. Schmidt. Light-traffic analysis for queues with spatially distributed arrivals. Mathematics of Operations Research, 21(1):pp. 135–157, 1996.

format: plain


text:
DeepAI

# Quantizing deep convolutional networks for efficient inference: A whitepaper

We present an overview of techniques for quantizing convolutional neural networks for inference with integer weights and activations. Per-channel quantization of weights and per-layer quantization of activations to 8-bits of precision post-training produces classification accuracies within 2 floating point networks for a wide variety of CNN architectures. Model sizes can be reduced by a factor of 4 by quantizing weights to 8-bits, even when 8-bit arithmetic is not supported. This can be achieved with simple, post training quantization of weights.We benchmark latencies of quantized networks on CPUs and DSPs and observe a speedup of 2x-3x for quantized implementations compared to floating point on CPUs. Speedups of up to 10x are observed on specialized processors with fixed point SIMD capabilities, like the Qualcomm QDSPs with HVX. Quantization-aware training can provide further improvements, reducing the gap to floating point to 1 also allows for reducing the precision of weights to four bits with accuracy losses ranging from 2 networks.We introduce tools in TensorFlow and TensorFlowLite for quantizing convolutional networks and review best practices for quantization-aware training to obtain high accuracy with quantized weights and activations. We recommend that per-channel quantization of weights and per-layer quantization of activations be the preferred quantization scheme for hardware acceleration and kernel optimization. We also propose that future processors and hardware accelerators for optimized inference support precisions of 4, 8 and 16 bits.

02/03/2021

### Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms

Convolutional Neural Networks (CNNs) have proven to be a powerful state-...
11/13/2021

### Iterative Training: Finding Binary Weight Deep Neural Networks with Layer Binarization

In low-latency or mobile applications, lower computation complexity, low...
02/08/2021

### VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference

Quantization enables efficient acceleration of deep neural networks by r...
03/19/2019

### Trained Uniform Quantization for Accurate and Efficient Neural Network Inference on Fixed-Point Hardware

We propose a method of training quantization clipping thresholds for uni...
06/30/2020

### EasyQuant: Post-training Quantization via Scale Optimization

The 8 bits quantization has been widely applied to accelerate network in...
08/02/2019

### U-Net Fixed-Point Quantization for Medical Image Segmentation

Model quantization is leveraged to reduce the memory consumption and the...
04/24/2020

### Quantization of Deep Neural Networks for Accumulator-constrained Processors

We introduce an Artificial Neural Network (ANN) quantization methodology...

## 1 Introduction

Deep networks are increasingly used for applications at the edge. Devices at the edge typically have lower compute capabilities and are constrained in memory and power consumption. It is also necessary to reduce the amount of communication to the cloud for transferring models to the device to save on power and reduce network connectivity requirements. Therefore, there is a pressing need for techniques to optimize models for reduced model size, faster inference and lower power consumption.

There is extensive research on this topic with several approaches being considered: One approach is to build efficient models from the ground up [1],[2] and [3]. Another technique is to reduce the model size by applying quantization, pruning and compression techniques [4], [5] and [6]. Faster inference has been achieved by having efficient kernels for computation in reduced precision like GEMMLOWP [7], Intel MKL-DNN [8] , ARM CMSIS [9], Qualcomm SNPE [10], Nvidia TensorRT [11] and custom hardware for fast inference [12], [13] and [14].

One of the simpler ways to reduce complexity of any model is to reduce the precision requirements for the weights and activations. This approach has many advantages:

• It is broadly applicable across a range of models and use cases. One does not need to develop a new model architecture for improved speed. In many cases, one can start with an existing floating point model and quickly quantize it to obtain a fixed point quantized model with almost no accuracy loss, without needing to re-train the model. Multiple hardware platforms and libraries support fast inference with quantized weights and activations, so there is no need to wait for new hardware development.

• Smaller Model footprint: With 8-bit quantization, one can reduce the model size a factor of 4, with negligible accuracy loss. This can be done without needing any data as only the weights are quantized. This also leads to faster download times for model updates.

• Less working memory and cache for activations: Intermediate computations are typically stored in cache for reuse by later layers of a deep network and reducing the precision at which this data is stored leads to less working memory needed. Having lower precision weights and activations allows for better cache reuse.

• Faster computation: Most processors allow for faster processing of 8-bit data.

• Lower Power: Moving 8-bit data is 4 times more efficient than moving 32-bit floating point data. In many deep architectures, memory access can dominate power consumption [2]. Therefore reduction in amount of data movement can have a significant impact on the power consumption.

All the factors above translate into faster inference, with a typical speedup of 2-3x due to the reduced precision for both memory accesses and computations. Further improvements in speed and power consumption are possible with processors and hardware accelerators optimized for low precision vector arithmetic.

## 2 Quantizer Design

In this section, we review different design choices for uniform quantization.

### 2.1 Uniform Affine Quantizer

Consider a floating point variable with range that needs to be quantized to the range where for 8-bits of precision. We derive two parameters: Scale () and Zero-point() which map the floating point values to integers (See [15]). The scale specifies the step size of the quantizer and floating point zero maps to zero-point [4]

. Zero-point is an integer, ensuring that zero is quantized with no error. This is important to ensure that common operations like zero padding do not cause quantization error.

For one sided distributions, therefore, the range is relaxed to include zero. For example, a floating point variable with the range (2.1,3.5) will be relaxed to the range (0,3.5) and then quantized. Note that this can cause a loss of precision in the case of extreme one-sided distributions.

Once the scale and zero-point are defined, quantization proceeds as follows:

 xint =round(xΔ)+z (1) xQ =clamp(0,Nlevels−1,xint) (2)

where

 clamp(a,b,x) =a x≤a =x a≤x≤b =b x≥b

The de-quantization operation is:

 xfloat=(xQ−z)Δ (3)

While the uniform affine quantizer allows for storing weights and activations at 8-bits of precision, there is an additional cost due to the zero-point. Consider a 2D convolution between a weight and an activation:

 y(k,l,n) =ΔwΔxconv(wQ(k,l,m;n)−zw,xQ(k,l,m)−zx) (4) y(k,l,n) =conv(wQ(k,l,m;n),xQ(k,l,m))−zwK−1∑k=0K−1∑l=0N−1∑m=0xQ(k,l,m) (5) −zxK−1∑k=0K−1∑l=0N−1∑m=0wQ(k,l,m;n)+zxzw (6)

A naive implementation of convolution, by performing the addition of zero-point prior to the convolution, leads to a 2x to 4x reduction in the throughput due to wider (16/32-bit) operands. One can do better by using the equation above and noting that the last term is a constant and each of the other terms requires N multiplies, which is 3x more operations than the 8-bit dot product. This can be further improved by noting that the weights are constant at inference and by noting that the sum over activations is identical for all convolutional kernels of the same size. However, this requires optimizing convolution kernels. For an indepth discussion, please see [16].

### 2.2 Uniform symmetric quantizer

A simplified version of the affine quantizer is the symmetric quantizer, which restricts zero-point to 0. With the symmetric quantizer, the conversion operations simplify to:

 xint =round(xΔ) (7) xQ =clamp(−Nlevels/2,Nlevels/2−1,xint) if signed (8) xQ =clamp(0,Nlevels−1,xint) if un-signed (9)

For faster SIMD implementation, we further restrict the ranges of the weights. In this case, the clamping is modified to:

 xQ =clamp(−(Nlevels/2−1),Nlevels/2−1,xint) if signed (10) xQ =clamp(0,Nlevels−2,xint) if un-signed (11)

Please see [4], Appendix B for more details.

The de-quantization operation is:

 xout=xQΔ

### 2.3 Stochastic quantizer

Stochastic quantization models the quantizer as an additive noise, followed by rounding. The stochastic quantizer is given by:

 xint =round(x+ϵΔ)+z,ϵ∼Unif(−12,12) xQ =clamp(0,Nlevels−1,xint)

The de-quantization operation is given by equation 3. Note that in expectation, the stochastic quantizer reduces to a pass-through of the floating point weights, with saturation for values outside the range. Therefore, this function is well behaved for purposes of calculating gradients. We do not consider stochastic quantization for inference as most inference hardware does not support it.

### 2.4 Modeling simulated quantization in the backward pass

For Quantization-aware training, we model the effect of quantization using simulated quantization operations, which consist of a quantizer followed by a de-quantizer, i.e,

 xout =SimQuant(x) (12) (13)

Since the derivative of a simulated uniform quantizer function is zero almost everywhere, approximations are required to model a quantizer in the backward pass. An approximation that has worked well in practice (see [5]) is to model the quantizer as specified in equation 14 for purposes of defining its derivative (See figure 1).

 xout=clamp(xmin,xmax,x) (14)

The backward pass is modeled as a ”straight through estimator” (see

[5]). Specifically,

 δout=δinIx∈SS:x:xmin≤x≤xmax (15)

where

is the backpropagation error of the loss with respect to the simulated quantizer output.

### 2.5 Determining Quantizer parameters

The quantizer parameters can be determined using several criteria. For example, TensorRT [11] minimizes the KL divergence between the original and quantized distributions to determine the step size. In this work, we adopt simpler methods. For weights, we use the actual minimum and maximum values to determine the quantizer parameters. For activations, we use the moving average of the minimum and maximum values across batches to determine the quantizer parameters. For post training quantization approaches, one can improve the accuracy of quantized models by careful selection of quantizer parameters.

### 2.6 Granularity of quantization

We can specify a single quantizer (defined by the scale and zero-point) for an entire tensor, referred to as per-layer quantization. Improved accuracy can be obtained by adapting the quantizer parameters to each kernel within the tensor

[17]. For example, the weight tensor is 4 dimensional and is a collection of 3 dimensional convolutional kernels, each responsible for producing one output feature map. per-channel quantization has a different scale and offset for each convolutional kernel. We do not consider per-channel quantization for activations as this would complicate the inner product computations at the core of conv and matmul operations. Both per-layer and per-channel quantization allow for efficient dot product and convolution implementation as the quantizer parameters are fixed per kernel in both cases.

## 3 Quantized Inference: Performance and Accuracy

Quantizing a model can provide multiple benefits as discussed in section 1. We discuss multiple approaches for model quantization and show the performance impact for each of these approaches.

### 3.1 Post Training Quantization

In many cases, it is desirable to reduce the model size by compressing weights and/or quantize both weights and activations for faster inference, without requiring to re-train the model. Post Training quantization techniques are simpler to use and allow for quantization with limited data. In this section, we study different quantization schemes for weight only quantization and for quantization of both weights and activations. We show that per-channel quantization with asymmetric ranges produces accuracies close to floating point across a wide range of networks.

#### 3.1.1 Weight only quantization

A simple approach is to only reduce the precision of the weights of the network to 8-bits from float. Since only the weights are quantized, this can be done without requiring any validation data (See figure 2). A simple command line tool can convert the weights from float to 8-bit precision. This setup is useful if one only wants to reduce the model size for transmission and storage and does not mind the cost of performing inference in floating point.

#### 3.1.2 Quantizing weights and activations

One can quantize a floating point model to 8-bit precision by calculating the quantizer parameters for all the quantities to be quantized. Since activations need to be quantized, one needs calibration data and needs to calculate the dynamic ranges of activations. (See figure 2) Typically, about 100 mini-batches are sufficient for the estimates of the ranges of the activation to converge.

#### 3.1.3 Experiments

For evaluating the tradeoffs with different quantization schemes, we study the following popular networks and evaluate the top-1 classification accuracy. Table 1 shows the wide variation in model size and accuracy across these networks. We note that Mobilenet-v1 [2] and Mobilenet-v2[1] architectures use separable depthwise and pointwise convolutions with Mobilenet-v2 also using skip connections. Inception-v3 [18] and NasNet [19]

use network in network building blocks with NasNet determining the architecture via reinforcement learning techniques. Resnets

[20] pioneered the idea of skip connections and consist of multiple blocks each making residual corrections to the main path with no transformations. Resnet-v2 [21] is an enhancement to the resnet architecture using pre-activation layers for improved accuracy. Note that all results are obtained using simulated quantization of weights and activations.

##### Weight only quantization:

We first quantize only the weights post training and leave the activations un-quantized. From figure 2, we note that per-channel quantization is required to ensure that the accuracy drop due to quantization is small, with asymmetric, per-layer quantization providing the best accuracy.

##### Weight and Activation Quantization:

Next, we quantize weights and activations to 8-bits, with per-layer quantization for activations. For weights we consider both symmetric and asymmetric quantizers at granularities of both a layer and a channel. We first show results for Mobilenetv1 networks and then tabulate results across a broader range of networks.

We also compare the post training quantization accuracies of popular convolutional networks: Inception-V3, Mobilenet-V2, Resnet-v1-50, Resnet-v1-152, Resnet-v2-50, Resnet-v2-152 and Nasnet-mobile on ImageNet in figure 4.

We make the following observations:

1. Per-channel quantization can provide good accuracy and can be a good baseline for post training quantization of weights and activations, with asymmetric quantization providing close to floating point accuracy for all networks.

2. Activations can be quantized to 8-bits with almost no loss in accuracy. The dynamic ranges of the activations are low due to a combination of:

1. Batch normalization with no scaling: Used in Inception-V3, which ensures that the activations of all feature maps have zero mean and unit variance.

2. ReLU6: Used in Mobilenet-V1, which restricts the activations to be in a fixed range (0,6) for all feature maps, thereby removing large dynamic range variations.

3. Networks with more parameters like Resnets and Inception-v3 are more robust to quantization compared to Mobilenets which have fewer parameters.

4. There is a large drop when weights are quantized at the granularity of a layer, particularly for Mobilenet architectures.

5. Almost all the accuracy loss due to quantization is due to weight quantization.

Weight quantization at the granularity of a layer causes large accuracy drops primarily due to batch normalization, which causes extreme variation in dynamic range across convolution kernels in a single layer. Appendix A has more details on the impact of batch normalization. Per-channel quantization side-steps this problem by quantizing at the granularity of a kernel, which makes the accuracy of per-channel quantization independent of the batch-norm scaling. However, the activations are still quantized with per-layer symmetric quantization.

Note that other approaches like weight regularization can also improve the accuracy of quantization post training, please see [22].

### 3.2 Quantization Aware Training

Quantization aware training models quantization during training and can provide higher accuracies than post quantization training schemes. In this section, we describe how quantization is modeled during training and describe how this can be easily done using automatic quantization tools in TensorFlow. We also evaluate the accuracies obtained for different quantization schemes with quantization aware training and show that even per-layer quantization schemes show high accuracies post training at 8-bits of precision. We also show that at 4 bit precision, quantization aware training provides significant improvements over post training quantization schemes.

We model the effect of quantization using simulated quantization operations on both weights and activations. For the backward pass, we use the straight through estimator (see section 2.4) to model quantization. Note that we use simulated quantized weights and activations for both forward and backward pass calculations. However, we maintain weights in floating point and update them with the gradient updates. This ensures that minor gradient updates gradually update the weights instead of underflowing. The updated weights are quantized and used for subsequent forward and backward pass computation. For SGD, the updates are given by:

 wfloat =wfloat−η∂L∂wout.Iwout∈(wmin,wmax) (16) wout =SimQuant(wfloat) (17)

Quantization aware training is achieved by automatically inserting simulated quantization operations in the graph at both training and inference times using the quantization library at [23] for Tensorflow [24]. We follow the approach outlined in [4] closely, with additional enhancements on handling batch normalization and in modeling quantization in the backward pass. A simple one-line change to the training or evaluation code automatically inserts simulated quantization operations into the training or eval graph.

For training, the code snippet is:

# Build forward pass of model.
...
loss = tf.losses.get_total_loss()

# Call the training rewrite which rewrites the graph in-place
# with FakeQuantization nodes and folds batchnorm for training.
# One can either fine tune an existing floating point model
# or train from scratch. quant_delay controls the onset
# of quantized training.

tf.contrib.quantize.create_training_graph(quant_delay=2000000)

# Call backward pass optimizer as usual.
optimizer.minimize(loss)

For evaluation, the code snippet is given below:

# Build eval model

logits, end_points = network_model(inputs,...)

# Call the eval rewrite which rewrites the graph in-place
# with FakeQuantization nodes and fold batchnorm for eval.
tf.contrib.quantize.create_eval_graph()

The high level conversion process is shown in figure 2.

The steps involved in training a quantized model are:

1. (Recommended): Fine tune from a floating point saved model: Start with a floating point pre-trained model or alternately train from scratch

2. Modify Estimator to add quantization operations: Add fake quantization operations to the model using the quantization rewriter at tf.contrib.quantize

3. Train model: At the end of this process, we have a savedmodel with quantization information (scale, zero-point) for all the quantities of interest. (weights and activations)

4. Convert model: The savedmodel with range information is transformed into a flatbuffer file using the tensorflow converter (TOCO) at: tf.contrib.lite.toco_convert. This step creates a flatbuffer file that converts the weights into integers and also contains information for quantized arithmetic with activations

5. Execute model: The converted model with integer weights can now be executed using the TFLite interpreter which can optionally execute the model in custom accelerators using the NN-API. One can also run the model on the CPU.

A simple example showing the graph transformation for a convolutional layer is shown in figure 5.

#### 3.2.1 Operation Transformations for Quantization

It is important to ensure that all quantization related artifacts are faithfully modeled at training time. This can make trivial operations like addition, figure 6 and concatenation , figure 7 non-trivial due to the need to rescale the fixed point values so that addition/concatenation can occur correctly.

In addition, it is important to ensure that fusion of operations at inference time is modeled correctly during training. For example, consider an add followed by a ReLU operation. In this case, one can fuse the addition and the ReLU operation at inference time in most platforms. To match this, fake quantization operations should not be placed between the addition and the ReLU operations.

#### 3.2.2 Batch Normalization

In this section, we describe several strategies for quantizing batch normalization layers. In section 4 and show that batch normalization with correction and freezing provides the best accuracy.

Batch normalization [25], is a popular technique that normalizes the activation statistics at the output of every layer, reducing dependencies across layers while significantly improving model accuracy.

Batch normalization is defined by the following equations:

 xbn=γ(x−μBσB)+β (18)

for training and

 xbn=γ(x−μσ)+β (19)

for inference.

Where and

are the batch mean and standard deviations.

and are the long term mean and standard deviations and are computed as moving averages of batch statistic during training.

For inference, we fold the batch normalization into the weights as defined by equations 20 and 21. Therefore, at inference there is no explicit batch normalization. The weights and biases are modified to account for batch normalization instead:

 Winf =γWσ (20) Biasinf =β−γμσ (21)

For quantized inference, we first consider folding batch norms and training them as shown in figure 8. We note that batch normalization uses batch statistics during training, but uses long term statistics during inference. Since the batch statistics vary every batch, this introduces undesired jitter in the quantized weights and degrades the accuracy of quantized models. (green curve in 14 and 15) A simple solution would be to switch to using long term moving averages during training, however, this eliminates batch normalization (i.e the mean and variance used do not correspond to the batch statistics) and causes instability in training. The graph rewriter implements a solution that eliminates the mismatch between training and inference with batch normalization (see figure 9):

1. We always scale the weights with a correction factor to the long term statistics prior to quantization. This ensures that there is no jitter in the quantized weights due to batch to batch variation.

 c =σBσ (22) wcorrected =c×γWσB (23)
2. During the initial phase of training, we undo the scaling of the weights so that outputs are identical to regular batch normalization. We also modify the bias terms correspondingly.

 y =conv(Q(wcorrected),x) (24) ycorrected =y/c (25) bias =β−γμB/σB (26) biascorrected =0 (27)
3. After sufficient training, switch from using batch statistics to long term moving averages for batch normalization, using the optional parameter freeze_bn_delay in

reate_experimental_training_graph() (about 300000 steps in figure \ref{fig:EMACorrBN} and 200000 in figure \ref{fig:MobV1BNfreeze}). Note that the long term averages are frozen to avoid instability in training.
This
orresponds to the normalization parameters used at inference and provides stable performance.

 y =conv(Q(wcorrected),x) (28) ycorrected =y (29) bias =β−γμB/σB (30) biascorrection =γ(μB/σB−μ/σ) (31)

#### 3.2.3 Experiments

Quantization aware training closes the gap to floating point accuracy, even for per-layer quantization of weights. We repeat the same experiments for quantized weights and activations with training, starting from a floating point check-point and with batch normalization freezing and obtain the results shown in figures 11 and 10 and Table 4.

All the experiments have the following settings:

• Fine tune from a floating point checkpoint, we used the models in [26].

• Use Stochastic Gradient Descent for fine tuning, with a step size of 1e-5.

We note that:

1. Training closes the gap between symmetric and asymmetric quantization.

2. Training allows for simpler quantization schemes to provide close to floating point accuracy. Even per-layer quantization shows close to floating point accuracy (see column 4 in Table 4)

#### 3.2.4 Lower Precision Networks

We note that at 8-bits of precision, post training quantization schemes provide close to floating point accuracy. In order to better understand the benefits of quantization aware training, we perform experiments to assess performance at 4 bit quantization for weights and activations.

We perform the following experiments:

• Experiment 1: Per-channel quantization is significantly better than per-layer quantization at 4 bits. We show that per-channel quantization provides big gains over per-layer quantization for all networks. At 8-bits, the gains were not significant as there were sufficient levels to represent the weights with high fidelity. At four bits, the benefits of per-channel quantization are apparent, even for post training quantization (columns 2 and 3 of Table 5).

• Experiment 2: Fine tuning can provide substantial accuracy improvements at lower bitwidths. It is interesting to see that for most networks, one can obtain accuracies within 5% of 8-bit quantization with fine tuning 4 bit weights (column 4 of Table 5). The improvements due to fine tuning are also more apparent at 4 bits. Note that activations are quantized to 8-bits in these experiments.

• Experiment 3: Lower precision activations: We investigate the accuracies obtained with 4-bit activations for all layers with and without fine tuning. Note that activations are quantized on a per-layer basis. The weights are quantized at 8-bits of precision with per-channel granularity. We note that fine tuning improves accuracy in this case also. The losses due to activation quantization are more severe than that of weight quantization (see Table 6). Note that the quantization granularity is different for activations and weights, so this is not a fair comparison of the impact of quantization. We hypothesize that quantizing activations introduces random errors as the activation patterns vary from image to image, while weight quantization is deterministic. This allows for the network to learn weight values to better compensate for the deterministic distortion introduced by weight quantization.

## 4 Training best practices

We experiment with several configurations for training quantized models: Our first experiment compares stochastic quantization with deterministic quantization. Subsequently, we study if training a quantized model from scratch provides higher accuracies than fine tuning from a floating point model. We also evaluate different methods for quantizing batch normalization layers and show that batch normalization with corrections provides the best accuracy. We also compare schemes that average weights during training with no averaging.

1. Stochastic Quantization does not improve accuracy: Stochastic quantization determines floating point weights that provide robust performance under stochastic quantization, which causes the quantized weights to vary from mini-batch to mini-batch. At inference, quantization is deterministic, causing a mismatch with training. We observe that due to this mis-match, stochastic quantization under-performs determinstic quantization (figure 12), which can be compensated better during training.

2. Quantizing a model from a floating point checkpoint provides better accuracy: The question arises as to whether it is better to train a quantized model from scratch or from a floating point model. In agreement with other work [27], we notice better accuracy when we fine tune a floating point model as shown in figure 13

. This is consistent with the general observation that it is better to train a model with more degrees of freedom and then use that as a teacher to produce a smaller model (

[28]).

3. Matching Batch normalization with inference reduces jitter and improves accuracy. We show results for two networks. In the first experiment (see figure 14), we compare training with naive batch norm folding, batch renormalization and batch normalization with correction and freezing for Mobilenet-v1_1_224. We note stable eval accuracy and higher accuracy with our proposed approach. In the second experiment, we compare naive batch norm folding and batch normalization with correction and freezing for Mobilenet_v2_1_224. We note that corrections stabilize training and freezing batch norms provides additional accuracy gain, seen after step 400000 in figure 15.

4. Use Exponential moving averaging for quantization with caution. Moving averages of weights [29] are commonly used in floating point training to provide improved accuracy [30]. Since we use quantized weights and activations during the back-propagation, the floating point weights converge to the quantization decision boundaries. Even minor variations in the floating point weights, between the instantaneous and moving averages can cause the quantized weights to be significantly different, hurting performance, see drop in accuracy for the EMA curve in figure 15.

## 5 Model Architecture Recommendations

In this section, we explore choices of activation functions and tradeoffs between precision and width of a network.

• Do not constrain activation ranges: One can get slightly better accuracy by replacing ReLU6 non-linearity with a ReLU and let the training determine the activation ranges (see figure 16)

• Explore tradeoff of width vs quantization: An over-parameterized model is more amenable to quantization. Even for leaner architectures like mobilenet, one can tradeoff the depth multiplier with the precisions of the weights. We compare the accuracies obtained with 4 bit per-channel quantization of weights with 8-bit quantization across different depth multipliers in figure 17. Note that this comparison allows us to evaluate a depth vs quantization tradeoff (see [31]). It is interesting to see that one can obtain a further 25% reduction in the model size for almost the same accuracy by moving to 4 bit precision for the weights.

## 6 Run-time measurements

We measure the run-times (Table 7) on a single large core of the Google Pixel 2 device for both floating point and quantized models. We also measure run-times using the Android NN-API on Qualcomm’s DSPs. We see a speedup of 2x to 3x for quantized inference compared to float, with almost 10x speedup with Qualcomm DSPs.

## 7 Neural network accelerator recommendations

In order to fully realize the gains of quantized networks, we recommend that neural network accelerators consider the following enhancements:

1. Aggressive operator fusion: Performing as many operations as possible in a single pass can lower the cost of memory accesses and provide significant improvements in run-time and power consumption.

2. Compressed memory access: One can optimize memory bandwidth by supporting on the fly de-compression of weights (and activations). A simple way to do that is to support lower precision storage of weights and possibly activations.

3. Lower precision arithmetic:

One can obtain further acceleration by supporting a range of precisions for arithmetic. Our recommendation is to support 4,8 and 16-bit weights and activations. While 4 and 8-bit precisions are sufficient for classification, higher precision support is likely needed for regression applications, like super-resolution and HDR image processing.

4. Per-layer selection of bitwidths: We expect that many layers of a network can be processed at lower precision. Having this flexibility can further reduce model size and processing time.

5. Per-channel quantization: Support for per-channel quantization of weights is critical to allow for:

1. Easier deployment of models in hardware, requiring no hardware specific fine tuning.

2. Lower precision computation.

## 8 Conclusions and further work

Based on our experiments, we make the following conclusions:

• Quantizing models

1. Use symmetric-per-channel quantization of weights with post training quantization as a starting point. Optionally fine tune if there is an accuracy drop.

2. Quantization aware training can narrow the gap to floating point accuracy and in our experiments, reduce the gap to within 5% of 8-bit quantized weights, even when all layers are quantized to 4 bits of precision.

• Performance

1. Quantized inference at 8-bits can provide 2x-3x speed-up on a CPU and close to 10x speedup compared to floating point inference on specialized processors optimized for low precision wide vector arithmetic, like the Qualcomm DSP with HVX.

2. One can obtain a model size reduction of 4x with no accuracy loss with uniform quantization. Higher compression can be obtained with non-uniform quantization techniques like K-means (

[6]).

• Training Techniques

1. Quantization aware training can substantially improve the accuracy of models by modeling quantized weights and activations during the training process.

2. It is critical to match quantized inference with the forward pass of training.

3. Special handling of batch normalization is required to obtain improved accuracy with quantized models.

4. Stochastic quantization during training underperforms deterministic quantization.

5. Exponential Moving Averages of weights may under-perform instantaneous estimates during quantization aware training and must be used with caution.

• Model architectures for quantization

1. There is a clear tradeoff between model size and compressibility. Larger models are more tolerant of quantization error.

2. Within a single architecture, one can tradeoff feature-maps and quantization, with more feature maps allowing for lower bitwidth kernels.

3. One can obtain improved accuracy by not constraining the ranges of the activations during training and then quantizing them, instead of restricting the range to a fixed value. In our experiments, it was better to use a ReLU than a ReLU6 for the activations.

Going forward, we plan to enhance our automated quantization tool to enable better quantization of networks by investigating the following areas:

1. Regularization techniques to better control the dynamic ranges of weights and activations can provide further improvements.

2. Distilled training to further improve the accuracy of quantized models [32].

3. Per-layer quantization of weights and activations to provide further compression and performance gains on hardware accelerators. Reinforcement learning has been applied successfully towards this problem in [33].

## 9 Acknowledgements

This work builds on the quantization scheme first introduced in [4]. Suharsh Sivakumar developed the [23] tool used for the quantization experiments, which extend the capabilities first described in [4]. Rocky Rhodes provided the performance measurement numbers for the models. We would also like to thank Cliff Young, Song Han, Rocky Rhodes and Skirmantas Kligys for their useful comments. Pete Warden provided useful input on the scope of the paper and suggested several experiments included in this document.

## References

• [1] M. Sandler, A. G. Howard, M. Zhu, A. Zhmoginov, and L. Chen, “Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation,” 2018.
• [2] A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” Apr. 2017.
• [3] F. N. Iandola, M. W. Moskewicz, K. Ashraf, S. Han, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size,” CoRR, vol. abs/1602.07360, 2016.
• [4] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference,” Dec. 2017.
• [5] M. Courbariaux, Y. Bengio, and J. David, “Binaryconnect: Training deep neural networks with binary weights during propagations,” 2015.
• [6] S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” CoRR, vol. abs/1510.00149, 2015.
• [7] GEMMLOWP, “Gemmlowp: a small self-contained low-precision GEMM library.”
• [8] Intel(R) MKL-DNN, “Intel(R) Math Kernel Library for Deep Neural Networks.”
• [9] ARM, “Arm cmsis nn software library.”
• [10] Qualcomm onQ blog, “How can Snapdragon 845’s new AI boost your smartphone’s IQ?.”
• [11] Nvidia, “8 bit inference with TensorRT.”
• [12] V. Sze, Y. Chen, T. Yang, and J. S. Emer, “Efficient processing of deep neural networks: A tutorial and survey,” CoRR, vol. abs/1703.09039, 2017.
• [13]

Nvidia, “The nvidia deep learning accelerator.”

• [14] S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, “EIE: efficient inference engine on compressed deep neural network,” 2016.
• [15] “Android Neural Network API.”
• [16] “Gemmlowp:building a quantization paradigm from first principles.”
• [17] A. Polino, R. Pascanu, and D. Alistarh, “Model compression via distillation and quantization,” 2018.
• [18]

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” Dec. 2015.

• [19] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning transferable architectures for scalable image recognition,” 2017.
• [20] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015.
• [21] K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” 2016.
• [22] T. Sheng, C. Feng, S. Zhuo, X. Zhang, L. Shen, and M. Aleksic, “A quantization-friendly separable convolution for mobilenets,” 2018.
• [23] “Tensorflow quantization library.”
• [24]

M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015.

Software available from tensorflow.org.
• [25] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” 2015.
• [26] “Slim model repository.”
• [27] A. K. Mishra and D. Marr, “Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy,” 2017.
• [28] G. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge in a Neural Network,” Mar. 2015.
• [29] B. Polyak, “New stochastic approximation type procedures,” Jan 1990.
• [30] “Exponential moving average.”
• [31] A. K. Mishra, E. Nurvitadhi, J. J. Cook, and D. Marr, “WRPN: wide reduced-precision networks,” CoRR, vol. abs/1709.01134, 2017.
• [32] A. K. Mishra and D. Marr, “Apprentice: Using knowledge distillation techniques to improve low-precision network accuracy,” 2017.
• [33] Y. He and S. Han, “ADC: automated deep compression and acceleration with reinforcement learning,” CoRR, vol. abs/1802.03494, 2018.

## Appendix A Impact of Batch Normalization on Quantization

To understand the impact of batch normalization on the dynamic range of the folded weights (W), we consider the following metrics:

1. SQNR: We calculate the Signal to quantization noise ratio defined as:

 SQNR=10log10∑W2∑(W−SimQuant(W))2

for different quantization schemes. We note that per-channel quantization provides significant improvement in SQNR over per-layer quantization, even if only symmetric quantization is used in the per-channel case.

2. Weight Power Distribution: We also plot the distribution of the sample weights, normalized by the average power, i.e we plot

 histogram(W2E[W2])

for the weights before and after folding. We note that after folding, there are much larger outliers which severely degrade performance.


text:
Visual Prompting via Image Inpainting

Amir Bar* 12,

UC Berkeley1
Tel Aviv University2

* equal contribution

NeurIPS 2022   [paper]   [BibTeX]   [dataset & code]

Visual prompting via Image Inpainting. Top: Prompting Image Inpainting Models. Given input-output example(s) $$(x_1, y_1)$$ and image query $$x_q$$, we construct a grid-like single image called a visual prompt $$x_{vp}$$. The visual prompt is composed of the desired task example(s) and a new query image (all in green). The inpainting model goal is then to predict the masked region (red) such that it is consistent with the example(s). Bottom: an inpainting model can solve this way various computer vision tasks, given that it was trained on the right data. The model predictions are annotated in red.

Abstract

How does one adapt a pre-trained visual model to novel downstream tasks without task-specific finetuning or any model modification? Inspired by prompting in NLP, this paper investigates visual prompting: given input-output image example(s) of a new task at test time and a new input image, the goal is to automatically produce the correct output image, consistent with the example(s) task. We show that posing this problem as a simple image inpainting task - literally just filling in a hole in a concatenated visual prompt image - turns out to be surprisingly effective, given that the inpainting algorithm has been trained on the right data. We train masked auto-encoding models on a new dataset that we curated - 88k unlabeled figures from academic papers sources on Arxiv. We apply visual prompting to these pretrained models and demonstrate results on various downstream tasks, including foreground segmentation, single object detection, colorization, edge detection, etc.

Our inpainting results

Visual prompting prediction examples. Each visual prompt was fed to an MAE-VQGAN model trained on the Figures dataset. For each visual prompt, the result is marked in red.

Computer Vision Figures Dataset

We curated a dataset of 88k unlabeled figures from Computer Vision academic papers. We train VQGAN-MAE on randomly sampled crops from these figures, without any additional parsing.

Random samples from our Computer Vision Figures dataset.

Inpainting using MAE-VQGAN

During training, an input image is patchified, masked and fed into a masked auto-encoder. For each masked token, the decoder outputs a distribution over a pretrained VQGAN codebook. The model is trained using cross entropy loss on random crops from our datasets.

Paper

 "Visual prompting via Image Inpainting", Amir Bar*, Yossi Gandelsman*, Trevor Darrell, Amir Globerson and Alexei A. Efros.

Supplementary Material


text:
# predict

Predict classification

## Syntax

```label = predict(tree,X)[label,score] = predict(tree,X)[label,score,node] = predict(tree,X)[label,score,node,cnum] = predict(tree,X)[label,...] = predict(tree,X,Name,Value)```

## Description

`label = predict(tree,X)` returns a vector of predicted class labels for a matrix `X`, based on `tree`, a trained full or compact classification tree.

```[label,score] = predict(tree,X)``` returns a matrix of scores, indicating the likelihood that a label comes from a particular class.

```[label,score,node] = predict(tree,X)``` returns a vector of predicted node numbers for the classification, based on `tree`.

```[label,score,node,cnum] = predict(tree,X)``` returns a vector of predicted class number for the classification, based on `tree`.

`[label,...] = predict(tree,X,Name,Value)` returns labels with additional options specified by one or more `Name,Value` pair arguments.

## Input Arguments

 `tree` A classification tree created by `fitctree`, or a compact classification tree created by `compact`. `X` A matrix where each row represents an observation, and each column represents a predictor. The number of columns in `X` must equal the number of predictors in `tree`.

### Name-Value Pair Arguments

Specify optional comma-separated pairs of `Name,Value` arguments. `Name` is the argument name and `Value` is the corresponding value. `Name` must appear inside single quotes (`' '`). You can specify several name and value pair arguments in any order as `Name1,Value1,...,NameN,ValueN`.

 `'Subtrees'` A vector of nonnegative integers in ascending order or `'all'`. If you specify a vector, then all elements must be at least `0` and at most `max(tree.PruneList)`. `0` indicates the full, unpruned tree and `max(tree.PruneList)` indicates the a completely pruned tree (i.e., just the root node). If you specify `'all'`, then `CompactClassificationTree.predict` operates on all subtrees (i.e., the entire pruning sequence). This specification is equivalent to using `0:max(tree.PruneList)`. `CompactClassificationTree.predict` prunes `tree` to each level indicated in `Subtrees`, and then estimates the corresponding output arguments. The size of `Subtrees` determines the size of some output arguments. To invoke `Subtrees`, the properties `PruneList` and `PruneAlpha` of `tree` must be nonempty. In other words, grow `tree` by setting `'Prune','on'`, or by pruning `tree` using `prune`. Default: `0`

## Output Arguments

 `label` Vector of class labels of the same type as the response data used in training `tree`. Each entry of `label` corresponds to the class with minimal expected cost for the corresponding row of `X`. See Predicted Class Label. If `Subtrees` has `T` elements, and `X` has `N` rows, then `labels` is an `N`-by-`T` matrix. The `i`th column of `labels` contains the fitted values produced by the `Subtrees(I)` subtree. `score` Numeric matrix of size `N`-by-`K`, where `N` is the number of observations (rows) in `X`, and `K` is the number of classes (in `tree.ClassNames`). `score(i,j)` is the posterior probability that row `i` of `X` is of class `j`. If `Subtrees` has `T` elements, and `X` has `N` rows, then `score` is an `N`-by-`K`-by-`T` array, and `node` and `cnum` are `N`-by-`T` matrices. `node` Numeric vector of node numbers for the predicted classes. Each entry corresponds to the predicted node in `tree` for the corresponding row of `X`. `cnum` Numeric vector of class numbers corresponding to the predicted `labels`. Each entry of `cnum` corresponds to a predicted class number for the corresponding row of `X`.

## Definitions

### Predicted Class Label

`predict` classifies so as to minimize the expected classification cost:

$\stackrel{^}{y}=\underset{y=1,...,K}{\mathrm{arg}\mathrm{min}}\sum _{k=1}^{K}\stackrel{^}{P}\left(k|x\right)C\left(y|k\right),$

where

• $\stackrel{^}{y}$ is the predicted classification.

• K is the number of classes.

• $\stackrel{^}{P}\left(k|x\right)$ is the posterior probability of class k for observation x.

• $C\left(y|k\right)$ is the cost of classifying an observation as y when its true class is k.

### Score (tree)

For trees, the score of a classification of a leaf node is the posterior probability of the classification at that node. The posterior probability of the classification at a node is the number of training sequences that lead to that node with the classification, divided by the number of training sequences that lead to that node.

For example, consider classifying a predictor `X` as `true` when `X` < `0.15` or `X` > `0.95`, and `X` is false otherwise.

Generate 100 random points and classify them:

```rng(0,'twister') % for reproducibility X = rand(100,1); Y = (abs(X - .55) > .4); tree = fitctree(X,Y); view(tree,'Mode','Graph') ```

Prune the tree:

```tree1 = prune(tree,'Level',1); view(tree1,'Mode','Graph') ```

The pruned tree correctly classifies observations that are less than 0.15 as `true`. It also correctly classifies observations from .15 to .94 as `false`. However, it incorrectly classifies observations that are greater than .94 as `false`. Therefore, the score for observations that are greater than .15 should be about .05/.85=.06 for `true`, and about .8/.85=.94 for `false`.

Compute the prediction scores for the first 10 rows of `X`:

```[~,score] = predict(tree1,X(1:10)); [score X(1:10,:)] ```
```ans = 0.9059 0.0941 0.8147 0.9059 0.0941 0.9058 0 1.0000 0.1270 0.9059 0.0941 0.9134 0.9059 0.0941 0.6324 0 1.0000 0.0975 0.9059 0.0941 0.2785 0.9059 0.0941 0.5469 0.9059 0.0941 0.9575 0.9059 0.0941 0.9649 ```

Indeed, every value of `X` (the right-most column) that is less than 0.15 has associated scores (the left and center columns) of `0` and `1`, while the other values of `X` have associated scores of `0.91` and `0.09`. The difference (score `0.09` instead of the expected `.06`) is due to a statistical fluctuation: there are `8` observations in `X` in the range `(.95,1)` instead of the expected `5` observations.

### True Misclassification Cost

There are two costs associated with classification: the true misclassification cost per class, and the expected misclassification cost per observation.

You can set the true misclassification cost per class in the `Cost` name-value pair when you create the classifier using the `fitctree` method. `Cost(i,j)` is the cost of classifying an observation into class `j` if its true class is `i`. By default, `Cost(i,j)=1` if `i~=j`, and `Cost(i,j)=0` if `i=j`. In other words, the cost is `0` for correct classification, and `1` for incorrect classification.

### Expected Cost

There are two costs associated with classification: the true misclassification cost per class, and the expected misclassification cost per observation.

Suppose you have `Nobs` observations that you want to classify with a trained classifier. Suppose you have `K` classes. You place the observations into a matrix `Xnew` with one observation per row.

The expected cost matrix `CE` has size `Nobs`-by-`K`. Each row of `CE` contains the expected (average) cost of classifying the observation into each of the `K` classes. `CE(n,k)` is

$\sum _{i=1}^{K}\stackrel{^}{P}\left(i|Xnew\left(n\right)\right)C\left(k|i\right),$

where

• K is the number of classes.

• $\stackrel{^}{P}\left(i|Xnew\left(n\right)\right)$ is the posterior probability of class i for observation Xnew(n).

• $C\left(k|i\right)$ is the true misclassification cost of classifying an observation as k when its true class is i.

### Predictive Measure of Association

The predictive measure of association between the optimal split on variable i and a surrogate split on variable j is:

${\lambda }_{i,j}=\frac{\text{min}\left({P}_{L},{P}_{R}\right)-\left(1-{P}_{{L}_{i}{L}_{j}}-{P}_{{R}_{i}{R}_{j}}\right)}{\text{min}\left({P}_{L},{P}_{R}\right)}.$

Here

• PL and PR are the node probabilities for the optimal split of node i into Left and Right nodes respectively.

• ${P}_{{L}_{i}{L}_{j}}$ is the probability that both (optimal) node i and (surrogate) node j send an observation to the Left.

• ${P}_{{R}_{i}{R}_{j}}$ is the probability that both (optimal) node i and (surrogate) node j send an observation to the Right.

Clearly, λi,j lies from –∞ to 1. Variable j is a worthwhile surrogate split for variable i if λi,j > 0.

## Examples

collapse all

### Predict Labels Using a Classification Tree

Examine predictions for a few rows in a data set left out of training.

```load fisheriris ```

Partition the data into training (50%) and validation (50%) sets.

```n = size(meas,1); rng(1) % For reproducibility idxTrn = false(n,1); idxTrn(randsample(n,round(0.5*n))) = true; % Training set logical indices idxVal = idxTrn == false; % Validation set logical indices ```

Grow a classification tree using the training set.

```Mdl = fitctree(meas(idxTrn,:),species(idxTrn)); ```

Predict labels for the validation data. Count the number of misclassified observations.

```label = predict(Mdl,meas(idxVal,:)); label(randsample(numel(label),5)) % Display several predicted labels numMisclass = sum(~strcmp(label,species(idxVal))) ```
```ans = 'setosa' 'setosa' 'setosa' 'virginica' 'versicolor' numMisclass = 3 ```

The software misclassifies three out-of-sample observations.

### Estimate Class Posterior Probabilities Using a Classification Tree

```load fisheriris ```

Partition the data into training (50%) and validation (50%) sets.

```n = size(meas,1); rng(1) % For reproducibility idxTrn = false(n,1); idxTrn(randsample(n,round(0.5*n))) = true; % Training set logical indices idxVal = idxTrn == false; % Validation set logical indices ```

Grow a classification tree using the training set, and then view it.

```Mdl = fitctree(meas(idxTrn,:),species(idxTrn)); view(Mdl,'Mode','graph') ```

The resulting tree has four levels.

Estimate posterior probabilities for the test set using subtrees pruned to levels 1 and 3.

```[~,Posterior] = predict(Mdl,meas(idxVal,:),'SubTrees',[1 3]); Mdl.ClassNames Posterior(randsample(size(Posterior,1),5),:,:),... % Display several posterior probabilities ```
```ans = 'setosa' 'versicolor' 'virginica' ans(:,:,1) = 1.0000 0 0 1.0000 0 0 1.0000 0 0 0 0 1.0000 0 0.8571 0.1429 ans(:,:,2) = 0.3733 0.3200 0.3067 0.3733 0.3200 0.3067 0.3733 0.3200 0.3067 0.3733 0.3200 0.3067 0.3733 0.3200 0.3067 ```

The elements of `Posterior` are class posterior probabilities:

• Rows correspond to observations in the validation set.

• Columns correspond to the classes as listed in `Mdl.ClassNames`.

• Pages correspond to the subtrees.

The subtree pruned to level 1 is more sure of its predictions than the subtree pruned to level 3 (i.e., the root node).

## Algorithms

`predict` generates predictions by following the branches of `tree` until it reaches a leaf node or a missing value. If `predict` reaches a leaf node, it returns the classification of that node.

If `predict` reaches a node with a missing value for a predictor, its behavior depends on the setting of the `Surrogate` name-value pair when `fitctree` constructs `tree`.

• `Surrogate` = `'off'` (default) — `predict` returns the label with the largest number of training samples that reach the node.

• `Surrogate` = `'on'``predict` uses the best surrogate split at the node. If all surrogate split variables with positive predictive measure of association are missing, `predict` returns the label with the largest number of training samples that reach the node. For a definition, see Predictive Measure of Association.


text:
#### Joint Location, Bandwidth and Power Optimization for THz-enabled UAV Communications

##### Luyao Xu, Ming Chen, Mingzhe Chen, Zhaohui Yang, Christina Chaccour, Walid Saad, Choong Seon Hong

In this paper, the problem of unmanned aerial vehicle (UAV) deployment, power allocation, and bandwidth allocation is investigated for a UAV-assisted wireless system operating at terahertz (THz) frequencies. In the studied model, one UAV can service ground users using the THz frequency band. However, the highly uncertain THz channel will introduce new challenges to the UAV location, user power, and bandwidth allocation optimization problems. Therefore, it is necessary to design a novel framework to deploy UAVs in the THz wireless systems. This problem is formally posed as an optimization problem whose goal is to minimize the total delays of the uplink and downlink transmissions between the UAV and the ground users by jointly optimizing the deployment of the UAV, the transmit power and the bandwidth of each user. The communication delay is crucial for emergency communications. To tackle this nonconvex delay minimization problem, an alternating algorithm is proposed while iteratively solving three subproblems: location optimization subproblem, power control subproblem, and bandwidth allocation subproblem. Simulation results show that the proposed algorithm can reduce the transmission delay by up to $59.3\%$, $49.8\%$ and $75.5\%$ respectively compared to baseline algorithms that optimize only UAV location, bandwidth allocation or transmit power control.

arrow_drop_up


text:
# Op amp signal processing of an accelerometer output for an ADC

I have been designing a board which will monitor vibration activity of a machine using MEMS accelerometers. So far, I have connected the unadulterated signal from the accelerometer to the ADC however, I was considering the fact that if there is very little vibration, if I removed the dc bias, amplified the signal and then re-added the bias, I could make better use of the ADC resolution. The bias is 1.65V and the sensitivity is 300mV/g so under normal conditions with no vibration, the output is 1.95V.

From what I have googled, I think I can use a differential amplifier to remove the bias and to amplify the signal followed by a summing amplifier to replace the bias. So I have created the following circuit where V1 is 1.95V and V2 is the accelerometer output. However, I have no idea of what value resistors to use and how to calculate gain. I would like to either use a digi-pot or a MUX to change resistance and control gain from unity, 2X, 4X, 8X, 16X and 32X

Can somebody give me a heads up please? What sort of values should I use, what voltage should I supply to the op amps - is $\pm 5\text{ V}$ sufficient for an accelerometer output of 3.3V maximum? Which resistor should be variable and how do I calculate the values to give me the gain I want?

• Specifying gain alone is insufficient. MEMs output ranges between 1.65 - 1.95v, but what's its output impedance? What peak amplitude is your expected signal? And what does the ADC expect for full-scale input signal range? Unipolar? Bipolar? Differential, with or without bias? – glen_geek Jan 27 '17 at 16:38
• The ADXL335 data sheet does not tell you its impedance. it is -3/+3g's @ 300mV/g so the peak amplitude can be 1.65+0.9 = 2.55V The expected range for the ADC is 0-3V3, it is unipolar and the bias needs to be added back in and removed digitally pre FFT. I also realise I have drawn the differential amp wrong, the inverting and non inverting inputs should be the other way round. I'm wondering if I should use a third op amp after the differential amp to do the actual amplification, use the same values for R1-R4 (above) so the output is just the difference X 1 – BaartCM Jan 27 '17 at 17:15
• This looks like an application for an instrumentation amplifier. If you use one, you won't have to worry about the output impedance of the accelerometer. Also, an in amp's gain is often set by only one resistor whereas the diff amp you show would need two resistors changed in order to adjust the gain. If you want to adjust the gain from unity to 32x, set the in amp to unity gain and follow it with a programmable gain amplifier. – Null Jan 27 '17 at 17:19
• An in amp also has a reference pin which you could set to the bias voltage (you wouldn't need a summing amplifier to add it back in). – Null Jan 27 '17 at 17:21
• @Null Thanks for the help. Not sure why I wouldn't need to add the bias back in. My thoughts are 1, convert from unipolar to bipolar (remove the bias) 2, amplify the signal, 3, convert back to unipolat by adding in the bias. If I use a unity gain instrumentation amp then a PGA to amplify, surely I still need to add the bias back. Any suggestions as to which PGA to use? – BaartCM Jan 27 '17 at 17:33

What sort of values [for the differential amplifier resistors] should I use?

A rule of thumb is that the input impedance of the differential amplifier should be at least ten times the output impedance of the accelerometer in order to avoid signal loss (the differential amplifier would ideally have infinite input impedance). The block diagram of the ADXL335 datasheet suggests that the output impedance is about $32\text{k}\Omega$, so you'd need high valued resistors.

The gain of the differential amplifier is set by the ratio of resistors (an example derivation is here). You need to set the resistors in your schematic to $$\frac{R_2}{R_1} = \frac{R_4}{R_3} = \text{desired gain}$$

The problem with this is that you need to adjust two resistors to adjust the gain.

You can solve both of these problems with an instrumentation amplifier. Conceptually, it's a difference amplifier with a pair of op amp buffers on each input:

The op amp buffers give you much higher input impedance than the difference amplifier alone, and the architecture allows the gain to be set by changing only one resistor. You can construct an in amp out of discrete op amps, but an IC in amp will have less gain error because ICs have better matching of the resistors. IC manufacturers offer a wide variety of instrumentation amplifiers so you should be able to find one which meets your needs.

As a bonus, IC in amps provide a reference pin which you can use to provide an offset to your output (e.g. by the bias voltage). In the above in amp schematic, the reference pin replaces the ground connection to $R_3$. An explanation for how to use this pin can usually be found in the in amp's datasheet; for example, the AD8221 datasheet says:

As shown in Figure 43, the reference terminal, REF, is at one end of a 10 kΩ resistor. The output of the instrumentation amplifier is referenced to the voltage on the REF terminal; this is useful when the output signal needs to be offset to a precise midsupply level. For example, a voltage source can be tied to the REF pin to level-shift the output so that the AD8221 can interface with an ADC. The allowable reference voltage range is a function of the gain, input, and supply voltage. The REF pin should not exceed either +VS or –VS by more than 0.5 V.

For best performance, source impedance to the REF terminal should be kept low, because parasitic resistance can adversely affect CMRR and gain accuracy.

You'd probably need to add a simple op amp buffer to the bias voltage so that the source impedance on the REF terminal is low enough.

what voltage should I supply to the op amps - is $\pm 5\text{ V}$ sufficient for an accelerometer output of 3.3V maximum?

It might be. It depends on the amplifier you choose. You need to make sure that the amplifier can operate on $\pm 5\text{ V}$, that its inputs will stay within its input common mode range, and that its output can swing close enough to its supply voltages. Consult its datasheet.

To handle the gain requirement of 1 to 32 in binary steps, I'd suggest using a programmable gain amplifier with binary weighted gains. For example, the PGA205 is a programmable gain instrumentation amplifier (both the in amp and programmable gain combined) with a gain selection of 1, 2, 4, or 8. Add another programmable gain operational amplifier with binary weighted gains (e.g. the LTC6910-2 to the output of the first one to achieve an overall gain of 1 to 32).