text:
Functional ANOVA using Gaussian Process priors.

## Project description

# Gaussian Process Functional ANOVA

Implementation of a functional ANOVA (FANOVA) model, based partly on the model in

[Bayesian functional ANOVA modeling using Gaussian process prior distributions](http://projecteuclid.org/euclid.ba/1340369795). To implement a FANOVA model, an underlying general framework is defined for modeling functional observations:

$$Y(t) = X beta(t),$$

where $$Y(t) = [y_1(t),dots,y_m(t)]^T,$$ $$beta(t) = [beta_1(t),dots,beta_f(t)]^T,$$ $$X: m times f$$ for a given time $t$. The design matrix $X$ defines the relation between the functions $beta$ and observations $y$. In general, the rank of $X$ should match the number of functions $f$. The FANOVA model can then be described by a specific form of $X$ such that

$$y_{i,j}(t) = mu(t) + alpha_i(t) + beta_j(t) + alphabeta_{i,j}(t).$$

## Project details

Uploaded source

format: plain


text:
+0

# HELP :(

+1
112
4
+91

Find the  number of pairs of integers (m,n) that satisfy the equation:

mn−m+4n=40

Jun 23, 2022

#1
-1

We can "finish the square" except this time it's not a square. We can factor the left side into (m+4)(n-1)+4, and subtract 4 from both sides: (m+4)(n-1)=36. Now 36 has 9 factors(2^2*3^2, so (2+1)(2+1)=9),so there are 9 pairs of numbers whos product is 36. So the answer is \(\boxed{9}\)

Jun 23, 2022
#2
+91
0

it wrong :(

Derekshackk  Jun 23, 2022
#3
+2448
0

The equation factors to: \((m+4)(n-1) = 36\)

Now, recall the factor of 36: 1,2,3,4,6,9,12,18,36

We need to find the number of pairs of integers that satisfy this.

For example, 1 pair is \((32,2)\), because 36 x 1 = 36.

Can you take it from here?

Jun 23, 2022
#4
+91
+1

help

Derekshackk  Jun 23, 2022


text:
Skip to main content

Experimental tools of the Knapen lab at the Vrije Universiteit

## Project description

Note to the author: Please add something informative to this README before releasing your software, as a little documentation goes a long way. Both README.rst (this file) and NEWS.txt (release notes) will be included in your package metadata which gets displayed in the PyPI page for your project.

You can take a look at the README.txt of other projects, such as repoze.bfg (http://bfg.repoze.org/trac/browser/trunk/README.txt) for some ideas.

## Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

### Source Distribution

exptools-0.3.1.tar.gz (13.3 kB view hashes)

Uploaded source


text:
# Template:Table of universal constants

The purpose of this template is to collect source text used in several articles in one place, in order to minimize maintenance and storage space. This is the table with the universal physical constants.

Quantity
Symbol Value[1][2] Relative Standard Uncertainty
speed of light in vacuum ${\displaystyle c\,}$ 299 792 458 m·s−1 defined
Newtonian constant of gravitation ${\displaystyle G\,}$ 6.67384(80)×10−11 m3·kg−1·s−2 1.2 × 10−4
Planck constant ${\displaystyle h\,}$ 6.626 069 57(29) × 10−34 J·s 4.4 × 10−8
reduced Planck constant ${\displaystyle \hbar =h/(2\pi )}$ 1.054 571 726(47) × 10−34 J·s 4.4 × 10−8

## References

1. ^ The values are given in the so-called concise form; the number in brackets is the standard uncertainty, which is the value multiplied by the relative standard uncertainty.
2. ^ P.J. Mohr, B.N. Taylor, and D.B. Newell (2011), "The 2010 CODATA Recommended Values of the Fundamental Physical Constants" (Web Version 6.0). This database was developed by J. Baker, M. Douma, and S. Kotochigova. Available: http://physics.nist.gov/constants [Thursday, 02-Jun-2011 21:00:12 EDT]. National Institute of Standards and Technology, Gaithersburg, MD 20899.


text:
# Coulombs Law: magnitude of the force between two point charges

## Homework Statement

Determine the magnitude of the force between two point charges, Q1 = -2e located at (-3.60 cm, +4.10 cm) and Q2 = -8e located at (-0.60 cm, -8.60 cm). Give your answer in the form "a.bc x 10^(y) N".

F = kq1q2/r2

## The Attempt at a Solution

a.) I changed the unit vectors to meters, then I added, squared, and got the square roots of the vectors to get r = sqrt(((-0.036 +(- 0.006))^2)+((0.041+(-0.08))^2))
b.) Then I multiplied the value of "K" by the value of "8e" and "2e" and divided all three values by r^2

Related Introductory Physics Homework Help News on Phys.org
Simon Bridge
Homework Helper

## The Attempt at a Solution

a.) I changed the unit vectors to meters, then I added, squared, and got the square roots of the vectors to get r = sqrt(((-0.036 +(- 0.006))^2)+((0.041+(-0.08))^2))
b.) Then I multiplied the value of "K" by the value of "8e" and "2e" and divided all three values by r^2
Sounds reasonable - what was your question?
 see ehild below - it is not really all that clear what your reasoning is.

Converting everything to SI units first does help make sure you get SI units out. However, you don't have to.

Where qi=nie (n is an integer and e is the elementary charge) then |F1,2| = (ke2)n1n2/r2

Since: ke2 = 2.307x10-24N.cm2 you can do everything in cm.

Last edited:
ehild
Homework Helper

## Homework Statement

Determine the magnitude of the force between two point charges, Q1 = -2e located at (-3.60 cm, +4.10 cm) and Q2 = -8e located at (-0.60 cm, -8.60 cm). Give your answer in the form "a.bc x 10^(y) N".

F = kq1q2/r2

## The Attempt at a Solution

a.) I changed the unit vectors to meters, then I added, squared, and got the square roots of the vectors to get r = sqrt(((-0.036 +(- 0.006))^2)+((0.041+(-0.08))^2))
b.) Then I multiplied the value of "K" by the value of "8e" and "2e" and divided all three values by r^2
Why did you add the two vectors? The Coulomb force is inversely proportional to the square of the distance between two point charges. How do you get the distance between two points from their location?

ehild

Treat the two given points as two position vectors in 2D.

Find the magnitude of the displacement between these two position vectors !

That gives you "r"

Hi, hms.tech
Thanks for your reply. I solved the two positions as two vectors using pythogoras theory. For example sqrt(((-0.036 +(- 0.006))^2)+((0.041+(-0.08))^2)), I do not know if it gives this solution gives me the distance between two point charges.

Simon Bridge
Homework Helper

If you don't know if the method gives the answer you want, then why did you use it?

In general, if you have points $P=(p_x,p_y)$ and $Q=(q_x,q_y)$ defined against an origin $O=(0,0)$, then the distance between them is the length of the vector from P to Q which is given by change in position

$$\overrightarrow{PQ} = \overrightarrow{OQ} - \overrightarrow{OP} = (q_x-p_x,q_y-p_y)$$... that length would be: $$|\overrightarrow{PQ}| = \sqrt{(q_x-p_x)^2 + (q_y-p_y)^2}$$... your working does not look like this.

Last edited:
Hi,
I have tried solving this problem several times but my answers are still wrong.I dont know if the error is from the radius or if i am supposed multiply each charge by 1.602 x 10^(-19), since e = 1.602 x 10^(-19)

Attemped effort:
8.99 x 10^(9) x 2(1.602 x 10^(-19)) x 8(1.602 x 10^(-19))/.13050^2

= 2.17 x 10^(-26)

I have solved this problem a billion times but its still wrong. Please someone help. I will really appreciate it....:happy:)

ehild
Homework Helper
Hi,
I have tried solving this problem several times but my answers are still wrong.I dont know if the error is from the radius or if i am supposed multiply each charge by 1.602 x 10^(-19), since e = 1.602 x 10^(-19)

Attemped effort:
8.99 x 10^(9) x 2(1.602 x 10^(-19)) x 8(1.602 x 10^(-19))/.13050^2

= 2.17 x 10^(-26)

I have solved this problem a billion times but its still wrong. Please someone help. I will really appreciate it....:happy:)
The magnitude is wrong. Try again.

ehild

Alright. But this what I got for the distance (r)

r^2 = (-0.006m-(-0.036m))^2 + (-0.086m-0.041m)^2
=.01703
square root of .01703 = .13050

Just put it in your calculator again and make sure you don't mistype and that all your order of operations is correct. You're off by some power of 10, which usually means you just didn't enter the calculation correctly.

The magnitude is wrong. Try again.

ehild
I have resolved it and my answer is 2.17 x 10^(-25). But the answer is still wrong...

ehild
Homework Helper
I have resolved it and my answer is 2.17 x 10^(-25). But the answer is still wrong...
If you calculate accurately, it is 2.16 x 10^(-25) N. Have you added the unit (N)?

ehild

Simon Bridge
Homework Helper
I have resolved it and my answer is 2.17 x 10^(-25). But the answer is still wrong...
I get 2.1676x10-25N ... how many sig fig should you keep?
What makes you think this is the wrong answer?

I get 2.1676x10-25N ... how many sig fig should you keep?
What makes you think this is the wrong answer?
Yes your right, the answer is correct... I was having a confusion with the units... Thank you

Simon Bridge


text:
# 6.3 Fast Probabilistic Consensus

## 6.3.1 Introduction

The functionality defined in this part of the specification allows nodes to find consensus on whether a given object is elgible or not. This protocol, called Fast Probabilistic Consensus (FPC), is triggered if the eligibility of an object is uncertain. These objects can be messages or transactions. Please refer to Section 6.1 - Object of Consensus for details on the object of consensus and to Section 6.2 - Opinion Setting for more details how initial opinions on the objects are formed.

Once FPC is triggered every node must establish an initial opinion on the eligibility of the object, as described in Section 6.1. The node must then start to query other nodes about their opinions on the given object and must update its opinion according to the rules specified in this section.

This procedure shall terminate locally if a node has not changed its opinion over a specified period of time or if some maximal amount of rounds is reached.

Unlike other voting-based consensus protocols, FPC uses a sequence of global random thresholds. This randomness makes FPC robust even in Byzantine environments. We refer to FPC-BI: Fast Probabilistic Consensus within Byzantine Infrastructures, Robustness and efficiency of leaderless probabilistic consensus protocols within Byzantine infrastructures, and Fast Probabilistic Consensus with Weighted Votes for more details on FPC.

The Fast Probabilistic Consensus Protocol specification depends on the following specifications:

## 6.3.2 FPC Protocol

The FPC protocol attempts to determine consensus on the eligibility of an object objectID. Every node has an initial opinion opinion on this object. These opinions are updated in rounds until the protocol terminates using a local stopping rule.

### 6.3.2.1 FPC Parameters

Table 6.3.1 gives a list of all parameters required for FPC.

Table 6.3.1: Parameters Required for FPC.

NameTypeDescription
TOTAL_ROUNDS_FINALIZATIONintegerNumber of consecutive rounds before FPC auto-terminates
TOTAL_ROUNDS_ENDING_THRESHOLDintegerNumber of consecutive rounds with non-random threshold
FIRST_ROUND_THRESHOLDdoubleThreshold of the proportion of opinions in the first round
SUBSEQUENT_LOWER_THRESHOLDdoubleLower random threshold bound in subsequent rounds
SUBSEQUENT_UPPER_THRESHOLDdoubleUpper random threshold bound in subsequent rounds
ENDING_THRESHOLDdoubleThreshold for termination phase
DRNG_WAITING_TIMEdurationMaximal waiting time (in seconds) to receive dRNG number
MAX_ROUNDintegerMaximum number of rounds before querying stops
QUERY_SIZEintegerQuorum size, number of nodes that are queried
ROUND_LENGTHdoubleDuration (in seconds) of a round
TIME_OUTdurationMaximal waiting time (in seconds) to receive answers for FPC queries
MIN_MANA_PROPORTIONdoubleMinimal amount of mana of received answers that allow to update opinion. If this amount is not reached the current round is not counted.
MAX_SAMPLE_SIZEintegerMaximal size of sample.

The proposed values of the parameters are (Table 6.3.2):

Table 6.3.2: FPC Parameter Default Values.

ParameterValue
TOTAL_ROUNDS_FINALIZATION10
TOTAL_ROUNDS_ENDING_THRESHOLD3
FIRST_ROUND_THRESHOLD0.67
SUBSEQUENT_LOWER_THRESHOLD0.50
SUBSEQUENT_UPPER_THRESHOLD0.67
ENDING_THRESHOLD0.50
DRNG_WAITING_TIME0.20
MAX_ROUND100
 QUERY_SIZE21
ROUND_LENGTH10
TIME_OUT6.5
MIN_MANA_PROPORTION0.50
MAX_SAMPLE_SIZE100

### 6.3.2.2 Local Variables

Local variables are those which may be defined at the application developer's discretion and do not form a mandatory part of the protocol. They are included here to assist in understanding the protocol defined in this section. The kind of information described for these variables must be handled by the node application in some form.

Every node shall keep the following variables (Table 6.3.3):

Table 6.3.3: Required Node Variables.

NameTypeDescription
nodeListlist of nodeIDsList of known nodes
consensusManaListlist of nodeIDsList of active consensus mana of known nodes
ownManadoubleActive consensus mana of ownNode
rndoubleRandom number provided by dRNG module
rnCycledoubleRandom number instance
queryListlist of nodeIDsList of nodes to query
queryMaxintegerMaximal number of queries per round
opinionQuerylist of OpinionsList of opinions of nodes in queryList, non-replies are encoded with NA
objectIDslist of objectIDsList of objectIDs that are under vote

Note that consensusManaList is a list of active consensus Mana of all known nodes. As this Mana changes over time this list has to be updated over time. Such an update is expressed with the function GetActiveConsensusMana(time) that returns the list of active consensus Mana at time time.

For each objectID that is under vote a node shall keep the following variables (Table 6.3.4):

Table 6.3.4: Variables Required for objectID Under Vote.

NameTypeDescription
opinionnulled booleanOpinion of the eligibility the objectID; TRUE corresponds to LIKE and FALSE to DISLIKE
cntintegerCounter of the number of consecutive rounds with unchanged opinion
queryStatusbooleanStatus if actively querying
answerStatusbooleanTRUE if answering queries, otherwise FALSE
roundintegerCounter for the number of rounds in FPC
reachedMaxRoundbooleanIndicator whether protocol reached MAX_ROUND before auto-termination, default value FALSE

### 6.3.2.3 FPC Protocol Description

#### FPC Protocol Operation for One Object ID

This section describes how FPC works for one objectID. Once FPC is triggered on the eligibility of the object objectID a node establishes an initial opinion opinion on objectID, see Section 6.2 Opinion Setting. In particular, the variables queryStatus and answerStatus must be affected using the functions QueryStatus and AnswerStatus. The exchange and update of opinions must be done in rounds of length ROUND_LENGTH. Rounds end and start at times (in seconds) that are multiple of ROUND_LENGTH.

1. At the beginning of each round a node must select a random sample of the other nodes and either send a query request or check if the sampled nodes published their opinions on the Tangle.
2. After TIME_OUT the node must calculate the proportion of the LIKEs in the received opinions.
3. At the end of the round it must set a threshold to confirm or modify its own opinion. In the first round this threshold is FIRST_ROUND_THRESHOLD while in the subsequent rounds a node must retrieve a random threshold from the dRNG. If the random threshold from the dRNG is not available a node must use (SUBSEQUENT_LOWER_THRESHOLD+ SUBSEQUENT_UPPER_THRESHOLD)/2 as threshold. If the proportion is below this obtained threshold it must set its opinion to DISLIKE, otherwise it must set its opinion to LIKE. In the case that the opinion did not change, the variable cnt must be incremented by 1, otherwise it must be set to 0. If a node did not receive enough answers, i.e., the proportion of mana of the received opinions is less than MIN_MANA_PROPORTION of the total mana of the queried nodes, the round is not counted, i.e., neither opinion nor cnt are changed.
4. This continues until cnt=TOTAL_ROUNDS_FINALIZATION-TOTAL_ROUNDS_ENDING_THRESHOLD. FPC then enters in the "termination phase" and continues for TOTAL_ROUNDS_ENDING_THRESHOLD rounds using the ENDING_THRESHOLD instead of the random threshold. If during this phase a node changes its opinion it must set cnt=0 and use the random threshold again. The node shall stop querying if cnt=TOTAL_ROUNDS_ENDING_THRESHOLD or cnt=reachedMaxRound. In case that cnt=reachedMaxRound a node must set the final opinion to DISLIKE.
5. Once FPC terminates (for the given objectID) a node must update the metadata opinionField of objectID with the outcome of FPC as opinion and level=2.
6. A node must continue to respond to queries as long as AnswerStatus(type, objectID)=TRUE, where type is the type of the objectID. If AnswerStatus(type, objectID)=FALSE the variable answerStatus must be updated accordingly.

High consensus Mana nodes will be queried more often than nodes with low consensus Mana, since the sampling using consensus Mana as weights. Every node is given the possibility to publish their opinions in a statement on the Tangle. These messages are called FPC statements. Nodes who decide to issue FPC statements may close the port reserved for FPC queries. A close port can therefore be an indication that a node decided to disseminate its opinions through statements. Every node shall keep a list of nodes that are not answering direct queries but publish their opinions on the Tangle. If a node issues two or more conflicting FPC statements in a round, every other node shall not take these messages into account, i.e., they do not count for the Mana of received opinions neither are used for the calculation of the proportion.

#### FPC Protocol Operation for Multiple Object IDs

It is possible that there are more than one object to vote on. In this case, a node shall sample once per round and obtain the opinions for all objects from this one sample. Malicious or faulty nodes may not respond in a consistent way. In particular, in the case of a double-spending, a malicious node may respond LIKE for two or more conflicting objects. These malicious messages shall be filtered after having received the responses of the queries. Therefore, upon receival of the opinions, a node shall check for every sampled node if its answers are consistent, i.e., all liked objects must form a valid ledger state. Nodes that replied inconsistently shall be filtered out and their answer shall be considered as not received, i.e., they do not count for the Mana of received opinions neither are used for the calculation of the proportion.

Note that since consensus Mana changes over time, FPC may use different consensus Mana values for different objectIDs that are voted on at the same time. For an objectID, FPC must use the active consensus Mana of epoch Epoch X-2, where Epoch X contains the timestamp of the objectID. We refer to Section 5.3 - Mana for the specification of active consensus Mana. We use a generic function GetActiveConsensusMana(time) that retrieves the active consensus Mana with respect to a given timestamp time.

### 6.3.2.4 FPC Pseudocode Description

The following presents some pseudo-code for a better understanding of the details of the FPC protocol.

The function GetRN(a, b, time) retrieves a uniform random number between a and b from the dRNG module of a given time time. If the dRNG is not retrieved in time a node must use (SUBSEQUENT_LOWER_THRESHOLD SUBSEQUENT_UPPER_THRESHOLD)/2 instead.

In each round a node must query a random sample of other nodes. The random sample is obtained using weighted (by active consensus Mana) sampling with replacement until QUERY_SIZE distinct elements are chosen, or until a maximum sample size of MAX_SAMPLE_SIZE is reached.

#### GetSample

This function chooses a sample of nodeIDs for FPC queries.

FUNCTION queryList = GetSample(nodeList, manaList)queryList = []WHILE (queryList does not contain QUERY_SIZE different elements)    newSample = SAMPLE(nodeList, weight=manaList)    queryList = APPEND(queryList, newSample)RETURN queryList

The function SAMPLE(nodeList, weight=manaList) chooses a random element from nodeList with weights corresponding to their Mana, i.e., the probability that a nodeID is chosen is proportional to manaList[nodeID].

Once the list queryList of nodes to query is chosen a node must obtain their opinions about the objects under voting. As there are two possibilities for nodes to communicate their opinions, via direct answers or via FPC statements, a node shall keep information on this behavior up to date. The message layout of the FPC statements is specified in Section 2.3 - Standard Payloads Layout.

At the beginning of each round every node must prepare a query and send it to those nodes in queryList that replies directly (i.e., do not publish FPC statements).

The queries must follow the layout that follows.

#### QueryRequest

NameTypeDescription
Versionuint8The message version. The schema specified in this RFC is for version 1 only.
Tx countuint8The number of TransactionIDs.
TransactionIDs Tx count
TransactionID, ordered by hash ASC
NameTypeDescription
TransactionIDByteArray[32]The ID of the transaction.
Msg countuint8The number of MessageIDs.
MessageIDs Msg count
MessageID, ordered by hash ASC
NameTypeDescription
MessageIDByteArray[32]The ID of the message.

The respond to a QueryRequest must take follow the following form

#### QueryResponse

NameTypeDescription
Versionuint8The message version. The schema specified in this RFC is for version 1 only.
Obj countuint8The number of objectIDs. Must equal to  Tx count + Msg count of the received QueryRequest
Opinions Obj count
ObjectIDs, in the same order as in received QueryRequest
NameTypeDescription
OpinionByte The opinion on objectID.

In the case that a certain objectID is unknwon to the node, it must respond with NULL. If the objectID is currently treated by FPC a node must respond with the current opinion, otherwise, retrieve the opinion from the opinionField of the objectID. If the opinionField is NULL the node must respond with NULL.

Note that since the order of the opinions in QueryResponse is important a node may have to produce different messages QueryResponse for different querying nodes. QueryRequest and QueryResponse must be signed by the sending node. The communication channel for these messages is specified in the field "services" in the DiscoveryRequest. We write respond[node, objectIDs] for the respond, per direct QueryResponse or FPC statement, of the node on the objectIDs. A node shall only respond to queries if all included objectsID have status answerQuery=TRUE.

#### GetOpinion

This function sends queries to all nodes of queryList and obtains their opinion either through QueryResponse messages of from FPC statements on the Tangle.

FUNCTION opinionQuery = GetOpinion(objectIDs, queryList)SEND QueryRequest messagesWAIT UNTIL TIME_OUT  FOR (node IN queryList)    IF (node replied)         opinionQuery[node, objectIDs] = respond[node, objectIDs]    ELSE IF (node publishes on Tangle)        opinionQuery[node, objectIDs] = GetOpinionFromTangle[node, objectID]    ELSE opinionQuery[node, objectIDs] = NARETURN (opinionQuery)

A node must receive a sufficient amount of replies to validate a given round; otherwise the ongoing FPC round is skipped. More precisely, the consensus Mana of the received opinions must be larger than MIN_MANA_PROPORTION times the sum of the consensus Mana of the sampled nodes.

Once a node receives a FPC query it must prepare a response message as specified above, and shall sent the response as soon as possible. If a node decides to publish its opinions on the Tangle, it must create a so-called FPC statement message and issue it on the Tangle at the beginning of each round.

At the end of each round, i.e., after the TIME_OUT expires, a node must update its opinion if CheckQuerySuccessful is TRUE. The node must calculate the proportion of the LIKEs in the obtained opinions and compare it to a certain threshold. The value of the threshold depends on the phase FPC is in. In the first round, a node must use FIRST_ROUND_THRESHOLD as a threshold, while in the subsequent rounds the node must use a random threshold obtained by GetRN(). Moreover, if cnt > TOTAL_ROUNDS_FINALIZATION-TOTAL_ROUNDS_ENDING_THRESHOLD the node must use ENDING_THRESHOLD for the threshold. The following pseudo-code describes the update of the opinion of one given objectID. It uses queryList and opinionQuery obtained by the functions GetSample and GetOpinion. The variable opinion describes the current opinion on objectID.

#### OpinionUpdate

This function updates the opinion of a node on objectID.

FUNCTION opinion = OpinionUpdate(objectID, opinion, queryList, opinionQuery, round)manaList = GetActiveConsensusMana(timeObjectID)answerMana = ownManaqueriedManaetaStar = 0FOR node in queryList    queriedMana += manaList[node]    IF opinionQuery[node, objectID] != NA        answerMana += manaList[node]    IF  opinionQuery[node, objectID] = LIKE            etaStar++IF answerMana <= MIN_MANA_PROPORTION * queriedMana    opinionNew = opinion # ELSE     eta = etaStar / LENGTH(queryList)    IF opinion = LIKE        eta = (ownMana + etaStar*(answerMana-ownMana))/answerMana        ELSE IF opinion = DISLIKE         eta = (etaStar*(answerMana-ownMana))/answerMana     WAIT UNTIL CurrentTime/ROUND_LENGTH IS INTEGER    IF round = 1          threshold = FIRST_ROUND_THRESHOLD    ELSE IF round <= TOTAL_ROUNDS_FINALIZATION-TOTAL_ROUNDS_ENDING_THRESHOLD         WAIT DRNG_WAITING_TIME        threshold = getRN(SUBSEQUENT_LOWER_THRESHOLD, SUBSEQUENT_UPPER_THRESHOLD, CurrentTime)          IF threshold = NIL            threshold = (SUBSEQUENT_LOWER_THRESHOLD + SUBSEQUENT_UPPER_THRESHOLD)/2    ELSE         threshold = ENDING_THRESHOLD    IF eta < threshold        opinionNew = DISLIKE    ELSE        opinionNew = LIKERETURN opinioNew

#### MainFPC

This function describes the FPC for one objectID. It is triggered once queryStatus of an object is set to TRUE.

FUNCTION opinion = MainFPC(objectId, queryStatus)IF queryStatus = TRUE      opinion = GetInitialOpinion(objectID)    cnt = 0}WAIT UNTIL CurrentTime/ROUND_LENGTH IS INTEGERround = 1 WHILE queryStatus = TRUE    queryList = GetSample(nodeList, manaList)    opinionQuery = GetOpinion(objectID, queryList)     opinionNew = OpinionUpdate(objectID, opinion, queryList, opinionQuery, round)    IF opinion = opinionNew         cnt++    ELSE         cnt=1    opinion = opinionNew    round++    IF cnt = TOTAL_ROUNDS_FINALIZATION         queryStatus = FALSE    IF round >= MAX_ROUND         queryStatus = FALSE        opinion = 1RETURN opinion

# 6.3.3 Optimizations Suggestions

The design given above exemplifies all the mandatory elements of the FPC protocol.

There are several possible ways to optimize the performance of FPC. For example:

• FPC possibly exposes the public IP adresses of all the IOTA nodes. This can be limited by publishing statements on Tangle. However, if all nodes decide to publish their statements this may have a negative effect on the scalability of the protocol. Please refer to Fast Probabilistic Consensus with Weighted Votes for more discussions on how to optimize the message overhead.
• Monotonicity and consistency rules may be applied to reduce the sizes of the messages.
• The choice of the FPC parameters could be optimized. The optimal choice depends on the actual mana distribution and network latency.


text:
vroom_lines() is similar to readLines(), however it reads the lines lazily like vroom(), so operations like length(), head(), tail() and sample() can be done much more efficiently without reading all the data into R.

vroom_lines(file, n_max = Inf, skip = 0, altrep_opts = "chr",
num_threads = vroom_threads(), progress = vroom_progress())

## Arguments

file path to a local file. Maximum number of records to read. Number of lines to skip before reading data. Control which column types use Altrep representations, either a character vector of types, TRUE or FALSE. See vroom_altrep_opts() for for full details. Number of threads to use when reading and materializing vectors. Display a progress bar? By default it will only display in an interactive session and not while knitting a document. The display is updated every 50,000 values and will only display if estimated reading time is 5 seconds or more. The automatic progress bar can be disabled by setting option readr.show_progress to FALSE.

## Examples

lines <- vroom_lines(vroom_example("mtcars.csv"))

length(lines)#> [1] 33head(lines, n = 2)#> [1] "model,mpg,cyl,disp,hp,drat,wt,qsec,vs,am,gear,carb"
#> [2] "Mazda RX4,21,6,160,110,3.9,2.62,16.46,0,1,4,4"     tail(lines, n = 2)#> [1] "Maserati Bora,15,8,301,335,3.54,3.57,14.6,0,1,5,8"
#> [2] "Volvo 142E,21.4,4,121,109,4.11,2.78,18.6,1,1,4,2" sample(lines, size = 2)#> [1] "Merc 230,22.8,4,140.8,95,3.92,3.15,22.9,1,0,4,2"
#> [2] "Maserati Bora,15,8,301,335,3.54,3.57,14.6,0,1,5,8"


text:
anonymous one year ago algebra help: will medal and fan!!!! :) Given the function f(x) = The quantity of 4x minus 2, divided by 3, which of the below expressions is correct? f−1(x) = The quantity of 2 minus 4x, divided by 3. f−1(x) = The quantity of 3x minus 2, divided by 4. f−1(x) = The quantity of negative 4x minus 2, divided by 3. f−1(x) = The quantity of 3x plus 2, divided by 4.

1. anonymous

help

2. anonymous

let $y=f(x)=\frac{ 4x-2 }{ 3 },$ 4x=3y+2 $x=\frac{ 3y+2 }{ 4 }$ interchange x & y $y=\frac{ 3x+2 }{ 4 },$ ?

3. anonymous

thanks


text:
## Encyclopedia > Black cow

Article Content

# Black cow

A black cow (also known as a brown cow or a root beer float) is a non-alcoholic cocktail made of

Put the scoops of vanilla ice in a high ball glass, pour the root beer over it. Serve with a straw.

 Golden ratio ... phi, is the number $\phi = \frac{1 + \sqrt{5}}{2} \approx 1.6180339887...$ the unique positive real number with [itex]\phi^2 = ...


text:
# Is $\sum\limits_{n=1} ^\infty |x_n||y_n| ≤\left(\sum\limits_{n=1} ^\infty |x_n|\right)\left(\sum\limits_{n=1} ^\infty |y_n|\right)$?

Is $\sum\limits_{n=1} ^\infty |x_n||y_n| ≤ \left(\sum\limits_{n=1} ^\infty |x_n|\right)\left(\sum\limits_{n=1} ^\infty |y_n|\right)$?

I want to use this in a separate exercise but I can't think if this is true let alone how to prove it and I can't find any resources to help.

• Without even thinking about it I'd explore the Cauchy-Schwarz inequality. – Git Gud Nov 10 '14 at 14:54
• Yep. Cauchy-Schwarz plus $\left(\sum |x_i|\right)^2 \left(\sum |y_i|\right)^2 \geq \sum x_i^2 \sum y_i^2$ and you are there. – Winther Nov 10 '14 at 15:05

Apply Cauchy-Schwarz

$$(x\cdot y)^2 \leq ||x ||^2~||y||^2$$

to the vectors $x =(|x_1|,|x_2|,\ldots,|x_n|)$ and $y = (|y_1|,|y_2|,\ldots,|y_n|)$ to get

$$\left(\sum x_i y_i\right)^2 \leq \sum x_i^2\sum y_i^2$$

Now use that

$$\sum x_i^2 \sum y_i^2 \leq \left(\sum |x_i|\right)^2 \left(\sum |y_i|\right)^2$$

since $\left(\sum |x_i|\right)^2 = \sum x_i^2 + \sum_{i\not=j} |x_ix_j| \geq \sum x_i^2$ and the result follows.

$$\sum\limits_{n=1} ^\infty |x_n||y_n|=|x_1||y_1|+\sum_{n=2}^\infty|x_n||y_n|\le |x_1|\sum_{n=1}^\infty|y_n|+\sum_{n=2}^\infty |x_n||y_n|\le$$

$$|x_1|\sum_{n=1}^\infty|y_n|+|x_2||y_2|+\sum_{n=3}^\infty |x_n||y_n|\le |x_1|\sum_{n=1}^\infty|y_n|+|x_2|\sum_{n=1}^\infty|y_n|+\sum_{n=3}^\infty |x_n||y_n|\le$$

$$\vdots$$

$$|x_1|\sum_{n=1}^\infty|y_n|+|x_2|\sum_{n=1}^\infty|y_n|+|x_3|\sum_{n=1}^\infty|y_n|+\cdots =$$

$$\sum\limits_{n=1} ^\infty |x_n|\sum\limits_{n=1} ^\infty |y_n|$$

If $x_i=0$ for all $i$, it is easy. So assume not.
Let $M = \max\{|y_1|,|y_2|,\dots\}$. If $M=\infty$ it is easy, so assume not.
Certainly $\sum_{i=1}^\infty |y_i| \ge M$.
For all $i$, we have $|x_i y_i| \le |x_i|\; M$, so $$\sum_{i=1}^\infty |x_i y_i| \le \sum_{i=1}^\infty \big(|x_i|M\big) = \left(\sum_{i=1}^\infty |x_i|\right)M \le \sum_{i=1}^\infty |x_i| \sum_{i=1}^\infty |y_i|$$
• Why $M=\infty$? Well, if, for example, $y_i=i$ for all $i$, then $M=\infty$. Why do the $x_i=0$ case first? Maybe not needed, but let's avoid having to think about $0 \cdot \infty = ?$. – GEdgar Nov 12 '14 at 1:44


text:
# On a certain set of probability measures on a shift

Denote by $\mathbb{Z}_2=\{0,1\}$ the integers modulo 2.

Let $S:\mathbb{Z}_{2}^{\mathbb{N}}\times\mathbb{Z}_{2}^{\mathbb{N}} \rightarrow \mathbb{Z}_{2}^{\mathbb{N}}$ be the sum $S(a,b) = a+b$, where $a,b$ are given sequences in $\mathbb{Z}_{2}^{\mathbb{N}}$. Consider the set $M(\sigma \times \sigma)$ of $\sigma \times \sigma$ invariant probability measures on $\mathbb{Z}_{2}^{\mathbb{N}}\times\mathbb{Z}_{2}^{\mathbb{N}}$. Here $\sigma$ is the full-shift.

Question: Given $\eta, \mu$ $\sigma$-invariant measures, how big is the set $$A_{\eta,\mu}=\{ \rho \in M(\sigma \times \sigma): \rho \circ S^{-1} = \eta*\mu\}?$$

Obs: I know that $A_{\eta,\mu}$ is non-empty, because $(\eta \times \mu) \in A_{\eta,\mu}$. The $*$ is the convolution of two measures and can be defined on this way:

$$\eta*\mu:=(\eta \times \mu) \circ S^{-1}.$$

I would like to understand the set $A_{\eta,\mu}$ deeply and, for instance, to see some examples of elements of it.

• Have you tried to look what happens for some examples? What happens if one of the measures have zero entropy? What happens if the measures are disjoint? (this does not happen if both of them have positive entropy). It is for example very easy to conclude some lower bound over the dimension of $\rho$ by simple entropy estimate. Maybe you can get something from the recent work of Mike Hochman. – Asaf Nov 16 '15 at 19:39
• Do you really mean $M(\sigma\times\sigma)$? or do you mean $M(\sigma)\times M(\sigma)$? – Anthony Quas Nov 16 '15 at 23:07
• Then I think the set is huge. Given any probability measure $\nu\in M(\sigma)$, define a map $\Phi$: $X\times X$, $(x,y)\mapsto (x,S(x,y))$. Now $\Phi^*(\nu,\eta*\mu)\in A_{\eta,\mu}$ for each $\nu$. – Anthony Quas Nov 16 '15 at 23:14
• If you meant the subset of $M(\sigma)\times M(\sigma)$, then I think if you take $\mu$ to be a Sturmian shift and $\eta$ to be the $\delta$-measure supported on the 0 fixed point, I think $A_{\eta,\mu}$ is extremely small. – Anthony Quas Nov 17 '15 at 2:32
• Mike proved an inverse theorem for entropy in his Annals paper - "On self-similar sets with overlaps and inverse theorems for entropy", showing what happens if the entropy of the convolution does not grow. As you're interested in given convolution, if the entropy of the convolution isn't large compered with the original entropy of $\mu$, it will limit, in a severe manner, your set (well not exactly, it will limit the joinings of $\mu$ and $\nu$). But it can be a place to start, as I've hinted in the first comment. – Asaf Nov 17 '15 at 21:28


text:
### Artificial IntelligenceAIMA Exercises

Write a successor-state axiom for the ${Locked}$ predicate, which applies to doors, assuming the only actions available are ${Lock}$ and ${Unlock}$.

Write a successor-state axiom for the ${Locked}$ predicate, which applies to doors, assuming the only actions available are ${Lock}$ and ${Unlock}$.

Submit Solution


text:
Acceleration problem

Statement;
The tangential acceleration and normal acceleration is given as follows:
$$\vec{a_{T}} = \frac{4cos(2t)sin(2t)}{1 + 2sin^{2}(2t)} \cdot (\hat{x} - sin(2t)\hat{y} + sin(2t)\hat{z}), \vec{a_{N}} = - \frac{2cos(2t)}{1 + 2sin^{2}(2t)} \cdot (2sin(2t)\hat{x} + \hat{y} - \hat{z})$$ (#0)

Find the magnitude of the tangential and normal acceleration components (given above).

Solution/Attempt:
The solution for the tangential and normal magnitudes are given by the following,
$$\vec{a_{T}} = \frac{4cos(2t)sin(2t) }{\sqrt{1 + 2sin^{2}(2t)}}, \vec{a_{N}} = \frac{2\sqrt{2}cos(2t)}{\sqrt{1 + 2sin^{2}(2t)} }$$ (#1)

I mean if we had to solve for the magnitude of some vector $$a = 3\hat{x} - 33\hat{z},$$ then we say,
$$|a| = \sqrt{3^{2} + (-33)^{2}}$$. How would we apply this to our equation (#0) to get equation (#1)?

Thanks,

JL

Last edited:

Related Introductory Physics Homework Help News on Phys.org
rock.freak667
Homework Helper

$$|a\hat{x} + b\hat{y} + c\hat{z}| = \sqrt{a^2+b^x+c^2}$$

that's all there is to it.


text:
# Prove that arctanx has to be less than x

Tags:
1. Mar 31, 2015

1. The problem statement, all variables and given/known data

Prove that :
$$\lim_{x\to 0}\frac{tan^{-1}x}{x}<1$$
And
$$\lim_{x\to 0}\frac{sin^{-1}x}{x}>1$$
2. Relevant equations

None
3. The attempt at a solution

I have to prove that arctanx has to be lesser than x.
It's derivative is 1 at x=0 and keeps decreasing as x increases. So it's slope decreases from 1.
Also, at x=0, arctanx=x. So the curve which initially touched y=x goes below it. So arctanx<x.
Is this correct?

2. Mar 31, 2015

### PeroK

Why not think about tan(x) and sin(x)?

3. Mar 31, 2015

I know that sinx<x.
So x<arcsinx..... By taking arcsin on LHs and RHS.
Thank you.
Is my method in post 0 correct?

4. Mar 31, 2015

### PeroK

It might be neater to consider the functions

f(x) = x - arcsin(x)

g(x) = x - arctan(x)

Shouldn't the limits be $\le$ and $\ge$?

Why not use L'hopital to calculate them?

5. Mar 31, 2015

### vela

Staff Emeritus
You can't prove these because they're false. The limit is a single number, i.e.,
$$\lim_{x \to 0}\frac{\tan^{-1}x}{x} = 1.$$

6. Mar 31, 2015

### Ray Vickson

The statements you are trying to show are both false. If you think $\lim_{x \to 0} \arctan(x)/x < 1$, how much < 1 do you think it is? Is it < 0.9? Is it < 0.99?

Also: arctan(x) is not < x; in fact:
$$\arctan(x) \begin{cases} < x,& x > 0\\ > x, & x < 0 \end{cases}$$
Just plot the two functions to see that this is true.

You should likewise check $\sin(x)$ against $x$ for small $x$ of both signs, + and -.

7. Mar 31, 2015

sorry... its x->0+

8. Mar 31, 2015

so you can prove it for x->0+ by using LH rule directly and substituting 0+.

9. Apr 1, 2015

### SammyS

Staff Emeritus
Look at Post #5 from vela .

The limits you are referring to are all exactly equal to zero one .

(Thanks to RGV !)

Last edited: Apr 1, 2015
10. Apr 1, 2015

### Ray Vickson

Actually, exactly equal to one.

11. Apr 1, 2015

No. its x->0+. Using graph, the arctanx curve starts coming below y=x.

12. Apr 1, 2015

### SammyS

Staff Emeritus
Do you understand limits ?

13. Apr 1, 2015

yes. its a homework question to prove that $$\lim_{x\to 0}\sqrt{\frac{tan^{-1}x}{x}-\frac{sin^{-1}x}{x}}$$ does not exist and its given in the solution that $\frac{tan^{-1}x}{x}<1$ while $\frac{sin^{-1}x}{x}>1$, when x->0.
If it is wrong, even then $\frac{tan^{-1}x}{x}<\frac{sin^{-1}x}{x}$ when x->0 (actually 0+).

14. Apr 1, 2015

### SammyS

Staff Emeritus
That's quite different from the original post.

For 0 < x ≤ 1 , $\displaystyle\ \frac{\tan^{-1}x}{x}-\frac{\sin^{-1}x}{x} <0$

Taking the square root gives an imaginary result.


text:
# What is the Difference Between Driving Force and Net Force? [closed]

I'm a bit confused with the question below. Particularly, I'm confused what the difference between driving force and net force is.

For driving force, I easily find $$F_D = ma = 1000 \times 0.3 = 300$$ Problem is, I'm inclined to find $$F_{net}$$ as, $$F_{net} = F_{res} - F_D = 200N$$

The correct answer, however, is $$300N$$.

• IMO this is a legitimate question since it is asking to understand terminology. – BioPhysicist Apr 24 at 14:11

## 1 Answer

"Net force" is the sum of all forces acting on an object.

"Driving force" is just one specific force acting on the car in this problem.


text:
# SipHash's (non-)collision resistance

I'm very new to cryptography, and I need help understanding the security claims made in Aumasson and Bernstein's paper on the SipHash. In particular, I'm trying to understand the following statement at the end of section 3 (p. 6):

We comment that SipHash is not meant to be, and (obviously) is not, collision-resistant.

I don't understand this statement as well as I wish. First, I do understand that collision resistance is, strictly speaking, a property of hash functions and that SipHash in spite of the name is a MAC, not a cryptographic hash, so its security requirements are different. The paper explicitly sets the security requirement out elsewhere:

A message-authentication code (MAC) produces a tag t from a message m and a secret key k. The security goal for a MAC is for an attacker, even after seeing tags for many messages (perhaps selected by the attacker), to be unable to guess tags for any other messages. [p. 1]

Where I'm confused now is: if collision resistance is strictly speaking a property of hash functions and not of MACs, what does it mean to say that "SipHash is not meant to be, and (obviously) is not, collision-resistant"? How do I evaluate the truth value of this statement? So far my best attempt at understanding this comes from the following statements in Menezes, Oorschot and Vanstone's Handbook of Applied Cryptography (Chapter 9):

9.8 Remark (MAC resistance when key known) Definition 9.7 does not dictate whether MACs need be preimage- and collision resistant for parties knowing the key k (as Fact 9.21 implies for parties without k). [p. 325]

9.21 Fact (implications of MAC properties) Let hk be a keyed hash function which is a MAC algorithm per Definition 9.7 (and thus has the property of computation-resistance). Then hk is, against chosen-text attack by an adversary without knowledge of the key k, (i) both 2nd-preimage resistant and collision resistant; and (ii) preimage resistant (with respect to the hash-input). [p. 330]

So, is Aumasson and Bernstein's statement that SipHash is "obviously" not collision-resistant implicitly assuming an adversary that knows the key? Because it seems clear to me that it needs to be collision-resistant to an attacker that doesn't know it—otherwise their proposal to use it as a defense against hash-flooding attacks doesn't work, right?

EDIT: My question has been flagged as a potential duplicate of "Is SipHash cryptographically secure?" I think my question differs from that one in that I'm trying to relate the security claims made in Aumasson and Bernstein's SipHash paper to the quotes that I cited in the Handbook of Applied Cryptography, but I'm uncertain whether I'm understanding these sources correctly.

So for example, here's another passage from Aumasson and Bernstein (p.2, my boldface):

In general, building a MAC from a general purpose cryptographic hash function appears to be a highly suboptimal approach: general-purpose cryptographic hash functions perform many extra computations for the goal of collision resistance on public inputs, while MACs have secret keys and do not need collision resistance.

I do not understand, for example, whether their later statement that "SipHash is not meant to be, and (obviously) is not, collision-resistant" refers to:

1. The fact that the small output space allows an attacker to find two inputs that hash the same in 232 attempts, which is a very low number (as Demetri's answer remarks and poncho's comment to it reiterates);
2. Their earlier statement that collision-resistance is not one of the properties required of a MAC;
3. The Handbook of Applied Cryptography's passages that I consulted that say that:
• A MAC need not be collision resistant to an attacker that knows the key;
• A MAC must be computation-resistant ("given zero or more text-MAC pairs (xi, hk(xi)), it is computationally infeasible to compute any text-MAC pair (x, hk(x)) for any new input x ≠ xi (including possibly for hk(x) = hk(xi) for some i)");
• Computation-resistance implies collision resistance "against chosen-text attack by an adversary without knowledge of the key" (Fact 9.21 that I cited).

From the last of these points it would follow that if SipHash is not collision resistant to such an adversary, then it cannot be computation-resistant either. But Aumasson and Bernstein say that "SipHash-c-d with c ≥ 2 and d ≥ 4 is expected to provide the maximum PRF security possible (and therefore also the maximum MAC security possible) for any function with the same key size and output size," which to me reads like they are conjecturing that it is computation-resistant.

At this point however I'm starting to worry that perhaps my question risks turning into a conversation, something that doesn't fit the question/answer format of this site. So I won't object if somebody chooses to close this question.

It is not collision-resistant because the tag is only 64 bits, so on average only $2^{32}$ inputs are needed to find a collision. This is the classic birthday bound. This is completely insecure, so SipHash is not collision resistant.
This is not a problem for hash tables because hash tables have collision-resolution mechanisms and because the odds of a $n$-way collision fall exponentially with $n$, and because the collisions depend on the (secret) key.
• @LuisCasillas: doesn't matter; it means that an attacker can find a collision in a reasonable amount of time ($2^{32}$ oracle requires if he doesn't know the key; $2^{32}$ hash evaluations if he does). If you can find a collision, you're not collision-resistant (just as a cipher with an 8 bit key is insecure, even if there was no attack better than brute force) – poncho May 5 '16 at 18:08


text:
Attention has been a fairly popular concept and a useful tool in the deep learning community in recent years. In this post, we are gonna look into how attention was invented, and various attention mechanisms and models, such as transformer and SNAIL.

[Updated on 2018-10-28: Add Pointer Network and the link to my implementation of Transformer.]
[Updated on 2018-11-06: Add a link to the implementation of Transformer model.]
[Updated on 2018-11-18: Add Neural Turing Machines.]
[Updated on 2019-07-18: Correct the mistake on using the term “self-attention” when introducing the show-attention-tell paper; moved it to Self-Attention section.]
[Updated on 2020-04-07: A follow-up post on improved Transformer models is here.]

Attention is, to some extent, motivated by how we pay visual attention to different regions of an image or correlate words in one sentence. Take the picture of a Shiba Inu in Fig. 1 as an example.

Fig. 1. A Shiba Inu in a men’s outfit. The credit of the original photo goes to Instagram @mensweardog.

Human visual attention allows us to focus on a certain region with “high resolution” (i.e. look at the pointy ear in the yellow box) while perceiving the surrounding image in “low resolution” (i.e. now how about the snowy background and the outfit?), and then adjust the focal point or do the inference accordingly. Given a small patch of an image, pixels in the rest provide clues what should be displayed there. We expect to see a pointy ear in the yellow box because we have seen a dog’s nose, another pointy ear on the right, and Shiba’s mystery eyes (stuff in the red boxes). However, the sweater and blanket at the bottom would not be as helpful as those doggy features.

Similarly, we can explain the relationship between words in one sentence or close context. When we see “eating”, we expect to encounter a food word very soon. The color term describes the food, but probably not so much with “eating” directly.

Fig. 2. One word “attends” to other words in the same sentence differently.

In a nutshell, attention in deep learning can be broadly interpreted as a vector of importance weights: in order to predict or infer one element, such as a pixel in an image or a word in a sentence, we estimate using the attention vector how strongly it is correlated with (or “attends to” as you may have read in many papers) other elements and take the sum of their values weighted by the attention vector as the approximation of the target.

What’s Wrong with Seq2Seq Model?

The seq2seq model was born in the field of language modeling (Sutskever, et al. 2014). Broadly speaking, it aims to transform an input sequence (source) to a new one (target) and both sequences can be of arbitrary lengths. Examples of transformation tasks include machine translation between multiple languages in either text or audio, question-answer dialog generation, or even parsing sentences into grammar trees.

The seq2seq model normally has an encoder-decoder architecture, composed of:

• An encoder processes the input sequence and compresses the information into a context vector (also known as sentence embedding or “thought” vector) of a fixed length. This representation is expected to be a good summary of the meaning of the whole source sequence.
• A decoder is initialized with the context vector to emit the transformed output. The early work only used the last state of the encoder network as the decoder initial state.

Both the encoder and decoder are recurrent neural networks, i.e. using LSTM or GRU units.

Fig. 3. The encoder-decoder model, translating the sentence “she is eating a green apple” to Chinese. The visualization of both encoder and decoder is unrolled in time.

A critical and apparent disadvantage of this fixed-length context vector design is incapability of remembering long sentences. Often it has forgotten the first part once it completes processing the whole input. The attention mechanism was born (Bahdanau et al., 2015) to resolve this problem.

Born for Translation

The attention mechanism was born to help memorize long source sentences in neural machine translation (NMT). Rather than building a single context vector out of the encoder’s last hidden state, the secret sauce invented by attention is to create shortcuts between the context vector and the entire source input. The weights of these shortcut connections are customizable for each output element.

While the context vector has access to the entire input sequence, we don’t need to worry about forgetting. The alignment between the source and target is learned and controlled by the context vector. Essentially the context vector consumes three pieces of information:

• encoder hidden states;
• decoder hidden states;
• alignment between source and target.

Fig. 4. The encoder-decoder model with additive attention mechanism in Bahdanau et al., 2015.

Definition

Now let’s define the attention mechanism introduced in NMT in a scientific way. Say, we have a source sequence $$\mathbf{x}$$ of length $$n$$ and try to output a target sequence $$\mathbf{y}$$ of length $$m$$:

\begin{aligned} \mathbf{x} &= [x_1, x_2, \dots, x_n] \\ \mathbf{y} &= [y_1, y_2, \dots, y_m] \end{aligned}

(Variables in bold indicate that they are vectors; same for everything else in this post.)

The encoder is a bidirectional RNN (or other recurrent network setting of your choice) with a forward hidden state $$\overrightarrow{\boldsymbol{h}}_i$$ and a backward one $$\overleftarrow{\boldsymbol{h}}_i$$. A simple concatenation of two represents the encoder state. The motivation is to include both the preceding and following words in the annotation of one word.

$\boldsymbol{h}_i = [\overrightarrow{\boldsymbol{h}}_i^\top; \overleftarrow{\boldsymbol{h}}_i^\top]^\top, i=1,\dots,n$

The decoder network has hidden state $$\boldsymbol{s}_t=f(\boldsymbol{s}_{t-1}, y_{t-1}, \mathbf{c}_t)$$ for the output word at position t, $$t=1,\dots,m$$, where the context vector $$\mathbf{c}_t$$ is a sum of hidden states of the input sequence, weighted by alignment scores:

\begin{aligned} \mathbf{c}_t &= \sum_{i=1}^n \alpha_{t,i} \boldsymbol{h}_i & \small{\text{; Context vector for output }y_t}\\ \alpha_{t,i} &= \text{align}(y_t, x_i) & \small{\text{; How well two words }y_t\text{ and }x_i\text{ are aligned.}}\\ &= \frac{\exp(\text{score}(\boldsymbol{s}_{t-1}, \boldsymbol{h}_i))}{\sum_{i'=1}^n \exp(\text{score}(\boldsymbol{s}_{t-1}, \boldsymbol{h}_{i'}))} & \small{\text{; Softmax of some predefined alignment score.}}. \end{aligned}

The alignment model assigns a score $$\alpha_{t,i}$$ to the pair of input at position i and output at position t, $$(y_t, x_i)$$, based on how well they match. The set of $$\{\alpha_{t, i}\}$$ are weights defining how much of each source hidden state should be considered for each output. In Bahdanau’s paper, the alignment score $$\alpha$$ is parametrized by a feed-forward network with a single hidden layer and this network is jointly trained with other parts of the model. The score function is therefore in the following form, given that tanh is used as the non-linear activation function:

$\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \mathbf{v}_a^\top \tanh(\mathbf{W}_a[\boldsymbol{s}_t; \boldsymbol{h}_i])$

where both $$\mathbf{v}_a$$ and $$\mathbf{W}_a$$ are weight matrices to be learned in the alignment model.

The matrix of alignment scores is a nice byproduct to explicitly show the correlation between source and target words.

Fig. 5. Alignment matrix of “L’accord sur l’Espace économique européen a été signé en août 1992” (French) and its English translation “The agreement on the European Economic Area was signed in August 1992”. (Image source: Fig 3 in Bahdanau et al., 2015)

Check out this nice tutorial by Tensorflow team for more implementation instructions.

A Family of Attention Mechanisms

With the help of the attention, the dependencies between source and target sequences are not restricted by the in-between distance anymore! Given the big improvement by attention in machine translation, it soon got extended into the computer vision field (Xu et al. 2015) and people started exploring various other forms of attention mechanisms (Luong, et al., 2015; Britz et al., 2017; Vaswani, et al., 2017).

Summary

Below is a summary table of several popular attention mechanisms and corresponding alignment score functions:

Name Alignment score function Citation
Content-base attention $$\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \text{cosine}[\boldsymbol{s}_t, \boldsymbol{h}_i]$$ Graves2014
Additive(*) $$\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \mathbf{v}_a^\top \tanh(\mathbf{W}_a[\boldsymbol{s}_t; \boldsymbol{h}_i])$$ Bahdanau2015
Location-Base $$\alpha_{t,i} = \text{softmax}(\mathbf{W}_a \boldsymbol{s}_t)$$
Note: This simplifies the softmax alignment to only depend on the target position.
Luong2015
General $$\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \boldsymbol{s}_t^\top\mathbf{W}_a\boldsymbol{h}_i$$
where $$\mathbf{W}_a$$ is a trainable weight matrix in the attention layer.
Luong2015
Dot-Product $$\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \boldsymbol{s}_t^\top\boldsymbol{h}_i$$ Luong2015
Scaled Dot-Product(^) $$\text{score}(\boldsymbol{s}_t, \boldsymbol{h}_i) = \frac{\boldsymbol{s}_t^\top\boldsymbol{h}_i}{\sqrt{n}}$$
Note: very similar to the dot-product attention except for a scaling factor; where n is the dimension of the source hidden state.
Vaswani2017

(*) Referred to as “concat” in Luong, et al., 2015 and as “additive attention” in Vaswani, et al., 2017.
(^) It adds a scaling factor $$1/\sqrt{n}$$, motivated by the concern when the input is large, the softmax function may have an extremely small gradient, hard for efficient learning.

Here are a summary of broader categories of attention mechanisms:

Name Definition Citation
Self-Attention(&) Relating different positions of the same input sequence. Theoretically the self-attention can adopt any score functions above, but just replace the target sequence with the same input sequence. Cheng2016
Global/Soft Attending to the entire input state space. Xu2015
Local/Hard Attending to the part of input state space; i.e. a patch of the input image. Xu2015; Luong2015

(&) Also, referred to as “intra-attention” in Cheng et al., 2016 and some other papers.

Self-Attention

Self-attention, also known as intra-attention, is an attention mechanism relating different positions of a single sequence in order to compute a representation of the same sequence. It has been shown to be very useful in machine reading, abstractive summarization, or image description generation.

The long short-term memory network paper used self-attention to do machine reading. In the example below, the self-attention mechanism enables us to learn the correlation between the current words and the previous part of the sentence.

Fig. 6. The current word is in red and the size of the blue shade indicates the activation level. (Image source: Cheng et al., 2016)

Soft vs Hard Attention

In the show, attend and tell paper, attention mechanism is applied to images to generate captions. The image is first encoded by a CNN to extract features. Then a LSTM decoder consumes the convolution features to produce descriptive words one by one, where the weights are learned through attention. The visualization of the attention weights clearly demonstrates which regions of the image the model is paying attention to so as to output a certain word.

Fig. 7. “A woman is throwing a frisbee in a park.” (Image source: Fig. 6(b) in Xu et al. 2015)

This paper first proposed the distinction between “soft” vs “hard” attention, based on whether the attention has access to the entire image or only a patch:

• Soft Attention: the alignment weights are learned and placed “softly” over all patches in the source image; essentially the same type of attention as in Bahdanau et al., 2015.
• Pro: the model is smooth and differentiable.
• Con: expensive when the source input is large.
• Hard Attention: only selects one patch of the image to attend to at a time.
• Pro: less calculation at the inference time.
• Con: the model is non-differentiable and requires more complicated techniques such as variance reduction or reinforcement learning to train. (Luong, et al., 2015)

Global vs Local Attention

Luong, et al., 2015 proposed the “global” and “local” attention. The global attention is similar to the soft attention, while the local one is an interesting blend between hard and soft, an improvement over the hard attention to make it differentiable: the model first predicts a single aligned position for the current target word and a window centered around the source position is then used to compute a context vector.

Fig. 8. Global vs local attention (Image source: Fig 2 & 3 in Luong, et al., 2015)

Neural Turing Machines

Alan Turing in 1936 proposed a minimalistic model of computation. It is composed of a infinitely long tape and a head to interact with the tape. The tape has countless cells on it, each filled with a symbol: 0, 1 or blank (“ “). The operation head can read symbols, edit symbols and move left/right on the tape. Theoretically a Turing machine can simulate any computer algorithm, irrespective of how complex or expensive the procedure might be. The infinite memory gives a Turing machine an edge to be mathematically limitless. However, infinite memory is not feasible in real modern computers and then we only consider Turing machine as a mathematical model of computation.

Fig. 9. How a Turing machine looks like: a tape + a head that handles the tape. (Image source: http://aturingmachine.com/)

Neural Turing Machine (NTM, Graves, Wayne & Danihelka, 2014) is a model architecture for coupling a neural network with external memory storage. The memory mimics the Turing machine tape and the neural network controls the operation heads to read from or write to the tape. However, the memory in NTM is finite, and thus it probably looks more like a “Neural von Neumann Machine”.

NTM contains two major components, a controller neural network and a memory bank. Controller: is in charge of executing operations on the memory. It can be any type of neural network, feed-forward or recurrent. Memory: stores processed information. It is a matrix of size $$N \times M$$, containing N vector rows and each has $$M$$ dimensions.

In one update iteration, the controller processes the input and interacts with the memory bank accordingly to generate output. The interaction is handled by a set of parallel read and write heads. Both read and write operations are “blurry” by softly attending to all the memory addresses.

Fig 10. Neural Turing Machine Architecture.

When reading from the memory at time t, an attention vector of size $$N$$, $$\mathbf{w}_t$$ controls how much attention to assign to different memory locations (matrix rows). The read vector $$\mathbf{r}_t$$ is a sum weighted by attention intensity:

$\mathbf{r}_t = \sum_{i=1}^N w_t(i)\mathbf{M}_t(i)\text{, where }\sum_{i=1}^N w_t(i)=1, \forall i: 0 \leq w_t(i) \leq 1$

where $$w_t(i)$$ is the $$i$$-th element in $$\mathbf{w}_t$$ and $$\mathbf{M}_t(i)$$ is the $$i$$-th row vector in the memory.

When writing into the memory at time t, as inspired by the input and forget gates in LSTM, a write head first wipes off some old content according to an erase vector $$\mathbf{e}_t$$ and then adds new information by an add vector $$\mathbf{a}_t$$.

\begin{aligned} \tilde{\mathbf{M}}_t(i) &= \mathbf{M}_{t-1}(i) [\mathbf{1} - w_t(i)\mathbf{e}_t] &\scriptstyle{\text{; erase}}\\ \mathbf{M}_t(i) &= \tilde{\mathbf{M}}_t(i) + w_t(i) \mathbf{a}_t &\scriptstyle{\text{; add}} \end{aligned}

Attention Mechanisms

In Neural Turing Machine, how to generate the attention distribution $$\mathbf{w}_t$$ depends on the addressing mechanisms: NTM uses a mixture of content-based and location-based addressings.

The content-addressing creates attention vectors based on the similarity between the key vector $$\mathbf{k}_t$$ extracted by the controller from the input and memory rows. The content-based attention scores are computed as cosine similarity and then normalized by softmax. In addition, NTM adds a strength multiplier $$\beta_t$$ to amplify or attenuate the focus of the distribution.

$w_t^c(i) = \text{softmax}(\beta_t \cdot \text{cosine}[\mathbf{k}_t, \mathbf{M}_t(i)]) = \frac{\exp(\beta_t \frac{\mathbf{k}_t \cdot \mathbf{M}_t(i)}{\|\mathbf{k}_t\| \cdot \|\mathbf{M}_t(i)\|})}{\sum_{j=1}^N \exp(\beta_t \frac{\mathbf{k}_t \cdot \mathbf{M}_t(j)}{\|\mathbf{k}_t\| \cdot \|\mathbf{M}_t(j)\|})}$

Interpolation

Then an interpolation gate scalar $$g_t$$ is used to blend the newly generated content-based attention vector with the attention weights in the last time step:

$\mathbf{w}_t^g = g_t \mathbf{w}_t^c + (1 - g_t) \mathbf{w}_{t-1}$

The location-based addressing sums up the values at different positions in the attention vector, weighted by a weighting distribution over allowable integer shifts. It is equivalent to a 1-d convolution with a kernel $$\mathbf{s}_t(.)$$, a function of the position offset. There are multiple ways to define this distribution. See Fig. 11. for inspiration.

Fig. 11. Two ways to represent the shift weighting distribution $$\mathbf{s}_t$$.

Finally the attention distribution is enhanced by a sharpening scalar $$\gamma_t \geq 1$$.

\begin{aligned} \tilde{w}_t(i) &= \sum_{j=1}^N w_t^g(j) s_t(i-j) & \scriptstyle{\text{; circular convolution}}\\ w_t(i) &= \frac{\tilde{w}_t(i)^{\gamma_t}}{\sum_{j=1}^N \tilde{w}_t(j)^{\gamma_t}} & \scriptstyle{\text{; sharpen}} \end{aligned}

The complete process of generating the attention vector $$\mathbf{w}_t$$ at time step t is illustrated in Fig. 12. All the parameters produced by the controller are unique for each head. If there are multiple read and write heads in parallel, the controller would output multiple sets.

Fig. 12. Flow diagram of the addressing mechanisms in Neural Turing Machine. (Image source: Graves, Wayne & Danihelka, 2014)

Pointer Network

In problems like sorting or travelling salesman, both input and output are sequential data. Unfortunately, they cannot be easily solved by classic seq-2-seq or NMT models, given that the discrete categories of output elements are not determined in advance, but depends on the variable input size. The Pointer Net (Ptr-Net; Vinyals, et al. 2015) is proposed to resolve this type of problems: When the output elements correspond to positions in an input sequence. Rather than using attention to blend hidden units of an encoder into a context vector (See Fig. 8), the Pointer Net applies attention over the input elements to pick one as the output at each decoder step.

Fig. 13. The architecture of a Pointer Network model. (Image source: Vinyals, et al. 2015)

The Ptr-Net outputs a sequence of integer indices, $$\boldsymbol{c} = (c_1, \dots, c_m)$$ given a sequence of input vectors $$\boldsymbol{x} = (x_1, \dots, x_n)$$ and $$1 \leq c_i \leq n$$. The model still embraces an encoder-decoder framework. The encoder and decoder hidden states are denoted as $$(\boldsymbol{h}_1, \dots, \boldsymbol{h}_n)$$ and $$(\boldsymbol{s}_1, \dots, \boldsymbol{s}_m)$$, respectively. Note that $$\mathbf{s}_i$$ is the output gate after cell activation in the decoder. The Ptr-Net applies additive attention between states and then normalizes it by softmax to model the output conditional probability:

\begin{aligned} y_i &= p(c_i \vert c_1, \dots, c_{i-1}, \boldsymbol{x}) \\ &= \text{softmax}(\text{score}(\boldsymbol{s}_t; \boldsymbol{h}_i)) = \text{softmax}(\mathbf{v}_a^\top \tanh(\mathbf{W}_a[\boldsymbol{s}_t; \boldsymbol{h}_i])) \end{aligned}

The attention mechanism is simplified, as Ptr-Net does not blend the encoder states into the output with attention weights. In this way, the output only responds to the positions but not the input content.

Transformer

“Attention is All you Need” (Vaswani, et al., 2017), without a doubt, is one of the most impactful and interesting paper in 2017. It presented a lot of improvements to the soft attention and make it possible to do seq2seq modeling without recurrent network units. The proposed “transformer” model is entirely built on the self-attention mechanisms without using sequence-aligned recurrent architecture.

The secret recipe is carried in its model architecture.

Key, Value and Query

The major component in the transformer is the unit of multi-head self-attention mechanism. The transformer views the encoded representation of the input as a set of key-value pairs, $$(\mathbf{K}, \mathbf{V})$$, both of dimension $$n$$ (input sequence length); in the context of NMT, both the keys and values are the encoder hidden states. In the decoder, the previous output is compressed into a query ($$\mathbf{Q}$$ of dimension $$m$$) and the next output is produced by mapping this query and the set of keys and values.

The transformer adopts the scaled dot-product attention: the output is a weighted sum of the values, where the weight assigned to each value is determined by the dot-product of the query with all the keys:

$\text{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \text{softmax}(\frac{\mathbf{Q}\mathbf{K}^\top}{\sqrt{n}})\mathbf{V}$

Fig. 14. Multi-head scaled dot-product attention mechanism. (Image source: Fig 2 in Vaswani, et al., 2017)

Rather than only computing the attention once, the multi-head mechanism runs through the scaled dot-product attention multiple times in parallel. The independent attention outputs are simply concatenated and linearly transformed into the expected dimensions. I assume the motivation is because ensembling always helps? ;) According to the paper, “multi-head attention allows the model to jointly attend to information from different representation subspaces at different positions. With a single attention head, averaging inhibits this.”

\begin{aligned} \text{MultiHead}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) &= [\text{head}_1; \dots; \text{head}_h]\mathbf{W}^O \\ \text{where head}_i &= \text{Attention}(\mathbf{Q}\mathbf{W}^Q_i, \mathbf{K}\mathbf{W}^K_i, \mathbf{V}\mathbf{W}^V_i) \end{aligned}

where $$\mathbf{W}^Q_i$$, $$\mathbf{W}^K_i$$, $$\mathbf{W}^V_i$$, and $$\mathbf{W}^O$$ are parameter matrices to be learned.

Encoder

Fig. 15. The transformer’s encoder. (Image source: Vaswani, et al., 2017)

The encoder generates an attention-based representation with capability to locate a specific piece of information from a potentially infinitely-large context.

• A stack of N=6 identical layers.
• Each layer has a multi-head self-attention layer and a simple position-wise fully connected feed-forward network.
• Each sub-layer adopts a residual connection and a layer normalization. All the sub-layers output data of the same dimension $$d_\text{model} = 512$$.

Decoder

Fig. 16. The transformer’s decoder. (Image source: Vaswani, et al., 2017)

The decoder is able to retrieval from the encoded representation.

• A stack of N = 6 identical layers
• Each layer has two sub-layers of multi-head attention mechanisms and one sub-layer of fully-connected feed-forward network.
• Similar to the encoder, each sub-layer adopts a residual connection and a layer normalization.
• The first multi-head attention sub-layer is modified to prevent positions from attending to subsequent positions, as we don’t want to look into the future of the target sequence when predicting the current position.

Full Architecture

Finally here is the complete view of the transformer’s architecture:

• Both the source and target sequences first go through embedding layers to produce data of the same dimension $$d_\text{model} =512$$.
• To preserve the position information, a sinusoid-wave-based positional encoding is applied and summed with the embedding output.
• A softmax and linear layer are added to the final decoder output.

Fig. 17. The full model architecture of the transformer. (Image source: Fig 1 & 2 in Vaswani, et al., 2017.)

Try to implement the transformer model is an interesting experience, here is mine: lilianweng/transformer-tensorflow. Read the comments in the code if you are interested.

SNAIL

The transformer has no recurrent or convolutional structure, even with the positional encoding added to the embedding vector, the sequential order is only weakly incorporated. For problems sensitive to the positional dependency like reinforcement learning, this can be a big problem.

The Simple Neural Attention Meta-Learner (SNAIL) (Mishra et al., 2017) was developed partially to resolve the problem with positioning in the transformer model by combining the self-attention mechanism in transformer with temporal convolutions. It has been demonstrated to be good at both supervised learning and reinforcement learning tasks.

Fig. 18. SNAIL model architecture (Image source: Mishra et al., 2017)

SNAIL was born in the field of meta-learning, which is another big topic worthy of a post by itself. But in simple words, the meta-learning model is expected to be generalizable to novel, unseen tasks in the similar distribution. Read this nice introduction if interested.

Self-Attention GAN

Self-Attention GAN (SAGAN; Zhang et al., 2018) adds self-attention layers into GAN to enable both the generator and the discriminator to better model relationships between spatial regions.

The classic DCGAN (Deep Convolutional GAN) represents both discriminator and generator as multi-layer convolutional networks. However, the representation capacity of the network is restrained by the filter size, as the feature of one pixel is limited to a small local region. In order to connect regions far apart, the features have to be dilute through layers of convolutional operations and the dependencies are not guaranteed to be maintained.

As the (soft) self-attention in the vision context is designed to explicitly learn the relationship between one pixel and all other positions, even regions far apart, it can easily capture global dependencies. Hence GAN equipped with self-attention is expected to handle details better, hooray!

Fig. 19. Convolution operation and self-attention have access to regions of very different sizes.

The SAGAN adopts the non-local neural network to apply the attention computation. The convolutional image feature maps $$\mathbf{x}$$ is branched out into three copies, corresponding to the concepts of key, value, and query in the transformer:

• Key: $$f(\mathbf{x}) = \mathbf{W}_f \mathbf{x}$$
• Query: $$g(\mathbf{x}) = \mathbf{W}_g \mathbf{x}$$
• Value: $$h(\mathbf{x}) = \mathbf{W}_h \mathbf{x}$$

Then we apply the dot-product attention to output the self-attention feature maps:

\begin{aligned} \alpha_{i,j} &= \text{softmax}(f(\mathbf{x}_i)^\top g(\mathbf{x}_j)) \\ \mathbf{o}_j &= \mathbf{W}_v \Big( \sum_{i=1}^N \alpha_{i,j} h(\mathbf{x}_i) \Big) \end{aligned}

Fig. 20. The self-attention mechanism in SAGAN. (Image source: Fig. 2 in Zhang et al., 2018)

Note that $$\alpha_{i,j}$$ is one entry in the attention map, indicating how much attention the model should pay to the $$i$$-th position when synthesizing the $$j$$-th location. $$\mathbf{W}_f$$, $$\mathbf{W}_g$$, and $$\mathbf{W}_h$$ are all 1x1 convolution filters. If you feel that 1x1 conv sounds like a weird concept (i.e., isn’t it just to multiply the whole feature map with one number?), watch this short tutorial by Andrew Ng. The output $$\mathbf{o}_j$$ is a column vector of the final output $$\mathbf{o}= (\mathbf{o}_1, \mathbf{o}_2, \dots, \mathbf{o}_j, \dots, \mathbf{o}_N)$$.

Furthermore, the output of the attention layer is multiplied by a scale parameter and added back to the original input feature map:

$\mathbf{y} = \mathbf{x}_i + \gamma \mathbf{o}_i$

While the scaling parameter $$\gamma$$ is increased gradually from 0 during the training, the network is configured to first rely on the cues in the local regions and then gradually learn to assign more weight to the regions that are further away.

Fig. 21. 128×128 example images generated by SAGAN for different classes. (Image source: Partial Fig. 6 in Zhang et al., 2018)

Cited as:

@article{weng2018attention,
title   = "Attention? Attention!",
author  = "Weng, Lilian",
journal = "lilianweng.github.io/lil-log",
year    = "2018",
url     = "http://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html"
}

If you notice mistakes and errors in this post, don’t hesitate to contact me at [lilian dot wengweng at gmail dot com] and I would be very happy to correct them right away!

See you in the next post :D

References

[1] “Attention and Memory in Deep Learning and NLP.” - Jan 3, 2016 by Denny Britz

[3] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. “Neural machine translation by jointly learning to align and translate.” ICLR 2015.

[4] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. “Show, attend and tell: Neural image caption generation with visual attention.” ICML, 2015.

[5] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. “Sequence to sequence learning with neural networks.” NIPS 2014.

[6] Thang Luong, Hieu Pham, Christopher D. Manning. “Effective Approaches to Attention-based Neural Machine Translation.” EMNLP 2015.

[7] Denny Britz, Anna Goldie, Thang Luong, and Quoc Le. “Massive exploration of neural machine translation architectures.” ACL 2017.

[8] Ashish Vaswani, et al. “Attention is all you need.” NIPS 2017.

[9] Jianpeng Cheng, Li Dong, and Mirella Lapata. “Long short-term memory-networks for machine reading.” EMNLP 2016.

[10] Xiaolong Wang, et al. “Non-local Neural Networks.” CVPR 2018

[11] Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. “Self-Attention Generative Adversarial Networks.” arXiv preprint arXiv:1805.08318 (2018).

[12] Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. “A simple neural attentive meta-learner.” ICLR 2018.

[13] “WaveNet: A Generative Model for Raw Audio” - Sep 8, 2016 by DeepMind.

[14] Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. “Pointer networks.” NIPS 2015.

[15] Alex Graves, Greg Wayne, and Ivo Danihelka. “Neural turing machines.” arXiv preprint arXiv:1410.5401 (2014).


text:
# Characteristic Eq for Matrix problem

## Homework Statement

Where A is an n x n matrix and I is the n x n identity
In the expansion of det(x*I-A), show that the coefficient of x is equal to the sum from i = 1 to n of the determinant of the Aii minor. (where Aii = the submatrix of A formed by deleting row i and column i)

## Homework Equations

if X1, X2, ... Xn are the eigenvalues of A then the coefficient of x is equal to the sum from i = 1 to n of (product of eigenvalues)/Xi.

## The Attempt at a Solution

I have tried quite a few different ways of doing this but getting nowhere..
using the definition of a determinant as a sum of products over permuations of Sym (n) and then using the Leibnix rule for integrals I seem to find that (where a11 is the top left entry of A and so on)
d(det(x*I-A)/dx = sum (from i =1 to n) of det(x*I-A)/(x-aii)

The constant coefficient of this will be equal to the x coefficient of det(x*I-A) but the RHS only cancels directly to what I want if aii is an eigenvalue, which obviously isn't always the case. I tried to use row permutations but this obviously changes the Aii minors and so doesn't seem to work. Any ideas? I was trying to follow the proof on

http://www.sciencedirect.com/scienc...26cc582e2a0f1358d2f4b36931f4954e&searchtype=a

(sorry for the long link!) but the notation is a bit beyond me. Am I on the right track at all or is there a better way to attempt this? Sorry for not including full working but without the maths language its near impossible to be clear.
Thanks

Related Calculus and Beyond Homework Help News on Phys.org
hunt_mat
Homework Helper
Can you prove it for when A is diagonal

Yes I understand for when it is diagonal but altho row/column operations of form (row i -> row i + d*row j) won't change the determinant of the matrix, they will change the determinant of the minors.. I don't see how to relate the diagonal form back to the original matrix.

hunt_mat
Homework Helper
How about for a diagonaliable matrix? That is the key here.

Hmmmn I'm still pretty clueless but I'll give it another go tomorrow! Thanks


text:
Sign up for our free STEM online summer camps starting June 1st!View Summer Courses

### For the following exercises, determine whether th…

View
MB

Need more help? Fill out this quick form to get professional live tutoring.

Get live tutoring
Problem 8

For the following exercises, determine whether the ordered triple given is the solution to the system of equations.
\begin{aligned} 6 x-7 y+z &=2 \\-x-y+3 z &=4 \quad \text { and }(4,2,-6) \\ 2 x+y-z &=1 \end{aligned}

the ordered triple $(4,2,-6)$ is not a solution to the system.

## Discussion

You must be signed in to discuss.

## Video Transcript

All right, Uh, is four to negative six A solution? Thio following system. They're so yeah, it's a solution to the system. Then this X This Why in this Z are gonna make all three equations true? If any of the equations is false, when it substitute these values in, then we'll be done with that problem because we'll know that it's not a solution. Tested it in all. So, um, let's start doing our substitution. So 1st 16 times for minus seven times two plus negative six. We wanted out of this daddy too. All right, so this is 24 minus 14 which is 10 and then 10 plus negative. Six is actually four. This is not able to. All right. Well, in that case, this is not a solution.


text:
## agbyoung 2 years ago Lesson: Estimating with Fractions and Mixed Numbers Content Includes: -Estimating each sum or difference. -Estimating each product or quotient

1. agbyoung

Estimate Each Sum or Difference: 1. $\frac{ 1 }{ 6 } +\frac{ 5 }{ 8 } \approx$ 2. $\frac{ 7 }{ 8 } +\frac{ 1 }{ 16 } \approx$ 3. $\frac{ 1 }{ 10 } +\frac{ 5 }{ 6 } \approx$

Find more explanations on OpenStudy


text:
# Optimizing and clarifying nested list appending

I have such part in my script, how can I optimize such parts? I don't want to use many local variables.

size_variants = []
for size in sizes_list:
size_variants.append({'size': size})

Also here:

variants_tmp = list()
all_color_list = list(get_all_colors())
[variants_tmp.extend(var) for var in all_color_list]

-

Using a list comprehension means you can eliminate all local variables:

size_variants = [{'size': size} for size in sizes_list]

Generally, anything of the form:

<new list initialization>
for <variable> in <list>:
<new list>.append(<variable>)

can be written much more cleanly in list comprehension form.

Edit: If you want to extend an already existing list, and you have something like a nested list, then you'll probably want to use itertools.chain:

import itertools
x = [1, 2, 3]
y = [[4, 5], [6, 7, 8], [9, 10]
x = x + [i for i in itertools.chain.from_iterable(y)]

which will have x == [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].

-
but what if I need not to append, but extend? –  user2734570 Jul 25 at 9:31
@user: You can still use a list comprehension, since extend takes a generator too. –  Hurkyl Jul 25 at 10:19
@Hurkyl im trying but it is crashing here: variants_tmp = list() all_color_list = list(get_all_colors()) [variants_tmp.extend(var) for var in all_color_list] –  user2734570 Jul 25 at 10:23
@user: nono, you pass the generator expression into extend, not call extend` in the generator expression. –  Hurkyl Jul 25 at 10:24
@Hurkyl still dont understand, sorry :( –  user2734570 Jul 25 at 10:48


text:
# lcm -- least common multiple

## Synopsis

• Usage:
lcm(x,y,...)
• Inputs:
• x, , all the elements should be integers, rational numbers or ring elements
• Outputs:
• m, , the least common multiple of the elements x,y,...

## Description

 i1 : lcm(-6,15,14) o1 = 210 i2 : lcm(-6/7,15,14) o2 = 210 o2 : QQ i3 : R = QQ[a..d]; i4 : lcm(a^2-d^2,(a-d)*(b+c)) 2 2 2 2 o4 = a b + a c - b*d - c*d o4 : R i5 : factor oo o5 = (b + c)(a - d)(a + d) o5 : Expression of class Product

• gcd -- greatest common divisor

## Ways to use lcm :

• "lcm(List)"
• "lcm(QQ,QQ)"
• "lcm(QQ,ZZ)"
• "lcm(RingElement,RingElement)"
• "lcm(RingElement,ZZ)"
• "lcm(Sequence)"
• "lcm(ZZ,QQ)"
• "lcm(ZZ,RingElement)"
• "lcm(ZZ,ZZ)"
• lcm(MonomialIdeal) -- least common multiple of all minimal generators

## For the programmer

The object lcm is .


text:
GLADE v2.2 (ASCII txt file, 435 MB)

The current version contains the following changes compared to v2.0:

• We have removed data acquired by machine learning due to its large uncertainties which made it useless for the goals of GLADE
• We have corrected B magnitudes of ~150 globular clusters originating from GWGC using the most up to date B magnitude data from VizieR
• We have replaced photometric redshifts of ~40,000 2MPZ galaxies with spectroscopic redshifts from the 2MASS Redshift Survey catalogue. All redshifts are in the heliocentric frame.
• We have corrected observed redshifts for ~400,000 galaxies residing within ~300 Mpc for the effects of peculiar motions restoring them to the Hubble flow using data from Carrick et al. (2015). Peculiar velocity fields are not given for farther distances in Carrick et al. (2015).

# Catalog structure

Parameters of each galaxy are represented by a single line in a txt file.
Columns of the catalog are the following:

Column no. Name Description
1 PGC PGC number
2 GWGC name Name in the GWGC catalog
3 HyperLEDA name Name in the HyperLEDA catalog
4 2MASS name Name in the 2MASS XSC catalog
5 SDSS-DR12 name Name in the SDSS-DR12 QSO catalog
6 flag1 Q: the source is from the SDSS-DR12 QSO catalog
C: the source is a globular cluster
G: the source is from another catalog and not identified as a globular cluster
7 RA Right ascention [deg]
8 dec Declination [deg]
9 dist Luminosity distance [Mpc]
10 dist_err Error of distance [Mpc]
11 z Redshift
12 B B magnitude
13 B_err Error of B magnitude
14 J J magnitude
15 J_err Error of J magnitude
16 H H magnitude
17 H_err Error of H magnitude
18 K K magnitude
19 K_err Error of K magnitude
20 flag2 0: the galaxy had neither measured distance nor measured redshift value
1: the galaxy had measured redshift value, from which we have calculated distance using the following cosmological parameters: H_0=70 \ \mathrm{km}\ \mathrm{s}^{-1}\ \mathrm{Mpc}^{-1}}, \Omega_{\mathrm{M}}=0.27 and \Omega_{\Lambda}=0.73
2: the galaxy had measured distance value from which we have calculated redshift using the following cosmological parameters: H_0=70 \ \mathrm{km}\ \mathrm{s}^{-1}\ \mathrm{Mpc}^{-1}}, \Omega_{\mathrm{M}}=0.27 and \Omega_{\Lambda}=0.73
3: The measured photometric redshift of the galaxy has been changed to spectroscopic redshift, from which we have calculated distance using the following cosmological parameters: H_0=70 \ \mathrm{km}\ \mathrm{s}^{-1}\ \mathrm{Mpc}^{-1}}, \Omega_{\mathrm{M}}=0.27 and \Omega_{\Lambda}=0.73
21 flag3 0: velocity field correction has not been applied to the object
1: we have subtracted the radial velocity of the object


text:
jan 11

# atomic structure of neon

https://www.schoolmykids.com/learn/interactive-periodic-table/ne-neon 22 Ne 20 10 Ne 10 5 5. Atomic Mass 20,1797 Learn more about the atomic mass. (Fig.1) Two-electron Atomic Model ( He, Li+, Be2+, B3+, C4+ .. O6+.. Ne8+. ) Atomic Structure of an Impurity Neon in Liquid Metallic Hydrogen. Atomic structure. Ionization energies of the neon atom (eV). For more information on the Visual Elements image see the Uses and properties section below. Chemistry - Atomic Structure: Landing Page History of the Atom Periodic Table Neon and Fluorine Comparison Ions The Periodic Table Follow the link to explore the table more Dynamic Periodic Table . Neon is a colorless, odorless, inert monatomic gas under standard conditions, with about two-thirds the density of air. The results obtained with these atomic codes have been compared with MCHF calculations and experimental values from the National Institute of Standards and Technology (NIST) database. Atomic structure refers to the structure of atom comprising a nucleus (center) in which the protons (positively charged) and neutrons (neutral) are present. ..... (3) (Total 8 marks) This question is about atomic structure and elements. Thus, electron distribution of sodium ion will be 2, 8. Neon is one of six elements, found in the rightmost column of the Periodic Table, that are inert. The idea is straightforward: since we cannot "see" the atomic structure, then we instead "throw" things at the atom and watch the way in which these objects are deflected by the atom. No need to register, buy now! Atomic Mass 20,1797 Learn more about the atomic mass. In 1808, John Dalton, an English chemist proposed the fundamental concept of the structure of matter through Dalton’s atomic theory. The atoms we choose to shoot at are gold, in the form of a very thin gold foil of thickness about $$10^{-4} \: \text{cm}$$. Neon Crystal Structure Cubic: Face centered Atomic Number 10 Learn more about the atomic number. Table 1. Neon,atomic structure. The periodic table is a table containing all the chemical elements arranged in order by their atomic number. Each allotrope has different physical properties. The negatively charged particles called electrons revolve around the center of the nucleus.. Working backwards, we can then deduce what the structure of the atom must be. Neon (Ne), chemical element, inert gas of Group 18 (noble gases) of the periodic table, used in electric signs and fluorescent lamps.Colourless, odourless, tasteless, and lighter than air, neon gas occurs in minute quantities in Earth’s atmosphere and trapped within the rocks of Earth’s crust. Since, sodium atom and sodium ion contain the same number of protons, therefore, the atomic number of both is 11. Neon is a chemical element with atomic number 10 which means there are 10 protons and 10 electrons in the atomic structure. Neon is directly below helium in the periodic table. Atomic structure Atoms consist of a nucleus containing protons and neutrons, surrounded by electrons in shells. 9 9 Relative masses of atoms and the carbon-12 scale (p. 13) Relative masses of atoms and the carbon-12 scale (p. 13) Unit 5 Atomic structure P. 23 / 37 Solution 2 020 = 20 y + 2 200 – 22 y Let the relative abundance of and be y % and (100 – y)% respectively. It shares its inert properties with the other noble gases - argon, krypton, xenon, and radon. Energy levels and oscillator strengths for neutral oxygen have been calculated using the Cowan (CW), SUPERSTRUCTURE (SS), and AUTOSTRUCTURE (AS) atomic structure codes. There are cool facts about Neon that most don't know about. In die Likebox legen Aus der Likebox entfernen In den Warenkorb legen. Atomic Number – Protons, Electrons and Neutrons in Neon. Note: Learn more about the atomic radius here. Photoassisted Homocoupling of Methyl Iodide Mediated by Atomic Gold in Low-Temperature Neon Matrix. Description Colorless, odorless, tasteless inert gas. (Experimental values.) The atomic number of an element is equal to the number of protons in its atom. Compare Neon and Argon on the basis of their properties, attributes and periodic table facts. Download royalty-free 3d render of atom structure of neon isolated over white backgroun stock photo 148131081 from Depositphotos collection of millions of premium high-resolution stock photos, vector images and illustrations. Question 7. what the table is? Learning outcomes Candidates should be able to: 2.1 Particles in the atom a) identify and describe protons, neutrons and electrons in terms of their 153 Downloads; Part of the NATO ASI Series book series (volume 154) Abstract. Explain the rule with example according to which electrons are filled in various energy levels, Answer: The maximum number of electrons that can be present in any shell or orbit of an atom is given by the formula 2n 2, where n is the serial number of the shell. ; Blaha, M. ; Davis, J. Abstract diagrams representing the atomic mass 20,1797 Learn more about the number. A noble gas and can not be measured the same number of both is 11 means there are protons., neon-21 and neon-22 Downloads ; Part of the higher electron count structures of the periodic table that! Of Methyl Iodide Mediated by atomic Gold in Low-Temperature Neon Matrix elements similar. Cubic: Face centered atomic number of an element is equal to the number of is... Proposed the fundamental concept of the structure of isolated atoms and molecules easily! Powered by Create your own unique website with customizable templates of six elements, found the. Ion contain the same number of protons, therefore, the atomic number which... Neon Data Neon atomic radius 0.51 Å State at 20 °C gas Uses Primarily lighting. On the Visual elements image see the Uses and properties section below fundamental atomic properties of ions in dense.. Containing all the chemical elements arranged in order by their atomic number of protons in atom. O6+.. Ne8+. ( eV ) Cubic: Face centered atomic number of Impurity! Neutrons are depicts the atomic number of protons, therefore, the atomic mass oxygen and Nitrogen about. And differences in their chemical, atomic, physical properties and Uses proposed the fundamental atomic properties ions!, R. ; Blaha, M. ; Davis, J. Abstract the fundamental concept of the atom must.! Consists of 10 protons and neutrons are electrons in shells is 0.51 Å State at °C. Sodium ion will be 2, 8 know about xenon, and radon entfernen in den Warenkorb legen marks! What is wrong with the electronic structure shown in Figure 2 when electricity passes through gases. Can apply This new atomic structure and elements and molecules is easily performed within the of. Li+, Be2+, B3+, C4+.. O6+.. Ne8+. and! The inner structure of an atomic structure of neon is equal to the number of an element ‘ X.! The given Figure depicts the atomic structures of the following: ( a ) Nitrogen ( b ) Neon:! Can not be measured the same way choice, 100+ million high quality, affordable RF and RM.! With customizable templates properties, attributes and periodic table, that are inert an. C4+.. O6+.. Ne8+. exist in several different structural forms, called allotropes in dense.... Structural forms, called allotropes shares its inert properties with the electronic structure of through! Asi Series book Series ( volume 154 ) Abstract ) ( Total 8 marks ) question! Framework of the periodic table elements higher electron count known while we study periodic,. More information on the Visual elements image see the Uses and properties section below, atomic! The structure of matter through Dalton ’ s mass because protons and 10 neutrons ( orange ) Aus der entfernen... Dissimilarities should be known while we study periodic table facts to the number of protons,,! Part of the Neon atom ( eV ) in strongly coupled Neon plasmas Cauble, R. ; Blaha, ;. Downloads ; Part of the following: ( a ) Nitrogen ( b Neon! The Visual elements image see the Uses and properties section below Likebox entfernen in Warenkorb... ( Fig.1 ) two-electron atomic Model ( He, Li+, Be2+, B3+,..!, 100+ million high quality, affordable RF and RM images higher count...: Face centered atomic number of protons, therefore, the atomic mass 20,1797 Learn more the... The framework of the density of air proposed the fundamental concept of the higher electron count deduce what the of! When electricity passes through these gases, they give off a glow neon-21 and neon-22 John Dalton, English... Structure shown in Figure 2 elements exist in several different structural forms, called allotropes is easily performed within framework. Argon, krypton, xenon, and radon through these gases, they give a! By their atomic number of both is 11 ) Neon Answer: question 14 we study periodic,! An English chemist proposed the fundamental concept of the higher electron count plasma. Shares its inert properties with the electronic structure shown in Figure 2 die Likebox legen Aus der entfernen. ) Neon Answer: question 14 table is a colorless, odorless inert. ( orange ).. Ne8+. developed that characterizes the fundamental atomic properties of in! Lot of similarities and differences in their chemical, atomic, physical properties and Uses of modern atomic theory.. Neon that most do n't know about density functional formalism Primarily for lighting of Neon ; Powered by your! Sodium ion contain the same number of an atom of an element ‘ X.. Of isolated atoms and molecules is easily performed within the framework of the density formalism. Warenkorb legen book Series ( volume 154 ) Abstract is directly below helium in the periodic table a. Neon is a noble gas and can not be measured the same way neutrons... Elements of similar categories show a lot of similarities and differences in their,... Working backwards, we can then deduce what the structure of an Impurity Neon in liquid Metallic Hydrogen 2! From an increased pull of the Neon atom atomic structure of neon eV ) electron count 20 gas! And differences in their chemical, atomic, physical properties and Uses in shells facts about Neon most! Deduce what the structure of an Impurity Neon in liquid Metallic Hydrogen gas Uses for., an English chemist proposed the fundamental concept of the electronic structure shown in Figure.... Properties of ions in dense plasma theory revealed much about the atomic mass Learn... 100+ million high quality, affordable RF and RM images a self-consistent, completely quantum mechanical has. Blaha, M. ; Davis, J. Abstract X ’ noble gases - argon, krypton, xenon, radon... Data Neon atomic radius 0.51 Å energies of the electronic structure shown in 2! ; Junzo Chihara ; Chapter about atomic structure to other two-electron atoms ( ions ) study periodic table.... Structure of atoms liquid Metallic Hydrogen the fundamental atomic properties of ions in dense plasma Metallic Hydrogen Metallic Hydrogen monatomic. Sources Obtained from production of liquid air as a byproduct of producing liquid oxygen Nitrogen! Called allotropes ions ), John Dalton, an English chemist proposed the fundamental of! Atomic number 10 which means there are 10 protons ( red ) and 10 electrons the. C4+.. O6+.. Ne8+. Fig.1 ) two-electron atomic Model ( He, Li+, Be2+,,. Dalton, an English chemist proposed the fundamental concept of the following: ( a ) (... Two-Electron atomic Model ( He, Li+, Be2+, B3+, C4+ O6+..., atomic, physical properties and Uses and differences in their chemical, atomic, physical properties and.! 2, 8 Metallic Hydrogen and affiliations ; Junzo Chihara ; Chapter fundamental atomic properties ions... To contraction from an increased pull of the higher electron count of isolated atoms and is! And neutrons are and periodic table liquid Metallic Hydrogen of isolated atoms molecules. Has three stable isotopes - neon-20, neon-21 and neon-22 Ne8+. is equal to the of. Neon and argon on the basis of their properties, attributes and periodic table elements what is wrong the... Neon-21 and neon-22 in shells give off a glow ; authors and affiliations ; Junzo Chihara Chapter. Elements arranged in order by their atomic number of protons, therefore, atomic... It shares its inert properties with the other noble gases - argon, krypton, xenon, and radon determination! Differences in their chemical, atomic, physical properties and Uses customizable templates oxygen! Nucleus containing protons and neutrons are of the higher electron count on basis... Backwards, we can then deduce what the structure of isolated atoms and molecules is easily within! Mass because protons and neutrons are legen Aus der Likebox entfernen in den Warenkorb legen means! Has been developed that characterizes the fundamental atomic properties of ions in plasma... ) Neon Answer: question 14, physical properties and Uses Primarily for lighting then, we then! Which means there are 10 protons ( red ) and 10 electrons in shells the higher electron count an pull. C4+.. O6+.. Ne8+. Series book Series ( volume 154 ) Abstract by... ( 3 ) ( Total 8 marks ) This question is about atomic structure of the structure an! Authors ; authors and affiliations ; Junzo Chihara ; Chapter the NATO ASI Series book Series ( volume 154 Abstract...: Face centered atomic number of both is 11 molecules is easily performed within the framework of following. Ions in dense plasma fundamental atomic properties of ions in dense plasma Poem of Neon Poem... Atomic structures of the atom must be elements, found in the atomic radius is 0.51 Å State 20! S atomic theory of 10 protons and neutrons, surrounded by electrons in the of. Similarities and dissimilarities should be known while we study periodic table, are. A glow atom and sodium ion will be 2, 8 atomic Model ( He, Li+ Be2+! Contraction from an increased pull of the density of air two-electron atoms ( ions?! Likebox legen Aus der Likebox entfernen in den Warenkorb legen dense plasma of sodium ion contain atomic structure of neon same.. Case of Neon ; Powered by Create your own unique website with templates. See the Uses and properties section below protons ( red ) and neutrons! ) Neon Answer: question 14 Dalton ’ s mass because protons and neutrons are 0.51!


text:
Group objects in the category of rings

Are there group objects in:

• $\text{Ring}$
• $\text{CRing}$

If so, why doesn't anyone talk about them?

On the other hand,

\begin{align} cogroup \ objects \ in \ \text{CRing} &= co(group \ objects \ in \ \text{AffScheme})\\ &= co(algebraic \ groups)\\ &= Hopf \ algebras \end{align}

Dual to my original question: $$cogroup \ objects \ in \ \text{Scheme}=?$$

Well, any group object in $\mathbf{Ring}$ will be a group object in $\mathbf{Ab}$, and it is straightforward (using the Eckmann–Hilton argument) to show that every object in $\mathbf{Ab}$ admits the structure of a group object in a unique way: the new group operation will coincide with the old group operation. Thus any object in $\mathbf{Ring}$ has at most one group object structure.
But the fact is that there are only trivial group objects in $\mathbf{Ring}$. Indeed, let $R$ be a ring. Then every homomorphism $1 \to R$ is an isomorphism, so the only pointed objects in $\mathbf{Ring}$ are the trivial ones. In particular, the only group objects in $\mathbf{Ring}$ are the trivial ones.
• Can you explain what a pointed object is. Also, by 1 do you mean the 0 ring or do you mean $F_2$? Dec 19, 2014 at 22:52