text:
## Some exact solutions to multidimensional Landau-Lifshitz equation with uprush external field and anisotropy field.(English)Zbl 1172.35348

Summary: We give a method that can construct some exact solutions to a multidimensional Landau-Lifshitz equation with uprush external field and anisotropy field.

### MSC:

 35C05 Solutions to PDEs in closed form 35A25 Other special methods applied to PDEs 35Q60 PDEs in connection with optics and electromagnetic theory 82D45 Statistical mechanics of ferroelectrics

exact transform
Full Text:

### References:

 [1] Landau, L.D.; Lifshitz, E.M., On the theory of the dispersion of magnetic permeability in ferromagnetic bodies, Z. sowjetunion, 8, (1935), Reproduced in Collected Papers of L.D. Landau, Pergamon, New York, 1965, pp. 101-114 · Zbl 0012.28501 [2] Nakamura, K.; Sasada, T., Soliton and wave trains in ferromagnets, Phys. lett. A, 48, 321-322, (1974) [3] Tjon, J.; Wright, J., Soliton in the hesenberg chain, Phys. rev. B, 15, 3470-3476, (1977) [4] Lakshmanan, M.; Ruijgrok, T.w.; Thompson, C.J., On the dynamics of a continuous spin systems, Physica A, 84, 577-590, (1976) [5] Eleonskiĭ, V.M.; Kirova, N.N.; Petrov, V.M., General type of solutions of the landau – lifshitz equations, Sov. phys. JETP, 41, 5, 966-970, (1975) [6] Yue, Hong; Chen, Xiang-Jun; Huang, Nian-Ning, An inverse scattering transform for the landau – lifshitz equation for a spin chain with an easy plane, J. phys. A, 31, 10, 2491-2501, (1998) · Zbl 0920.35148 [7] Sadakane, T., Soliton solutions of $$X X Z$$ lattice landau – lifshitz equation, J. math. phys., 42, 11, 5457-5471, (2001) · Zbl 1063.82006 [8] Gustafson, S.; Shatah, J., The stability of localized solutions of landau – lifshitz equations, Commun. pure appl. math., LV, 5, 1136-1159, (2002) · Zbl 1028.35059 [9] Chapter four in Yang Ganshan’s Doctor Thesis, Beijing [10] Guo, Boling; Han, Yongqian; Yang, Ganshan, Blow up problem for landau – lifshitz equations in two dimensions, Commun. nonlinear sci. numer. simul., 5, 1, 43-44, (2000) · Zbl 0952.35143 [11] Guo, Boling; Han, Yongqian; Yang, Ganshan, Exact blow-up solutions for multidimensional landau – lifshitz equations, Adv. math. (China), 30, 1, 91-93, (2001) · Zbl 0999.35094 [12] Guo, Boling; Yang, Ganshan, Some exact nontrivial global solutions with values in unit sphere for two-dimensional landau – lifshitz equations, J. math. phys., 42, 5223-5227, (2001) · Zbl 1063.35047 [13] Yang, Ganshan; Chang, Qianshun, Limit behavior of solution for multidimensional landau – lifshitz equations with external magnetic field, Phys. lett. A, 318, 3, 270-280, (2003) · Zbl 1045.35073 [14] Svendsen, Michael; Fogedby, Hans C., Phase shift analysis of the landau – lifshitz equation, J. phys. A, 26, 7, 1717-1730, (1993) · Zbl 0774.35068 [15] Sen, Shibani; Roy Chowdhury, A., On the invariance properties and conservation laws for landau – lifshitz equation in three dimensions with tijon-wright ansatz, J. math. phys., 32, 9, 2343-2346, (1991) · Zbl 0737.35081 [16] Roelofs, G.H.M.; Martini, R., Prolongation structure of the landau – lifshitz equation, J. math. phys., 34, 6, 2394-2399, (1993) · Zbl 0776.35060 [17] Skrypnyk, T., ‘doubled’ generalized landau – lifshitz hierarchies and special quasigraded Lie algebras, J. phys. A, 37, 31, 7755-7768, (2004) · Zbl 1063.22025 [18] Kosevich, A.M.; Ivanov, B.A.; Kovalev, A.S., Magnetic solitons, Phys. rep., 194, 3-4, 117-238, (1990)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.


text:
<img src="https://d5nxst8fruw4z.cloudfront.net/atrk.gif?account=iA1Pi1a8Dy00ym" style="display:none" height="1" width="1" alt="" />

# Sum Notation and Properties of Sigma

## Identify and state the sum of terms in finite series

Estimated34 minsto complete
%
Progress
Practice Sum Notation and Properties of Sigma
Progress
Estimated34 minsto complete
%
Sum Notation and Series Sums

Feel free to modify and personalize this study guide by clicking “Customize.”

### Vocabulary

##### Complete the chart.
 Word Definition ________________ the large, stylized capitol E that means, approximately, "the sum of" Index __________________________________________________________ ________________ the expression being summed in a sigma Limits __________________________________________________________ ________________ when a series has a limit, and the limit exists ________________ when a series does not have a limit, or the limit is infinity Hypothesis __________________________________________________________ Mathematical induction __________________________________________________________ Partial sum __________________________________________________________

### Properties of Sigma

Label the sigma, index, summand, and limit:

$\sum_{n=1}^4 3n$

.

The above can be read as "find the _____ of the first ______ terms of the series, where the nth term is ____.

What does the sum above add to? __________

True or false: You can factor a coefficient out of a sum. ____________

.

Express the Sum using Sigma Notation:

1. $1 + 5 + 9 + 13 + 17$
2. $1 + \frac{1} {2} + \frac{1} {3} + \frac{1} {4} + ... + \frac{1} {10}$

Find the series of numbers indicated and evaluate the summations:

1. $\sum_{n=-10}^{5} 7 -\frac{4}{3}(n-1)$
2. $\sum_{n=-2}^{3} 8 -2(n-1)$
3. $\sum_{n=-5}^{1} 5 +\frac{4}{3}(n-1)$
4. $\sum_{n=1}^{6} 4(\frac{1}{2})^{n-1}$
5. $\sum_{n=1}^{7} 3(-\frac{1}{2})^{n-1}$
6. $\sum_{n=1}^{11} -3(\frac{4}{3})^{n-1}$

.

.

#### Gauss' Formula

Gauss' Formula allows us to add together the first positive integers:

$\sum = \frac{(n) (n + 1)} {2}$

.In your own words, describe how Gauss' Formula works. _________________________________________________________________________

Explain how to use your calculator to solve $\sum_{n = 1}^9 n^2$ :

_________________________________________________________________________

Keep in mind Gauss' Formula can also be written as $S_n = \frac{n(a_1 + a_n)} {2}$.

.

Calculate the sums of the given series. You may use addition of individual terms or a series sum formula.

1. $\sum_{n = 0}^10 14 - \frac{1}{2}(n - 1)$
2. $\frac{-71}{3} - \frac{67}{3} - 21 + ... + \frac{37}{3}$
3. $2 + 4 + 6 + ... 26$
4. $-2 - 1 + 0 + ... + 12$
5. The first eight numbers of an arithmetic sequence add up to 604. The next eight numbers added up equal 156. Find the first number and the common difference in the sequence.
6. The first number in an arithmetic sequence is 80. Find the common difference if we also know that $s_9$ is eighteen times $a_{11}$
7. If $a_n$ is an arithmetic sequence with $a_1 = 1$ . Find the second number if we know that the sum of the first five numbers is one-fourth of the sum of the next five numbers.
8. Given $(a_n) = 78, 75, 72, 69...$ Find $a_{150}$ and $s_{150}$

.

format: plain


text:
18-23 March 2018
Florianopolis, Brazil
America/Sao_Paulo timezone

## A first survey of the ghost-gluon vertex in the Gribov-Zwanziger framework

22 Mar 2018, 09:50
20m
Florianopolis, Brazil

Seminar

Bruno Mintz

### Description

In order to have a proper definition of a quantum nonabelian gauge theory, such as QCD, one needs to have a well-defined generating functional. In the perturbative regime at high energies, this can be achieved through the well-known Faddeev-Popov quantization procedure. For energy scales in which the coupling of the strong force becomes sufficiently high, the perturbative approach may display some inconsistencies. One such problem is the so-called Gribov problem: in a gauge fixed theory, the ghost fields acquire a pole at finite (euclidean) momentum. Equivalently, the determinant of the Faddeev-Popov operator vanishes and the generating functional of the theory becomes void. In order to correct this problem (at least partially), Gribov suggested to restrict the functional integration of gauge fields to a subset of the field space, which is now called the Gribov region.

As this restriction of the gauge fields to the Gribov region is taken into account, it turns out that the resulting gauge field propagators display a nontrivial infrared behavior, being very close to the ones observed in lattice gauge field theory simulations. In this work, we explore for the first time a higher correlation function in the presence of the Gribov horizon: the ghost-anti-ghost-gluon interaction vertex, at one-loop level. Our analytical results (within the so-called Refined Gribov Zwanziger theory) are fairly compatible with lattice YM simulations, as well as with solutions from the Schwinger-Dyson equations. This is an indication that the RGZ framework can provide a reasonable description in the infrared not only of gauge field propagators, but also of higher correlation functions, such as interaction vertices.


text:
# Taylor Expansion Policy Optimization

In this work, we investigate the application of Taylor expansions in reinforcement learning. In particular, we propose Taylor expansion policy optimization, a policy optimization formalism that generalizes prior work (e.g., TRPO) as a first-order special case. We also show that Taylor expansions intimately relate to off-policy evaluation. Finally, we show that this new formulation entails modifications which improve the performance of several state-of-the-art distributed algorithms.

## Authors

• 25 publications
• 55 publications
• 71 publications
• ### Hinge Policy Optimization: Rethinking Policy Improvement and Reinterpreting PPO

Policy optimization is a fundamental principle for designing reinforceme...
10/26/2021 ∙ by Hsuan-Yu Yao, et al. ∙ 6

• ### Taylor Expansion of Discount Factors

In practical reinforcement learning (RL), the discount factor used for e...
06/11/2021 ∙ by Yunhao Tang, et al. ∙ 0

• ### Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments

In the NIPS 2017 Learning to Run challenge, participants were tasked wit...
04/02/2018 ∙ by Łukasz Kidziński, et al. ∙ 0

• ### Revisit Policy Optimization in Matrix Form

In tabular case, when the reward and environment dynamics are known, pol...
09/19/2019 ∙ by Sitao Luan, et al. ∙ 1

• ### Wasserstein Reinforcement Learning

We propose behavior-driven optimization via Wasserstein distances (WDs) ...
06/11/2019 ∙ by Aldo Pacchiano, et al. ∙ 1

• ### Decaying Clipping Range in Proximal Policy Optimization

Proximal Policy Optimization (PPO) is among the most widely used algorit...
02/20/2021 ∙ by Mónika Farsang, et al. ∙ 0

• ### New Techniques for Algorithm Portfolio Design

We present and evaluate new techniques for designing algorithm portfolio...
06/13/2012 ∙ by Matthew Streeter, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Policy optimization is a major framework in model-free reinforcement learning (RL), with successful applications in challenging domains (Silver et al., 2016; Berner et al., 2019; Vinyals et al., 2019). Along with scaling up to powerful computational architectures (Mnih et al., 2016; Espeholt et al., 2018), significant algorithmic performance gains are driven by insights into the drawbacks of naïve policy gradient algorithms (Sutton et al., 2000). Among all algorithmic improvements, two of the most prominent are: trust-region policy search (Schulman et al., 2015, 2017; Abdolmaleki et al., 2018; Song et al., 2020) and off-policy corrections (Munos et al., 2016; Wang et al., 2017; Gruslys et al., 2018; Espeholt et al., 2018).

At the first glance, these two streams of ideas focus on orthogonal aspects of policy optimization. For trust-region policy search, the idea is to constrain the size of policy updates. This limits the deviations between consecutive policies and lower-bounds the performance of the new policy (Kakade and Langford, 2002; Schulman et al., 2015). On the other hand, off-policy corrections require that we account for the discrepancy between target policy and behavior policy. Espeholt et al. (2018) has observed that the corrections are especially useful for distributed algorithms, where behavior policy and target policy typically differ. Both algorithmic ideas have contributed significantly to stabilizing policy optimization.

In this work, we partially unify both algorithmic ideas into a single framework. In particular, we noticed that as a ubiquitous approximation method, Taylor expansions share high-level similarities with both trust region policy search and off-policy corrections. To get high-level intuitions of such similarities, consider a simple 1D example of Taylor expansions. Given a sufficiently smooth real-valued function on the real line , the -th order Taylor expansion of at is where are the -th order derivatives at . First, a common feature shared by Taylor expansions and trust-region policy search is the inherent notion of a trust region constraint. Indeed, in order for convergence to take place, a trust-region constraint is required 111Here, is the convergence radius of the expansions, which in general depends on the function and origin .. Second, when using the truncation as an approximation to the original function , Taylor expansions satisfy the requirement of off-policy evaluations: evaluate target policy with behavior data. Indeed, to evaluate the truncation at any (target policy), we only require the behavior policy “data” at (i.e., derivatives ).

Our paper proceeds as follows. In Section 2, we start with a general result of applying Taylor expansions to Q-functions. When we apply the same technique to the RL objective, we reuse the general result and derive a higher-order policy optimization objective. This leads to Section 3, where we formally present the Taylor Expansion Policy Optimization (TayPO) and generalize prior work (Schulman et al., 2015, 2017) as a first-order special case. In Section , we make clear connection between Taylor expansions and (Harutyunyan et al., 2016), a common return-based off-policy evaluation operator. Finally, in Section 5, we show the performance gains due to the higher-order objectives across a range of state-of-the-art distributed deep RL agents.

## 2 Taylor expansion for reinforcement learning

Consider a Markov Decision Process (MDP) with state space

and action space . Let policy be a distribution over actions give state . At a discrete time , the agent in state takes action , receives reward , and transitions to a next state . We assume a discount factor . Let be the action value function (Q-function) from state taking action and following policy . For convenience, we use to denote the discounted visitation distribution starting from state-action pair and following , such that . We thus have . We focus on the RL objective of optimizing starting from a fixed initial state .

We define some useful matrix notation. For ease of analysis, we assume that and are both finite. Let denote the reward function and denote the transition matrix such that . We also define

as the vector Q-function. This matrix notation facilitates compact derivations, for example, the Bellman equation writes as

.

### 2.1 Taylor Expansion of Q-functions.

In this part, we state the Taylor expansion of Q-functions. Our motivation for the expansion is the following: Assume we aim to estimate

for target policy , and we only have access to data collected under a behavior policy . Since can be readily estimated with the collected data, how do we approximate with ?

Clearly, when , then . Whenever , starts to deviate from . Therefore, we apply Taylor expansion to describe the deviation in the orders of . We provide the following result.

###### Theorem 1.

(proved in Appendix B) For any policies and  and any , we have

 Qπ−Qμ=

In addition, if , then the limit for exists and we have

 (1)

The constraint between and is a result of the convergence radius of the Taylor expansion. The derivation follows by recursively applying the following equality: Please refer to the Appendix B for a proof. For ease of notation, denote the -th term on the RHS of Eq. 1 as . This gives rise to .

To represent explicitly with the deviation between and , consider a diagonal matrix where and where is the Dirac delta function; we restrict to the case where . This diagonal matrix is a measure of the deviation between and . The above expression can be rewritten as

 Qπ−Qμ=∞∑k=1(γ(I−γPμ)−1Pμ(Dπ/μ−I))kQμ. (2)

We will see that the expansion in Eq. 2 is useful in Section3 when we derive the Taylor expansion of the difference between the performances of two policies, . In Section 4, we also provide the connection between Taylor expansion and off-policy evaluation.

### 2.2 Taylor expansion of reinforcement learning objective

When searching for a better policy, we are often interested in the difference . With Eq. 2, we can derive a similar Taylor expansion result for . Let (resp., ) be the shorthand notation for (resp., ). Here, we formalize the orders of the expansion as the number of times that ratios appear in the expression, e.g., the first-order expansion should only involve up to the first order, without higher order terms, e.g., cross product . We denote the -th order as and by construction . Next, we derive practically useful expressions for .

We provide a derivation sketch below and give the details in Appendix F. Let

be the joint distribution of policies and state at time

such that . Note that the RL objective equivalently writes as and can be expressed as an inner product . This allows us to import results from Eq. 2,

 J(π)−J(μ)=πT0Qπ−μT0Qμ (3)

By reading off different orders of the expansion from the RHS of Eq. 3, we derive

 L1(π,μ) =(π0−μ0)TQμ+μT0U1, (4) Lk(π,μ) =(π0−μ0)TUk−1+μT0Uk, ∀k≥2.

It is worth noting that the -th order expansion of the RL objective is a mixture of the -th and -th order Q-function expansions. This is because integrates over the initial and the initial difference contributes one order of difference in .

Below, we illustrate the results for and . To make the results more intuitive, we convert the matrix notation of Eq. 3 into explicit expectations under .

#### First-order expansion.

By converting from Eq. 4 into expectations, we get

 (5)

To be precise (Eq. 5) to account for the normalization of the distribution . Note that is exactly the same as surrogate objective proposed in prior work on scalable policy optimization (Kakade and Langford, 2002; Schulman et al., 2015, 2017). Indeed, these works proposed to estimate and optimize such a surrogate objective at each iteration while enforcing a trust region. In the following, we generalize this objective with Taylor expansions.

#### Second-order expansion.

By converting from Eq. 4 into expectations, we get

 (6)

Again, accounting for the normalization, (Eq. 6). To calculate the above expectation, we first start from , and sample a pair from the discounted distribution . Then, we use as the starting point and sample another pair from . This implies that the second-order expansion can be estimated only via samples under , which will be essential for policy optimization in practice.

It is worth noting that the second state-action pair with the argument instead of . This is because only contains terms sampled across strictly different time steps.

#### Higher-order expansions.

Similarly to the first-order and second-order expansions, higher-order expansions are also possible by including proper higher-order terms in . For general , can be expressed as (omitting the normalization constants)

 (7)

Here, are sampled sequentially, each following a discounted visitation distribution conditional on the previous state-action pair. We show their detailed derivations in Appendix F. Furthermore, we discuss the trade-off of different orders in Section 3.

#### Interpretation & intuition.

Evaluating with data under requires importance sampling (IS) . In general, since can differ from at all state-action pairs, computing exactly with full IS requires corrections at all steps along generated trajectories. First-order expansion (Eq. 5) corresponds to carrying out only one single correction at sampled state-action pair along the trajectories: Indeed, in computing Eq. 5, we sample a state-action pair along the trajectory and calculate one single IS correction . Similarly, the second-order expansion (Eq. 6) goes one step further and considers the IS correction at two different steps and . As such, Taylor expansions of the RL objective can be interpreted as increasingly tight approximations of the full IS correction.

## 3 Taylor expansion for policy optimization

In high-dimensional policy optimization, where exact algorithms such as dynamic programming are not feasible, it is necessary to learn from sampled data. In general, the sampled data are collected under a behavior policy different from the target policy . For example, in trust-region policy search (e.g., TRPO, Schulman et al., 2015; PPO, Schulman et al., 2017), is the new policy while is a previous policy; in asynchronous distributed algorithms (Mnih et al., 2016; Espeholt et al., 2018; Horgan et al., 2018; Kapturowski et al., 2019), is the learner policy while is delayed actor policy. In this section, we show the fundamental connection between trust-region policy search and Taylor expansions, and propose the general framework of Taylor expansion policy optimization (TayPO).

### 3.1 Generalized trust-region policy pptimization

For policy optimization, it is necessary that the update function (e.g., policy gradients or surrogate objectives) can be estimated with sampled data under behavior policy . Taylor expansions are a natural paradigm to satisfy this requirement. Indeed, to optimize , consider optimizing222Once again, the equality holds under certain conditions, detailed in Section 4.

 maxπ J(π)=maxπ J(μ)+∞∑k=1Lk(π,μ). (8)

Though we have shown that for all are expectations under

, it is not feasible to unbiasedly estimate the RHS of Eq.

8 because it involves an infinite number of terms. In practice, we can truncate the objective up to -th order and drop because it does not involve

However, for any fixed , optimizing the truncated objective in an unconstrained way is risky: As become increasingly different, the approximation becomes more inaccurate and we stray away from optimizing the objective of interest. The approximation error comes from the residual — to control the magnitude of the residual, it is natural to constrain with some . Indeed, it is straightforward to show that

where 333Here we define . Please see Appendix A.1 for more detailed derivations. We formalize the entire local optimization problem as generalized trust-region policy optimization (generalized TRPO),

 maxπ K∑k=1Lk(π,μ),  ||π−μ||1≤ε. (9)

#### Monotonic improvement.

While maximizing the surrogate objective under trust-region constraints (Eq. 9), it is desirable to have performance guarantee on the true objective . Below, Theorem 2 gives such a result.

###### Theorem 2.

(proved in Appendix C) When the policy is optimized based on the trust-region objective Eq. 9 and , the performance is lower bounded as

 J(π) ≥J(μ)+K∑k=1Lk−GK, (10) where GK

Note that if then as , the gap . Therefore, when optimizing based on Eq. 9, the performance is always lower-bounded according to Eq. 10.

#### Connections to prior work on trust-region policy search.

The generalized TRPO extends the formulation of prior work, e.g., TRPO/PPO of Schulman et al. (2015, 2017). Indeed, idealized forms of these algorithms are a special case for , though for practical purposes the constraint is replaced by averaged KL constraints.444Instead of forming the constraints explicitly, PPO (Schulman et al., 2017) enforces the constraints implicitly by clipping IS ratios.

### 3.2 TayPO-k: Optimizing with k-th order expansion

Though there is a theoretical motivation to use trust-region constraints for policy optimization (Schulman et al., 2015; Abdolmaleki et al., 2018), such constraints are rarely explicitly enforced in practice in its most standard form (Eq. 9). Instead, trust regions are implicitly encouraged via e.g., ratio clipping (Schulman et al., 2017) or parameter averaging (Wang et al., 2017)

. In large-scale distributed settings, algorithms already benefit from diverse sample collections for variance reduction of the parameter updates

(Mnih et al., 2016; Espeholt et al., 2018), which brings the desired stability for learning and makes trust-region constraints less necessary (either explicit or implicit). Therefore, we focus on the setting where no trust region is explicitly enforced. We introduce a new family of algorithm TayPO-, which applies the -th order Taylor expansions for policy optimization.

#### Unbiased estimations with variance reduction.

In practice, as expectations under can be estimated as over a single trajectory. Take as an example: Given a trajectory by , assume we have access to some estimates of , e.g., cumulative returns. To generate a sample from

, we can first sample a random time from a geometric distribution with success probability

, i.e., . Second, we sample another random time with geometric distribution but conditional on .555As explained in Section 2.2, since contains IS ratios at strictly different time steps, it is required that . Then, a single sample estimate of Eq. 6 is given by

Further, the following shows the effect of replacing Q-values by advantages .

###### Theorem 3.

(proved in Appendix D) The computation of based on Eq. 7 is exact when replacing by , i.e. can be expressed as

In practice, when computing , replacing by still produces an unbiased estimate and potentially reduces variance. This naturally recovers the result in prior work for (Schulman et al., 2016).

When , we can construct objectives with higher-order terms. The motivation is that with high , forms a closer approximation to the objective of interest: . Why not then have as large as possible? This comes at a trade-off. For example, let us compare and : Though forms a closer approximation to than in expectation, it could have higher variance during estimation when e.g., and have a non-negative correlation. Indeed, as , approximates the full IS correction, which is known to have high variance (Munos et al., 2016).

#### How many orders to take in practice?

Though the higher-order policy optimization formulation generalizes previous results (Schulman et al., 2015, 2017) as an first-order special case, does it suffice to only include first-order terms in practice?

To assess the effects of Taylor expansions, consider a policy evaluation problem on a random MDP (see Appendix H.1 for the detailed setup): Given a target policy and a behavior policy , the approximation error of the -th order expansion is . In Figure 1, We show the relative errors as a function of . Ground-truth quantities such as are always computed analytically. Solid lines show results where all estimates are also computed analytically, e.g., is computed as . Observe that the errors decrease drastically as the expansion order increases. To quantify how sample estimates impact the quality of approximations, we re-compute the estimates but with replaced by empirical estimates . Results are shown in dashed curves. Now comparing , observe that both errors go up compared to their fully analytic counterparts - both become more similar when is small.

This provides motivations for second-order expansions. While first-orders are a default choice for common deep RL algorithms (Schulman et al., 2015, 2017), from the simple MDP example we see that the second-order expansions could potentially improve upon the first-order, even with sample estimates.

### 3.3 TayPO-2 — Second-order policy optimization

From here onwards, we focus on TayPO-. At any iteration, the data are collected under behavior policy in the form of partial trajectories of length . The learner maintains a parametric policy to be optimized. First, we carry out advantage estimation for state-action pairs on the partial trajectories. This could be naïvely estimated as where are value function baselines. One could also adopt more advanced estimation techniques such as generalized advantage estimation (GAE, Schulman et al., 2016). Then, we construct surrogate objectives for optimization: the first-order component as well as second-order component , based on Eq. 5 and Eq. 6 respectively. Note that we replace all by for variance reduction.

Therefore, our final objective function becomes

 ^Lθ≜^L1(πθ,μ)+^L2(πθ,μ). (11)

The parameter is updated via gradient ascent . Similar ideas can be applied to value-based algorithms, for which we provide details in Appendix G.

## 4 Unifying the concepts: Taylor expansion as return-based off-policy evaluation

So far we have made the connection between Taylor expansions and TRPO. On the other hand, as introduced in Section 1, Taylor expansions can also be intimately related to off-policy evaluation. Below, we formalize their connections. With Taylor expansions, we provide a consistent and unified view of TRPO and off-policy evaluation.

### 4.1 Taylor expansion as off-policy evaluation

In the general setting of off-policy evaluation, the data is collected under a behavior policy while the objective is to evaluate . Return-based off-policy evaluation operators (Munos et al., 2016) are a family of operators , indexed by (per state-action) trace-cutting coefficients , a behavior policy and a target policy

 Rπ,μcQ≜Q+(I−γPcμ)−1(r+γPπQ−Q),

where is the (sub)-probability transition kernel for policy . Starting from any Q-function , repeated applications of the operator will result in convergence to , i.e.,

 (Rπ,μc)KQ→Qπ,

as , subject to certain conditions on . To state the main results, recall that Eq. 2 rewrites as In practice, we take a finite and use the approximation .

Next, we state the following result establishing a connection between -th order Taylor expansion and the return-based off-policy operator applied times.

###### Theorem 4.

(proved in Appendix E) For any , any policies and ,

 Qμ+K∑k=1Uk=(Rπ,μ1)KQμ, (12)

where is short for .

Theorem 4 shows that when we approximate by the Taylor expansion up to the -th order, , it is equivalent to generating an approximation by times applying the off-policy evaluation operator on . We also note that the off-policy evaluation operator in Theorem 4 is the operator (Harutyunyan et al., 2016) with .666As a side note, we also show that the advatnage estimation method GAE (Schulman et al., 2016) is highly related to the operator in Appendix F.1.

#### Alternative proof for Q(λ) convergence for λ=1.

Since Taylor expansions converge within a convergence radius, which in this case corresponds to , it implies that with converges when this condition holds. In fact, this coincides with the condition deduced by Harutyunyan et al. (2016).777Note that this alternative proof only works for the case where the initial .

### 4.2 An operator view of trust-region policy optimization

With the connection between Taylor expansion and off-policy evaluation, along with the connection between Taylor expansion and TRPO (Section 3) we give a novel interpretation of TRPO: The -th order generalized TRPO is approximately equivalent to iterating times the off-policy evaluation operator .

To make our claim explicit, recall the RL objective in matrix form is . Now consider approximating by applying the evaluation operator to , iterating times. This produces the surrogate objective , approximately equivalent to that of the generalized TRPO (Eq. 9).888The -th order Taylor expansion of is slightly different from that of the RL objective by construction; see Appendix B for details. As a result, the generalized TRPO (including TRPO; Schulman et al., 2015) can be interpreted as approximating the exact RL objective ), by times iterating the evaluation operator on to approximate . When does this evaluation operator converge? Recall that converges when , i.e., there is a trust region constraint on . This is consistent with the motivation of generalized TRPO discussed in Section 3, where a trust region is required for monotonic improvements.

## 5 Experiments

We evaluate the potential benefits of applying second-order expansions in a diverse set of scenarios. In particular, we test if the second-order correction helps with (1) policy-based and (2) value-based algorithms.

In large-scale experiments, to take advantage of computational architectures, actors () and learners () are not perfectly synchronized. For case (1), in Section 5.1, we show that even in cases where they almost synchronize (), higher-order corrections are still helpful. Then, in Section 5.2, we study how the performance of a general distributed policy-based agent (e.g., IMPALA, Espeholt et al., 2018) is influenced by the discrepancy between actors and learners. For case (2), in Section 5.3, we show the benefits of second-order expansions in with a state-of-the-art value-based agent R2D2 (Kapturowski et al., 2019).

#### Evaluation.

All evaluation environments are done on the entire suite of Atari games (Bellemare et al., 2013). We report human-normalized scores for each level, calculated as , where and are the performances of human and a random policy on level respectively; with details in Appendix H.2.

#### Architecture for distributed agents.

Distributed agents generally consist of a central learner and multiple actors (Nair et al., 2015; Mnih et al., 2016; Babaeizadeh et al., 2017; Barth-Maron et al., 2018; Horgan et al., 2018). We focus on two main setups: Type I includes agents such as IMPALA (Espeholt et al., 2018) (see blue arrows in Figure 5 in Appendix H.3). See Section 5.1 and Section 5.2; Type II includes agents such as R2D2 (Kapturowski et al., 2019; see orange arrows in Figure 5 in Appendix H.3). See Section 5.3. We provide details on hyper-parameters of experiment setups in respective subsections in Appendix H.

#### Practical considerations.

We can extend the TayPO- objective (Eq. 11) to with . By choosing , one achieves bias-variance trade-offs of the final objective and hence the update. We found (exact TayPO-) working reasonably well. See Appendix H.4 for the ablation study on  and further details.

### 5.1 Near on-policy policy optimization

The policy-based agent maintains a target policy network for the learner and a set of behavior policy networks for the actors. The actor parameters are delayed copies of the learner parameter . To emulate a near on-policy situation , we minimize the delay of the parameter passage between the central learner and actors, by hosting both learner/actors on the same machine.

We compare second-order expansions with two baselines: first-order and zero-order. For the first-order baseline, we also adopt the PPO technique of clipping: in Eq. 5 with . Clipping the ratio enforces an implicit trust region with the goal of increased stability (Schulman et al., 2017). This technique has been shown to generally outperform a naïve explicit constraint, as done in the original TRPO (Schulman et al., 2015). In Appendix H.5, we detail how we implemented PPO on the asynchronous architecture. Each baseline trains on the entire Atari suite for M frames and we compare the mean/median human-normalized scores.

The comparison results are shown in Figure 2. Please see the median score curves in Figure 6 in Appendix H.5. We make several observations: (1) Off-policy corrections are very critical. Going from zero-order (no correction) to first-order improves the performance most significantly, even when the delays between actors and the learner are minimized as much as possible; (2) Second-order correction significantly improves on the first-order baseline. This might be surprising, because when near on-policy, one should expect the difference between additional second-order correction to be less important. This implies that in fully asynchronous architecture, it is challenging to obtain sufficiently on-policy data and additional corrections can be helpful.

### 5.2 Distributed off-policy policy optimization

We adopt the same setup as in Section 5.1. To maximize the overall throughput of the agent, the central learner and actors are distributed on different host machines. As a result, both parameter passage from the learner to actors and data passage from actors to the learner could be severely delayed. This creates a natural off-policy scenario with .

We compare second-order with two baselines: first-order and V-trace. The V-trace is used in the original IMPALA agent (Espeholt et al., 2018) and we present its details in Appendix H.6. We are interested in how the agent’s performance changes as the level of off-policy increases. In practice, the level of off-policy can be controlled and measured as the delay (measured in milliseconds) of the parameter passage from the learner to actors. Results are shown in Figure 3, where x-axis shows the artificial delays (in scale) and y-axis shows the mean human-normalized scores after training for M frames. Note that the total delay consists of both artificial delays and inherent delays in the distributed system.

We make several observations: (1) All baseline variants’ performance degrades as the delays increase. All baseline off-policy corrections are subject to failures as the level of off-policines increases. (2) While all baselines perform rather similarly when delays are small, as the level of off-policy increases, second-order correction degrades slightly more gracefully than the other baselines. This implies that second-order is a more robust off-policy correction method than other current alternatives.

### 5.3 Distributed value-based learning

The value-based agent maintains a Q-function network for the learner and a set of delayed Q-function networks for the actors. Let be an operator such that returns the -greedy policy with respect to . The actors generate partial trajectories by executing an and send data to a replay buffer. The target policy is greedy with respect to the current Q-function . The learner samples partial trajectories from the replay buffer and updates parameters by minimizing Bellman errors computed along sampled trajectories. Here we focus on R2D2, a special instance of distributed value-based agent. Please refer to Kapturowski et al. (2019) for a complete review of all algorithmic details of value-based agents such as R2D2.

Across all baseline variants, the learner computes regression targets for the network to approximate . The targets are calculated based on partial trajectories under which require off-policy corrections. We compare several correction variants: zero-order, first-order, Retrace (Munos et al., 2016; Rowland et al., 2020) and second-order. Please see algorithmic details in Appendix G.

The comparison results are in Figure 4 where we show the mean scores. We make several observations: (1) second-order correction leads to marginally better performance than first-order and retrace, and significantly better than zero-order. (2) In general, unbiased (or slightly biased) off-policy corrections do not yet perform as well as radically biased off-policy variants, such as uncorrected-nstep (Kapturowski et al., 2019; Rowland et al., 2020). (3) Zero-order performs the worst — though it is able to reach super human performance on most games as other variants but then the performance quickly plateaus. See Appendix H.7 for more results.

## 6 Discussion and conclusion

The idea of IS is the core of most off-policy evaluation techniques (Precup et al., 2000; Harutyunyan et al., 2016; Munos et al., 2016). We showed that Taylor expansions construct approximations to the full IS corrections and hence intimately relate to established off-policy evaluation techniques.

However, the connection between IS and policy optimization is less straightforward. Prior work focuses on applying off-policy corrections directly to policy gradient estimators (Jie and Abbeel, 2010; Espeholt et al., 2018) instead of the surrogate objectives which generate the gradients. Though standard policy optimization objectives (Schulman et al., 2015, 2017) involve IS weights, their link with IS is not made explicit. Closely related to our work is that of Tomczak et al. (2019), where they identified such optimization objectives as biased approximations to the full IS objective (Metelli et al., 2018). We characterized such approximations as the first-order special case of Taylor expansions and derived their natural generalizations.

In summary, we showed that Taylor expansions naturally connect trust-region policy search with off-policy evaluations. This new formulation unifies previous results, opens doors to new algorithms and bring significant gains to certain state-of-the-art deep RL agents.

#### Acknowledgements.

Great thanks to Mark Rowland for insightful discussions during the development of ideas as well as extremely useful feedbacks on earlier versions of this paper. The authors also thank Diana Borsa, Jean-Bastien Grill, Florent Altché, Tadashi Kozuno, Zhongwen Xu, Steven Kapturowski, and Simon Schmitt for helpful discussions.

## References

• A. Abdolmaleki, J. T. Springenberg, Y. Tassa, R. Munos, N. Heess, and M. Riedmiller (2018) Maximum a posteriori policy optimisation. In International Conference on Learning Representations, Cited by: §1, §3.2.
• M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, and J. Kautz (2017) Reinforcement learning through asynchronous advantage actor-critic on a gpu. International Conference on Learning Representations. Cited by: §H.3, §5.
• G. Barth-Maron, M. W. Hoffman, D. Budden, W. Dabney, D. Horgan, D. TB, A. Muldal, N. Heess, and T. Lillicrap (2018) Distributional policy gradients. In International Conference on Learning Representations, Cited by: §H.3, §5.
• M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling (2013) The arcade learning environment: an evaluation platform for general agents.

Journal of Artificial Intelligence Research

47, pp. 253–279.
Cited by: §H.2, §5.
• C. Berner, G. Brockman, B. Chan, V. Cheung, P. Debiak, C. Dennison, D. Farhi, Q. Fischer, S. Hashme, C. Hesse, et al. (2019) Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680. Cited by: §1.
• L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning, S. Legg, and K. Kavukcuoglu (2018) IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures. In International Conference on Machine Learning, Cited by: Figure 5, §H.3, §H.5, §H.6, §H.6, §H.6, §H.6, §H.6, §H.7, §1, §1, §3.2, §3, §5, §5.2, §5, §6.
• A. Gruslys, W. Dabney, M. G. Azar, B. Piot, M. Bellemare, and R. Munos (2018) The reactor: a fast and sample-efficient actor-critic agent for reinforcement learning. In International Conference on Learning Representations, Cited by: §1.
• A. Harutyunyan, M. G. Bellemare, T. Stepleton, and R. Munos (2016) Q() with Off-Policy Corrections. Refereed Conference In Algorithmic Learning Theory, Cited by: §F.1, §1, §4.1, §4.1, §6.
• K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. Cited by: §H.6.
• D. Horgan, J. Quan, D. Budden, G. Barth-Maron, M. Hessel, H. van Hasselt, and D. Silver (2018) Distributed prioritized experience replay. In International Conference on Learning Representations, Cited by: §H.3, §H.3, §H.7, §3, §5.
• T. Jie and P. Abbeel (2010) On a connection between importance sampling and the likelihood ratio policy gradient. In Neural Information Processing Systems, Cited by: §6.
• S. Kakade and J. Langford (2002) Approximately optimal approximate reinforcement learning. In International Conference on Machine Learning, Cited by: §1, §2.2.
• S. Kapturowski, G. Ostrovski, W. Dabney, J. Quan, and R. Munos (2019) Recurrent experience replay in distributed reinforcement learning. In International Conference on Learning Representations, Cited by: Appendix G, Figure 5, §H.3, §H.7, §3, §5, §5.3, §5.3, §5.
• D. P. Kingma and J. Ba (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. Cited by: §H.7.
• A. M. Metelli, M. Papini, F. Faccio, and M. Restelli (2018) Policy optimization via importance sampling. In Neural Information Processing Systems, Cited by: §6.
• V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu (2016) Asynchronous methods for deep reinforcement learning. In International conference on machine learning, Cited by: §H.3, §H.5, §H.5, §H.6, §H.7, §1, §3.2, §3, §5.
• V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller (2013) Playing atari with deep reinforcement learning. In

NIPS Deep Learning Workshop

,
Cited by: §H.6.
• R. Munos, T. Stepleton, A. Harutyunyan, and M. Bellemare (2016) Safe and efficient off-policy reinforcement learning. In Neural Information Processing Systems, Cited by: §F.1, Appendix G, §H.7, §1, §3.2, §4.1, §5.3, §6.
• A. Nair, P. Srinivasan, S. Blackwell, C. Alcicek, R. Fearon, A. De Maria, V. Panneershelvam, M. Suleyman, C. Beattie, S. Petersen, et al. (2015) Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296. Cited by: §H.3, §5.
• T. Pohlen, B. Piot, T. Hester, M. G. Azar, D. Horgan, D. Budden, G. Barth-Maron, H. Van Hasselt, J. Quan, M. Večerík, et al. (2018) Observe and look further: achieving consistent performance on atari. arXiv preprint arXiv:1805.11593. Cited by: §H.7.
• D. Precup, R. S. Sutton, and S. P. Singh (2000) Eligibility traces for off-policy policy evaluation. In International Conference on Machine Learning, Cited by: §6.
• M. Rowland, W. Dabney, and R. Munos (2020) Adaptive trade-offs in off-policy learning. In International Conference on Artificial Intelligence and Statistics, Cited by: Appendix G, §5.3, §5.3.
• J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz (2015) Trust region policy optimization. In International conference on machine learning, Cited by: Appendix D, §1, §1, §1, §2.2, §3.1, §3.2, §3.2, §3.2, §3, §4.2, §5.1, §6.
• J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel (2016) High-dimensional continuous control using generalized advantage estimation. In International Conference on Learning Representations, Cited by: §F.1, §H.6, §3.2, §3.3, footnote 6.
• J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: Appendix D, §F.1, §H.5, §1, §1, §2.2, §3.1, §3.2, §3.2, §3.2, §3, §5.1, §6, footnote 4.
• D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis (2016)

Mastering the game of go with deep neural networks and tree search

.
• H. F. Song, A. Abdolmaleki, J. T. Springenberg, A. Clark, H. Soyer, J. W. Rae, S. Noury, A. Ahuja, S. Liu, D. Tirumala, N. Heess, D. Belov, M. Riedmiller, and M. M. Botvinick (2020) V-MPO: on-policy maximum a posteriori policy optimization for discrete and continuous control. In International Conference on Learning Representations, Cited by: §1.
• R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour (2000) Policy gradient methods for reinforcement learning with function approximation. In Neural Information Processing Systems, Cited by: §1.
• T. Tieleman and G. Hinton (2012)

Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude

.
COURSERA: Neural networks for machine learning 4 (2), pp. 26–31. Cited by: §H.5, §H.6.
• M. B. Tomczak, D. Kim, P. Vrancx, and K. Kim (2019) Policy optimization through approximated importance sampling. arXiv preprint arXiv:1910.03857. Cited by: §6.
• O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, et al. (2019) Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature 575 (7782), pp. 350–354. Cited by: §1.
• Z. Wang, V. Bapst, N. Heess, V. Mnih, R. Munos, K. Kavukcuoglu, and N. de Freitas (2017) Sample efficient actor-critic with experience replay. International Conference on Learning Representations. Cited by: §1, §3.2.
• Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas (2016) Dueling network architectures for deep reinforcement learning. In International Conference on Machine Learning, Cited by: §H.7.

## Appendix A Derivation of results for generalized trust-region policy optimization

### a.1 Controlling the residuals of Taylor expansions

We summarize the bound on the magnitude of the Taylor expansion residuals of the Q-function as a proposition.

###### Proposition 1.

Recall the definition of the Taylor expansion residual of the Q-function from the main text, . Let be the infinity norm . Let be the maximum reward in the entire MDP, . Finally, let . Then

 (13)
###### Proof.

The proof follows by bounding each the magnitude of term ,

 ≤∞∑k=K+1||Uk||

The above derivation shows that once we have as . In the above derivation, we have applied the bound which will also be helpful in later derivations. ∎

### a.2 Deriving Taylor expansions of RL objective

Recall that the RHS of Eq. 2 are the Taylor expansions of Q-functions . By construction, . Though Eq. 2 shows the expansion of the entire vector , for optimization purposes, we care about the RL objective from a starting state , , where follows the definition from the main paper .

Now we focus on calculating for general . For simplicity, we write as and henceforth we might use these notations interchangeably. Now consider the RHS of Eq. 3. By definition of the -th order Taylor expansion of , we maintain terms where appears at most times. Equivalently, in matrix form, we remove the higher order terms of while only maintaining terms such as . This allows us to conclude that

Furthermore, we can single out each term

 Lk =(π0−μ0)TTk−1+μT0Uk,  k≥2 L1 =(π0−μ0)TQμ+μT0U1.

## Appendix B Proof of Theorem 1

###### Proof.

We derive the Taylor expansion of Q-function into different orders of . For that purpose, we recursively make use of the following matrix equality

 (I−γPπ)−1=(I−γPμ)−1+γ(I−γPμ)−1(Pπ−Pμ)(I−γPπ)−1,

which can be derived either from matrix inversion equality or directly verified. Since , we can use the previous equality to get

 Qπ =(I−γPμ)−1R+γ(I−γPμ)−1(Pπ−Pμ)(I−γPπ)−1R =Qμ+γ(I−γPμ)−1(Pπ−Pμ)Qπ.

Next, we recursively apply the equality times,

 Qπ

Now if then we can bound the sup-norm in of the above term as

 ∥γ(I−γPμ)−1(Pπ−Pμ)∥∞=γ1−γ||π−μ||1<1,

thus the -th order residual term vanishes when . As a result, the limit is well defined and we deduce

 Qπ

## Appendix C Proof of Theorem 2

###### Proof.

To derive the monotonic improvement theorem for generalized TRPO, it is critical to bound . We achieve this by simply bounding each term separately. Recall that from Appendix A.1 we have . Without loss of generality, we first assume for ease of derivations.

This leads to a bound over the residuals

Since we have the equality for we can deduce the following monotonic improvement,

 (14)

To write the above statement in a compact way, we define the gap

To derive the result for general , note that the gap has a linear dependency on . Hence the general gap is

which gives produces the monotonic improvement result (Eq. 10) stated in the main paper. ∎

## Appendix D Proof of Theorem 3

###### Proof.

It is known that for , replacing by in the estimation can potentially reduce variance (Schulman et al., 2015, 2017) yet keeps the estimate unbiased. Below, we show that in general, replacing by renders the estimate of unbiased for general .

As shown above and more clearly in Appendix F, can be written as

 (15)

Note that for clarity, in the above expectation, we omit an explicit sequence of discounted visitation distributions (for detailed derivations of this sequence of visitation distributions, see Appendix F). Next, we leverage the conditional expectation with respect to to yield

 LK(π,μ) (16)

The above derivation shows that indeed, replacing by does not change the value the expectation, while potentially reducing the variance of the overall estimation. ∎

## Appendix E Proof of Theorem 4

###### Proof.

From the definition of the return off-policy evaluation operator , we have

 Rπ,μ1Q

Thus is a linear operator, and

Applying this step times, we deduce

Applying the above operator to we deduce that

which proves our claim. ∎

## Appendix F Alternative derivation for Taylor expansions of RL objective

In this section, we provide an alternative derivation of the Taylor expansion of the RL objective. Let . In cases where (e.g., for the trust-region case), . To calculate using data from , a natural technique is employ importance sampling (IS),

 J(π)

To derive Taylor expansion in an intuitive way, consider expanding the product , assuming that this infinite product is finite. Assume all with some small . A second-order Taylor expansion is

 (17)

Now, consider the term associated with ,

 (18)

Note that in the last equality, the factor is absorbed into the discounted visitation distribution . It is then clear that this term is exactly the first-order expansion shown in the main paper.

Similarly, we could derive the second-order expansion by studying the term associated with .

 (19)

Note that similar to the first-order expansion, the discount factor is absorbed into the discounted visitation distribution and respectively. Here note that the second discounted visitation distribution is instead of — this is because by construction and we need to sample the second state conditional on the time difference to be . The above is exactly the second-order expansion .

By a similar argument, we can derive expansion for all higher-order expansion by considering the term associated with . This would introduce


text:
# Properties of Multiplication of Fractional Numbers

The properties of multiplication of fractional numbers are discussed here.

Property 1: If two fractional numbers are multiplied in either order, the product remains the same.

For Example:

(i) $$\frac{2}{3}$$ × $$\frac{7}{5}$$

= $$\frac{2 × 7}{3 × 5}$$

= $$\frac{14}{15}$$

And now if you interchange the place of the fractional numbers the product does not change.

$$\frac{7}{5}$$ × $$\frac{2}{3}$$

= $$\frac{7 × 2}{5 × 3}$$

= $$\frac{14}{15}$$

We observe that the product in both the cases are same.

So, $$\frac{2}{3}$$ × $$\frac{7}{5}$$ = $$\frac{7}{5}$$ × $$\frac{2}{3}$$.

Note: From the above example we understand that, changing the order of fractional numbers does not change the product.

(ii) (4$$\frac{2}{3}$$ × 5$$\frac{1}{3}$$) × $$\frac{1}{5}$$ = 4$$\frac{2}{3}$$ (5$$\frac{1}{3}$$ × $$\frac{1}{5}$$)

Property 2: If a fractional number is multiplied by one, the product is the fractional number itself.

For Example:

(i) $$\frac{7}{9}$$ × 1

= $$\frac{7}{9}$$ × $$\frac{1}{1}$$

= $$\frac{7 × 1}{9 × 1}$$

= $$\frac{7}{9}$$

So, we observe that a fraction multiplied by 1 is the fraction itself.

(ii) $$\frac{5}{8}$$ × 1

= $$\frac{5}{8}$$ × $$\frac{1}{1}$$

= $$\frac{5 × 1}{(8 × 1}$$

= $$\frac{5}{8}$$

(iii) $$\frac{15}{19}$$ × 1

= $$\frac{15}{19}$$ × $$\frac{1}{1}$$

= $$\frac{15 × 1}{(19 × 1}$$

= $$\frac{15}{19}$$

Property 3: If a fractional number is multiplied by zero, the product is zero.

For Example:

(i) $$\frac{3}{11}$$ × 0

= $$\frac{3 × 0}{11}$$

= 0

(ii) $$\frac{7}{15}$$ × 0

= $$\frac{7 × 0}{15}$$

= 0


text:
# What are the minimum and maximum prerequisites to study Stochastic Processes?

Suppose, I never studied random variables. This is the syllabus:

Lecture contents

Review of important notions of probability theory (4h).

A few remarks on stochastic processes : Definition of a Stochastic process, Notion of the state and realization of the process, Classification of Stochastic processes.Probability and Moment generating functions and their properties (2h).

Branching processes Galton process :Probability of extinction , Applications in demography and nuclear physics (4h).

Poisson processes and its applications. Exponentioal distribution and its properties, . Poisson Process and their properties : Distribution of periods between successive calls , Summing independent Poisson processes, Conditional distributions of inter-arrival times , Generalizations of Poisson processes , nonuniform distribution , Composed Poisson process (6h).

Simple queuing systems: M/M/c systems without and with queue.: Probability of blocking, probability of the delay and average waiting time (4h).

Renewal Processes (6h).

Review (4h).

What are the minimum and maximum prerequisites to study Stochastic Processes?

That is, what things should I know beforehand to start studying Stochastic Processes?

• If you have never studied random variables, it will be close to impossible. What is your background in probability theory? – luka5z Oct 6 '15 at 21:02
• If you have a "basic working knowledge" of random variables. rest assured that you do not have the necessary background to get a "basic working knowledge" of stochastic processes in a week. This is, for the most part, true even if you actually took a course based on a text with a title such as Probability and Statistics for Engineers (or for Scientists and Engineers). These types of books are mostly about statistics, typically consist of just a lot of cookbook formulas about buzzwords such as $t$-tests and $z$-scores and $p$-values. – Dilip Sarwate Oct 6 '15 at 21:47
• @anonymouse - your best try is wikipedia en.wikipedia.org/wiki/Stochastic_process – luka5z Oct 7 '15 at 7:09
• Are you good with basic probability theory?? Info is required? If no then follow this if you would like to start from the basics link math.stackexchange.com/questions/31838/… . You dont have to study whole things in the text book but the central concepts(be good at it) and skip other things(if you dont have sufficient time) – Jasser Dec 28 '15 at 15:17

You have basically answered your own question. Random variables are the single most important prerequisite to start learning about processes. And underneath that, basic probability theory (the infinite kind, based on $\sigma$-algebras).

Apart from that it depends on what kinds of processes are going to be the focus of your study or work. For finite-state, discrete-time processes some matrix calculus might come in handy. For continuous-time real-valued processes you want to review calculus and the properties of the real numbers.

Based on the course description I recommend the first chapters (say, 2 and 3) of Arnold Allen, Probability, Statistics, and Queueing Theory. A bit superficial on the basic probability stuff but you don't want to spend 20 hours preparing for a 4h lecture, right? Bonus: chapters 4 and 5 seem to match some of your course content rather nicely.

• " first chapters of Arnold Allen, Probability, Statistics, and Queuing Theory. " --- First Chapter or First Part? – user6704 Dec 31 '15 at 1:49
• I have made the recommendation more specific now. – Justpassingby Dec 31 '15 at 8:02

I had been absent from the university for >10 years and made a comeback this semester starting with a course about stochastic processes.

I bought the book "Schaum's Outline of Probability, Random Variables, and Random processes" by Hwei P. Hsu and made the exercises of the first 4 chapters and some exercises in the other chapters and after that i had no problems during the course.

• "some exercises in the other chapters " --- which other chapter? – user6704 Dec 27 '15 at 17:39
• @anonymous Well, those chapters are usually part of the course and not really necessary to do before the course starts but if you want to be well prepared spend some time studying chapter 5, 6 and 9. 5 is Random processes, 6 is Analysis and processing of random processes and 9 is Queueing theory. – JKnecht Dec 27 '15 at 18:16
• @anonymous I can add that the book has solutions to almost every exercise which is nice. – JKnecht Dec 27 '15 at 18:16
• Just curious, why did you need to come back to the university after >10 years? – user6704 Dec 27 '15 at 22:53
• @anonymous I didnt need to. I left the university because i got a good job. Then i had a career there and retired just a few months ago. Now i am back at the university just because its fun. – JKnecht Dec 28 '15 at 7:23


text:
# Divergence of electric field of point dipole

Am I correct to say that since the divergence of the following E field

$$\mathbf{E} = \frac{1}{4\pi\epsilon_0}\left[ \frac{3(\mathbf{p_0}\cdot\hat{r})\hat{r}-\mathbf{p_0}}{r^3} \right] - \frac{1}{3\epsilon_0}\mathbf{p_0} \delta^3(\mathbf{r})$$

satisfies Gauss’ law for electricity where

$$\rho=-\mathbf{p_0}\cdot\nabla\delta^3(\mathbf{r})$$

for an electric point dipole, the usual electric field associated with the point dipole

$$\mathbf{E} = \frac{1}{4\pi\epsilon_0}\left[ \frac{3(\mathbf{p_0}\cdot\hat{r})\hat{r}-\mathbf{p_0}}{r^3} \right]$$

is thus incomplete?

Incidentally, what is the divergence of the usual electric field of point dipole?

• If I'm understanding your question, the usual electric field you've written is the correct one everywhere except for the origin and usually that's where we're interested to look. The divergence will then be, as you said, given by Gauss' law, just for $\boldsymbol{r}\neq0$ Jan 19, 2021 at 12:11
• Does that mean that if I want to include $\mathbf{r}=0$, I will need the Dirac term? Also, what is the divergence of the usual electric field?
– user161157
Jan 19, 2021 at 12:16
• The delta term fixes spatial integrals of the electric field to physically correct value (so they are the limit of what we get with physical non-point dipoles when the distance is shrinked to 0). You only ever need this term when you calculate spatial integrals of electric field vector (for example, for spatial averaging). Jan 19, 2021 at 12:49
• @JánLalinský That sounds like an answer. Jan 19, 2021 at 13:57

## 1 Answer

As @Ján Lalinský mentions, the addition of the $$\delta$$ contribution comes out when discussing a volume integral of the field. A finer point that I think is worth mentioning is that the addition of the $$\delta$$ term comes as relevant once we set the volume integral of the singular non-$$\delta$$ term to $$0$$ (you can see this in the discussion at the end of section $$4.1$$ in Jackson's Classical Electrodynamics book). As mentioned there, this makes sense when looking at the angular integration, but it is less obvious when one looks at the radial integration.

When asking about the divergence of the field, it is somehow more straightforward to talk about the Laplacian of the point dipole potential, which you can find in this answer. When you compute the electric field via $$-\nabla \Phi$$ from that potential, you find only the non-$$\delta$$ term, as long as you ignore all subtleties with the singular point at the location of the dipole. Interestingly enough, when computing $$-\nabla \Phi$$, it is possible to somehow motivate the $$\delta$$ term, by splitting up double partial derivatives $$\partial_i \partial_j$$ into a trace and a traceless terms, in which the trace brings out a Laplacian of a $$\frac{1}{r}$$. In the end, this is an attempt to fully capture the singular behavior of the expression.

You can also address the issue of the divergence with a tangential computation: by definition of the dipole moment, $$\mathbf{p} = \int \mathbf{x'} \rho(\mathbf{x'}) d^3x'$$. Demanding $$\mathbf{p} = \epsilon_0 \int \mathbf{x'} \,\nabla'\cdot \mathbf{E}(\mathbf{x'}) d^3x'$$ does give you another angle to tackle the problem and test your expressions for the field and charge density. This is another integral that is ultimately related to the volume integral of the field. It also touches on the importance of preserving the contributions from the boundary terms. By imposing this constraint and using the prescription set for the volume integral, you can find some consistency in the end.


text:
• anonymous
Wouldn't this be B? Joe earns $40 for playing music on Saturday nights at the local coffee house. He makes an average of$3.25 in tips per hour. Write the function of Joe's earnings, and solve for how much he would make after 5 hours. y = 3.25x + 40; $43.25 y = 3.25x + 40;$56.25 y = 40x + 3.25; $58.25 y = 40x + 3.25;$203.25
Algebra

Looking for something else?

Not the answer you are looking for? Search for more explanations.


text:
# Thread: mechanics help , a block is on an incline need to find the period and amplitude of mo

1. ## mechanics help , a block is on an incline need to find the period and amplitude of mo

a block P of mass m lies on a smooth plane AB that is inclined at angle of (alpha) to the horizontal. the block is attatched to the bottom of the plane,A, by a spring of stiffness 2k and natural length Lo . a second model spring of stifness 3k and natural length 2Lo is now attatched to the block, and its other end is fixed at B at a distance 5Lo up the plane from A.
a) the block is initially released from rest from the equilibrium position. i need to find the particular solution of the differential equation that satisfies these initial conditions.
b)i need to write down the period and amplitude of the subsequent motion

if some one could please go through it with me step by step i would be very thankfull .

2. Originally Posted by longy
a block P of mass m lies on a smooth plane AB that is inclined at angle of (alpha) to the horizontal. the block is attatched to the bottom of the plane,A, by a spring of stiffness 2k and natural length Lo . a second model spring of stifness 3k and natural length 2Lo is now attatched to the block, and its other end is fixed at B at a distance 5Lo up the plane from A.
a) the block is initially released from rest from the equilibrium position. i need to find the particular solution of the differential equation that satisfies these initial conditions.
b)i need to write down the period and amplitude of the subsequent motion

if some one could please go through it with me step by step i would be very thankfull .
I don't have much time, but I'll give you an outline.

Let the origin for x be at the $L_0$ position for spring A. Then the "stretch" of spring A is x and the "stretch" of spring B is 2L_0 - x. I am calling down the slope positive.

Set up your Newton's second to find a. Remember we have a weight term to consider here depending on the angle, but the spring forces don't. So we get
$a = \frac{1}{m} \left [ mg~sin(\theta) + 2k(L_0 + x) - 3k(2L_0 - x) \right ]$

So
$\frac{d^2x}{dt^2} - \frac{5k}{m} \cdot x = g~sin(\theta) - \frac{4kL_0}{m}$
(Double check each of these to make sure I'm not making any mistakes. As I said, I'm running through this quickly.)

The right hand side is just a constant, so the particular solution will be in the form of x = C.

To get the period etc. solve the homogeneous equation
$\frac{d^2x}{dt^2} - \frac{5k}{m} \cdot x = 0$

This will be a sine function of the form $A~sin(kx + \phi )$ which you can get your period and amplitude information from.

-Dan


text:
# How to configure G-API launcher¶

G-API launcher runs model using graph-based evaluation approach proposed by OpenCV G-API module and OpenVINO as inference backend.

For enabling G-API launcher you need to add framework: g-api in launchers section of your configuration file and provide following parameters:

• device - specifies which device will be used for infer. Supported: CPU, GPU, FPGA, MYRIAD, HDDL, Heterogeneous plugin as HETERO:target_device,fallback_device and Multi device plugin as MULTI:target_device1,target_device2.

If you have several MYRIAD devices in your machine, you are able to provide specific device id in such way: MYRIAD.<DEVICE_ID> (e.g. MYRIAD.1.2-ma2480)

It is possible to specify one or more devices via -td, --target devices command line argument. Target device will be selected from command line (in case when several devices provided, evaluations will be run one by one with all specified devices).

• model - path to xml file with model for your topology or compiled executable network.

• weights - path to bin file with weights for your topology (Optional, the argument can be omitted if bin file stored in the same directory with model xml or if you use compiled blob).

• adapter - approach how raw output will be converted to representation of dataset problem, some adapters can be specific to framework. You can find detailed instruction how to use adapters here.

• outputs - list of model output names. You also should specify all inputs for your model with their shapes to write inputs, using specific parameter: inputs. Each input description should has following info:

• name - input layer name in network

• type - type of input values, it has impact on filling policy. Available options:

• CONST_INPUT - input will be filled using constant provided in config. It also requires to provide value.

• IMAGE_INFO - specific key for setting information about input shape to layer (used in Faster RCNN based topologies). You do not need to provide value, because it will be calculated in runtime. Format value is list with N elements of the form [H, W, S], where N is batch size, H - original image height, W - original image width, S - scale of original image (default 1).

• ORIG_IMAGE_INFO - specific key for setting information about original image size before preprocessing.

• PROCESSED_IMAGE_INFO - specific key for setting information about input size after preprocessing.

• IGNORE_INPUT - input which should be stayed empty during evaluation.

• INPUT - network input for main data stream (e. g. images). If you have several data inputs, you should provide regular expression for identifier as value for specifying which one data should be provided in specific input.

• shape - shape of input layer described as comma-separated of all dimensions size except batch size.

Optionally you can determine layout in case when your model was trained with non-standard data layout (For G-API default layout is NHWC) and precision (Supported precisions: FP32 - float, FP16 - signed shot, U8 - unsigned char, U16 - unsigned short int, I8 - signed char, I16 - short int, I32 - int, I64 - long int).

G-API launcher config example:

launchers:
- framework: g-api
device: CPU
model: path_to_model/alexnet.xml
weights: path_to_weights/alexnet.bin
inputs:
- name: 'data'
type: INPUT
shape: 3, 227, 227
outputs:
- prob
adapter: classification


text:
# General linear group of a subspace

I am confused by the definition of the general linear group of a vector space. Let $V$ be a vector space of dimension $3$ and $W$ be a subspace of $V$ with dimension $2$. (For example, $V$ can be the three dimensional Euclidean space, and $W$ can be a plane in $R^3$.) We can observe that elements of $V$ and $W$ are going to be $3$-tuples.

When the general linear group of a vector space is defined, the size of matrix elements is given by the dimension of the vector space. So $GL(V,R)$ will have invertible real matrices of size $3\times3$ as elements. However, I am really confused when I am looking at $GL(W,R)$. Using the definition it seems its elements are invertible $2\times2$ real matrices. However, these matrices should be able to act on elements of $W$, so if $A$ is in $GL(W,R)$ and $w$ is in $W$ (note that $w$ is a vector in $R^3$). Then $A*w$ should be well defined, but it's not because $A$ is $2\times2$ and $w$ is $3\times1$.

I am pretty sure the elements of $GL(W,R)$ should be $3\times3$ matrices. However, how does this fulfill the definition of the general linear group of the finite dimensional vector space $W$? The dimension of $W$ is certainly not $3$. There is something I am misinterpreting, please any help would be appreciated. Thanks!

• $GL(W)$ is the set of linear mappings from $W$ to $W$, not a set of matrices. Once a particular basis for $W$ has been selected, then you can of course write all those linear mappings in that basis, and you'll get the set of all invertible $2 \times 2$ matrices. When $V$ is a general three-dimensional vector space, the same problem arises: you need to select a basis in order to identify elements of $GL(V)$ with matrices. When $V = R^3$, you have a ready-made choice: the standard basis. When $W$ is a plane in $R^3$, there's no basis that is really singled out that way. – user49640 Jun 2 '17 at 3:46
• The set of linear maps is a set of matrices.... but... After re-reading your original post. $W$ has a two dimensional basis, and so a $2\times 2$ matrix $A$ can act on vectors in the vector space $W$ even though $W$ is represented by 3-tuples in the basis of $V$ – Doug M Jun 2 '17 at 3:50
• Sorry, I should have said the set of bijective linear mappings. That was a mistake. – user49640 Jun 2 '17 at 7:24

As others have pointed out in the comments, $\DeclareMathOperator{\GL}{GL} \GL(V)$ is the set of invertible linear maps $V \to V$; only by choosing a basis do we get an isomorphism $\GL(V) \cong \GL_3(\mathbb{R})$. Nonetheless, I think there is a way in which your question does make sense. Given $\varphi \in \GL(W)$, $\varphi: W \to W$ is only defined on $W$, not on all of $V$, but there is a way to extend $\varphi$ to all of $V$.
Choose a basis $\mathcal{B} = \{e_1, e_2\}$ for $W$ and let $[\varphi]_\mathcal{B}$ denote the matrix representation of $\varphi$ with respect to $\mathcal{B}$. By the basis extension theorem, we can extend the basis $\mathcal{B} = \{e_1, e_2\}$ of $W$ to a basis $\mathcal{C} = \{e_1, e_2, e_3\}$ of $V$. Define $\widetilde{\varphi} : V \to V$ by $\widetilde{\varphi}(e_i) = \varphi(e_i)$ for $i = 1,2$ and $\widetilde{\varphi}(e_3) = e_3$, i.e., $\widetilde{\varphi}$ acts as $\varphi$ on $W$ and $\widetilde{\varphi}$ fixes $e_3$. Then the matrix representation of $\widetilde{\varphi}$ with respect to $\mathcal{C}$ is block-diagonal: $$[\widetilde{\varphi}]_\mathcal{C} = \left( \begin{array}{c|c} [\varphi]_\mathcal{B} & 0 \\ \hline 0 & 1 \end{array} \right) \, .$$
So in some sense we have have embedded $\GL(W) \subseteq \GL(V)$, which in coordinates amounts to embedding $2 \times 2$ matrices in $3 \times 3$ matrices. (More generally, this sort of block diagonal representation will arise whenever there is a $\varphi$-invariant subspace.)
• Your $\widetilde\varphi$ is not linear! In fact, you should never define a linear map piecewise (but rather by choosing complements or bases). – Claudius Jun 2 '17 at 7:10
• @user218931 Yes, you're right, my attempt to write $\widetilde{\varphi}$ without coordinates was incorrect. The definition I gave in coordinates was what I intended, so I've kept that. – Richard D. James Jun 2 '17 at 8:01


text:
#### DOCUMENTA MATHEMATICA, Extra Volume: Kazuya Kato's Fiftieth Birthday (2003), 761-788

Christophe Soulé

A Bound for the Torsion in the $K$-Theory of Algebraic Integers

Let $m$ be an integer bigger than one, $A$ a ring of algebraic integers, $F$ its fraction field, and $K_m (A)$ the $m$-th Quillen $K$-group of $A$. We give a (huge) explicit bound for the order of the torsion subgroup of $K_m (A)$ (up to small primes), in terms of $m$, the degree of $F$ over $\mathbf Q$, and its absolute discriminant.

2000 Mathematics Subject Classification:

Keywords and Phrases:

Full text: dvi.gz 35 k, dvi 87 k, ps.gz 487 k, pdf 201 k.


text: each series of the transition elements from left to right. Describe the variable oxidation state of 3d series elements. The oxidation states of 4d and 5d metals vary from +3 for Y and La to +8 for Ru and Os. ii) Which element has the highest m.p? Long Answer Type Questions. • Although the first ionisation enthalpy, in general, increases, the magnitude of … iii) Which elements shows only +3 oxidation state ? Electron arrangements of the first transition series; 3. In some cases, the average oxidation state of an element is a fraction, such as 8/3 for iron in magnetite (Fe 3 O 4). Maximum number of oxidation states of transition metal is derived from the following configuration [MP PET/PMT 1988] A) ns electron done clear. The highest oxidation state of 4d and 5d elements are found in their compounds with the higher electronegative elements like O, F and Cl. The atomic number of an element is 22. (i) fluorine is more electronegative than oxygen. Transition elements show magnetic moment due to spin and orbital motion of electrons. * Variable oxidation states: a) Transition metals make use of their ns and (n-1) d electrons for bonding as their energies are close. (ii) Manganese exhibits the highest oxidation state of+7 among the 3d series of transition elements Answer: (i) Refer Ans. In transition elements, the oxidation state can vary from +1 to the highest oxidation state by removing all its valence electrons. The highest observed oxidation state was that of iridium in [IrO4]+, which was +9, but predictions were made about platinum having an even higher (+10) in [PtO4]2+. The highest known oxidation state is +8 in the tetroxides of ruthenium, xenon, osmium, iridium, hassium, and some complexes involving plutonium; the lowest known oxidation state is −4 for some elements in the carbon group. The tendency to show highest oxidation state increases from Sc to Mn, then decreases due to pairing of electrons in 3d subshell. The maximum oxidation number in the first row of transition metals is equal to the number of valence electrons from scandium (+3) up to manganese (+7). Introduction to General Properties of the Transition Elements. Often, the highest oxidation state of a transition element … Question 64. • Ionisation enthalpies give some guidance concerning the relative stabilities of oxidation states. The highest oxidation state exhibited by it in its compounds is [MP PMT 1996] A) 1 … All transition metals except the first and the last in the series exhibit a great variety of oxidation states. The charge of a transition element and its oxidation state are equal. For nonmetals like N, O, P, S, Cl, Br and I, the lowest possible state is the valency of the element with a minus sign. Some of these oxidation states are common because they are relatively stable. Elements in first transition series form ions … The elements with incompletely filled d-subshell in their ground state or most stable oxidation state are named as D-block elements.They are additionally named as transition elements.The partially filled subshells incorporate the (n-1) d subshell.All the d-block elements have a similar number of electrons in the furthest shell. Elements (metals) of the first transition series; 2. Also, in transition elements, the oxidation states differ by 1 (Fe 2+ and Fe 3+; Cu + and Cu 2+). Iron has two common oxidation states (+2 and +3) in, for example, Fe 2+ and Fe 3+. Compare the stability of +2 oxidation state for the elements of the first transition series. Reason (R): Osmium is a 5d-block element. when the number of unpaired valence electrons increases, the d-orbital increase & the highest oxidation state increases. Iron. The significance of the 4s subshell; 5. Examples of variable oxidation states in the transition metals. From the table, TiX 4, VF 5 and CrF 6 have the highest oxidation numbers.. Some properties of the first row transition elements; 4. Solution: (b) The highest oxidation state of osmium is +8 due to its ability to expand their octet by using its all 8 electrons. The stability of a particular oxidation state depends upon the nature of the element with which the transition metal forms the compound. Also, in transition elements, the oxidation states differ by 1 (Fe 2+ and Fe 3+; Cu + and Cu 2+). The highest oxidation state ever achieved by an element is +8. to Q.9 (ii). The stabilization of the highest oxidation states of transition Cr042-, metals as their complex oxyanion is well known, e.g., TiOa2-, vo43-, The Mn04-, Fe04*-,as well as the neutral Hu04and oso4. 9. The highest oxidation state +7, for manganese is not seen in simple halides, but MnO 3 F is known.. VF 5 is stable, while the other halides undergo hydrolysis to give oxohalides of the type VOX 3.. Fluorine stabilises higher oxidation states either because of its higher lattice energy or higher bond enthalpy. (iii) fluorine stabilises lower oxidation state. values are intermediate to those of s-block & p-block elements. The elements of group 12 i.e., Zinc, Cadmium, and Mercury are generally not regarded as transition elements as their atoms and all ions formed have completely filled d-orbitals i.e., these do not have partially filled d-orbitals in atomic state or common oxidation state (Zn 2+, Cd 2+, Hg 2+). Variable oxidation states are mainly shown by many nonmetals and most transition metals. In non-transition elements, the oxidation states differ … The highest oxidation states of the 3d transition metal elements. • I.E. VARIABLE OXIDATION STATE. oxidation number or state is defined as the charge present on an atom or ion. Highest oxidation state of manganese in fluoride is +4 (MnF 4) but highest oxidation state in oxides is +7 (Mn 2 O 7) because (i) fluorine is more electronegative than oxygen. Oxidation numbers (states) of transition metals; 6. The most common oxidation states of the first series of transition metals are given in the table below. Question 63. Iron forms oxidation states from 2+ to 6+. Fe 3+ and Fe 2+, Cu 2+ and Cu +. As an example, $\ce{[Fe(CO)4]^2-}$ with an iron oxidation state of $\mathrm{-II}$ is known. Furthermore, the oxidation states change in units of one, e.g. Highest oxidation state of manganese in fluoride is +4 (MnF 4) but highest oxidation state in oxides is +7 (Mn 2 O 7) because _____. (a) (i) In transition elements, the oxidation states can vary from +1 to highest oxidation state by removing all its valence electrons and the oxidation states differ by 1, e.g., Fe 2+ and Fe 3+ while in p-block elements, the oxidation states differ by 2, e.g., +2 and +4 or +3 and +5, etc. in case of transition metals, there are five orbitals in the d subshell . Answer the following: i) Write the element which shows a maximum number of oxidation states.Give reason. Colour and transition metal compounds; 7. because of these properties Oxygen are able to oxidise the metal to its highest oxidation state. All elements of the first transition series have oxidation state (+2) because after losing the electrons of (4s) sublevel at first (except for scadium), while in the higher oxidation states they lose the electron of (3d) in sequence.. For example, the oxidation state of Iron is between 2+ to 6+. Assertion (A): The highest oxidation state of osmium is +8. The highest possible oxidation state depends on the given element as well as on the used ligand system and highly electronegative ligands generally favor high-oxidation numbers. Question 8.14. b) Most common oxidation state is +2 c) Highest common oxidation state is + 2 d) Highest oxidation state shown in 4d series in +8 by Ruthenium (less stable). for example: RuO 4, OsO 4 and WCl 6. Download : Download full-size image; Fig. However, it decreases in the latter elements. (b) In the highest oxidation states, the transition metals show basic character and form cationic complexes. Answer: 1. (ii) fluorine does not possess d-orbitals. In the second and third rows, the maximum oxidation number is that of ruthenium and osmium (+8). The elements scandium through manganese (the first half of the first transition series), show the highest oxidation state as their valence shell shows loss of all of the electrons in both the s and d orbitals. 2.5 Transition Metals General properties of transition metals transition metal characteristics of elements Sc Cu arise from an incomplete d sub-level in atoms or ions Sc 1s22s22p63s23p6 4s23d1 Ti 1s22s22p63s23p6 4s23d2 V 1s22s22p63s23p6 4s23d3 Cr 1s22s22p63s23p6 4s13d5 Mn 1s22s22p63s23p6 4s23d5 Fe 1s22s22p63s23p6 4s23d6 Co 1s22s22p63s23p6 4s23d7 Transition metals and complex ion formation; 8. Compare the stability of +2 oxidation state for the elements of the first transition series. Solution: The common oxidation state of 3d series elements is + 2 which arises due to participation of only 4s electrons. (ii) Cu 2+ (aq) is much more stable than Cu + (aq). The elements Sc through Mn show display the highest oxidation states because they lose all the s and d orbitals electrons from valence shell. Some oxidation states, however, are more common than others. Oxygen is strong oxidising agent due to its high electronegtivity and smaller size. Of the familiar metals from the main groups of the Periodic Table, only lead and tin show variable oxidation state to any extent. Highest experimentally known oxide species, highest experimentally known fluoride species, doubtful reports of fluoride species. The highest oxidation states are found in compounds of fluorine and oxygen. In transition elements, the oxidation state can vary from +1 to the highest oxidation state by removing all its valence electrons. Among transition metals, the highest oxidation state is exhibited in oxoanions of a metal. All of this complicates the analysis strongly. (ii) Mn has electronic configuration (Ar) 4s2 3d5 and all the electrons in ‘s’ as well as ‘d’ orbitals can take part in bond formation, therefore, it shows + 7 highest oxidation state… Efforts to explain the apparent pattern in this table ultimately fail for a combination of reasons. In non-transition elements, the oxidation states differ by 2, for example, +2 and +4 or +3 and +5, etc. The table below shows the different oxidation states of the first row of transition metals and the colours of their solutions. The elements of 3d transition series are given as: Sc Ti V Cr Mn Fe Co Ni Cu Zn. All of them form ions with a charge of +2 and +3 but many of them form more oxidation states up to +7. Until much more research has been performed, you should probably not attempt to predict maximum and minimum oxidation states of these elements. The first transition metal Scandium exhibits only +3 oxidation state, but all other transition elements exhibit variable oxidation states by loosing electrons from (n-l)d orbital and ns orbital as … The highest oxidation states of the 4d transition metal elements. One of the most striking features of the transition elements is that the elements usually exist in several different oxidation states. 19. NCERT DC Pandey Sunil Batra HC Verma Pradeep Errorless. (a) In addition to the normal oxidation states, the zero oxidation state is also shown by these elements in complexes. , then decreases due to its high electronegtivity and smaller size Mn show display the highest oxidation state ever by. And WCl 6 6 have the highest oxidation states are found in of! Charge of +2 oxidation state for the elements of the familiar metals the. Case of transition metals show basic character and form cationic complexes +2 and +3 ) the. First series of the first row transition elements ; 4 unpaired valence electrons only +3 oxidation state increases from to... The stability of +2 oxidation state ever achieved by an element is +8 doubtful. Sc through Mn show display the highest oxidation states of the first transition series form with! Research has been performed, you should probably not attempt to predict maximum and minimum states. Its highest oxidation state of+7 among the 3d transition metal elements Cu 2+ and Cu + ( )! Table, TiX 4, VF 5 and CrF 6 have the highest oxidation state of+7 among the series! Oxygen are able to oxidise the metal to its high electronegtivity and smaller size states.Give reason the familiar metals the! Given in the highest oxidation numbers differ by 2, for example: RuO 4, OsO and! ( R ): the common oxidation states are mainly shown by these elements +8. Shows the different oxidation states in the transition metal elements magnetic moment due its! 5D-Block element number is that of ruthenium and osmium ( +8 ) particular oxidation state by removing all its electrons... +3 ) in, for example: RuO 4, OsO 4 and WCl 6 to predict maximum and oxidation... Batra HC Verma Pradeep Errorless fluorine is more electronegative than oxygen some guidance concerning the relative stabilities oxidation! Increase & the highest oxidation state is also shown by many nonmetals and most transition metals as: Ti! Ncert DC Pandey Sunil Batra HC Verma Pradeep Errorless, OsO 4 and WCl 6 vary from +3 for and! Dc Pandey Sunil Batra HC Verma Pradeep Errorless ions … each series transition. Lead and tin show variable oxidation states are mainly shown by these elements complexes. Wcl 6 of s-block & p-block elements 4s electrons elements Sc through Mn show display the highest state! And 5d metals vary from +3 for Y and La to +8 for Ru and.... Osmium is a 5d-block element state of 3d series elements is + which! Oso 4 and WCl 6 ) Manganese exhibits the highest oxidation states are found in compounds of fluorine oxygen...: osmium is a 5d-block element more research has been performed, you should probably attempt... To Mn, then decreases due to its highest oxidation numbers ( states ) of the series. & p-block elements any extent in, for example, +2 and or! Decreases due to pairing of electrons in 3d subshell Sc to Mn, decreases. Among transition metals ; 6 of variable oxidation states s-block & p-block elements between to! Most common oxidation states of the 3d transition metal elements 2+ and Fe 2+, Cu 2+ Fe! Then decreases due to participation of only 4s electrons Write the element with which transition! Than oxygen, then decreases due to participation of only 4s electrons charge present on atom. ): osmium is +8 main groups of the familiar metals from the main groups of the metal. Series form ions … each series of transition metals ; 6 is much more stable than Cu + aq!