text:
# Electric Current, Voltage, and Resistance Overview | Three Basic Electrical Quantities

Want create site? Find Free WordPress Themes and plugins.

Electric Current, voltage, and resistance are three of the fundamental electrical properties. Stated simply,

current: is the directed flow of charge through a conductor.
Voltage: is the force that generates the current.
Resistance: is an opposition to current that is provided by the material, component, or circuit.

Electric Current, Voltage, and resistance are the three primary properties of an electrical circuit. The relationships among them are defined by the fundamental law of circuit operation, called Ohm’s law.

## Electric Current

As you know, an outside force can break an electron free from its parent atom. In copper (and other metals), very little external force is required to generate free electrons. In fact the thermal energy (heat) present at room temperature (220C) can generate free electrons. The number of electrons generated varies directly with temperature. In other words, higher temperatures generate more free electrons.

The motion of the free electrons in copper is random when no directing force is applied. That is, the free electrons in copper are random when no directing force is applied. That is, the free electrons move in every direction, as shown in Figure 1. Since, the free electrons are moving in every direction, this net flow of electrons in any direction is zero.

Figure 1 Random electron motion in copper

Figure 2 Illustrates what happens when an external force causes all of the electrons to move in the same direction. In this case, a negative potential is applied to one end of the copper and a positive potential is applied to the other. As a result, the free electrons all move from negative to positive, and we can say that we have a directed flow of charge (electrons). This directed flow of electrons is called electric current.

Figure 2 Directed electron motion in copper.

Let’s look at what happens on a larger scale when electron motion is directed by an outside force. In Figure 3, the negative potential directs electron flow (current) toward the positive potential. The current passes through the lamp, causing it to produce light and heat. The more intense the current (meaning the greater its value), the greater the light and heat produced by the bulb.

Figure 3 Current through a basic lamp circuit.

Electric Current is represented in formulas by the letter I (For intensity). The intensity of current is determined by the amount of charge flowing per second. The greater the flow of charge per second, the more intense the current.

Coulombs and Amperes

The change on a single electron is not sufficient to provide in a practical unit of measure for charge. Therefore, the Coulomb (C) is used as the basic unit of charge One coulomb equals the total charge on 6.25 × 1018 electrons. When, one coulomb of charge passes a point in one second, we have one Ampere (A) of electric current. In other words,

$\begin{matrix}1\text{ ampere=1 Coulomb per second} \\Or \\1\text{ A=1 C/s} \\\end{matrix}$

The total current passing a point (in amperes) can be found by dividing the total charge ( in coulombs) by the time ( in seconds) . By Formula

$\begin{matrix}I=\frac{Q}{t} & {} & \left( 1 \right) \\\end{matrix}$

Where

I= the intensity of electric current in amperes

Q= the total charge, in coulombs

T= the time it takes the charge to pass a point, in seconds

This relationship is illustrated in Example 1.

Example 1

Three coulombs of charge pass through a copper wire every second. What is the value of electric current?

Solution

Using equation 1, the current is found as

$I=\frac{Q}{t}=\frac{3C}{1s}=3{}^{C}/{}_{s}=3A$

Example 1 is included here to help you understand the relationship between amperes, coulombs, and seconds. In practice, electric current is not calculated using Equation 1 because you cannot directly measure coulombs in charge. As you will learn, there are far more practical ways to calculate current.

Two Theories: Conventional Current and Electron Flow.

There are two theories that describe electric current, and you will come across both in practice.

The Conventional Current theory defines current as the flow of charge from positive to negative. This theory is called “conventional current” because it is the older of the two approaches to current, and for many years was the only one taught outside of military and trade schools.

Electron Flow is the newer of the two current theories. Electron flow theory defines current as the flow of electrons from negative to positive.

The two electric current theories are contrasted in Figure 4. Each circuit contains a battery and a lamp. Conventional current begins at the positive battery terminal, passes through the lap, and returns to the battery through its negative terminal. Electron flow is in the opposite direction: It begins at the negative terminal, passes through the lamp, and returns to the battery through its positive terminal.

Figure 4 Conventional current and electron flow.

It is worth nothing that the two circuits in Figure 4 are identical. The only difference between the two is how we describe the electric current. In practice, how you view current does not affect any circuit calculations, measurements, or test procedures. Even so you should get comfortable with both viewpoints, since both are used by many engineers technicians, and technical publications. In this text, we take the electron flow approach to current. That is, we will assume current is the flow of electrons from negative to positive.

Direct Current (DC) Versus Alternating Current (AC)

Current is generally classified as being either Direct Current (DC) or Alternating Current (AC). The differences between direct current and alternating current are illustrated in Figure 5.

Figure 5 Direct current (DC) and alternating current (AC).

Direct current is unidirectional. That is, the flow of charge is always in the same direction. The term direct current usually implies that the current has a fixed value. For example, the graph in Figure 5a shows that the current has a constant value of 1A. While a fixed value is implied, direct current can change in value. However, the direction of current does not change.

Alternating Current is Bidirectional. That is, the direction of current changed continually. For example, in figure 5b, the graph shows that the current builds to a peak value in one direction and then builds to a peak value in the other direction. Note that the alternating current represented by the graph not only changes direction but is constantly changing in value.

Electric Current Produces Heat

Whenever electric current is generated through a component or circuit, heat is produced. The amount of heat varies with the level of current: The greater the current, the more heat it produces. This is why many high-current components, like motors, get hot when they are operated. Some High-current circuits get so hot that they have to be cooled.

The heat produced by electric current is sometimes a desirable thing. Toasters, electric stoves, and heat lamps are common items that take advantage of the heat produced by current.

Figure 6 High current causes a stove heating element burner to glow red.

Putting it all together

Free electrons are generated in copper at room temperature. When undirected, the motion of these free electrons is random, and the net flow of electrons in any one direction is zero.

When directed by an outside force, free electrons are forced to move in a uniform direction. This directed flow of charge is referred to as electric current.

Electric Current is represented by the letter I, which stands for intensity. The intensity of current depends on the amount of charge moved and the time required to move it.

Electric Current is measured in amperes (A). When one coulomb of current passes a point every second, you have one ampere of current.

There are two current theories. The electron flow theory describes current as the flow of charge (electrons) from negative to positive. The conventional current theory describes current as the flow of charge from positive to negative. Both approaches are widely followed. The way you view current does not affect the outcome of any circuit calculations, measurements, or test procedures.

Most electrical and electronic systems contain both direct current (DC) and alternating current (AC) circuits. In DC circuits, the current is always in the same direction. In AC circuits, current continually changes direction.

Review Questions

How are free electrons generated in a conductor at room temperature?

The thermal energy (heat) present at room temperature is enough to generate free electrons.

What is electric current? What factors affect the intensity of electric current?

Current is the directed flow of electrons in a material. The intensity of current depends on the amount of charge moved and the time required to move it.

What is a coulomb?

One coulomb equals the total charge on 6.25 × 1018 electrons.

What is he basic unit of electric current?

The ampere is the basic unit of electric current. It is defined as 1 coulomb per second or 1 A = 1 C/s.

Contrast the electron flow and convention current theories.

Conventional current theory defines current as the flow of charge from positive to negative. Electron flow is the flow of charge from negative to positive.

## Voltage

Voltage can be described as a force that generates the flow of electrons (current) through a circuit. In this section, we take a detailed look at voltage and how it generates current.

Generating Current with a Battery

Battery in Figure 7a has two terminals. The positive (+) terminal has an excess of positive ions and is described as having a positive potential. The negative (-) terminal has an excess of electrons and is described as having a negative potential.

Figure 7 A difference of potential and a resulting current.

Thus there is a Difference of Potential, or voltage (V), between the two terminals.

If we connect the two terminals of the battery with the copper wire and lamp (Figure7b), a current is produced as the electrons are drawn to the positive terminal of the battery. In other words, there is a directed flow of electrons from the negative (-) terminal to the positive (+) terminal of the battery.

There are several important points that need to be made:

1. Voltage is a force that moves electrons, for this reason, it is often referred to as Electrical Force (E) or Electromotive Force (EMF).

2. Current and voltage is not the same thing. Current is the directed flow of electrons from negative to positive. Voltage is the electrical force that generates current. In other words, current occurs as a result of an applied voltage (electric force).

The volt (V) is the unit of measure for voltage. Technically defined, one volt is the amount of electrical force that uses one joule (J) of energy to move one coulomb (C) of charge that is,

\begin{align}& \text{1 volt= 1 joule per coulomb} \\& \text{or} \\& \text{1 V=1 }{}^{J}/{}_{C} \\\end{align}

Review Questions

What is voltage?

Voltage is the force that generates current in a circuit.

How does voltage generate a current through a wire?

A voltage source has an excess of electrons (negative charge) on one terminal and an excess of positive ions on the other. This is referred to as a potential difference. The excess electrons at the negative terminal are attracted by the positive ions on the positive terminal. This results in the flow of charge in any wire that connects the two terminals of the voltage source.

What is the unit of measure for voltage? How is it defined?

The unit of measure for voltage is the volt. One volt is the amount of electrical force that uses one joule (J) of energy to move one coulomb (C) of charge. 1 V = 1 J/C.

How would you define a coulomb in terms of voltage and energy?

1 coulomb equals 1 joule per volt, 1 C = 1 J/V

How would you define a joule in terms of voltage and charge?

1 joule equals 1 V times 1 coulomb, 1 J = 1 V × 1 C

## Resistance

All elements provide some opposition to current. This opposition to current is called resistance. The higher the resistance of an element, component, or circuit, the lower the current produced by the given voltage.

Resistance (R) is measured in Ohms. Ohms are represented using the Greek letter omega (Ω). Technically defined, one ohm is the amount of resistance that limits current to one ampere when one volt of electrical force is applied. This definition is illustrated in Figure 8.

Figure 8 A basic electric circuit.

The schematic diagram in Figure 8 shows a battery that is connected to a resistor. A resistor is a component that provides a specific amount of resistance. As shown in the figure, a resistance of 1Ω limits the current to 1A when 1V is applied. Note that the long end-bar on the battery schematic symbol represents the battery’s positive terminal and the short end bar represents its Negative terminal.

Putting It All Together

We have now defined charge, current, voltage and resistance. For convenience, these electrical properties are summarized in Table 1.

Table 1: Basic Electrical Properties

Many of the properties listed in Table 1 can be defined in terms of the others. For example, in our discussion on resistance, we said that one ohm is the amount of resistance that limits current to one amp when one volt of electrical force is applied. By the same token, we can redefine the ampere and the volt as follows:

1. One ampere is the amount of current that is generated when one volt of electrical force is applied to one ohm of resistance.
2. One volt is the amount of electrical force required to generate one amp of current through one
ohm of resistance.

Review Question

What is resistance?

Resistance is the opposition to current.

What is the basic unit of resistance and how is it defined?

The unit of resistance is the Ohm (Ω). One ohm is the amount of resistance that limits current to 1 ampere when one volt is applied. 1 Ω = 1 A/V.

Define each of the following values in terms of the other two: current, voltage, and resistance.

1 V is the force required to cause 1 ampere of current through 1 ohm of resistance.
1 A is the current that results from when 1 volt is applied to 1 ohm of resistance.
1 Ω is the resistance that limits current to 1 ampere when 1 volt is applied.

Did you find apk for android? You can find new Free Android Games and apps.


text:
# LogP of halogen derivatives of aromatics and aliphatics explanation

Benzene logP = 2.13 (logP=log([benzene in octan-1-ol]/[benzene in water])) Fluorobenzene logP = 2.27 (slight increase, slightly more hydrophobic) Chlorobenzene logP = 2.84 Bromobenzene logP = 2.99

So as we go down group 7, logP or hydrophobicity increases. What can explain these trends? Furthermore on aliphatic compounds, we observe this trend which confuses matters more.

-Fluorine DECREASES logP slightly (by -0.17) on an aliphatic compound. The book doesn't give a reference as to which alpiphatic compound this is and which hydrogen is substituted, but oh well. -Chlorine increases logP (by 0.39) on the same aliphatic compound. -Bromine increases logP even more (by 0.60) on the same aliphatic compound.

So a few questions: 1. Why does a fluorine substitution decrease logP slightly on the aliphatic but increase slightly on the aromatic? 2. Why do chlorine and bromine substitutions increase logP in aliphatics and slightly more in aromatics.

Answers to these trends would be greatly appreciated, thanks.

Here is a chart of the solubility of the species in water and the log of the partition coefficient you noted.

                           solubility       solubility
MW       g/100ml water    moles/liter   log(P)
Benzene         78.11    0.179 (1)        0.0229         2.13
Flororbenzene   96.103   0.155 (2)        0.0161         2.27
Cholorbenzene  112.56    0.050 (2)        0.0044         2.84
Bromobenzene   157.01    0.041 (2)        0.0026         2.99

(1) Wikipedia

(2) Cation Complexes of Compounds Containing Carbon-Carbon Double Bonds. VI. The Argentation of Substituted Benzenes
L. J. Andrews, R. M. Keefer
J. Am. Chem. Soc., 1950, 72 (7), pp 3113–3116
DOI: 10.1021/ja01163a085

There is a Wikipedia chart in article on Ionic radii.

 Ion     radii (picometers)
F-      119
Cl-     167
Br-     182

The halide on a benzene ring is going to be relatively covalently bound. It definitely won't dissociate to form ions and only very moderately participate in hydrogen bonding.

So as the size of the halide increases the "volume" of the molecule increases. Thus the molecule would preferentially like to be in the non-polar phase.

Here is a scaled image of fluorobenzene molecule:

and a scaled image of the bromobenzene molecule.

Haloalkanes

The "anomalous" fluoroalkane is explained in a different manner. To get an n-alkane that has a boiling point above room temperature we need to go to n-pentane.

                   BP                 solubility      solubility
(deg C)   MW       g/100ml water   millimoles/liter
n-Pentane         36        72.15    0.040 (1)         0.55
1-Fluoropentane   63        90.14    1.270 (2)         14.1
1-Chloropentane  108       106.6     0.197 (3)         1.85
1-Bromopentane   130       151.05    0.127 (4)         0.838

(1) Wikipedia
(2) NIH
(3) Wikipedia
(4) Handbook of Aqueous Solubility Data, Second Edition By Samuel H. Yalkowsky, Yan He, Parijat Jain

There are two competing effects.

(1) The size of atom bound to the alkane.

Br > Cl > F > H

Scaled model of Fluoroethane

(2) The second effect is the electronegativity of the X group in the $\text{Alkane}\text{X}$ bond.

 F    >   Cl   >   Br   >   H
4.0      3.2      3.0      2.2

The fluorine atom has a slightly larger radius in a fluoroalkane than the corresponding C-H group. But the halide groups allow more dipole forces. So by "volume" the overall effect is:

 fluoroalkane
alkane
choloroalkane
bromoalkane

• Why does increased volume lead to the molecule preferring the non polar phase? I don't understand how increased dipole forces reduce the volume of the fluoroalkane. Furthermore why wouldn't the volume of chloro and bromoalkanes be reduced in this case? Also, why is it that fluoroalkanes actually prefer water MORE than alkanes? – Laksh Nov 6 '15 at 1:44
• See this problem for an answer to the volume effect. chemistry.stackexchange.com/questions/40108/… – MaxW Nov 6 '15 at 1:53

format: plain


text:
Difficulty:IntermediateLength:LongLanguages:

Charts are a great way to present data. They make data more digestible by making it visually appealing. In WordPress, there is no built-in method for getting posts and pages data in a graphical form.

Although, there are certain plugins available which integrate Google Analytics with WordPress, but they are overkill if you want to get only a portion of that data.

Also, nothing should keep you from learning new techniques and to dive straight into the subject is the best way to learn.

## What We Are Going to Do

This tutorial is aimed at new WordPress developers who want to get insight into some useful techniques which we can use in our development process. In this tutorial, we're going to make our own simple plugin that will present us data graphically within the WordPress admin. We will use standard WordPress classes and functions to fetch data and then display it graphically in the WordPress admin with the help of a jQuery plugin called HighCharts.

We will prepare our basic plugin, add a page for it in the wp-admin and then enqueue our plugin's JavaScript only on that page. We will then pass the data that we had fetched to an external script through the use of wp_localize_script. In turn, the script will display our data graphically.

The statistics that we plan to show are:

1. Most popular posts
2. Top five categories by number of posts
3. Breakup of categories by posts

## HighCharts

According to their site:

Highcharts is a charting library written in pure HTML5/JavaScript, offering intuitive, interactive charts to your web site or web application. Highcharts currently supports line, spline, area, areaspline, column, bar, pie, scatter, angular gauges, arearange, areasplinerange, columnrange, bubble, box plot, error bars, funnel, waterfall and polar chart types.

You can grab yourself a copy from their website.

That said, let's begin working on our plugin.

## Getting Started With Our Plugin

### Initial Setup

We will first make a directory inside our wp-content/plugins folder named "admin-charts". Inside that let's make the initial directory structure for our plugin.

Copy the file highcharts.js from the directory in the ZIP file you downloaded from the official website, to a js folder:

In the root directory, we will make a index.php file and inside that we will add the initial declaration for our plugin:

Now if you go to WP-Admin > Plugins, you will notice that the plugin is being shown there, but it's not functional yet.

Our next step would be to add a page for the plugin inside the admin where we would perform all the operations. For that, we will use the WordPress action admin_menu. This action triggers after the basic admin panel menu structure is in place and thus can be used to add more menus or submenus in the admin. The basic usage is as follows:

We would add a function chart_add_admin_page inside our index.php and then hook it into this action:

Inside our chart_add_admin_page function we will call the native WordPress function add_plugins_page:

The first argument is the page title that will be shown in the <title></title> tags of the page. The second argument is the menu title. The third and fourth arguments are the user capability and unique slug for referring to this menu. The last argument is the name of the callback function that will be used to render the contents of this page.

Now if you activate the plugin and hover over the "Plugins" menu, you will notice that we have successfully added a menu for our plugin:

### Rendering the Contents of the Plugin Page

At this stage, we have successfully added an empty page for our plugin. It's time to make it functional by outputting some content.

In our previous call to add_plugins_page we have referred to a callback function render_admin_charts_page. This is the function we have planned to output all the content that we want to have on our page. So let's write the function.

Below the chart_add_admin_page function, add the following code:

We are just adding some simple HTML here. We have added a heading and a form inside WordPress' native CSS class "wrap".

Inside the form, we have added a select box that currently has only one option for showing most popular posts by comment count. Below the form we have added a container div for our charts.

Our page is now taking shape:

Tip: You should always try to integrate with WordPress' native UI. We have a great series by Stephen Harris covering the topic.

It's time to register the necessary scripts and styles so we can enqueue them later. For that, we will use the wp_register_script and wp_register_style function that work in collaboration with the action hook admin_enqueue_scripts if we want to enqueue them on the admin side.

But before that, let's add a constant for our plugin root directory so we can refer to it later when we are defining paths for our scripts and styles. So, at the top of the page below the plugin declaration, add this bit of code:

We can now define our function to register our scripts and styles:

First, we have registered the HighCharts script that we had downloaded earlier. We have given it a handle "highCharts". For the next argument, we have defined the path where it exists.

Next, we have passed an array of scripts upon which our script depends, in this case it's jQuery since we would be manipulating the DOM through jQuery. In this way, we won't have to worry about enqueuing jQuery, it will be enqueued automatically whenever we enqueue the highCharts script.

For the third argument, we have defined a version number and for the last argument, we have told the wp_register_script to enqueue the script in the footer of the page after the main content. In the same way, we have registered our second script where we would be adding all of our necessary JavaScript code.

We can now actually enqueue our scripts and styles on our plugin page, but we don't want them to be enqueued on every single page in the admin where they are not needed.

For that reason, we will check for a condition before we enqueue our scripts:

The function that we hook with admin_enqueue_scripts actually receives a parameter for the admin page that we are currently on. In our case, it's "plugins_page_admin-charts". You can always check this parameter by echoing it in your development process.

Now that we have prepared a base for our plugin, we can begin working on our core objective i.e. fetching data and displaying statistics.

## Fetching and Displaying Statistics

We want to fetch three types of data:

1. Most popular posts (column chart)
2. Top five categories by number of posts (column chart)
3. Breakup of categories by posts (pie chart)

### Most Popular Posts

For this type of data, we can use the WP_Query class to fetch five posts that have the highest number of comments. The WP_Query class comes in handy when we need to fetch posts based on different criteria. Using this class, we can list posts in any way we want. The query for fetching posts with the highest number of comments can be written as:

We now have five posts that have the highest number of comments, in the form of objects inside the $posts array. You can always echo the variable to see what you are working with. We need to pass this object along with other essential data to the JavaScript. For that purpose, we will first prepare an array that will contain the data type, the chart type that we need to draw, and finally the posts that we have just fetched through WP_Query. We can pass this variable to JavaScript through wp_localize_script: The first argument in the function call is the handle of the script to which we need to pass the data. We had registered this script earlier. The second argument is the name of the object that will be available in JavaScript and the final argument is the data itself that we need to pass. The final index.php should look like this: But we need to make sure that WP_Query and wp_localize_script only get called when we have submitted the form; therefore, we are enclosing them inside another if statement that checks if the form has been submitted: The only thing left now is to grab the data inside our JavaScript file and draw the chart. Inside the js/admin_charts.js file, add the following code: The $data array we had passed through index.php has turned into an object inside JavaScript. We can thus manipulate it like any other JavaScript object.

We are first checking for the data type that's coming in:

Then we have initialized two empty arrays for post titles and post comment count respectively:

And finally we have iterated through the posts and grabbed the titles and comment count in the arrays we have initialized:

Now it's time to actually draw the chart using the data we have fetched, for that we have used the HighCharts API:

Now go back to your Plugins > Admin Charts and after selecting an option from the drop down, click the submit button, you should now have a working column chart.

You might want to go back to your posts, add some comments and then come back to see the updated statistics:

We can now add support for more data types in our plugin.

### Top Five Categories by Number of Posts

All we have to do now is to get five categories that have the highest number of posts associated. But before that, let's add an option for this data type in our select dropdown. So go back to the render_admin_charts_page function we had defined earlier and update it as follows:

We can use the native WordPress function get_categories and pass in some arguments:

We then fetch our data the same way we had done before:

So the final piece of code should be like this:

In the admin_charts.js, insert this code after the if statement:

We are doing the same thing as before, but this time we have changed the chart title and the captions for the axes. We should now have another chart displaying the top five categories with most number of posts:

### Breakup of Categories by Posts

Our last data type is the breakup of categories by number of posts. For this type of data, we will use the pie chart as it's most appropriate in this case. Also note that a single post might belong to multiple categories.

We will use the same get_categories function but this time we are not limiting the number of categories we get, instead we need to get all the categories to show the breakup.

So we will begin by checking if the option has been selected for this data type and then call our function to retrieve the categories, prepare our array, and then pass it to the JavaScript:

It's relatively simple to draw a pie chart. In admin_charts.js, add the following code after the existing else-if statement:

Note that we have formatted the tooltip to show percentage instead of an integer. We are almost done except for a little utility function we can add to each of our select options so that it persists when the page loads after submitting the form.

Add this bit of code inside index.php after the render_admin_charts_page function:

And then call the function inside each of our select options:

Our plugin is now completed and you now have three working charts for different types of data. Feel free to play around and add more charts for more data types.

## Conclusion

In this tutorial we made our own plugin from the ground up and added a separate page in the wp-admin. We have also looked at some of the techniques that can be useful when developing for WordPress.

These include different ways to get the required information about the posts and categories and then passing that to the JavaScript so we could use it for different purposes.

I hope that you have found this tutorial useful and don't forget to provide your constructive feedback in the comments.


text:
How should an oxygen system be properly prepared in a dental office? Can you give me a long answer please:(

Question:

How should an oxygen system be properly prepared in a dental office? Can you give me a long answer please:(

In Russia, . politicians are interested in using the educational system to shape minds, unlike in other countries there is vigorous

In Russia, . politicians are interested in using the educational system to shape minds, unlike in other countries there is vigorous competition among textbook publishers schools are extremely critical of Russia's past politicians are interested in using the educational system to shape minds, as in o...

Consider a three-step reversible cycle consisting of (a) an isothermal expansion at temperature T1, (b) a constant-volume

Consider a three-step reversible cycle consisting of (a) an isothermal expansion at temperature T1, (b) a constant-volume cooling to temperature T2 and (c) an adiabatic compression to the initial state. If one mole of an ideal gas is used, what is the total work obtained from the operation of this c...

-33 < -7n-12 ≤ -26 compound inequality

-33 < -7n-12 ≤ -26 compound inequality...

How does the audience know that the play is moving on to a new scene?The play ends.The actors take a break.The scenery, lighting, and

How does the audience know that the play is moving on to a new scene? The play ends. The actors take a break. The scenery, lighting, and props change. The characters continue their dialogue without pausing....

2. in your own words, summarize this law

2. in your own words, summarize this law...

Read the passage below, and then answer the question that follows. in our last issue, we featured an

Read the passage below, and then answer the question that follows. in our last issue, we featured an in-depth comparison of several of the latest popular automobiles, only one of which was a sedan. we focused on five features: interior room, handling, power, appearance, and price. which of the foll...

Why does the specific quadrant that an angle is found in matter when solving trigonometric functions?

Why does the specific quadrant that an angle is found in matter when solving trigonometric functions?...

If you want to use a method of dispute resolution other than a trial and you want someone else to decide the issues between you and a business partner,

If you want to use a method of dispute resolution other than a trial and you want someone else to decide the issues between you and a business partner, what type of dispute resolution would you select? O A. Arbitration O B. Mediation O C. Conciliation O D. Negotiations...

Finish the lyrics lil peep editioni know that u like me kiss me with ur eyes closed

Finish the lyrics lil peep edition i know that u like me kiss me with ur eyes closed...

Round off 34162 to one significant figure

Round off 34162 to one significant figure...

An effective chart to clarify understanding. true false

An effective chart to clarify understanding. true false...

The average temperature for a country during September 2013 was 52.2 degrees fahrenheit. this is 1.6 degrees fahreinheit above

the average temperature for a country during September 2013 was 52.2 degrees fahrenheit. this is 1.6 degrees fahreinheit above the 20th century average temperature for September. What was the countries average temperature for september?...

Rewrite the equation y=2|x−2|+4 as two linear functions f and g with restricted domains.

Rewrite the equation y=2|x−2|+4 as two linear functions f and g with restricted domains....

The first electric generators used direct current, so they needed to be reversed manually on a regular

The first electric generators used direct current, so they needed to be reversed manually on a regular basis in order to work properly. nikola tesla saw this as an inefficient way to produce electricity. as a result, tesla used the idea of the rotating magnetic field, which had been developed by fra...

PLEASEEE HELP value of x $PLEASEEE HELPPPP value of x$...

A Production Facility has produced 615,000 products and would like to allocate these products to their customers. Given the following

A Production Facility has produced 615,000 products and would like to allocate these products to their customers. Given the following requested customer demand, calculate how much of the EXCESS product should be sent to Customer 3. Customer 1 Demand...

Consider the ideals, values, and themes that early american writers considered to be important and worthy

Consider the ideals, values, and themes that early american writers considered to be important and worthy of inclusion in their journals, sermons, and poems. what are these ideals and values? why were they important? how do different writers incorporate them into their works? did certain values o...

What is the difference between mature writing and immature writing? immature writing is longer than mature writing because of

What is the difference between mature writing and immature writing? immature writing is longer than mature writing because of descriptions. immature writing uses more descriptive phrases, but a simple vocabulary. mature writing uses a broader vocabulary and more descriptive phrases. mature writing ...
25 points i need in calculating the total cost of packaging for both shapes, the image below: $25 points i need in calculating the total cost of packaging for both shapes, the$...


text:
# Stochastic Gradient MCMC for State Space Models

State space models (SSMs) are a flexible approach to modeling complex time series. However, inference in SSMs is often computationally prohibitive for long time series. Stochastic gradient MCMC (SGMCMC) is a popular method for scalable Bayesian inference for large independent data. Unfortunately when applied to dependent data, such as in SSMs, SGMCMC's stochastic gradient estimates are biased as they break crucial temporal dependencies. To alleviate this, we propose stochastic gradient estimators that control this bias by performing additional computation in a `buffer' to reduce breaking dependencies. Furthermore, we derive error bounds for this bias and show a geometric decay under mild conditions. Using these estimators, we develop novel SGMCMC samplers for discrete, continuous and mixed-type SSMs. Our experiments on real and synthetic data demonstrate the effectiveness of our SGMCMC algorithms compared to batch MCMC, allowing us to scale inference to long time series with millions of time points.

## Authors

• 6 publications
• 12 publications
• 11 publications
• 27 publications
• ### Stochastic Gradient MCMC for Nonlinear State Space Models

State space models (SSMs) provide a flexible framework for modeling comp...
01/29/2019 ∙ by Christopher Aicher, et al. ∙ 12

• ### Stochastic Gradient MCMC Methods for Hidden Markov Models

Stochastic gradient MCMC (SG-MCMC) algorithms have proven useful in scal...
06/14/2017 ∙ by Yi-an Ma, et al. ∙ 0

• ### Scalable approximate inference for state space models with normalising flows

By exploiting mini-batch stochastic gradient optimisation, variational i...
10/02/2019 ∙ by Tom Ryder, et al. ∙ 0

• ### Scalable MCMC for Mixed Membership Stochastic Blockmodels

We propose a stochastic gradient Markov chain Monte Carlo (SG-MCMC) algo...
10/16/2015 ∙ by Wenzhe Li, et al. ∙ 0

• ### Mini-batch Metropolis-Hastings MCMC with Reversible SGLD Proposal

Traditional MCMC algorithms are computationally intensive and do not sca...
08/08/2019 ∙ by Tung-yu Wu, et al. ∙ 1

• ### Factorized Inference in Deep Markov Models for Incomplete Multimodal Time Series

Integrating deep learning with latent state space models has the potenti...
05/30/2019 ∙ by Zhi-Xuan Tan, et al. ∙ 0

• ### Rao-Blackwellized Stochastic Gradients for Discrete Distributions

We wish to compute the gradient of an expectation over a finite or count...
10/10/2018 ∙ by Runjing Liu, et al. ∙ 0

## Code Repositories

### sgmcmc_ssm_code

Python code for SGMCMC for Time Series SSMs

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

State space models (SSMs) are ubiquitous in the analysis of time series in fields as diverse as biology [69], finance and economics [40, 74], and systems and control [26]. As a defining feature, SSMs augment the observed time series with a latent state sequence

to model complex time series dynamics with a latent Markov chain dependence structure. Given a time series, inference of model parameters involves sampling or marginalizing this latent state sequence. Unfortunately, both the runtime and memory required scale with the length of the time series, which is prohibitive for long time series (e.g. high frequency stock prices

[33], genome sequences [25], or neural impulse recordings [16]). In practice, given a long time series, one could ‘segment’ or ‘downsample’ to reduce length; however, this preprocessing can destroy or change important signals and computational considerations should ideally not limit scientific modeling.

To help scale inference in SSMs, we consider stochastic gradient Markov chain Monte Carlo (SGMCMC), a popular method for scaling Bayesian inference to large data sets [12, 47, 68]. The key idea of SGMCMC is to employ stochastic gradient estimates based on subsets or ‘minibatches’ of data, avoiding costly computation of gradients on the full dataset, such that the resulting dynamics produce samples from the posterior distribution over SSM parameters. This approach has found much success in independent data models, where the stochastic gradients are unbiased estimates of the true gradients. However, when applying SGMCMC to SSMs, naive stochastic gradients are biased, as subsampling the data breaks dependencies in the SSM’s latent state sequence. This bias can destroy the dynamics of SGMCMC causing it to fail when applied to SSMs. The challenge is to correct these stochastic gradients for SSMs while maintaining the computational benefits of SGMCMC.

In this work, we develop computationally efficient stochastic gradient estimators for inference in general discrete-time SSMs. To control the bias of stochastic gradients, we marginalize the latent state sequence in a buffer around each subsequence, propagating critical information from outside each subsequence to its local gradient estimate while avoiding costly full-chain computations. Similar buffering ideas have been previously considered for belief propagation [32], variational inference [27]

, and in our earlier work on SGMCMC for hidden Markov models (HMMs)

[48], but all are limited to discrete latent states. Here, we present buffering as an approximation to Fisher’s identity [9], allowing us to naturally extend buffering trick to continuous and mixed-type latent states.

We further develop analytic bounds on the bias of our proposed gradient estimator that, under mild conditions, decay geometrically in the buffer size. To obtain these bounds we prove that the latent state sequence posterior distribution has an exponential forgetting property [9, 17]. However unlike classic results which prove a geometric decay between the approximate and exact marginal posterior distributions in total variation distance, we use Wasserstein distance [66] to allow analysis of continuous and mixed-type latent state SSMs. Our approach is similar to proofs of Wasserstein ergodicity in homogeneous Markov chains [24, 49, 57]; however we extend these ideas to the nonhomogeneous Markov chains defined by the latent state sequence posterior distribution. These geometrically decaying bounds guarantee that we only need a small buffer size in practice, allowing scalable inference in SSMs.

Although our proposed gradient estimator can be generally applied to any stochastic gradient method, here, we develop SGMCMC samplers for Bayesian inference in a variety of SSMs such as HMMs, linear Gaussian SSMs (LGSSM), and switching linear dynamical systems (SLDS) [9, 29]. We also derive preconditioning matrices to take advantage of information geometry, which allows for more rapid mixing and convergence of our samplers [31, 52]. Finally, we validate our algorithms and theory on a variety of synthetic and real data experiments, finding that our gradient estimator can provide orders of magnitude run-time speed ups compared to batch sampling.

This paper significantly expands upon our initial work [48], by (i) connecting buffering to Fisher’s identity, simplifying its presentation and analysis, (ii) non-trivially generalizing the approach to SSMs beyond the HMM, including continuous and mixed-type latent states, (iii) developing a general framework for bounding the error of buffered gradient estimators using Wasserstein distance, and (iv) providing extensive validation on a number of real and synthetic datasets.

The paper is organized as follows. First, we review background on SSMs and SGMCMC methods in Section 2. We then present our framework of constructing buffered gradient estimators to extend SGMCMC to SSMs in Section 3. We prove the geometrically decaying bounds for our proposed buffered gradient estimate in Section 4. We apply our framework and error bounds to discrete, continuous and mixed-type latent state SSMs in Section 5. Finally, we investigate our algorithms on both synthetic and real data in Section 6.

## 2 Background

### 2.1 State Space Models for Time Series

State space models (SSMs) for time series are a class of discrete-time bivariate stochastic process , , consisting of a latent state sequence generated by a homogeneous Markov chain and an observation sequence generated independently conditioned on  [9]. Examples of state space models include: HMMs, LGSSMs, and SLDSs (see Section 5

for details). For a generic SSM, the joint distribution of

and factorizes as

 Pr(y,u|θ)=T∏t=−TPr(yt|ut,θ)Pr(ut|ut−1,θ), (1)

where are model-specific parameters. As the latent state sequence is unobserved, the likelihood of given only the observations (marginalizing ) is

 Pr(y|θ)=∫T∏t=−TPr(yt|ut,θ)Pr(ut|ut−1,θ)du, (2)

Unconditionally, the observations are not independent and the graphical model of this marginal likelihood, Eq. (2), has many long term dependencies, Figure 1 (right). In contrast, when conditioned on the observations are independent and the complete-data likelihood, Eq. (1), has a simpler chain structure, Figure 1 (left).

To infer given , we can maximize the marginal likelihood or, given a prior , sample from the posterior . However, traditional inference methods for

, such as expectation maximization (EM), variational inference, or Markov chain Monte Carlo (MCMC), take advantage of the conditional independence structure in

, Eq. (1), rather than working directly with , Eq. (2[5, 58]. To use with unobserved , these methods rely on sampling or taking expectations of from the posterior . As an example, gradient-based methods take advantage of Fisher’s identity [9]

 ∇logPr(y|θ)=Eu|y,θ[∇logPr(y,u|θ)]=Eu∼γ[∇logPr(y,u|θ)], (3)

which allows gradients of Eq. (2) to be computed in terms of Eq. (1). To compute the posterior , these methods use the well-known forward-backward algorithm [9, 58]. The algorithm works by recursively computing a sequence of forward messages and backward messages which are used to compute the pairwise marginals of . More specifically,

 αt(ut) :=Pr(ut,y≤t|θ)=∫Pr(yt,ut|ut−1,θ)αt−1(ut−1)dut−1 (4) βt(ut) :=Pr(y>t|ut,θ)=∫Pr(yt+1,ut+1|ut,θ)βt+1(ut+1)dut+1 (5) γt−1:t(ut−1,ut) :=Pr(ut−1,ut|y,θ)∝αt−1(ut−1)Pr(yt,ut|ut−1,θ)βt(ut). (6)

When message passing is tractable (i.e., when Eqs. (4)-(5) involve discrete or conjugate likelihoods), the forward-backward algorithm can be calculated in closed form. When message passing is intractable, the messages can be approximated using Monte-Carlo sampling methods (e.g. blocked Gibbs sampling [10, 28], particle methods [2, 21, 60]). In both cases, when the length of the time series is much larger than the dimension of , the forward-backward algorithm (running over the entire sequence) requires time and memory at each iteration.

The SSM challenge is to scale inference of model parameters to long time series when the computation and storage per iteration is prohibitive.

One popular method for scalable Bayesian inference is stochastic gradient Markov chain Monte Carlo (SGMCMC) [12, 47, 68]. The idea behind gradient-based MCMC is to simulate continuous dynamics of a potential energy function such that the dynamics generate samples from the posterior distribution . For example, the Langevin diffusion over is given by the stochastic differential equation (SDE)

 dθs=−∇U(θ)ds+√2dWs, (7)

where is Brownian motion, , and indexes continuous time. As , the distribution of converges to the SDE’s stationary distribution, which by the Fokker-Planck equation is the posterior  [47, 55]. Because we cannot perfectly simulate Eq. (7), in practice we use a discretized numerical approximation. One straightforward approximation is the Euler-Mayurma discretization

 θ(s+1)←θ(s)−h∇U(θ(s))+N(0,2h), (8)

where is the stepsize and indexes discrete time steps. This recursive update defines the Langevin Monte-Carlo (LMC) algorithm. Typically, a Metropolis-Hastings correction step is added to account for the discretization error [55].

For large datasets, computing at every step in Eq. (8) is computationally prohibitive. To alleviate this, the key ideas of stochastic gradient Langevin dynamics (SGLD) are to replace

with a quick-to-compute unbiased estimator

and to use a decreasing stepsize to avoid costly Metropolis-Hastings correction steps [68]

 θ(s+1)←θ(s)−h(s)∇˜U(θ(s))+N(0,2h(s)). (9)

For i.i.d. data, an example of is to use a random minibatch ,

 ∇ˆU(θ)=−1Pr(S)∑t∈S∇logPr(yt|θ)−∇logPr(θ), (10)

which only requires time to compute. When is unbiased and with an appropriate decreasing stepsize schedule , SGLD converges to the posterior distribution [12, 61]. However, in practice one uses a small, finite step-size for greater efficiency, which introduces a small bias [15].

A Riemannian extension of SGLD (SGRLD) simulates the Langevin diffusion over a Riemannian manifold with metric by preconditioning the gradient and noise of Eq. (9) by . By incorporating geometric information about structure of , SGRLD aims for a diffusion which mixes more rapidly. Suggested examples of the metric are the Fisher information matrix or a noisy Hessian estimate  [31, 52]. Given , each step of SGRLD is

 θ(s+1)←θ(s)−h[D(θ(s))∇~U(θ(s))+Γ(θ(s))]+N(0,2hD(θ(s))) (11)

where the vector

is a correction term to ensure the dynamics converge to the target posterior [47, 70]. Many other SGMCMC have been proposed [4, 12, 13, 19, 43]. Although our ideas extend to these formulations as well, we focus on the popular SGLD and SGRLD algorithms.

To apply SGMCMC to SSMs, we must choose whether to use the complete-data loglikelihood or the marginal data loglikelihood in the potential . If we use the complete-data loglikelihood, then we treat as the parameters. Although the observations conditioned on are independent, we must calculate gradients for at each iteration, which is prohibitive for long sequences and intractable for discrete or mixed-type . On the other hand, if we use the marginal loglikelihood, then we only need to take gradients in . However, the observations conditioned on alone are not independent and therefore the minibatch gradient estimator Eq. (10) breaks crucial dependencies causing it to be biased. Our SGMCMC challenge is correcting the bias in stochastic gradient estimates when applied to SSMs.

## 3 General Framework

We now present our framework for scalable Bayesian inference in SSMs with long observation sequences. Our approach is to extend SGMCMC to SSMs by developing a gradient estimator that ameliorates the issue of broken temporal dependencies. In particular, we develop a computationally efficient gradient estimator that uses a buffer to avoid breaking crucial dependencies, only breaking weak dependencies. We first present a (computationally prohibitive) unbiased estimator of for SSMs using Fisher’s identity. We then derive a general computationally efficient gradient estimate that accounts for the dependence in observations using a buffer. We also propose preconditioning matrices for SGRLD with SSMs. Finally, we present our general SGMCMC pseudocode for SSMs.

The main challenge in constructing an efficient estimate of for SSMs is handling the lack of independence (marginally) in . Because the observations in SSMs are not independent, we cannot produce an unbiased estimate of with a randomly selected subset of data points as in Eq.(10). For example, a naive estimate for is to take the gradient of a random contiguous subsequence with

 ∇ˆU(θ)=−1Pr(S)∇logPr(yS|θ)−∇logPr(θ). (12)

This estimate only requires time compared to the for . However because the marginal likelihood does not factorize as in the independent observations case, this estimate is biased .

To obtain an unbiased estimate for , we use Fisher’s identity Eq. (3) to rewrite in terms of the complete-data loglikelihood as a sum over time points

 ∇U(θ) =−∇logPr(y|θ)−∇logPr(θ) (13) =−Eu|y,θ[∇logPr(y,u|θ)]−∇logPr(θ) =−∑t∈TEu|y,θ[∇logPr(yt,ut|ut−1,θ)]−∇logPr(θ)

From this, we straightforwardly identify an unbiased estimator for a subsequence

 ∇¯¯¯¯U(θ)=−1Pr(S)∑t∈SEu|y,θ[∇logPr(yt,ut|ut−1,θ)]−∇logPr(θ). (14)

Although Eq. (14) reduces the number of gradient terms to compute from to , the summation terms require calculating expectations of . More specifically, Eq. (14) requires expectations with respect to the pairwise marginal posteriors for . Recall that computing these marginals take time to pass messages over the entire sequence . This defeats the purpose of using a subsequence. If we instead only pass messages over the subsequence , then the pairwise marginals are and we return to the naive biased gradient estimator

 ∇ˆU(θ)=−1Pr(S)∑t∈SEu|yS,θ[∇logPr(yt,ut|ut−1,θ)]∇logPr(yS|θ)−∇logPr(θ). (15)

We instead propose passing messages over a buffered subsequence for some positive buffer size , with (see Figure 2). The idea is that there exists a large enough such that . Our buffered gradient estimator sums only over , but takes expectations over instead of

 ∇~U(θ)=−1Pr(S)∑t∈SEu|yS∗,θ[∇logPr(yt,ut|ut−1,θ)]−∇logPr(θ). (16)

When this is equivalent to the naive biased estimator of Eq. (15). When this is equivalent to the unbiased estimator of Eq. (14).

The trade-off between accuracy (bias) and runtime depends on the size of the buffer and current model parameters . Intuitively, when produces pairwise marginals that are similar to i.i.d. data, we can use a small buffer . When produces strongly dependent pairwise marginals, we must use a larger buffer . In Section 4, we analyze, for a fixed value of , how quickly the bias between and decays with increasing . We show a geometric decay

 ES∥∇¯¯¯¯U−∇˜U∥2≤Cθρ−Bθ, for some Cθ>0, (17)

where is large for i.i.d. data and small for strongly dependent data. For a gradient accuracy of , we only need a logarithmic buffer size 111As . Therefore our buffered gradient estimator reduces the computation time from to . In our experiments (Section 6), we find that modest buffers are both necessary and sufficient to correct for bias.

The correction factor in Eq. (16) needs to be calculated carefully to ensure points near the beginning and end of the sequence

are properly weighted. Furthermore, an average over gradient estimates of a collection of subsequences can be used to control the variance of

. For our analysis, we consider the simple case of uniformly sampling a single sequence from separate subsequences (). Details for more complex sampling schemes are provided in the Supplement.

### 3.3 Preconditioning and Fisher Information

The desirable properties for the preconditioning matrix for SGRLD are (i) the resulting dynamics takes advantage of the geometric structure of , (ii) both and can be efficiently computed, and (iii) neither nor are numerically unstable.

The expected Fisher information is the Riemannian metric proposed in [31]

 D−1(θ)=G(θ)=Iy=Ey|θ[∇2logPr(y|θ)]. (18)

This corresponds to the Riemannian manifold over with distance measured by the symmetric Kullback-Liebler (KL) divergence between marginal likelihoods. Unfortunately for SSMs, the lack of independence in the marginal likelihood requires a double sum over to compute , which is computationally intractable for long time series. We instead replace with the complete data Fisher information

 Iu,y=Eu,y|θ[∇2logPr(y,u θ)]=|T|⋅Eu,y|θ[∇2logPr(yt,ut|ut−1,θ)]. (19)

Similar to , using corresponds to the manifold with distance measured by the symmetric KL divergence between complete-data likelihoods. Because can be calculated analytically for the SSMs we consider (Section 5), we use when possible. In our experiments, we find that in practice, using the complete data Fisher information matrix works well and outperforms vanilla SGLD.

### 3.4 Algorithm Pseudocode

Algorithms 1 and 2 summarize our generic SGMCMC method for SSMs222Python code for our method will be available at https://github.com/aicherc/sgmcmc_ssm_code.

## 4 Buffered Gradient Estimator Error Bounds

In this section, we establish a bound on the expected error between the full data gradient and our buffered gradient estimator Eq. (16). Given such a bound, we can control the overall error in our SGLD or SGRLD scheme when the SGMCMC dynamics possess a contraction property [36]. Using the triangle inequality, we decompose the error of using the unbiased gradient estimator Eq. (14)

 ES∥∇U(θ)−∇˜U(θ)∥2≤ES∥∇U(θ)−∇¯¯¯¯U(θ)∥2+ES∥∇¯¯¯¯U(θ)−∇˜U(θ)∥2, (20)

where the expectation is taken over the random subsequence . The first term is the error from subsampling and is the same error from minibatching in SGMCMC for i.i.d. data, Eq. (10[68]. The second term is the error from approximating the latent state posterior, which is unique to our buffered gradient estimator. Because error in the first term is studied in depth in the existing stochastic gradient MCMC literature [4, 11, 22, 61], we restrict our focus to bounding the second term. In particular, we will uniformly bound for any .

Our approach is to bound in terms of the Wasserstein distance between the exact posterior and our approximate posterior and then show this Wasserstein distance decays geometrically. To bound the Wasserstein distance, we follow existing work on bounding Markov processes in Wasserstein distance [24, 49, 57]. However, unlike previous work that focuses on the homogeneous Markov process of the joint model , we instead focus on the induced nonhomogeneous Markov process of the conditional model . To do so, we use the forward () and backward () random maps of  [18]

 ut∼Pr(ut|y,θ) ⇒(ft(ut),ut)∼Pr(ut+1,ut|y,θ) (21) ut∼Pr(ut|y,θ) ⇒(bt(ut),ut)∼Pr(ut−1,ut|y,θ), (22)

If and satisfy a contractive property, then we can bound the Wasserstein distance between in terms of and respectively. Bounding the error of the induced nonhomogeneous Markov process has been previously studied in the SSM literature using total variation (TV) distance [9, 17, 42, 62]. These works bound the error in total variation distance by quantifying how quickly the smoothed posterior forgets the initial condition. However, these bounds typically require stringent regularity conditions, which are hard to prove outside of finite or compact spaces. In particular, these bounds are not immediately applicable for LGSSMs. In contrast, we bound the error in Wasserstein distance by proving contraction properties of and , allowing us to extend results to continuous and mixed-type SSMs such as the LGSSM (Section 5.3.1).

Our main result is that if, for fixed , the gradient of satisfies a Lipschitz condition and the random maps all satisfy a contraction property, then the error decays geometrically in the buffer size .

###### Theorem 1.

Let and be the 1-Wasserstein distances between and at the left and right ends of respectively. Let . If the gradients of are all Lipschitz in with constant , and random maps and are all Lipschitz in with constant , then we have

 ∥∇¯¯¯¯U(θ)−∇˜U(θ)∥2≤|T|⋅LU⋅1+L1−L⋅1−LSS⋅LB⋅2ϵ1. (23)

As , Theorem 1 states that the error of the buffered gradient estimator decays geometrically as . Therefore, the required buffer size for an error tolerance of scales logarithmically as . In contrast, the error of the gradient estimator decays only linearly in the subsequence length, ; therefore much longer subsequences, , are required to reduce bias. This agrees with the intuition that the bias is dominated by the error at the endpoints of subsequence. A similar result for when the gradient of the complete data loglikelihood is Lipschitz in instead of (as needed for LGSSM) will be proved in Section 4.3.

The remainder of this section is as follows. First, in Section 4.1, we show how to bound the error in in terms of Wasserstein distances between . Second, in Section 4.2, we show these Wasserstein distances decay geometrically in . Finally, in Section 4.3, we prove our main results: Theorems 1 and 2. To keep the presentation clean, we leave proofs of Lemmas to the Supplement.

### 4.1 Functional Bound in terms of Wasserstein

We first review the definition of Wasserstein distance. Let be the -Wasserstein distance

 Wp(γ,˜γ):=[infξ∫∥u−˜u∥p2dξ(u,˜u)]1/p (24)

where is a joint measure or coupling over with marginals and . Wasserstein distance satisfies all the properties of a metric. A useful property of the -Wasserstein distance is the following Kantorovich-Rubinstein duality formula for the difference of expectations of Lipschitz functions [66]

 W1(γ,˜γ)=sup∥ψ∥Lip≤1{∫ψdγ−∫ψd˜γ}⇒|Eγ[ψ]−E˜γ[ψ]|≤∥ψ∥Lip⋅W1(γ,˜γ), (25)

where denotes the Lipchitz constant of .

We connect the error to the Wasserstein distances between , by applying this duality formula Eq. (25) to the difference of Eqs. (14) and (16)

 ∇¯¯¯¯U−∇˜U=|T|S∑t∈SEγt−1:t[∇logPr(yt,ut|ut−1,θ)]−E˜γt−1:t[∇logPr(yt,ut|ut−1,θ)]. (26)

Applying the triangle inequality gives Lemma 1.

###### Lemma 1.

If are Lipschitz in with constant ,

 ∥∇¯¯¯¯U(θ)−∇˜U(θ)∥2≤|T|S⋅LU⋅∑t∈SW1(γt−1:t,˜γt−1:t). (27)

If is not Lipschitz in , but is Lipschitz in (as in LGSSMs), then the following Lemma lets us bound the -Wasserstein distance of in terms of the -Wasserstein distance of .

###### Lemma 2.

Let be the distribution of . Let be the distribution of . Let . (Note implies .) Then,

 W1(γ′,˜γ′)≤(2√M+1)⋅max{W2(γ,˜γ)1/2,W2(γ,˜γ)}.

### 4.2 Geometric Wasserstein Decay

We first review why contractive random maps induce Wasserstein bounds. If two distributions have identically distributed random maps , that is there exists a random function satisfying

 u∼γt and u′∼γ′t⇒ft(u)∼γt+1 and ft(u)∼γ′t+1, (28)

then we can bound the Wasserstein distance of in terms of the Wasserstein distance of given a bound on the random map’s Lipschitz constant

 Wp(γt+1,γ′t+1)p =infξt+1∫∥ut+1−u′t+1∥p2dξt+1(ut+1,u′t+1) (29) ≤infξt∫∥ft(ut)−ft(u′t)∥p2dξt(ut,u′t)dft ≤infξt∫Lp⋅∥ut−u′t∥p2dξt(ut,u′t)=Lp⋅Wp(γt,γ′t)p.

Unfortunately for SSMs, Eq. (29) does not apply as the random maps of and of are not identically

distributed. To see this, we first review the conditional probability distributions used to define

. The forward random map draws from the forward smoothing kernel

 Ft(ut+1|ut):=Pr(ut+1|ut,y>t)=Pr(ut+1|ut)Pr(yt+1|ut+1)βt+1(ut+1)/βt(ut) (30)

and the backward random map draws from the backward smoothing kernel

 Bt(ut−1|ut):=Pr(ut−1|ut,y≥t)=Pr(ut|ut−1)Pr(yt|ut)αt−1(ut−1)/αt(ut). (31)

Because uses different forward and backward messages , in Eqs. (30) and (31), the kernels are not identical to (and the random maps are not identically distributed). This is unlike homogeneous Markov chains, where the kernels are identical at each time (and the random maps are identically distributed).

Instead of connecting to directly, we use the triangle inequality to connect them through an intermediate distribution

 Wp(γ,˜γ)≤Wp(γ,ˆγ)+Wp(ˆγ,˜γ). (32)

Introducing this particular intermediate distribution is the key step for our Wasserstein bounds between and . Because conditions on all after , and have identical backward messages and therefore identically distributed forward random maps . Similarly, because does not condition on before , and have identical forward messages and identically distributed backward random maps . Therefore, we can bound using and bound using with the contraction trick Eq. (29) giving us Lemma 3.

###### Lemma 3.

If there exists such that for all , and , then for all we have

 Wp(γt−1:t,ˆγt−1:t) ≤(1+Lpf)1/p⋅Wp(γt−1,ˆγt−1) (33) ≤(1+Lpf)1/p⋅LB+t−1f⋅Wp(γ−B,ˆγ−B) Wp(ˆγt−1:t,˜γt−1:t) ≤(1+Lpb)1/p⋅Wp(ˆγt,˜γt) (34) ≤(1+Lpb)1/p⋅LB+S−tb⋅Wp(ˆγS+B,˜γS+B)

We show sufficient conditions for the random maps to be contractions (i.e. ) for specific models in Section 5.1.1 (HMMs) and 5.3.1 (LGSSMs).

### 4.3 Proof of Main Theorems

Putting together the results of the previous two subsections gives us our geometric error bounds: Theorem 1 when the gradient terms are Lipschitz in and Theorem 2 when the gradient terms are Lipschitz in . Both theorems require the random maps of the forward and backward smoothing kernels are contractions. We first prove Theorem 1.

###### Proof of Theorem 1.

Combining Lemmas 1 and 3 with some algebra

 ∥∇¯¯¯¯U(θ)−∇˜U(θ)∥2 ≤|T|S⋅LU⋅∑t∈SW1(γt−1:t,˜γt−1:t) ≤|T|S⋅LU⋅S∑t=1(1+Lf)LB+t−1fϵ1+(1+Lb)LB+S−tbϵ1 ≤|T|⋅LU⋅1+L1−L⋅1−LSS⋅LB⋅2ϵ1,

where . ∎

We now prove a similar result for when is Lipschitz in .

###### Theorem 2.

Let . If the gradients are Lipschitz in with constant , and there exists for Lemma 3, then with and

 ∥∇¯¯¯¯U(θ)−∇˜U(θ)∥2≤|T|⋅LU⋅√1+L21−L1/2⋅1−LS/4S/2⋅LB/2⋅maxr∈{1/2,1}(2ϵ2)r.

Similar to Theorem 1, Theorem 2 states that the squared error of the buffered gradient estimator decays geometrically if the complete-data loglikelihood is Lipschitz in instead of . However, the price we pay is a square-root: the error decays instead of .

###### Proof of Theorem 2.

Applying Lemmas 2 and 3, we have

 ∥∇¯¯¯¯U(θ)−∇˜U(θ)∥2 ≤|T|S⋅LU⋅S∑t=1maxr∈{1/2,1}[W2(γt−1:t,ˆγt−1:t)+W2(ˆγt−1:t,˜γt−1:t)]r ≤|T|S⋅LU⋅S∑t=1maxr∈{1/2,1}[(LB+t−1+LB+S−t)√1+L2


text:
# another circle geometry question~~~ really urgent

• Jul 2nd 2007, 10:40 PM
shosho
another circle geometry question~~~ really urgent
in a traingle ABC, <BAC=80 and Q is the point on BC such that AQ bisects the angle BAC. There is also a point P on AB such that <QPC=40 and P is distinct from A. Prove that PQ=QC

any help would be awesome!!
• Jul 2nd 2007, 11:04 PM
red_dog
We have $\widehat{QPC}=\widehat{QAC}$, then the quadrilater $ACQP$ can be inscribed in a circle.
Yields $\widehat{PAQ}=\widehat{PCQ}=\widehat{QPC}$, so the triangle $PQC$ is isosceles, then $PQ=QC$


text:
## Problem B. Brackets ≡

 Author: NEERC 2005 Time limit: 2 sec Input file: brackets.in Memory limit: 64 Mb Output file: brackets.out

### Statement

Let us consider arithmetic expressions that consist of variables denoted by lower-case letters "a" to "z"; four binary arithmetic operations: addition ("+"), subtraction (" − "), multiplication ("*"), and division ("/"); opening ("(") and closing (")") round brackets. The normal order of precedence is used multiplication and division have the highest precedence, addition and subtraction have the lowest precedence. Operations of the same precedence are evaluated from left to right (for example a − b + c = (a − b) + c).

Thus, the grammar for the expressions is the following:

 expression → term | expression + term | expression  −  term term → factor | term * factor | term / factor factor → variable | (expression) variable → a | b | ... | z

Your task is to rewrite the given expression so that its semantics is not changed, but the resulting expression has the minimal number of round brackets.

You can remove any excessive brackets that do not change the order of evaluation, for example

 (a + b) + (c) ⇒ a + b + c, (a * b) / (c) ⇒ a * b / c,
and you can rewrite expressions using the following rules:
• If A and B are arbitrary expressions, you can change A + (B) to A + B, for example
a − g / h + (b + c − d + e * (f + h − i))⇒ a − g / h + b + c − d + e * (f + h − i).
• If A and B are arbitrary expressions, you can change A − (B) to A − B where B is obtained from B by replacing all top-level "+" operations to "-" operations and vice versa, for example
a − g / h − (b + c − d + e * (f + h − i))⇒ a − g / h − b − c + d − e * (f + h − i)).
• If A and B are arbitrary terms, you can change A * (B) to A * B, for example
x / (y + z) * (a * (b − c) / d / (e / f))⇒ x / (y + z) * a * (b − c) / d / (e / f).
• If A and B are arbitrary terms, you can change A / (B) to A / B, where B is obtained from B by replacing all top-level "*" operations to "/" operations and vice versa, for example
x / (y + z) / (a * (b − c) / d / (e / f))⇒ x / (y + z) / a / (b − c) * d * (e / f).

You can think about these transformations as ones that only use "+" and "*" associativity, the fact that "-" is the reverse operation to "+", "/" is the reverse operation to "*", and nothing else. You can apply the described transformations and remove excessive brackets as many times as you need to get the expression with the minimal number of round brackets.

### Input file format

The input file contains a single line with the expression.

### Output file format

Write to the output file a single line with the same expression that is rewritten with the minimal number of round brackets. Do not write any spaces.

### Constraints

Expression does not have any leading, trailing, or inner spaces and consists of at most 1000 characters.

### Sample tests

No. Input file (brackets.in) Output file (brackets.out)
1
((a-b)-(c-d)-(z*z*g/f)/(p*(t))*((y-u)))
a-b-c+d-z*z*g/f/p/t*(y-u)

0.179s 0.017s 13


text:
# Increasing space between paragraphs [duplicate]

I want to know how to increase the distance between two paragraphs but not between two lines of the same paragraph. For example, if I type a long line, I want to wrap that line into a paragraph, but when I deliberately press Enter or type `\\` and begin another paragraph, I want the default distance to be higher. Is it possible to set this for the whole document without using `\\[10pt]`?

This is obtained by setting nonzero `\parskip`, e.g. `\parskip10pt`. Please keep in mind that such paragraphs should not be indented (you should set `\parindent0pt`), because mixing these two is considered a bad style.


text:
# Trigonometry : Complete a Proof Using Sums, Differences, or Products of Sines and Cosines

## Example Questions

### Example Question #1 : Sum, Difference, And Product Identities

True or false:

.

False

True

Cannot be determined

False

Explanation:

The sum of sines is given by the formula .

### Example Question #2 : Sum, Difference, And Product Identities

True or false: .

False

True

Cannot be determined

False

Explanation:

The difference of sines is given by the formula .

### Example Question #1 : Complete A Proof Using Sums, Differences, Or Products Of Sines And Cosines

True or false: .

Cannot be determined

False

True

False

Explanation:

The sum of cosines is given by the formula .

### Example Question #2 : Complete A Proof Using Sums, Differences, Or Products Of Sines And Cosines

True or false: .

Cannot be determined

True

False

False

Explanation:

The difference of cosines is given by the formula .

### Example Question #5 : Sum, Difference, And Product Identities

Which of the following correctly demonstrates the compound angle formula?

Explanation:

The compound angle formula for sines states that .

### Example Question #6 : Sum, Difference, And Product Identities

Which of the following correctly demonstrates the compound angle formula?

Explanation:

The compound angle formula for cosines states that .

### Example Question #7 : Sum, Difference, And Product Identities

Simplify by applying the compound angle formula:

Explanation:

Using the compound angle formula, we can rewrite each half of the non-coefficient terms in the given expression. Given that   and , substitution yields the following:

This is the formula for the product of sine and cosine, .

### Example Question #8 : Sum, Difference, And Product Identities

Simplify by applying the compound angle formula:

Explanation:

Using the compound angle formula, we can rewrite each half of the non-coefficient terms in the given expression. Given that  and , substitution yields the following:

This is the formula for the product of two cosines, .

### Example Question #9 : Sum, Difference, And Product Identities

Using  and the formula for the sum of two sines, rewrite the sum of cosine and sine:

Explanation:

Substitute  for :

Apply the formula for the sum of two sines, :

### Example Question #10 : Sum, Difference, And Product Identities

Using  and the formula for the difference of two sines, rewrite the difference of cosine and sine:


text:
# Children of The Sun

Yet, this is not the end of the story. The progress in modern biology, complex system, nonlinear science has drawn another humanitarian epic narrative (Wilson, 1999), out of the Darwin jungle one — indeed we are stardust, but we are also children of the sun: “Human social existence, unlike animal sociality, is based on the genetic propensity to form long-term contracts that evolve by culture into moral precepts and law. They evolved over tens or hundreds of millennia because they conferred upon the genes prescribing their survival and the opportunity to be represented in future generations. We are not errant children who occasionally sin by disobeying instructions from outside our species. We are adults who have discovered which covenants are necessary for survival, and we have accepted the necessity of securing them by sacred oath.” --- Artificial Intelligence and Humanities in Measure, Manifold, Learning, and Optimization: A Theory Of Neural Networks

## What Have I Lived For

I have sought truth first, in the universal abstract world of mathematics, and the constructive concrete world of hacking. Meanwhile, I sought the salvation of humanities, and myself, in the world of philosophers who have sought in the time before me, and the stories weaved with love and tears in the history of mankind. In doing so, I hope to find the answer to the existence of life, and the answer to seemingly unbearable human suffering.
I found loneliness,
I found the solitary pain
that gnaws the heart,
But also peace I do find.

Sorrow is knowledge; they who know the most
Must mourn the deepest over the fatal truth.
Yet, the tree of Knowledge is also that of Life.

It reveals
Love as our Nature,
and Reason is our Blessing;
Ethereal can weave with Mundane,
and Peace
Lies in Making.

Now, Old yet still Young
I have known you.
And, knowing you,
I have found both ecstasy & peace;
I know the rest.

After so many solitary years,
I know what Life may be,
though I may still not know Love.

Now, if I could keep moving,
I see Possibilities.
I shall Sail,
As if I am still Young.

## What I Do

Intellectually, I do three things roughly right now.
• build a theory of artificial neural networks (Panel *Research*)
• build a coherent epistemological, ethic, moral, and political philosophy of information age that forges order in the chaotic fragmentation of postmodernism through integration of Science and classic philosophy, e.g, Confucianism.
• make small bricks that prepare for a system that is capable of identifying unaware (for particular users of the system) high quality cultural intellectual artistic contents, e.g., books, articles, websites, journals, thus to democratize culture.


text:
The DeclareDesign Blog

## An instrument does not have to be exogenous to be consistent

We often think of an instrumental variable ($$Z$$) as a random shock that generates exogenous variation in a treatment of interest $$X$$. The randomness of $$Z$$ lets us identify the effect of $$X$$ on $$Y$$, at least for units for which $$Z$$ perturbs $$X$$ in a way that’s not possible by just looking at the relationship between $$X$$ and $$Y$$. But surprisingly, we think, if effects are constant the instrumental variables estimator can be consistent for the effect of $$X$$ on $$Y$$ even when the relationship between the instrument ($$Z$$) and the endogenous variable ($$X$$) is confounded (for example, Hernán and Robins (2006)). That’s the good news. Less good news is that when there is effect heterogeneity you can get good estimates for some units but it can be hard to know which units those are (Swanson and Hernán 2017). We use a declaration and diagnosis to illustrate these insights.

Consider the causal graph below (a directed acyclic graph, or DAG). Here $$U_2$$ confounds the relationship between $$X$$ and $$Y$$. That’s the usual thing and the reason for why you need something like an instrument in the first place. But now a binary $$U_1$$ also confounds the relationship between $$Z$$ and $$X$$ (see for instance Fig 2d Swanson et al. (2018)). To emphasize, in this set up, the instrumental variable and the instrumented variable share a common cause in $$U_1$$.

Using DeclareDesign, we write down a declaration that is consistent with this DAG. Our declaration specifies some “compliance types” or “principal strata.” In the standard IV setup with a binary instrument $$Z$$ and a binary treatment $$X$$, there are only four types (corresponding to all conceivable ways that the endogenous variable can respond to the exogenous variable): compliers, never-takers, always-takers, and defiers.1 But here, the binary $$U_1$$ and binary $$Z$$ both affect $$X$$, so there are actually sixteen conceivable ways to comply!2

We simplify by imagining a world with only three types: $$U_1$$-compliers, $$Z$$-compliers, and never-takers.3 $$U_1$$-compliers have $$X = 1$$ if and only if $$U_1 = 1$$. $$Z$$-compliers have $$X = 1$$ if and only if $$Z = 1$$. Never-takers have $$X = 0$$ regardless of the values of $$U_1$$ or $$Z$$. $$U_2$$ takes three values, 1-3, and plays two roles. First, it determines whether you are a $$U_1$$-complier, $$Z$$-complier, or never-taker with respect to $$X$$. Second, it is positively correlated with your value in $$Y$$.

We’ll define five estimands, which is a lot.

1. The Average Treatment Effect (ATE) of $$X$$ on $$Y$$.
2. The Intention-To-Treat (ITT) effect. The average effect of $$Z$$ on $$Y$$.
3. The first-stage effect. This is the average effect of $$Z$$ on $$X$$.
4. The Local Average Treatment Effect (LATE_U1) of $$X$$ on $$Y$$, among the subgroup of units that are $$U_1$$-compliers
5. The Local Average Treatment Effect (LATE_Z) of $$X$$ on $$Y$$, among the subgroup of units that are $$Z$$-compliers

We’ll use three estimators:

1. The ITT estimator, which is an OLS regression of $$Y$$ on $$Z$$.
2. The first-stage estimator, which is an OLS regression of $$X$$ on $$Z$$.
3. The IV estimator, which is two-stage least squares regression of $$Y$$ on $$X$$, instrumented by $$Z$$.

The declaration also includes two additional parameters (bear with us!) that we’ll vary down below. They are:

1. heterogeneity: the amount of treatment effect heterogeneity. When this is 0, the effect of $$X$$ on $$Y$$ is the same for $$U_1$$-compliers as it is for $$Z$$-compliers. Otherwise, the effect for the two subgroups is different.
2. path_weight: describes whether the U1 --> X or the Z --> X path is stronger. This is equivalent to the fraction of compliers that are $$U_1$$-compliers.

OK, here’s the design:

heterogeneity <- 0   # how much does LATE_Z differ from LATE_U1?
path_weight   <- .5  # what fraction of compliers are U1 compliers?

design <-

# U2 is the compliance "type" (U1 complier, Z complier, or Never Taker)
declare_population(N = 2000,
U2 = complete_ra(
N,
prob_each = c(path_weight * 0.6, (1 - path_weight) * 0.6, 0.4),
conditions = c("U1_c", "Z_c", "NT")),
noise = rnorm(N)) +

# U1 is binary
declare_assignment(prob = 0.5, assignment_variable = "U1") +

# Z is affected by U1: among U1 = 0, Z has probability 0.3; among U1 = 1, probability 0.6
declare_assignment(blocks = U1, block_prob = c(0.3, 0.6), assignment_variable = "Z") +

# X potential outcomes in terms of U1 and Z, given U2 (there are **four**)
declare_potential_outcomes(X ~ as.numeric((U2 == "U1_c") * U1 +
(U2 == "Z_c")  * Z),
assignment_variables = c(U1, Z)) +

# X potential outcomes in terms of Z given U1 and U2 (there are **two**)
declare_potential_outcomes(X ~ as.numeric((U2 == "U1_c") * U1 +
(U2 == "Z_c")  * Z),
assignment_variables = Z) +

declare_reveal(X, assignment_variables = c(U1, Z)) +

# Y potential outcomes in terms of X (there are **two**)
declare_potential_outcomes(Y ~ X * (1 + heterogeneity * as.numeric(U2)) + noise,
assignment_variables = X) +

# Y potential outcomes in terms of Z (there are **two**)
# this is same as above, but X is written in terms of Z
declare_potential_outcomes(Y ~ as.numeric((U2 == "U1_c") * U1 + (U2 == "Z_c") * Z) *
(1 + heterogeneity * as.numeric(U2)) + noise,
assignment_variables = Z) +

# reveal Y (doesn't matter "how" this is revealed, either via the X pos or the Z pos)
declare_reveal(Y, assignment_variables = X) +

# Use X POs to learn about the two complier definitions
declare_step(
U1_complier = X_U1_0_Z_0 == 0 & X_U1_0_Z_1 == 0 & X_U1_1_Z_0 == 1 & X_U1_1_Z_1 == 1,
Z_complier  = X_U1_0_Z_0 == 0 & X_U1_1_Z_0 == 0 & X_U1_0_Z_1 == 1 & X_U1_1_Z_1 == 1,
handler = fabricate) +

# 5 (!) estimands
declare_estimand(
ATE =         mean(Y_X_1 - Y_X_0),
LATE_U1 =     mean(Y_X_1[U1_complier] - Y_X_0[U1_complier]),
LATE_Z =      mean(Y_X_1[Z_complier] - Y_X_0[Z_complier]),
ITT =         mean(Y_Z_1 - Y_Z_0),
first_stage = mean(X_Z_1) - mean(X_Z_0)) +

# Three estimators
declare_estimator(Y ~ Z, estimand = "ITT", model = lm_robust, label =  "itt") +
declare_estimator(X ~ Z, estimand = "first_stage", model = lm_robust, label =  "first") +
declare_estimator(Y ~ X | Z, estimand = c("ATE", "LATE_U1", "LATE_Z"),
model = iv_robust, label =  "iv_robust")

Let’s now diagnose this design to show that – pretty counterintuitively for us at least! – the IV estimator still recovers the LATE even when $$Z$$ and $$X$$ have a common cause.

diagnosis_1 <-
diagnose_design(
design, diagnosands = declare_diagnosands(select = c(bias, mean_estimate, mean_estimand)))
Design Label Estimand Label Estimator Label Term N Sims Bias Mean Estimate Mean Estimand
design ATE iv_robust X 200 -0.00 1.00 1.00
(0.01) (0.01) (0.00)
design first_stage lm_robust_first_stage Z 200 0.09 0.39 0.30
(0.00) (0.00) (0.00)
design ITT lm_robust_itt Z 200 0.09 0.39 0.30
(0.00) (0.00) (0.00)
design LATE_U1 iv_robust X 200 -0.00 1.00 1.00
(0.01) (0.01) (0.00)
design LATE_Z iv_robust X 200 -0.00 1.00 1.00
(0.01) (0.01) (0.00)

Let’s take the diagnosis estimand-by-estimand:

1. The ATE is well-estimated by the instrumental variables estimator. This, despite the fact that:
2. The first stage regression estimates are biased for the average effect of $$Z$$ on $$X$$ because of the unobserved confounder $$U_1$$.
3. The ITT estimates are also biased because of the confounding between $$Y$$ and $$Z$$.
4. The instrumental variables performs well for the LATE, as defined here. But in this case the LATE is the same as the ATE so this is not surprising.

You see in the estimator column that the bias in the ITT and first stage estimates are proportionate, such that, in the ratio, the bias cancels out. We provide some intuition for this result below, where we show how it can also break down.

The upshot is that when assessing the validity of IV you might be imposing an additional assumption that you might not need to impose. This may be good news in some specific research settings where the more stringent assumption of exogenous assignment of $$X$$ to $$Z$$ is not well justified.4

# Allowing heterogeneity

We show here that we lose the ability to estimate the effect of $$X$$ on $$Y$$ when this effect varies among compliance types – the $$U_1$$-compliers, the $$Z$$-compliers, and the never-takers.5 We can quickly redesign to allow heterogeneity, and see that now, we are biased for all the estimands!

design_h <- redesign(design, heterogeneity = 1)

diagnosis_2 <-
diagnose_design(
design_h, diagnosands = declare_diagnosands(select = c(bias, mean_estimate, mean_estimand)))
Design Label heterogeneity Estimand Label Estimator Label Term N Sims Bias Mean Estimate Mean Estimand
design_h 1 ATE iv_robust X 200 -0.33 2.77 3.10
(0.01) (0.01) (0.00)
design_h 1 first_stage lm_robust_first_stage Z 200 0.09 0.39 0.30
(0.00) (0.00) (0.00)
design_h 1 ITT lm_robust_itt Z 200 0.18 1.08 0.90
(0.00) (0.00) (0.00)
design_h 1 LATE_U1 iv_robust X 200 0.77 2.77 2.00
(0.01) (0.01) (0.00)
design_h 1 LATE_Z iv_robust X 200 -0.23 2.77 3.00
(0.01) (0.01) (0.00)

Why was the bias so much lower under homogeneous effects of $$X$$ on $$Y$$ for compliers?

Here’s some intuition. In both the heterogeneous and homogeneous cases, we misattribute some of the effect of $$U_1$$ on $$X$$ to the effect of $$Z$$ on $$X$$, because those for whom $$U_1 = 1$$ are also more likely to have $$Z = 1$$. Thus, we overestimate the first stage in proportion to this misattribution. In both the homogeneous and heterogeneous cases, we estimate that $$Z$$ increases the probability of $$X = 1$$ by 39pp on average, whereas the true effect is 30pp. When we compute the effect of $$Z$$ on $$Y$$ for the ITT, those $$U_1$$-compliers that we mistook for $$Z$$-compliers also produce an effect on $$Y$$ that we again misattribute to $$Z$$, though in fact it was caused by the effect of $$U_1$$ running through $$X$$. The extent of this misattribution in the ITT is the same as the extent of the misattribution in the first stage when the effects of $$X$$ on $$Y$$ are homogeneous. Thus, even though we wanted to divide 0.3 / 0.3 to get 1, we ended up dividing 0.39 / 0.39 – and we still got 1! When the effects are heterogeneous, however, we lose this proportionality: because the effect of $$X$$ on $$Y$$ is bigger now for $$Z$$-compliers than for $$U_1$$ compliers, our estimate of the ITT is not biased enough!. We would have wanted to estimate an ITT of 1.20 so that our IV estimator would have given us the true ATE via 1.20 / 0.39 = 3.10. But instead, we misattribute the $$U_1$$-compliers’ effect of $$X$$ on $$Y$$ to the $$Z$$-compliers in a way that is disproportionate to the misattribution of the first stage, $$Z$$ on $$X$$. We don’t have enough bias to unbias our estimates!

# With non causal instruments the “compliers” are those that comply with respect to the unobserved causal instrument

Although we have described $$Z$$ as being the instrument, you can see in this DAG that both $$Z$$ and $$U_1$$ can affect $$X$$. So perhaps $$U_1$$ should thought of as an instrument even if it is unobserved? And perhaps compliers should be thought of as units that comply with $$U_1$$ and not $$Z$$?

To explore this possibility, we diagnose design modifications that shift probability mass between two special cases of the DAG.

The first special case is the standard IV setup with no arrow between $$U_1$$ and $$X$$. The second special case is one discussed in Swanson and Hernán (2017) in which $$Z$$ is a “noncausal instrument” and the true causal instrument is the unobserved $$U_1$$. We get this by removing the path from $$Z$$ to $$X$$. Swanson and Hernán (2017) show that even though $$Z$$ does not itself affect $$X$$, instrumenting $$X$$ with $$Z$$ will recover good estimates of a well-defined estimand. You guessed it, that estimand is the LATE for $$U_1$$ compliers. Amazing!

These cases are the two extremes of a continuum of the relative strength of the two paths. To be clear those two paths are U1 --> X and Z --> X. We can adjust the strength of those two paths with the path_weight parameter in our design. For this last simulation, we’ll consider 5 values for this parameter between 0 and 1.

designs <- redesign(design, heterogeneity = 1, path_weight = seq(0.01, .99, length = 5))

diagnosis_3 <- diagnose_design(
designs, diagnosands = declare_diagnosands(select = c(bias, mean_estimate, mean_estimand)))

Let’s consider the graph from left to right.

All the way to the left, all the compliers are $$Z$$ compliers. Equivalently, all the way to the left, $$U_1$$ has no effect on $$X$$. That’s the standard IV setup in which $$Z$$ is as-if randomized, so our estimates of the ITT, the first stage, and the LATE are all on target.

All the way to the right, all the compliers are $$U_1$$-compliers, because $$Z$$ has no effect on $$X$$. That’s the “non causal instrument” case described in Swanson and Hernán (2017) (see also Brito and Pearl (2002)). The estimates of everything else – the ITT, the first stage, and the ATE are all quite biased, but the IV estimator does a good job of estimating the LATE among $$U_1$$-compliers.

In between the leftmost and rightmost extremes, however, our estimates of all of the estimands are biased. Indeed, the middle case (at the 0.50 mark) is exactly the situation we described in the “Allowing heterogeneity” section above. It’s possible (and we’d love to hear from you if you have ideas along these lines!) that an alternative estimation strategy would recover good estimates of one or more of these estimands.

# Blog update

This will be the last of our weekly blog posts on declaredesign.org. When we started blogging, we set ourselves a goal of six months of posts. We really enjoyed putting these posts together, but we’re also excited to shift attention to other parts of the project. We expect to continue to post occasionally here and to put up more informal posts on http://discuss.declaredesign.org and we’d love it if you joined us there.

Yours in declaration and diagnosis,

Graeme, Jasper, Alex, and Macartan

# References

Brito, Carlos, and Judea Pearl. 2002. “Generalized Instrumental Variables.” In Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, 85–93. Morgan Kaufmann Publishers Inc.

Hernán, Miguel A, and James M Robins. 2006. “Instruments for Causal Inference: An Epidemiologist’s Dream?” Epidemiology, 360–72.

Swanson, Sonja A, and Miguel A Hernán. 2017. “The challenging interpretation of instrumental variable estimates under monotonicity.” International Journal of Epidemiology 47 (4): 1289–97.

Swanson, Sonja A, Miguel A Hernán, Matthew Miller, James M Robins, and Thomas S Richardson. 2018. “Partial Identification of the Average Treatment Effect Using Instrumental Variables: Review of Methods for Binary Instruments, Treatments, and Outcomes.” Journal of the American Statistical Association 113 (522): 933–47.

1. Denoting $$x$$ the value that the endogenous variable $$X$$ takes, compliers have $$x = f_X(Z) = Z$$, never-takers have $$x = f_X(Z) = 0$$, always-takers have $$x = f_X(Z) = 1$$, and defiers have $$x = f_X(Z) = (1 - Z)$$.↩

2. The value $$X$$ takes could depend on all four potential combinations of $$U_1$$ and $$Z$$. So if $$x = f_X(U_1,Z)$$, you have to think about every possible way in which $$f_X(0,0)$$, $$f_X(1,0)$$, $$f_X(0,1)$$ and $$f_X(0,0)$$ can map into values of $$x$$.↩

3. The assumption that there are no types who “defy” their $$U_1$$ or $$Z$$ assignments amounts to a monotonicity assumption common in the IV framework.↩

4. Of course, the result also holds when $$Z$$ and $$X$$ are unconfounded, which also allows for unbiased estimation of the ITT and the first stage, which may be of practical or theoretical interest in and of themselves.↩

5. More precisely, weaker conditions than homogeneity can be sufficient for identification. As put by Hernán and Robins (2006), a sufficient assumption is that “the X-Y causal risk difference is the same among treated subjects with $$Z=1$$ as among treated subjects with $$Z=0$$, and similarly among untreated subjects.” See other conditions discussed in Swanson et al. (2018).↩