text:
# String Types in Julia

A shortish post about the various string type in Julia 0.6, and it’s packages. This post covers Base.String, Base.SubString, WeakRefStrings.jl, InternedStrings.jl, ShortStrings.jl and Strs.jl; and also mentioneds StringEncodings.jl. Thanks to Scott P Jones, who helped write the section on his Strs.jl package.

All strings are subtypes of AbstractString, which provides most of the functionality.

## Base: String

The standard string type is String. This is an UTF8 string. It is immutable, and in memory is is a pointer-type (This is in-contrast to structs which are immutable value types, and mutable structs which are mutable pointer-types.) It is binary compatible with C, it is null-terminated. But it also records the length along-side.

If you want to read or write non-UTF8 strings, use StringEncodings.jl. That will let you load and write strings in any encoding and any codepage. It is really easy to use, especially if you are willing for them to be converted to UTF8 during loading. (At one point IIRC in 0.3, it was impossible to load non-UTF8 strings, and it was bad, but StringEncodings.jl make that easy.)

## Base: SubString

SubString is the other string type included in Base. It is returned by the substring function, and by other functions such as split, and various regex matching. It holds a reference to the original string, as well as an offset and a length recording. This saves on allocating memory for the split-up string. However, that reference to the “parent” string, is a real reference. While the substring is referenced, the parent string can not be garbage collected. Further, if the substring is serialised (either to disk, or over the network via e.g. pmap), the parent string also must be serialised. I’ve been caught out by that a few times, wondering why my vocabulary set, containing just 10,000 unique words is >5GB on disk, only to open up the serialised JLD file and find the whole corpus is stored along side it, to resolve that reference.

## WeakRefStrings.jl: WeakRefString

WeakRefStrings.jl are not really for casual users. They are more or less just like SubStrings except that they have only a weak reference to the parent string. This means that holding a reference to the WeakRefString will not prevent the parent string from being garbage collected. At which point trying to read the WeakRefString will likely result is garbage or crashing with a segfault. They true advantage of not having a strong reference is that it allows the WeakRefString to be stack allocated. Right now, if a composite type has an actual (strong) reference (like in SubString) it will always be allocated on the heap. Which is slower, and has garbage collection related things. If we ever fully get stack allocation for structs with heap references, then WeakRefStrings.jl will likely be replaced with just using SubString.

## InternedStrings.jl: InternedString

v0.4 of InternedStrings.jl introduces a type InternedString which represents a String that has been interned. InternedStrings have the property that only a single unique instance for a given string’s content exists. All instances of such a string are just references to the same data. When the last reference is gone, that string can/will be garbage collected.

In the just released v0.5.0+ InternedStrings.jl does the same thing, except that it no longer has a type associated with it. One simply called intern(str) on the string type of choice and it either gives back a reference to a copy from the pool; or adds it to the pool (and acts as the identity). Because of these changes v0.5.0 of InternedStrings is compatible with (basically) any string type. Further, because it is not doing an allocation for the wrapper type, it is really fast (and it wasn’t crazy slow before), and uses even less memory.

Interning strings has a solid advantage if you are going to be dealing with a lot of copies of the same string. Which is common in natural language tokenization because of Zipf’s law, particularly if one eliminates the rare words.

The other thing it gets you is really fast equality checking, since if two interned strings are not the same reference then they can’t be equal. In InternedStrings.jl v0.4 this applies to all checks since it overloads == into ===. In v0.5+ this has to be done explicitly via ===. (It might be worth defining intern_eq(a,b)= a===b || intern!(a)==intern!(b) or similar)

## ShortStrings.jl: ShortString{SZ}

Since a String in julia is binary just like a string C, that means it has a pointer to character memory, and that memory then contains the content. On a 64bit build that pointer has 8 bytes, (4 bytes on 32bit). For you can store a significant number of words in 8 bytes. Pointer indirection takes time.

ShortStrings.jl does strings without pointers, where all the values are inline. It does this by basically reinterpretting the binary data behind a primitive type, per byte. It thus provides strings in 3 sizes: ShortString15 ontop of a UInt128, able to hold up to 15 bytes. ShortString7 ontop of a UInt64, able to hold up to 7 bytes. ShortString3 ontop of a UInt32, able to hold up to 3 bytes. If your cointing bytes, you’ll see the obvious question of “where is the extra byte going?”. The last byte of the memory is used to store the length of the string. (The actual length, not the maximum length).

## Strs.jl

Strs.jl is the only project I am aware of that is basically out to replace Base.String outright.

Strs.jl provides string types for valid ASCII, Latin1, UCS2, UTF-8, UTF-16, and UTF-32 encoded strings, Binary strings, 8-,16-,32-bit strings where the encoding is not known, and “raw” UTF-8 strings (like String, where the encoding is purportedly UTF-8, but it is not necessarily valid. There is also a type UniStr, which is meant to be used in place of String, which encodes the string with 1, 2, or 4 byte codeunits, depending on the contents of the string. Strs.jl’s goals are to comply with the recommendations of the Unicode organization, IETF, and W3C in the handling of invalid strings (to avoid known security exploits), to improve performance substantially (major gains are possible because of dealing with validated strings), to provide greater flexibility (for handling other encodings, and for interfacing with other languages, libraries, and file/database formats, with UTF-16 and UTF-32 support), easy to use (except for UTF8Str and UTF16Str, all of the Str string types are indexed directly by characters, not codeunits), more robust (many of the bugs that continue to be found with String are caused by the complexity of dealing with variable-length encodings, having to use lastindex, thisind, nextind, prevind correctly, or consistency problems caused by allowing invalid characters (which means that memcmp cannot be used)), more functionality, a cleaner API, and better memory utilization (UTF-8 takes a lot more space than UCS-2 for the languages of 3/4s of the world’s population, and more space than Latin1 for Western European languages). Strs.jl also provides a Chr type, which is used to create a set of types, ASCIIChr, LatinChr, UCS2Chr, UTF32Chr, Text1Chr, Text2Chr, Text4Chr, optimized for the characters in different string types. These are validated, and operating on them is much faster than using Char. You can see some outdated benchmarks here. The current version is now almost universally much faster than Base.String.

### Common Unicode Encodings

(This is not just relevant to Strs.jl, but as the main package being made for Unicode, it mostly is. When you start thinking about Unicode encodings while working in julia Strs.jl is almost certainly where you are going.)

ASCII and Latin1 (ISO-8859-1) are fixed width 1 byte encodings, which can be used to represent any of the characters in Western European languages. ASCII is compatible with both UTF-8 and Latin1 encoding, which is a useful property when doing conversions. It is a useful optimization (used by Python, Swift, and the Strs.jl package), to use 1-byte encodings when possible, and keep track of whether a string is pure ASCII or not. They are of-course theoretically really optimal if all your data can indeed be encoded in Latin1/ASCII; being fixed-width and still rather supported by legacy applications and legacy data.

UTF-16 is variable width, with each character being either 1 or 2 2-byte codeunits. UCS-2, for the purposes of this discussion, is UTF-16, with no surrogate characters present, such that all characters can be represented in fixed size 2-bytes code, and O(1) to index, and O(1) to get the length of a string. It was claimed that >99% of the world’s text can be encoded using just the Unicode BMP (i.e. UCS-2) (while that may not be true if you are talking about text from Twitter or other chat apps, where people frequently use Emojis)

One could say that UTF-16 is the worst of both worlds, or you could say it is a good middle ground. Microsoft really likes it and to use the Windows API you basically have to use it. It is used as the internal format for many languages / libraries, because 1) it is significantly faster for processing than UTF-8 encoding, 2) by keeping track of whether a string only holds BMP characters, indexing operations and getting the length are O(1), 3) for many human languages (Asian, Indian, African ones in particular), UTF-8 takes 50% more space than UCS-2 / UTF-16, and for languages that can be expressed using (non-ASCII) Latin, Hebrew, Arabic, Cyrillic characters, UTF-8 and UTF-16 take the same space. It is used by many libraries with C/C++ APIs, such as ICU, many programming languages such as Java, Objective-C, Swift, Python, C# as well as any based on JVM or .NET, and other heavily used ones.

UTF-32 takes 4 bytes per codeunit / character, and is faster to process (like ASCII, Latin1, UCS-2, and even UTF-16) than UTF-8, especially if you are using characters from outside the first 256 UTF-8 characters. The encoding is trivial, O(1) to index and determine the length, however it suffers from taking more space than other encodings, which can affect performance negatively, depending on the text. In general, it’s best to use only when there are non-BMP characters present in the string.

UTF-8 is a variable width encoding, from 1-4 bytes, where ASCII characters take up 1 byte, most non-ASCII characters in European languages, Arabic, Hebrew, languages using the Cyrillic alphabet take 2 bytes (compared to 1 byte for Western European languages using Latin1 encoding), most of the rest take 3 bytes (which would take only 2 bytes for UCS-2 or UTF-16), and non-BMP characters take 4 bytes per character, the same as both UTF-16 and UTF-32 encodings. It is the best encoding for dealing with JavaScript, JSON, sending data to/from web applications, and external text files, although it is not the best for doing text processing. Processing it is fairly complex, and there are many possibly invalid sequences, including ones which were valid prior to 2003, and those can be used for security exploits.

LegacyStrings.jl also provides unicode string types, however, I believe it is being maintained primarily for compatibility reasons. Most things in JuliaArchive are not being maintained, and I suspect LegacyStrings will join them after julia 1.0. Also as mentioned above one can use StringEncodings.jl to convert between basdically any string encodings and the UTF-8 as implemented by Base.String during IO, if you just want to avoid the whole question (and just want the mostly good enough Base.String).


text:
## Blog Archives

#### Categories

May 6, 2017 - Comments Off on Commonplace #0050

## Commonplace #0050

Extraordinary claims demand extraordinary proof.

If you eliminate the impossible, whatever remains, however improbable, must be the truth.

Among competing hypotheses, the one with the fewest assumptions should be selected. In simpler language, the simplest solution is correct.

The simpler a hypothesis is, the more easily it can be proved or falsified. A complex explanation for a phenomenon involves many factors which can be difficult to test or lead to issues with the repeatability of an experiment. As a consequence, the simplest solution which is consistent with the existing data is preferred.

format: plain


text:
# Computing p-values

I am a computer engineer, living in a digital world. I have no probabilistic background besides a probability course in the University and tried to grasp the Fisher's idea and have got the following questions.

Why is this room called Ten Fold and Corss validation? Wikipedia says that "To compute a p-value from the test statistic, one must simply sum (or integrate over) the probabilities of more extreme events occurring". Thereby, we are warned against interpreting p-values as type I errors. But, AFAIK, $a$-values-based Type I errors just mean exactly that you cut the tail distribution whereas correct result resides in it, which is exactly Type 1 error.

Secondly, I could not find the formulas or examples computing p-values. Instead WP tells that they were invented by Fisher, who Rather than using a table of p-values, inverted the CDF, publishing a list of values of the test statistic for given fixed p-values. Does it say that I must know the p-value in advance? How do I compute it if I must know it in advance?

Third, Wikipedia gives a hint on a way to compute the p-value: "Hypothesis tests, such as Student's t-test, typically produce test statistics whose sampling distributions under the null hypothesis are known. To compute a p-value from the test statistic, one must simply sum (or integrate over) the probabilities of more extreme events occurring." This begs a question: do we need convert t- into p-value and apply Fisher test if we can use the t-value and Student test immediately? Why is Fisher's approach better?

Finally, Wikipedia says that the formula for p-value is $2/2^n$. But, it is stated in coin flipping context and it is not clear if it is true in every context. Thereby, it points out that rolling two dies, which yield 6+6=12 has probability of 1/36, which is sufficient to overcome the 5% threshold but it is not statistically significant. This leaves me with uncertainty: how can p-value determine the statistical significance after that?

A $p$-value is the probability of someone getting a test statistic as far from 0 as it is observed or farther, given the sample size at hand. It is not related with type I errors. To use MacKay's definition in his book Information Theory, Inference, and Learning Algorithms:

p-value is the probability, given a null hypothesis for the probability distribution of the data, that the outcome would be as extreme as, or more extreme than, the observed outcome.

Here is a tiny example of computing $p$-values in the case of a simplistic linear model:

set.seed(10);
x = 1:20;
y = 2*x + rnorm(20);
lm1 <- lm(y ~ x)
summary(lm1)
Call:
lm(formula = y ~ x)

Residuals:
Min       1Q   Median       3Q      Max
-1.49158 -0.35334 -0.02612  0.59079  1.13746

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.58248    0.35503  -1.641    0.118
x            2.04971    0.02964  69.160   <2e-16 ***
Residual standard error: 0.7643 on 18 degrees of freedom

To restate our problem with actually numbers: the $p$ value (<2e-16) is the probability that the value of our test statistic (t value here) will be "as extreme as, or more extreme than" the observed value (69.160), given our null hypothesis ($t$-distribution).

So we want the $Prob(x>|69.160|)$. Fine; that probability requires a distribution to be meaningful. Fine again; will use a $t$-distribution because we are using a $t$-test after all. $t$-distributions require degrees of freedom(DF) though. Fine once more, we have the DF's of our model (here the number of data points minus the number of model parameters) 18. So, we want pt(df=18,-69.160)? Almost, remember we want the absolute in this case (we are looking at a two-way test) so 2*pt(df=18,-69.160)? Yeap. Pretty much. (I hope you understand R-syntax :D )

2*pt(df=18,-69.160) < 2e-16
[1] TRUE

So no, you don't know the p-value in advance you calculate it given you model at hand. There are certain tables of $p$-values that are handy when you are not having access to a computer but in general nowadays you can get $p$-values on the fly. This link http://www.cyclismo.org/tutorial/R/pValues.html is rather helpful actually.

For your final question: The null hypothesis of a fair dice means you have a uniform (discrete) distribution $[1,6]$ for a single dice. And a non-uniform distribution between $[2,12]$ (you can convince yourself for that; you have 36 different roll outcomes and only a single one yields a value of 12). As such a single sample from that distribution to equate to 12 appears is unlikely ($1/36$). And based on that you would reject the claim that your dices are fair (aka. you would reject your null-hypothesis). What wikipedia then says is that you totally ignore sample size effects if you do that. It would be wrong to use a $p$-value generated this way to determine the statistical significance of your null hypothesis. You need more rolls (samples) to assume as your expected roll outcome and test it against your null hypothesis.

Just to restate the take-home message. Given you decide your model/null hypothesis the $p$-value will be the probability of your test statistic to be as extreme as, or more extreme than, what you observed for your sample at hand. If your sample is not large enough to approximate the population behaviour (as for instance your single two-dice roll case) statistical inference will be meaningless (if not outright wrong).

• Excellent answer. I was confused about how to calculate the t value in your example. However, Wikipedia explains why: en.wikipedia.org/wiki/…. In R code: denom <- sqrt((1/(length(x)-2))*sum(resid(lm1)^2)) / sqrt(sum((x-mean(x))^2)); coef(lm1)['x']/ denom. This is equal to 69.1599 – Robert Smith Sep 28 '13 at 19:06

I will try for a more intuitive explanation:

Consider the case where you are asked to compare 2 distances to see which is further, but one of the distances is given to you in miles and the other in kilometers. You cannot compare the 2 numbers directly, but first need to convert at least one of them. But, it does not matter if you convert the kilometers to miles and compare miles to miles, or convert miles to kilometers and compare kilometers to kilometers, or even convert both to leagues and compare leagues to leagues. As long as all the conversions are correct we will come to the same conclusion as to which distance is further.

The same is true with hypothesis testing, on one end we have the data, on the other end we have our type I error rate ($\alpha$), and between we have hypotheses and assumptions.

Simplifying a bit there are 2 directions of conversion:

data -> test statistic -> P-value

???? <- critical value <- alpha ($\alpha$)

We can compare a p-value to alpha to declare significance, or we can compare the test statistic to the critical value. Sometimes we can even compare a statistic that is not the usual test statistic to a different critical value (the ???? above). The hypothesis and assumptions determine which test statistic we compute and how we convert either from test statistic to p-value or from alpha to critical value.

For a more concrete example lets say that we want to test if a coin is fair by flipping it 10 times. The null hypothesis is that the probability of heads is 0.5 and the alternative is that it is not 0.5 in either direction. We can do the test using the binomial distribution, or the normal approximation to the binomial (probably not a good idea with n=10, possibly reasonable if we flipped the coin 20 times, would be fine if we flip it 100 times) or a couple of other methods (this is why you sometimes see computations using the t distribution or the f distribution or the ? distribution).

In this example there are only 11 possible outcomes (number of heads out of 10 flips), so if we use an alpha of 0.05 then the true probability of a type I error is about 0.021 (probability of flipping 0, 1, 9, or 10 heads). If we now do the experiment and get 7 heads then we would calculate the p-value as the probability of getting 7, 8, 9, or 10 heads plus the probability of getting 3, 2, 1, or 0 heads since those are as extreme in the opposite direction. This p-value is 0.344 and we would compare that to our alpha of 0.05 and not reject the null hypothesis.

Or we could have looked at the probabilities of the different outcomes when the null is true and created a critical region. We do this by starting at the extremes and working in as far as we can without exceeding alpha. In this case the critical region says we will reject if we see 0, 1, 9, or 10 heads and not reject if we see other values. Then we flip the coin and see 7 heads (same as above) and from that we don't reject, same conclusion.

Either way will give us the same results, we will see a p-value < 0.05 if we flip 0, 1, 9, or 10 heads and a p-value > 0.05 otherwise. Personally the critical value approach is more intuitive to me, but computers have made the p-value approach more common (and there are advantages there as well).

Why do we start from the extremes when calculating the critical region, or why do we include all the more extreme values in calculating the p-value? Well consider this case, with 10 flips of a fair coin there is a 0.0439 probability of flipping exactly 8 heads, so we could define our critical region as flipping 8 heads. But if we expand the region to 8 or 9 heads then the probability is greater than 0.05 (0.0537 to be more precise), so that does not work. We could set the rejection region to be 0, 8, or 10 heads and that keeps us below 0.05 (0.0459). But does that really make sense as a critical region? before flipping the coin we are already saying that if we see 0, 8, or 10 heads we will reject the null, but if we see 9 heads (or 1 head) we will not reject the null, this works mathematically but not intuitively that if 8 is extreme enough then 9 should be even more evidence, not less.


text:
# Using negative pressure flow to generate electricity from a large body of water

While preparing to drain an "above ground" pool with a regular garden hose, I was thinking about how the hose actually generated a "suction" force. Place one end at the bottom of the pool, connect the other end to a water flow (water spigot), disconnect from spigot, and then place that end at an equal or slightly lower elevation to generate suction.

At this point, the hose is draining the pool of all its water until air stops the suction force at the submerged end. In this scenario, the hose is acting as an extension (increases the volume) of the source of water and the suction force should be just a partial vacuum (relative negative pressure).

Scaling this idea up, use the setup except now the ocean is the source of water. The atmosphere of the earth is pushing down on the entire 2-dimensional surface of the ocean. The water itself has mass that is pushing on all sides of the submerged hose opening. Once the flow has been reversed from adding to the ocean to removing, would it be possible to generate electricity with only the "negative flow"?

• Just curious: what will you drain the ocean into? If you provide a drawing it will be easier to explain what's missing in your reasoning. – S. McGrew Apr 19 '20 at 13:49
• It's not what the ocean is draining into which could be itself at different elevation, but the flow rate itself can be used to generate electricity. The flow rate is being provided by the "ocean" sink. – Doug Apr 19 '20 at 13:51
• The flow rate is provided by the difference in gravitational potential energy at the top of the ocean and at the bottom of the siphon tube. There will be no flow if the bottom of the siphon tube is below the surface of a second reservoir whose surface is the same height as the ocean's surface. – S. McGrew Apr 19 '20 at 14:29
• I agree with your statement 100%. If you don't care about where the water drains (since all water leads back to the ocean or the nearby source), aren't there elevations near the coast where it is below sea level? (Think about New Orleans or the Netherlands). – Doug Apr 19 '20 at 14:39
• It kinda sounds like you're basically just describing hydro-power, which is very much used. – JMac Apr 19 '20 at 15:34

The first part of your question essentially refers to a so-called siphon, schematised below:

This is a 'machine' that converts potential (gravity) energy into kinetic energy. It is well described by Bernouilli's Principle:

$$P_1+\frac{\rho v_1^2}{2}+\rho gz_1=P_2+\frac{\rho v_2^2}{2}+\rho gz_2\tag{1}$$ where: $$P_1=P_0+\rho gd$$ where $$P_0$$ is the atmospheric pressure. If $$d\ll z$$, then $$P_1\approx P_2=P_0$$.

And if the higher reservoir is large, then $$v_2 \gg v_1$$.

In that case $$\text{Eq. (1)}$$ reduces to:

$$\frac{\rho v_2^2}{2}\approx \rho gz$$ or: $$v_2\approx \sqrt{2 gz}$$

In principle some of the kinetic energy could be converted to (for example) electrical energy by means of a suitably sized turbine.

Scaling this idea up, use the setup except now the ocean is the source of water.

But this raises the question: where will the siphoned flow flow into?

• I can't deny your math, you are right to ask where the flow will flow into. In my example, I was trying to imagine that hose pouring back into the ocean. Now using the entire earth as the source, if the hose was long enough, wouldn't the curvature of the earth come into effect? – Doug Apr 19 '20 at 15:11
• @Doug Gravity is part of a central force field. Its magnitude is dependant ONLY on a point's distance to the CoG of the Earth (causing small variations in $g$ across the Earth). So you 'can't beat the system' with curvature. Have look at the Hoover dam to see how that 'machine' works. – Gert Apr 19 '20 at 15:18
• I'm not trying to beat gravity with curvature tricks, but with potential gravitational energy. As long as the hose mouths are at different relative elevations, doesn't that still count? – Doug Apr 19 '20 at 15:21
• @Doug As long as the hose mouths are at different relative elevations, doesn't that still count? Of course that works, as I've shown. But how would it work siphoning off an ocean? Where would the lower outlet be? And even if you used a disused mine shaft (or such like) it would quickly fill up. And to empty it again would cost the same energy as was liberated filling it up! – Gert Apr 19 '20 at 15:26
• Your best bet to exploit surface water remains to create a dam. As the rising water gains potential energy it can be used to drive turbines. – Gert Apr 19 '20 at 15:52


text:
## Tuesday, March 3, 2015

### Who Controls the Pace in Basketball, Offense or Defense?

During a recent chat with basketball analyst Seth Partnow, he mentioned a topic that came up during a discussion at the recent MIT Sloan Sports Analytics Conference. Who  controls the pace of a game more, the offense or defense? And what is the percentage of pace responsibility for each side? The analysts came up with a rough consensus opinion, but is there a way to answer this question analytically? I came up with one approach that examines the variations in possession times, but it suddenly occurred to me that this question could also be answered immediately by looking at the offense-defense asymmetry of the home court advantage.

As you can see in the R output of my NCAA team model code in one of my public basketball repositories, the offense at home scores points at a rate about $$e^{0.0302} = 1.031$$ times the rate on a neutral court, everything else the same. Likewise, the defense at home allows points at a rate about $$e^{-0.0165} = 0.984$$ times the rate on a neutral court; in both cases the neutral court rate is the reference level. Notice the geometric asymmetry; $$1.031\cdot 0.984 = 1.015 > 1$$. The implication is that the offense is responsible for about the fraction $\frac{(1.031-1)}{(1.031-1)+(1-0.984)} = 0.66$ of the scoring pace. That is, offensive controls 2/3 of the pace, defense 1/3 of the pace. The consensus opinion the analysts came up with at Sloan? It was 2/3 offense, 1/3 defense! It's nice when things work out, isn't it?

I've used NCAA basketball because there are plenty of neutral court games; to examine the NBA directly we'll have to use a more sophisticated (but perhaps less beautiful) approach involving the variation in possession times. I'll do that next, and I'll also show you how to apply this new information to make better game predictions. Finally, there's a nice connection to some recent work on inferring causality that I'll outline.


text:
Antidotes in De..

Hypoglycemia is a common cause of altered mental status. Although classically associated with tachycardia, tremor, and diaphoresis, the predictive value of these manifestations is too low to be relied upon. As a result, all patients with altered consciousness require either rapid point-of-care testing of their glucose concentrations or empiric treatment for presumed hypoglycemia. When rapidly diagnosed and treated, hypoglycemic patients typically recover without sequelae. Delayed or incomplete therapy may lead to permanent neurologic dysfunction.

Adenosine triphosphate (ATP) provides the metabolic energy that fuels all critical cellular processes in all organs. In the adult brain, the anaerobic and aerobic metabolism of glucose through glycolysis and the citric acid cycle, respectively, are the primary sources of ATP (see Chap. 12). Although the adult brain can use fatty acids, amino acids, and ketones as alternate substrates for ATP synthesis, these are not adequate to sustain normal cerebral function in the setting of glucose deprivation. In the brain of fetuses and neonates, glucose is the only substrate for ATP production.81,99Hypoglycemia is defined by organ dysfunction in the setting of inadequate concentrations of glucose; it cannot be safely defined by strict numerical values. The onset of hypoglycemia in both adults and children is followed rapidly by global cerebral dysfunction. In individuals with diabetes, the density of neuronal insulin receptors varies as a function of glycemic control so that diabetics with poor glycemic control have fewer neuronal glucose receptors and may experience hypoglycemic symptoms at much higher concentrations of glucose than those who are normally euglycemic. An important study of diabetics demonstrated that the mean blood glucose concentration for symptomatic hypoglycemia in poorly controlled diabetics was 78 ± 5 mg/dL compared with 53 ± mg/dL in normal control subjects.11 Hypoglycemia may cause neuropsychiatric sequelae effects that are clinically indistinguishable from those of other toxic-metabolic and structural brain injuries, which may include focal stroke syndromes, movement disorders, seizures, irritability, confusion, delirium, coma, and irreversible encephalopathy.3,33,59,67,69,70,85,103,108 The heart is partially dependent on glucose for energy production. Hypoglycemia causes myocardial stress that may manifest as angina and or dysrhythmias. This is aggravated by the systemic catecholamine response to hypoglycemia.34,62,68,71,72

The history and physical examination do not reliably detect patients who are hypoglycemic.47 Tachycardia, diaphoresis, pallor, hypertension, tremors, hunger, and restlessness tend to predominate when the decline in blood glucose concentration is rapid. These signs and symptoms may be blunted or absent from the use of β-adrenergic antagonists. Central nervous system signs of neuroglycopenia are nonspecific. They include visual disturbances, psychiatric disturbances, confusion, stupor, coma, seizures, and focal neurologic findings.67,86 In children, the only sign of neuroglycopenia may be lethargy or irritability.110

In most cases, the rapid correction of hypoglycemia by the administration of 0.5 to 1.0 g/kg of concentrated intravenous (IV) dextrose (6536232) immediately reverses these neurologic and ...

Sign in to your MyAccess profile while you are actively authenticated on this site via your institution (you will be able to verify this by looking at the top right corner of the screen - if you see your institution's name, you are authenticated). Once logged in to your MyAccess profile, you will be able to access your institution's subscription for 90 days from any location. You must be logged in while authenticated at least once every 90 days to maintain this remote access.

Ok

## Subscription Options

### AccessEmergency Medicine Full Site: One-Year Subscription

Connect to the full suite of AccessEmergency Medicine content and resources including advanced 8th edition chapters of Tintinalli’s, high-quality procedural videos and images, interactive board review, an integrated drug database, and more.


text:
## Friday, April 27, 2012

### Tim Duy — Bernanke's Shift

Bottom Line:  Bernanke is being disingenuous in his defense.  Despite his claims that his earlier views only applied to deflation, his writings still appear at odds with his willingness to embrace a new price and aggregate demand paths.  Moreover, the Fed's own forecasts clearly do not support his contention that the target is symmetric, but indeed a hard ceiling.  The Fed must also know that the by reducing the path of inflation they have knowing altered the distribution of outcomes in a way that is likely to slow the pace of recovery. Finally, with inflation near 2 percent, I suspect the bar toward another round of QE is higher than many believe.
Read it at Tim Duy's Fed Watch
Bernanke's Shift
by Tim Duy


text:
This is an old revision of the document!

# Lecture 40: Final Review Lecture

## Young's double slit experiment

If we treat each of the slits as a point source of circular wavefronts. The condition for constructive interference (bright fringes) is

$d\sin\theta=m\lambda$       (m=0,1,2,..)

and for destructive interference (dark fringes)

$d\sin\theta=(m+\frac{1}{2})\lambda$       (m=0,1,2,..)

## Reflection from a transparent medium

As in the case of a wave on a rope that is incident on a heavier rope and is reflected with a 180o phase change when a light wave is reflected from a more optically dense media a 180o phase change occurs. This effect is important when we want to consider interference effects in thin films.

## Anti reflective coating

As the two reflections both occur from more optically dense media they both experience a $\frac{\lambda}{2}$ phase change on reflection. To have the light be out of phase we need to light that goes through the coating to have advanced by $\frac{\lambda}{2}$ for destructive interference to occur. Critically, when destructive interference occurs the light is not lost, but is instead transmitted. As the wavelength of light in a medium is given by $\lambda=\frac{\lambda_{0}}{n}$ where $n$ is the refractive index of the medium and $\lambda_{0}$ is the wavelength of the light in free space, the thickness of the coating should be $\frac{\lambda}{4n_{2}}$.

In practice the light incident will not all be the same wavelength, so the thickness of the coating is typically chosen to work optimally in the center of the visible band (~550nm).

## Air wedge

For a single wavelength dark rings will occur whenever

$2t=m\lambda$      (m=0,1,2,..)

and bright rings will occur whenever

$2t=(m+\frac{1}{2})\lambda$      (m=0,1,2,..)

For white light different colors will experience constructive interference at different thicknesses, leading to the colorful lines we see when an air gap is under normal light.

## Intensity pattern for a single slit

Rather than writing our equation in terms of k

$I_{\theta}=I_{0}\mathrm{sinc^{2}}(\frac{Dk}{2}\sin\theta)$

it is convenient to write it in terms of the wavelength, using $k=\frac{2\pi}{\lambda}$

$I_{\theta}=I_{0}\mathrm{sinc^{2}}(\frac{D\pi}{\lambda}\sin\theta)$

This function has a central maxima and then minima at

$D\sin\theta=m\lambda$       $m=\pm 1,\pm 2,\pm 3,..$

## Diffraction in the double slit

Now we have an expression for the intensity from a slit of width $D$

$I_{\theta}=I_{0}\mathrm{sinc^{2}}(\frac{D\pi}{\lambda}\sin\theta)$

we can use consider slits as sources instead of the point sources we considered earlier which gave

$I_{\theta}=I_{0}\cos^{2}(\frac{d \pi}{\lambda}\sin \theta)$

The combination of these gives

$I_{theta}=I_{0}\mathrm{sinc^{2}}(\frac{D\pi}{\lambda}\sin\theta)\cos^{2}(\frac{d\pi}{\lambda}\sin \theta)$

## Diffraction grating

Irrespective of the number of slits $n$ the condition for maxima is the same as for the double slit

$d\sin\theta=m\lambda$       (m=0,1,2,..)

but that the larger the number of slits from which diffraction occurs the sharper the maxima will be.

## Polarizers

The polarization of the light after it has passed through the polarizer is the direction defined by the polarizer, so the way to approach the calculation of the magnitude of the intensity that passes through the polarizer is by finding the component of the electric field which is in that direction

$E=E_{0}\cos\theta$

the intensity is then

$I=E^{2}=E_{0}^{2}\cos^{2}\theta=I_{0}\cos^{2}\theta$

This formula is known as Malus' Law.


text:
• March 02, 2021, 03:37:00 AM
• Welcome, Guest

Pages: [1] 2 3   Go Down

### AuthorTopic: Tail wagging. I’m getting frustrated.  (Read 9487 times)

0 Members and 1 Guest are viewing this topic.

#### brat

• member
• Offline
• Posts: 20
##### Tail wagging. I’m getting frustrated.
« on: July 23, 2011, 11:09:31 PM »

Spitfires and others --- P40/P51/F6F/F4U/Mosquito/Fury - - - -
I have been playing around for ages with my joystick input under hardware setup and the profiler to calibrate my twist handle Extreme 3D Pro but in the series of oscillations that I am experiencing in IL2 it is hard to tease apart the various interrelated effects. I have only managed to find a small reduction in amplitude and the irregularity of the motion. The linear relationship between yawing and this crazy oscillation has eluded me.
Self having ratings on Tiger Moth, Harvard, Chipmunk to name a few tail draggers and having the good fortune to fly a TR9 Spit from the back seat last year I found it had excellent directional stability and that there was very little oscillation when centering the ball in hard maneuvers.
The aircraft was stable with positive stability characteristics in all axis. However during maneuvers the dynamic and static stability mix nicely towards the negative side which gives a lively feel to the aircrafts nature and does get you looking up sun and feeling for the firing button.
With seat of the pants flying I believe us full size pilots often damp out yaw/pitch oscillations so unconsciously that we are not aware that an oscillation was even starting. Even so, I have never experienced anything like the crazy motions I am now experiencing in our sim.
I am aware that the FM’s are modeled on some very accurate data by some clever people but using a twist grip rudder on a little stick where there is no feedback is so far removed from a full size aircraft when gunnery aiming or balancing the turn result in a very loose aircraft that is so difficult to fly with any sort of accuracy and realism. The old classic iL2 was never this much a problem.
Some people might be happy with these fm’s. Perhaps those with top of the line expensive flight controllers do not have this problem, but for me it just doesn’t work anymore.
I know there are others with similar views from some of the comments on the UP site and if there are other members experiencing the same please post a reply and give us your thoughts.

Logged

#### Wildchild

• member
• Offline
• Posts: 993
• Bf 109 Killer
##### Re: Tail wagging. I’m getting frustrated.
« Reply #1 on: July 23, 2011, 11:43:11 PM »

+100000 I find it impossible for me to keep the ball in the center even when doing simple turns
Logged

#### CWMV

• Kalashnikov connoisseur
• Modder
• member
• Offline
• Posts: 2706
• A free people ought to be armed and disciplined.
##### Re: Tail wagging. I’m getting frustrated.
« Reply #2 on: July 23, 2011, 11:47:14 PM »

Me too, until I got rudder pedals.
Logged

#### SAS~Anto

• Aussie Bush Pilot
• Editor
• member
• Offline
• Posts: 4434
• Retired from modding
##### Re: Tail wagging. I’m getting frustrated.
« Reply #3 on: July 24, 2011, 01:08:36 AM »

Me too, until I got rudder pedals.
+1. Sorry guys, you can't complain about flight models unless you are using a proper setup. I can sympathize with you all as I have been in the same setup (used to crash my 109s almost every time on landing). Most twist sticks are cheap and have crappy sensors. You can always adjust the difficulty settings so you don't have to worry about yaw control.

Honestly the easiest solution is for you to dish out and buy some pedals. This does sound a little high and mighty, but the alternative (creating new FMs and then supporting multiple versions of multiple incompatible modpacks) is simply not sustainable and a waste of resources.
Logged

#### RGA

• member
• Offline
• Posts: 81
##### Re: Tail wagging. I’m getting frustrated.
« Reply #4 on: July 24, 2011, 01:30:08 AM »

Indeed rudder pedals are MUST. My flying skill increased 10-fold after I bought myself a crappy rudder pedals set.
But actually, I also find IL-2 aircrafts somewhat instable. With pedals and trimming I can keep my ball centered most of the time, but keeping it completely standing is next to impossible. Curiously, P-51B's ball is quite sensitive, it swings like crazy even in the mildest maneuvers (perhaps that's why it's placed right before your eyes). On the other hand, P-51D's ball is very stable, never have to look at it during the dogfight, feeling is enough.
Logged

#### Wildchild

• member
• Offline
• Posts: 993
• Bf 109 Killer
##### Re: Tail wagging. I’m getting frustrated.
« Reply #5 on: July 24, 2011, 01:32:28 AM »

Good point anto. Do you know were I can find an inexpensive set of rudder pedals?
Logged

#### CWMV

• Kalashnikov connoisseur
• Modder
• member
• Offline
• Posts: 2706
• A free people ought to be armed and disciplined.
##### Re: Tail wagging. I’m getting frustrated.
« Reply #6 on: July 24, 2011, 01:43:30 AM »

Define cheap.
Got mine for around 60 bucks.
Logged

#### Wildchild

• member
• Offline
• Posts: 993
• Bf 109 Killer
##### Re: Tail wagging. I’m getting frustrated.
« Reply #7 on: July 24, 2011, 01:47:49 AM »

Let's go for around 30... Budget flyer
Logged

#### CWMV

• Kalashnikov connoisseur
• Modder
• member
• Offline
• Posts: 2706
• A free people ought to be armed and disciplined.
##### Re: Tail wagging. I’m getting frustrated.
« Reply #8 on: July 24, 2011, 02:01:11 AM »

Try eBay.
Logged

#### SAS~Tom2

• SAS Honourable Member
• member
• Offline
• Posts: 7954
• no sweat
##### Re: Tail wagging. I’m getting frustrated.
« Reply #9 on: July 24, 2011, 09:00:20 AM »

My 2 cents..
I also use a low-price but very proven Logitech Wingman Ex Dig 3D...have bought it 6-7 times in the last 10 years..since all other sticks I used sucked or had less spring strength and feel.
But twist rudder really sucks with IL-2. A fact.
It works very well with FreeFalcon, modern jet sim, where rudder inputs are basically ironed out above 300 Knots.

But the twist stick will always compromise your inputs in IL-2...
For that reason and since I mainly fly jets...I simply cut off all my rudder inputs from the twist grip in IL-2.
I fly it like a modern jet sim-with as little rudder input as necessary.
This also means totally reinventing your flying skills-when I "customized" this way of flying when first flying Falcon4.0-it was really hard to aim, land and shoot with it.
Once you're accustomed to it-it becomes your 2nd nature.

For a passionate prop-fighter WWII sim pilot, I would recommend dedicated rudders..or at least deactivate rudders from your twist grip joystick in your IL-2 setup ingame and assign them to a key on your stick.-That way you at least have an alternative for emergencies like stalls, spins, landings or battle situations.

If your fun depends on flying WWII- or WWI fighters, I would suggest to buy yourself rudders. Even for modern jet sims they make sense, albeit in RL jets like F-16s and Mirage 2000s would see very limited use of rudders AFAIK.

Logged

#### SAS~Anto

• Aussie Bush Pilot
• Editor
• member
• Offline
• Posts: 4434
• Retired from modding
##### Re: Tail wagging. I’m getting frustrated.
« Reply #10 on: July 24, 2011, 09:41:14 AM »

Well most modern jets have a yaw dampening system which means you only really need that rudder for emergencies, tough dogfights and crosswind landings.

Wildchild:
I've tried 3 of the main rudder pedal sets out on the market and this my opinion on them. You are looking at around US$220 new for most of these (more for G940 system), get them as cheap as$60 second hand.

Saitek ProFlight Pedals
- Decent quality build, comfortable, adjustable to many sizes of feet
- Dead zone was a little large
- Good range of movement

CH Products
- Decent build quality but not adjustable
- Had extreme difficulty tuning this to work with Microsoft Flight Sim but worked okay (if a little too sensitive) in Il-2
- Not my first choice

G940 Pedals
- Very solid build, adjustable friction
- Carpet spikes! With other two, had to use block of wood to stop them sliding to the wall
- Smaller range of movement than Saitek
- Disadvantage is that these only come in set with G940 stick (which is pretty decent for a Logitech joystick)

I personally say go for the Saitek but prices second hand tend to be around US\$150. Sometimes you are lucky like Uwe and will pickup some cheap. I was a poor student (now I'm slightly less poor) and had to sacrifice a fortnights pay for my pedals. Tell you what, despite eating Mee Goreng for a week, the pedals brought a few years of enjoyment
Logged

#### CWMV

• Kalashnikov connoisseur
• Modder
• member
• Offline
• Posts: 2706
• A free people ought to be armed and disciplined.
##### Re: Tail wagging. I’m getting frustrated.
« Reply #11 on: July 24, 2011, 10:44:24 AM »

Luck had nothing to do with it, just shopped around, but probably easier for us stateside than yall down under.
I'll post up the link at provantage in a dew minutes.
Logged
Pages: [1] 2 3   Go Up

Page created in 0.055 seconds with 24 queries.


text:
Version:

# Debugging Lua Scripts

O3DE provides Lua scripts with several functions to make debugging easier.

## Logging to the Console

To print text to O3DE Editor and the Console, use the Debug.Log() function.

The following example shows the use of the Debug.Log() function.

local LoggingTest = { }
function LoggingTest:OnActivate()
componentName = "MyComponent"
Debug.Log(componentName .. " has been activated.")
end

return LoggingTest

## Using an assert to detect potential issues

You can use the assert function to display an error message in the console when conditions are detected that might result in an execution fault. The assert function takes two arguments: a condition that evaluates to true or false, and a message to display if the condition is false.

The following example shows the use of the assert function.

function SampleScript:DoStuff()
-- This value should never be negative
assert( self.positiveValue >= 0, "Expected a positive value! Got: " .. self.positiveValue )
end

-- Console output when the value of self.positiveValue is -5:
-- [Error] Lua error (2 - [string "q:/lyengine/branches/systems/dev/samplespro..."]:61: Expected a positive value! Got: -5) during call samplescript:DoStuff

## Communicating errors

You can use the Debug.Error() function to display an error in the console and halt execution of the current script function. This does not halt all execution of the script. If you have active handlers, they can still be called when the engine posts notifications. The Debug.Error() function takes arguments similar to the Debug.Assert function: a condition and a message. The message is displayed in bright red and execution halts only if the condition is false.

The following example shows the use of the Debug.Error() function.

function SampleScript:CheckAndError()
-- This value should never be negative
Debug.Error( self.positiveValue >= 0, "Detected a negative value: " .. self.positiveValue )
end

-- Console output when the value of self.positiveValue is -5:
-- [Error] Error on argument 0: Detected a negative value: -5

## Displaying a warning when user attention is required

A script condition can occur that does not adversely affect the execution of the script but might be useful for the user to know about. The Debug.Warning() function uses arguments similar to those of the Error and Assert functions but just displays an orange warning message in the console. It does not halt execution.

The following example shows the use of the Debug.Warning() function.

function SampleScript:CheckValue()
-- This value should probably never be negative
Debug.Warning( self.positiveValue >= 0, "Detected a negative value: " .. self.positiveValue )
end

-- Console output when the value of self.positiveValue is -5:
-- [Warning] Warning on argument 0: Detected a negative value: -5


text:
More topics in Digital Logic Design and Digital Electronics Course

# Comparator – Designing 1-bit, 2-bit and 4-bit comparators using logic gates

A Comparator is a combinational circuit that gives output in terms of A>B, A<B, and A=B. This is entirely expected from the name. A digital comparator’s purpose is to compare numbers and represent their relationship with each other.

In this post, we will make different types of comparators using digital logic gates. We will begin by designing a simple 1-bit and 2-bit comparators. The circuit for a 4-bit comparator will get slightly more complex. Let’s begin.

Contents

## How to design a simple 1-bit comparator?

A 1-bit comparator compares two single bits.

 A B A>B A

Let’s apply a shortcut to find the equations for each of the cases. Normally, we can use a K-map. But this shortcut is efficient and handy when you understand it.

For A>B, there is only one case when the output is high when A=1 and B=0. Let’s call this X. We can write the equation as follows

X(which stands for A>B) = AB’

Similarly, denote A<B with Y and A=B with Z.

Since Y is high when A=0 and B=1, we get the following equation

Y = A’B

Since Z is high in two cases, there will be an OR gate. This is because the logic behind an OR gate is that a high output can be achieved in one or more cases. Z is high when A=0 and B=0, it is also high when A=1 and B=1. Therefore,

Z = A’B’ + AB

This is similar to the equation of an EXNOR gate. Hence,

Z = A$\oplus$B

## How to design a 2-bit comparator?

Let’s plot the truth table for a 2-bit comparator

 A1 A0 B1 B0 A>B A

The shortcut that we saw above can be used here too. But notice that since we have four variables (A1, A0, B1, B0) and each of the three outputs is high at least four times, the equations that we will get will have four terms of 4 variables. At least. So, though applying the shortcut is possible, we won’t.

K-maps come in handy in situations like these. Because it is possible to achieve the most straightforward equation using them, and remember, the simpler the equation, the lesser the logic gates required. That is the aim of any designing process – to obtain the simplest hardware implementation.

Using K-maps

we get the following equations

A>B = A1B1′ + B0′(A0B1′ + A0A1)

A<B = B1A1′ + B0B1A0′ + A1’A0’B0

A=B = $\overline { A0\oplus B0 }$$\overline { A1\oplus B1 }$

## How to design a 4–bit comparator?

The truth table for a 4-bit comparator would have 4^4 = 256 rows. So we will do things a bit differently here. We will compare each bit of the two 4-bit numbers, and based on that comparison and the weight of their positions, we will draft a truth table.

 A3B3 A2B2 A1B1 A0B0 A>B AB3 x x x 1 0 0 A3B2 x x 1 0 0 A3=B3 A2B1 x 1 0 0 A3=B3 A2=B2 A1B0 1 0 0 A3=B3 A2=B2 A1=B1 A0

Now let’s derive the equations for the three outputs.

A=B

Recall the 1-bit comparator circuit we saw above. The equation for the A=B condition was A$\oplus$B. Let’s call this x. Taking a look at the truth table above, A=B is true only when A3=B3 and A2=B2 and A1=B1 and A0=B0. Hence

Z (A=B) = A3B3 . A2B2 . A1.B1 . A0.B0 = x3x2x1x0

A>B

Since there are multiple occasions where this particular condition is high, we will OR (add) each of those individual occasions. We find the first instance of A>B at the top of the table where A3>B3. For this to be possible in a binary system, A3 has to be equal to 1, and B3 has to be equal to 0. Since there are only 0s and 1s in a binary system. We can represent this as A3.B3′. Moving on to the next instance of A>B, we can see that it occurs at A3=B3 and A2>B2. From the equation for A=B above, A3=B3 can be represented as x3. And this entire instance can be written as x3A2B2′. Similarly, deriving equations for the remaining instances, we get the following equation

X(A>B) = A3B3′ + x3A2B2′ + x3x2A1B1′ + x3x2x1A0B0′

A<B

Employing the same principles we used above, we get the following equation

Y(A<B) = A3’B3 + X3A2’B2 + X3X2A1’B1 + X3X2X1A0’B0

Designing the circuit based on the above equations, we get the following logic diagram for a 4-bit comparator.

## How do I remember this?

This is the exact question I had when I first studied this truth table. I felt that this truth table was made only because whoever made it knew that it had to be made this way. How would I, as a student, be expected to devise a new system for a truth table? The answer is, you don’t have to. You are entirely free to do it the old way with 256 rows. But this is a more natural way to deal with when you have many variables that will end up in a vast truth table. You can remember it and maybe use it elsewhere when the need arises. If not, that’s okay, too; you can bookmark this page and refer to it when you are tasked with making a huge truth table.

Note:

X More topics in Digital Logic Design and Digital Electronics Course
 Digital Number Systems And Base Conversions Boolean Algebra – All the Laws, Rules, Properties and Operations Binary Arithmetic – All rules and operations Sequential and Combinational logic circuits – Types of logic circuits Logic Gates using NAND and NOR universal gates Half Adder, Full Adder, Half Subtractor & Full Subtractor Comparator – Designing 1-bit, 2-bit and 4-bit comparators using logic gates Multiplier – Designing of 2-bit and 3-bit binary multiplier circuits 4-bit parallel adder and 4-bit parallel subtractor – designing & logic diagram Carry Look-Ahead Adder – Working, Circuit and Truth Table Multiplexer and Demultiplexer – The ultimate guide Code Converters – Binary to Excess 3, Binary to Gray and Gray to Binary Priority Encoders, Encoders and Decoders – Simple explanation & designing Flip-Flops & Latches – Ultimate guide – Designing and truth tables Shift Registers – Parallel & Serial – PIPO, PISO, SISO, SIPO Counters – Synchronous, Asynchronous, up, down & Johnson ring counters Parity Generator and Parity Checker Memories in Digital Electronics – Classification and Characteristics Programmable Logic Devices – A summary of all types of PLDs Difference between TTL, CMOS, ECL and BiCMOS Logic Families
CLOSE


text:
# integral

• Jul 28th 2009, 07:25 PM
lsutigers
integral

Prove the equality: int[(t^n)(e^-t)] from 0 to infinity = n!, for n=0,1,2,...

(Evaluate int(e^(-xt)) dt from 0 to infinity and consider whether repeatedly differentiating under the integral is permitted)
• Jul 28th 2009, 07:52 PM
mr fantastic
Quote:

Originally Posted by lsutigers

Prove the equality: int[(t^n)(e^-t)] from 0 to infinity = n!, for n=0,1,2,... Mr F says: Proof by induction would be a simple approach (combined with integration by parts).

(Evaluate int(e^(-xt)) dt from 0 to infinity and consider whether repeatedly differentiating under the integral is permitted) Mr F says: Is this a seperate question? Or is it related to the above?

..
• Jul 28th 2009, 08:26 PM
Random Variable
I'll give it a try.

We know that $\int^{\infty}_{0} e^{-at} \ dt = \frac {1}{a}$

$(\text{-}1)^{n}\frac {\partial^{n}}{\partial a^{n}} \int^{\infty}_{0} e^{-at} \ dt = (\text{-}1)^{n}\int^{\infty}_{0} \frac {\partial^{n}(e^{-at})}{\partial a^{n}} \ dt$ $= \int^{\infty}_{0}t^{n}e^{-at} \ dt$

and $(\text{-1})^{n}\frac {\partial^{n}}{\partial a^{n}}\frac{1}{a} = \frac{n!} {a^{n-1}}$

so $\int^{\infty}_{0}t^{n}e^{-at} \ dt = \frac{n!}{a^{n-1}}$

now let a =1

then $\int^{\infty}_{0}t^{n}e^{-t} \ dt = n!$
• Jul 29th 2009, 06:27 AM
HallsofIvy
But where do you show that "repeatedly differentiating under the integral is permitted"?

I think I would be inclined to use "proof by induction", avoiding differentiating under the integral.

If t= 0, $\int_0^\infty e^{-t}dx= 1= 0!$

Assume that for some k, [tex]\int_0^\infty t^ke^{-t}dx= k!. Then do $\int_0^\infty t^{k+1}e^{-t}dt$ using integration by parts: let $u= t^{k+1}$, $dv= e^{-t}dt$ so that [tex]du= (k+1)t^k dt[tex] and $v= -e^{-t}$. Then the integral becomes $\left[-t^{k+1}e^{-t}\right]_0^\infty+ (k+1)\int_0^\infty t^k e^{-t}dt$[tex]= (k+1)k!= (k+1)!

(Ahhh! I see that mr. fantastic already suggested that.)
• Jul 29th 2009, 08:06 AM
Random Variable
Quote:

But where do you show that "repeatedly differentiating under the integral is permitted"?
I think it's permitted because $f(t, a) = e^{-at}$ and the first n derivatives of $f(t,a)$ are continuous in both t and a. Am I mistaken?


text:
# Tag Info

## About optional-arguments

{optional-arguments} concerns commands and environments that accept optional arguments. For example the \sqrt command can be used as \sqrt[3]{x}.


text:
# 1 Introduction

This project is made for a competition, which I received grand prize. The key objective of this challenge is to build a machine learning model that handles given data set, with the most suitable encoding schemes and the least set of feature variables. Also, it requires no correlated variables in the set of predictors. The model should be built based on the Challenge dataset, and to predict the observations in Evaluation dataset.

Here is a data sample for Challenge dataset.

# 2 Data Evaluation

The received datasets files are “Challenge Data Set.xlsx” (called Challenge dataset) and “Evaluation Data Set.xlsx” (called Evaluation dataset). In the Challenge dataset, there are 3000 records with 31 variables. Among those variables, only one variable with label “XC”, is categorical with 5 levels – “A”, “B”, “C”, “D” and “E”, and the rest are numerical variables. For the binary response “y”, its mean is 0.31, so the class 0 and 1 is unbalanced. Also, there are 7000 records in the Evaluation dataset. For both of the datasets, there are no missing values. Furthermore, in terms of the variable “XC”, you can find the distribution of 5 levels in Evaluation dataset, and Challenge dataset in Table 1, as well as the distribution of response “y” grouped by those levels. From the table, you can see the 5 levels are nearly evenly distributed in both datasets. Later, We will transform this categorical variable by replacing each level with their related the average of response “y”.

# 3 Method

## 3.1 Motivation

For this project, we proposed a novel hybrid model, which is a combination of neural network and LightGBM (NN +lightGBM). Here are my motivations on combining those two models

1. Comparing with other widely used machine learning models, lightGBM has its own unique advantages for this classification assignment. First, as a Gradient Boosting Decision Tree (GBDT) method, it is very convenient to apply lightGBM to demonstrate the importance of features. Meanwhile, unlike other GBDT algorithms, such as XGboost, lightGBM can deal with categorical data. It gives me more flexibility on dealing with categorical variables. Also, lightGBM is better in training speed and space efficiency than other GBDT algorithms. Furthermore, unlike some traditional machine learning methods, lightGBM can handle data with imbalanced classes

2. For this supervised learning , neural networks can be used to perform feature learning, since they learn a representation of their input at the hidden layer(s) which is subsequently used for classification at the output layer. The later experiments show by adopting those newly extracted features, the model’s performance increases significantly.

Overall, the instability of the single model prediction and the shallow model’s unsatisfactory data feature mining inspired me to move toward deep learning and ensemble learning. Hence, we decided to adopt this fusion model. This model has shown promising performance in terms of the F1 score.

## 3.2 Pipeline Details

The model pipeline can be divided into three parts

### 3.2.1 Data Preparation (CleanData.py)

In this part, the data is divided into test, training and validation set randomly with ratio, 2:6:2. Meanwhile, all sets’ variable “XC” is transformed into numerical type, by replacing that level with the average of “Y” under that level in the training set. This transformation will be applied to all sets, including the Evaluation dataset.

### 3.2.2 Model (Model.py)

This part has 3 steps.

Step 1-Feature Selection by lightGBM: The goal is to limit the number of features used in the final model based on features’ importance and correlation with others. The averaged importance score for each feature was calculated by using lightGBM . Next, we calculate the correlation within those features. Then, I select features which have high importance scores (higher than an importance score threshold) and low correlation (lower than a correlation threshold). For the features which are highly correlated, the one with highest importance score will be chosen from correlated features sets. The feature selection is done in the training set.

Step 2- Exact features by neural networks:After obtaining important features, this pipeline will generate new features from those selected features, by extracting the last hidden layers from a neural network built on a training set. The hyperparameters of this neural network were tuned by grid-searching in the validation set. Finally, the training, test, validation and Evaluation set will be transformed with the last hidden layers to get new features. Notice all observations from all sets are normalized based on the training set.

Step 3- LightGBM tuning and testing:After combining selected features and extracted features from step 1 and 2, the transformed training and validation data will be used for training and hyperparameter tuning. The tuned model will be saved for later deployment.

The overall operations on the Evaluation data:

1. Dropping the unimportant features in step 1, and keep the set of selected features, S1
2. Normalizing S1 by the std and mean of the training set, call it S2
3. Extracting hyper features from S2 by the neural network trained in step2, call it S3
4. Combining S1 and S3 to get the final feature set, S4
5. Using the trained lightGBM to predict the Evaluation dataset’s labels based on S4.

### 3.2.3 Model Deployment (Wrapper.py)

The pipeline will load the saved model to predict the label for the transformed Evaluation dataset.

## 3.3 Model Details

Features’ Importance Score: In lightGBM, The importance of a feature is the number of times the feature is used in a model. In terms of feature importance score, the train set is divided into several folds. In each fold, we calculate the feature importance for each feature, and average the importance for each feature over all folds. The averaged result will be the feature importance score. The logic behind this is using randomness in each run of lightGBM fitting. Therefore, the ensemble mean can be strong evidence to reflect the importance of features. Here is the feature importance graph.

As the figure shows, there is a big jump in the feature importance score between “X20” and “X23”. Hence, features can be easily grouped into two sets based on importance score. This indicates how important the feature is and how to control the number of features when building the model.

Neural Networks:This neural network has 5 layers- one input layer, two dense hidden layers, one output layer and there is a drop-out layer to avoid over-fitting. Figure 3 shows the final selected neural network’s structure.

The hyperparameters of the NN include the size of the neurons, the activation function and the drop-out rates. The output layer uses sigmoid function, thus for each sample, it predicts a value between 0 to 1 – the probability that this sample belongs to class 1.

# 4 Performance

For testing purposes, we run the model for 146 times. For getting a convincing result, in each run, the pipeline will randomly split the training, test, and validation dataset. Also, for all models’ hyperparameters and feature sets will be refitted, and reselected. As a comparison to NN+lightGBM, the prediction results from the neural network, which used to generate features, were also recorded. Figure 4 shows the distribution of the F1 score from all runs. Both model’s averaged F1-scores are above 0.95, but NN+lightGBM has better performance overall and has a lower standard deviation in F1-scores. Thus NN+lightGBM is a more robust model compared to NN. It also means if you are willing to sacrifice some F1-score, you can use the trained neural network to predict for gaining some efficiency.

In terms of dealing with imbalance class, ROC curves may present an overly optimistic view of an algorithm’s performance if there is a large skew in the class distribution. However, Precision-Recall (PR) curves, often used in Information Retrieval , have been cited as an alternative to ROC curves for tasks with a large skew in the class distribution.

Here is the precision and recall curve on one run. The area under the curve gets to 0.998. Therefore, the model performs extremely well under the imbalance situation. we also attached a ROC curve as a comparison. In terms of limitation of the model, there is a potential risk to have overfitting, since we are using neural network and lightGBM. Also, it is hard to interpret the newly extracted features by the neural networks.

# 5 Code

The related code can be found from here

# 6 Reference

• Maura E Stokes, Charles S Davis, and Gary G Koch.Categorical dataanalysis using SAS. SAS institute, 2012.
• Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma,Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boostingdecision tree. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus,S. Vishwanathan, and R. Garnett, editors,Advances in Neural InformationProcessing Systems 30, pages 3146–3154. Curran Associates, Inc., 2017.
• Juliaflower. Feature selection lgbm with python, Dec 2018.
• Jason Brownlee. Roc curves and precision-recall curves for imbalanced clas-sification, Jan 2020.
comments powered by Disqus


text:
# Motion problem

From Levíc to Košíc go car at speed 81 km/h.
From Košíc to Levíc go another car at speed 69 km/h.

How many minutes before the meeting will be cars 27 km away?

Result

t =  10.8 min

#### Solution:

$v_{ 1 } = 81 \ km/h \ \\ v_{ 2 } = 69 \ km/h \ \\ s = 27 \ km \ \\ \ \\ v = v_{ 1 }+v_{ 2 } = 81+69 = 150 \ km/h \ \\ t_{ 1 } = \dfrac{ s }{ v } = \dfrac{ 27 }{ 150 } = \dfrac{ 9 }{ 50 } = 0.18 \ h \ \\ t = t_{ 1 } \rightarrow h = t_{ 1 } \cdot \ 60 \ h = 10.8 \ h = \dfrac{ 54 }{ 5 } = 10.8 \ \text { min }$

Leave us a comment of this math problem and its solution (i.e. if it is still somewhat unclear...):

Be the first to comment!

#### Following knowledge from mathematics are needed to solve this word math problem:

Need help calculate sum, simplify or multiply fractions? Try our fraction calculator. Do you have a linear equation or system of equations and looking for its solution? Or do you have quadratic equation? Do you want to convert length units?

## Next similar math problems:

1. An equilateral triangle
The perimeter of an equilateral triangle is 33cm. How long is each side?
Calculate the radius of the circle whose length is 107 cm larger than its diameter
3. A baker
A baker has 5 1/4 pies in her shop. She cut the pies in pieces that are each 1/8 of a whole pie. How many pieces of pie does she have?
4. Sum of the digits
How many are two-digit natural numbers that have the sum of the digits 9?
5. 45th birthday
This year Mrs. Clever celebrated her 45th birthday. Her three children are now 7.11 and 15 years old. In how many years will Mrs. Clever's age be equal to the sum of her children's years?
6. Coal mine
The towing wheel has a diameter of 1.7 meters. How many meters does the elevator cage lower when the wheel turns 32 times?
7. Video game
Nicole is playing a video game where each round lasts 7/12 of an hour. She has scheduled 3 3/4 hours to play the game. How many rounds can Nicole play?
8. Forestry workers
In the forest is employed 56 laborers planting trees in nurseries. For 8 hour work day would end job in 37 days. After 16 days, 9 laborers go forth? How many days are needed to complete planting trees in nurseries by others, if they will work 10 hours a d
9. Three cats
If three cats eat three mice in three minutes, after which time 260 cats eat 260 mice?
10. Common divisors
Find all common divisors of numbers 30 and 45.
11. Numbers division
Which number should be divided by 2 3/4 to get 11/12?
12. Simple equation 10
35= 7*3*x what is x?


text:
## Wednesday, May 12, 2010

### Freezing in Space I - Blackest Night

In the last post I made, I discussed the fact that humans radiate energy.  In that post I calculated that we actually radiate quite a lot of power.  This immediately raises a few questions, the most obvious one being: How long would it take you to freeze in space?  This question is multifaceted, and I'm going to split it between two parts.  This first part, 'Blackest Night' is how quickly we'd freeze if we were completely lost in space, nothing anywhere near.  The second part, 'Turn On The Sun!' will address what would happen in near earth orbit.

We need to clarify what we mean when we say freezing in space.  The fatal temperature change for a human is (according to the all knowing internet), roughly a drop of 7C.  If you remember my previous post, we calculated that we would radiate about 860 W.  Now, we have to ask how much energy it takes to change our temperature by 7 C.  Well, as I've discussed a few times, the energy it takes to change the temperature is given by
$Q=mc\Delta T$
The mass of a human is ~75 kg.  We're mostly water, so let's just assume that we are all water.  This gives us a specific heat of 4.2 kJ/kg*K.  The energy needed for our 7 C temperature change is then
$Q=(75kg)(4.2kJ/kg*K)(7K)=2.2 MJ$

By the time we have dropped to 30 C, we are only radiating a power of
$P=eA\sigma T^4\=(.97)(1.7m^2)(5.67\cdot10^{-8}W/m^2K^4)(303K)^4=790W$
Let us assume that the average power radiated is the geometric average of these two powers,
$P_{avg}=\sqrt{(860W)(790W)}\approx825W$
This gives us a time to freeze of
$t=\frac{2.2MJ}{825W}\approx2700s$
This is ~45 minutes.  So you've got 45 minutes until a deadly freeze in deep space.  Seems a rather long time, does it not?  I'm fairly certain that you'd freeze much faster in antarctica than deep space.  Why?  Because in Antarctica you have more cooling mechanisms that just radiation, you have conduction in the air around you and convection of that warmer air away from your skin.  If you're adrift in space, for all that it is rather cold, the good news is that you'll asphyxiate before you freeze!

All of this was done assuming that there's no energy gain from anywhere.  That is, that we're stuck somewhere in the deepest space, the blackest night.  Tomorrow we'll consider what would happen if you were in near earth orbit, with all of these lovely energy sources around, particularly the sun.

#### 1 comment:

1. You also have internal mechanisms generating heat; you are metabolizing food and you'll start to shiver involuntarily at some point.


text:
# Likelihod ratio test: $f_0(x)=2x$ vs $f_1(x)=3x^2$ : $2n$ degree of freedom?

Let $\left\{ X_i \right \}$ be an $n$-sample with pdf $f$. Show that the likelihood ratio test statistic for \begin{align} H_0 &: f(x) = 2x \\ H_1 &: f(x)= 3x^2 \end{align} has a $\chi^2$ distribution with $2n$ degrees of freedom.

The likelihood ratio test statistic is $$\lambda = \frac{\prod f_0(x_i)}{\prod f_1(x_i)} = \frac{(2/3)^n}{t^n},$$ with $t=\prod x_i$. Then I should show $$\lim_{n\to\infty} -2\ln \lambda \stackrel{d}= \chi^2(2n).$$ How can the degree of freedom be $2n$?
I don't see any parameter in $H_0$ and $H_1$, to apply the theorem which says $$\lim_{n\to\infty} -2\ln \lambda \stackrel{d}= \chi^2(\#\Theta_1-\#\Theta_0).$$

-
$\lambda$ does not seem to have $\chi^2$ distribution under either $H_0$ or $H_1$. Probably you want to show that statement asymptotically. –  passerby51 Feb 24 at 16:56
@passerby51 Thanks, I meant asymptotically but didn't write clearly. I tried to copy a proof on the asymptotics of $\lambda$, but all the proof I know makes reference to the parameter space $\Theta_1$ and $\Theta_0$. –  Nicolas Essis-Breton Feb 24 at 19:41
The statement is still not quite correct. The RHS of your limit ($\chi^2(2n)$) depends on $n$ which shouldn't. Probably you want to show that $-2 \ln \lambda_n - Y_n$ goes to zero in probability, where $Y_n \sim \chi^2(2n)$ is some sequence of random variables. I am still not quite sure this is true. Are you sure you are not missing anything in the problem? –  passerby51 Feb 27 at 16:27
@passerby51 I think everything of the problem is here. I will try to find such a sequence $Y_n$. I tried to view the LRT as a goodness-of-fit test with $n$ cells in $H_0$ and $2n$ cells in $H_1$ ($max(f_0(X_i),f_1(X_i))$). Then the degree of freedom is $2n-n=n$. –  Nicolas Essis-Breton Feb 27 at 18:55
Asked and answer on statSE. @passerby51 the trick is that $-ln X_i$ has an exponential distribution under $H_0$, that is a $\chi^2$ with $2$ degrees of freedom. –  Nicolas Essis-Breton Mar 4 at 13:46


text:
[–] 1 point2 points

sorry, this has been archived and can no longer be voted on

(i) [; (0,1) = \{x \in \mathbb{R} | 0 < x < 1\} ;], but [; 0 < 0 < 1 ;] is false, thus [; 0 \notin (0,1) ;]. Other 3 are done similarly.

(ii) [; [a,b] \setminus (a,b) = \{x \in \mathbb{R} | a \leq x \leq b\} \setminus \{x \in \mathbb{R} | a < x < b\} = \{a,b\} ;]

(iii) We need to prove [; (a,b) = \emptyset \Rightarrow a \geq b ;] and [; a \geq b \Rightarrow (a,b) = \emptyset ;], we'll begin with [; (a,b) = \emptyset \Rightarrow a \geq b ;] and use contrapositive proof, thus [; a < b ;] and then [; (a,b) = \{x \in \mathbb{R} | a < x < b\} \neq \emptyset ;], this proves [; (a,b) = \emptyset \Rightarrow a \geq b ;]. Now we have to prove [; a \geq b \Rightarrow (a,b) = \emptyset ;], so we have [; a \geq b ;] and [; (a,b) = \{x \in \mathbb{R} | a < x < b\} ;], do you think such [; x ;] exists?

Don't have time for fourth, will do it later in case no one does it.

[–][S] 1 point2 points

sorry, this has been archived and can no longer be voted on

sorry i dont quite understand (iii) and what does mathbb mean?

[–] 1 point2 points

sorry, this has been archived and can no longer be voted on

Look immediately to your right and you'll see instructions on how to install a plugin to make the LaTeX render properly.

[–][S] 1 point2 points

sorry, this has been archived and can no longer be voted on

thank you very much

[–] 1 point2 points

sorry, this has been archived and can no longer be voted on

Great post.

One super-small thing, but it's a pet peeve of mine. The vertical bar in set builder notation is \mid and not |. It's just a difference of spacing:

\{\, x \in \mathbb{R} | x \leq 5 \,\} gives [; \{\, x \in \mathbb{R} | x \leq 5 \,\} ;]

\{\, x \in \mathbb{R} \mid x \leq 5 \,\} gives [; \{\, x \in \mathbb{R} \mid x \leq 5 \,\} ;]

[–] 0 points1 point

sorry, this has been archived and can no longer be voted on

Thanks, is there any case in which to use '|' as opposed to '\mid'? (I mean divisibility, absolute values, etc..)

[–] 1 point2 points

sorry, this has been archived and can no longer be voted on

| is used for absolute values and for evaluation at the limits of antiderivative in a definite integral

[; |x| = 4 \qquad \int_0^1 \!x^2\, \mathrm{d}x = \left. \frac{x^3}{3} \right|_0^1 ;]

is typed as |x| = 4 \qquad \int_0^1 \! x^2 \, \mathrm{d}x = \left. \frac{x^3}{3} \right|_0^1

\mid is used in set builder notation and divisibility

[; \{\, x \mid x>2 \, \} \qquad 2 \mid x \Rightarrow x \text{ is even} ;]

is typed as \{\, x \mid x>2 \, \} \qquad 2 \mid x \Rightarrow x \text{ is even}

The rule of thumb is that you use | if the vertical line is "attached" to the symbol to its left or right, use \mid if it is "separating" the symbols to its left and right.

[–][S] 0 points1 point

sorry, this has been archived and can no longer be voted on

If possible could anyone help me with this, really struggling and need the anwser for tomorrow!

[–] 0 points1 point

sorry, this has been archived and can no longer be voted on

Sorry to be useless but sweet we're at the same University doing the same course, although I'm first year and not up to this yet.

[–][S] 0 points1 point

sorry, this has been archived and can no longer be voted on

im first year isnt this 1st year stuff?may be different course then

[–] 0 points1 point

sorry, this has been archived and can no longer be voted on

Oh, you may be a lecture or two ahead, do you have Charles Eaton?

[–][S] 0 points1 point

sorry, this has been archived and can no longer be voted on

nope Prof. N. Ray


text:
# B Decoherence Clarification

1. Aug 21, 2015

### Feeble Wonk

Despite the best efforts of some of PF's finest, I continue to struggle with the general concept of spontaneous quantum state reduction by means of environmentally induced decoherence.

I think that much of my confusion lies in the confounding degree of ambiguity in the delineation between the "system" and the "environment". On a cosmological scale, this differentiation often seems to me to be somewhat arbitrary.

Maybe part of my challenge is in simply not understanding the terms well enough. So, with regard to the three primary constituents of the decoherence process... System/Apparatus/Environment... could the physicists in the room please try to give me a conceptual definition using actual words (prohibiting any use of numbers, formulas or references to matrices). I recognize that this might feel like trying to teach me French without speaking French, but I'd greatly appreciate the effort. Consider it a charitable attempt at "No Fool Left Behind".

A possible example of what I'm looking for might be something like (pending your correction of this concept)... "The apparatus is the thing by which a preferred basis of observation is isolated and/or determined".

Any takers?

2. Aug 21, 2015

### jambaugh

Hmmm... I could use some more details as to your confusion but let me start with a general expo of my understanding. The system is of course that which you are representing with, presumably a density operator. As to the meaning of "the environment" that is any other system or systems which may interact with "The System". One would represent interaction by constructing a composite system representation (tensor product of Hilbert spaces and form density operator in the composite space). The interaction is represented by a joint Hamiltonian in which this composite system evolves. To revert to the original system description you would need to trace over the external system or environment component of the composite operator space. Decoherence manifest here in that entanglement occurs between system and environment and when we ignore the environment we see a less coherent system.

Now it is hard to explain this without dragging you through the density operators and partial trace operations and such but let me try it this way.
You meet a cute girl (or guy) who is a math geek and they left you their phone number in the form of two numbers on a piece of paper which add up to your number.
Together the numbers represent "coherent" information but they then tear the paper in two giving you only one of the two numbers. You now have what amounts to a random number. This is not a perfect analogy because quantum mechanical correlation (entanglement) can be stronger than classical in a way we can't model with classical pieces of paper and classical information written on them. But the decoherence stage is analogous to this and not very much different from thermal randomization as described in the classical domain. The system entropy goes up. What the quantum mechanical description allows is that the entropy of the parts is greater than the whole because you can have the maximal information in the whole encoded in a way that is not compatible with subdividing the system into those particular parts (that's incompatible in the sense of momentum vs position type complementarity).

So just as, say, a particle can evolve from a sharp state (or rather mode) of definite position to a sharp state where position is not well defined, a composite system can evolve from a sharp state describable as a composite of sharp states of its parts (system + environment e.g.) to another ***sharp*** state (of the whole) where in the best description of either part is not sharp. It has experienced decoherence.

Now given you knew the original system state and exactly how it evolved you could in principle set things up to reverse this however it is usually the case that we do not know the sharp initial state of the secondary system since it is "everything else" and once interaction has occurred it the original system is entangled with an ever expanding sphere of the electromagnetic field in space. You physically cannot catch up with this in order to work with the whole composite system and reverse the decoherence.

I'm fond of saying when I take this over the top "The entropy of the universe is 0! It's only when we look at separate parts that we get positive entropy!!!"

I hope this clarifies more than it confuses.

3. Aug 21, 2015

### Feeble Wonk

That was a very good effort! Thank you.

But before I ask a follow up question, I'd like to make sure I understand this part. When you say "The system entropy goes up", are you referring to the COMPOSITE system or the original system?

4. Aug 21, 2015

### jambaugh

The original system, in the example I gave the composite retains zero entropy (sharpness is retained in the unitary dynamic evolution).

5. Aug 21, 2015

### Feeble Wonk

.

6. Aug 21, 2015

### Feeble Wonk

I think this is getting at my confusion regarding the mixed/pure state designations.

Before interaction with the environment, we can say that the system was in a "pure" state with zero (informational) entropy. But after interaction (is considered), the original system becomes "mixed", with increased entropy. Is that right?

But, isn't this really just a result of our ignorance of the quantum state, because if the quantum state is actually reduced as a result of the interaction, then the resultant state is still "pure" and has zero entropy.

What am I missing here?

Last edited: Aug 21, 2015
7. Aug 27, 2015

### jambaugh

Yes you have that right w.r.t. the first question. In your either/or question both are correct depending on answering the question "the state of what?" of the system vs of the system and environment together as a larger composite system.

Both are occurring, the system "state" is reduced (but its really not a higher entropy state or reality but a higher entropy class of states) and the resulting state of the larger system is still "pure".

Keep in mind that our wave functions and density operators (i. in the orthodox interp. and ii. especially when considering "mixed" "states") represent classes of possible systems and not specific systems. It is an important philosophical distinction.

8. Dec 9, 2015

### Feeble Wonk

I've been rolling this around in my head, and I keep hitting the same cognitive stumbling block.

I think that the "pure" vs "mixed" designation continues to lie at the heart of my confusion. Again... IF environmental decoherence triggers state reduction (I'm purposefully avoiding the "collapse" term to avoid unnecessary debate), then it seems to me that the reduced state should be considered pure and have zero (informational) entropy at the moment of reduction. I'm not sure what "reality" refers to in your statement here. However, IF decoherence can be considered the triggering mechanism by which state reduction occurs, then I would think it would be accurate to say that one of the potential quantum states of the reduction would be "realized".

Yet, this portion of your post seems to imply that the wave function is not "really" reduced by decoherence, but that decoherence simply limits which system states CAN occur upon reduction.

Can anyone please clarify this distinction further?

Last edited: Dec 9, 2015
9. Dec 9, 2015

### ddd123

Maybe what you're missing is this? Suppose a system is comprised of two parts as the tensor product of the corresponding Hilbert spaces, and is in a pure state with the corresponding density matrix; in general, the density matrix of one of the two single parts (the trace over the corresponding Hilbert space of the single part) represents a mixed state, except for a specific non-entangled case (which doesn't apply here).

If you want I can prove this to you, but there will be math. Or you can take it as a dogma if you don't want the math. Honestly I don't think it's possible to explain this detail without the math, but if you just believe it I don't see the problem (or I don't see by which intuition it would have to be otherwise, it's not like the converse would be intuitive either).

So once you know this, apply it to a system + environment case: the environment interacts with the system and becomes entangled with it, the total is a pure state but the system is now a mixed state.

Last edited: Dec 9, 2015
10. Dec 9, 2015

### Feeble Wonk

The system is in a "mixed" state because there are multiple "potential" states of the system?

11. Dec 9, 2015

### ddd123

No. "Mixed state" and "pure state" are technical terms that refer to properties of the density matrix associated to a quantum state. A pure state's density matrix is just a projector. Both the pure and mixed states represent a probability amplitude for an observable outcome, if that's what you mean. In fact, if you write both the pure and the mixed states with a state vector and a density matrix respectively, using a particular eigenbasis, the probability outcomes of a measurement of an observable with that eigenbasis can be exactly the same. Only with a non-commuting observable do you see the difference, which lies in an interference term given by the relative phase between the pure state vector coefficients (which are complex numbers); whereas a mixed state density matrix is written using real numbers, which, in the above case of there being the same probability distribution, correspond to the former complex numbers squared. That's why the density matrix has less information than a state vector.

So you should not confuse the quantum-informational entropy with the quantity of measurement outcomes: after all, if you had only one outcome for an observable, you could get more with another observable not commuting with the first (which is typical with polarization, for example). Quantum-informational zero entropy refers to the uniqueness of the state vector, represented by complex coefficients in some basis; whereas a mixed state's density matrix, which can be seen as a mix of pure states, has real coefficients (the square of the pure states' complex coefficients, losing the phase information) for all observables, which means it represents an incoherent mix.

Edit: if by "potential states" you mean the pure states the mix is comprised of, then yes.

Last edited: Dec 9, 2015
12. Dec 9, 2015

### StevieTNZ

I think these passages from 'Quantum Enigma' by Bruce Rosenblum and Fred Kuttner may assist (pg 209, 2nd edition):

(without referencing the experimental set-up discussed)
and

EDIT: realised the emphasis on some words, as per the book, were not present.

Last edited: Dec 9, 2015
13. Dec 9, 2015

### Feeble Wonk

I'm afraid that it's precisely this ambiguity that confuses me. I thought that is what I meant, but now I'm not sure.
Let me try this a different way, and maybe it will help you help me (and I do very much appreciate the effort).

My confusion initially began during a previous PF thread (can't put my finger on it immediately) that was discussing the ontological "reality" of the wave function. I was trying to understand how this might relate to the process of environmental decoherence on a cosmological scale.

In that thread, there seemed to be a school of thought among some of the participants that, on a cosmological scale, the delineation between the "system" and the "environment" might be somewhat arbitrary. The question of exactly when, and why, actual state reduction occurred in an ontologically "real" wave function (secondary to decoherence) lead to the discussion of mixed vs pure quantum states of the environment/system complex. It seemed that some were suggesting that the "mixed" state (mixture of potential pure states) after interaction was a reflection of ignorance of outcome, and that the "pure" (realized?) state could not yet be determined. Others appeared to argue that the differentiation was mathematically irrelevant, and any suggestion to the contrary was utterly philosophical.

Now, I am fully aware that I probably misunderstood the discussion, and "proper" vs "improper" mixes were additional sources of confusion for me, so it's entirely possible that I've also confused the "pure/mixed" and the "proper/improper" terms. Also, I'm confident that the answer to my question is highly interpretation dependent. However, IF we are considering a "universal" wave function on a cosmological scale... and IF we are trying to consider this wave function as being ontologically "real" (whatever that means)... I still have trouble understanding how decoherence "triggers", by direct causation, quantum state collapse (or state reduction, if you prefer). It seems, to my befuddled brain anyway, that the mathematical formalism of decoherence simply places logical limitations on what quantum states can be observed... defining the possible "subsystems" of the universal wave function describing the cosmological system/environment complex.

Is this assessment even remotely close to reasonable? If not, can you identify where (in the likely long chain of errors) my conception is in error?

Last edited: Dec 9, 2015
14. Dec 9, 2015

### Feeble Wonk

Both of these postings seem to suggest a similar concept, though without the necessity of the wave function being ontologically "real".

15. Dec 10, 2015

### ddd123

Okay. Indeed, the mixed state is the analogue of the classical probability density in phase space. Let me break this down more clearly.

Classical mechanics: an ideal state is a point in phase space, the corresponding density is a Dirac delta in phase space. A statistical density (of microstates) instead assign a probability distribution to possible ideal states the system is in: it's a "smeared cloud" in phase space if you want to visualize it. A classical physical state is always non-ideal due to measurement errors.

Quantum mechanics: a pure state is a ray (or orbit, if you take Gauge transformations into account) in Hilbert space, which for simplicity we call a state vector. The density is now a "matrix" (actually can also have a term with an integral over continuous observables) which is just a projector over that pure state vector. A statistical mixture - a mixed state - assigns probabilities (hence real numbers) for a quantum system to be found in a number of pure states. A quantum pure state CAN be realized physically.

As you can see there are fundamental differences: in quantum mechanics, a pure state is already of statistical nature it representing a probability amplitude. So you can consider a mixed state to be just another state: the pure state was a superposition of observable eigenstates, the mixed state is a superposition of pure states. In the formalism, there's much less "ontological" distinction between pure and mixed states than there is between ideal and ensemble states in classical mechanics. However, you can devise an interpretation in which such an ontological distinction is recovered (I guess, for a psi-ontologists that's evident).

Now, if your question is, how does a psi-ontologist justify the fact that a composite pure state has its parts taken separately that are mixed states, I guess the trivial answer is "you're not addressing the whole wave-function but only a part, hence whatever you do to a part behaves as a statistical mix to you but the whole is still a pure state".

Be warned that this is NOT the mathematical formalism of decoherence per se. It's the mathematical formalism of all of quantum mechanics. It's just a mathematical lemma: given a vector in a tensor product of two Hilbert spaces, in general the density operator on just one of the two spaces is not a projector (and is never a projector if the total state is entangled). This means that in all interpretations, in all of quantum mechanics, an entangled pure state is always a mixed state for just one part of it. So the "direct causation" you're talking about is simply the entanglement that happens between the environment and the system (which is inevitable): the "logical limitations" you talk about are inevitable maths, decoherence theory didn't postulate them or anything.

The point of decoherence is that keeping track of pure states becomes impossible since they're scattered in bits and pieces all over the place, while you want to look at just one system. Suppose your system is hit by a number of photons: each of them becomes entangled with it and then scatters away. You'd have to run after each photon to recover and study the pure state, looking at the system you'll see a mixed state.

16. Dec 11, 2015

### Demystifier

I think you are right. The split into the "system" and the "environment" is quite arbitrary. In practical calculations this is usually not the problem because physicists have a good intuition about what is a "natural" split in given circumstances. So if you think of QM only as a practical mental tool for human physicists (including cosmologists), then there is no any serious problem. But if you think of QM as a fundamental law obeyed by nature itself, irrespective of humans, then there is a deep problem.

How to resolve the problem? Well, to resolve it, the minimal quantum formalism is not enough. You must use some interpretation of QM, and any choice of interpretation is somewhat controversial.

Personally, I like the Bohmian interpretation. Among other things, this interpretation gives a preferred status to the position observable, which circumvents the problem of "arbitrary split into system and environment".

17. Dec 11, 2015

### ddd123

Can you provide an example in which only an intuitive but not arbitrary choice provides a correct prediction, while another choice is wrong? Of course if calculations turned out correct in all arbitrary choices there wouldn't be a problem.

18. Dec 11, 2015

### Demystifier

There are some examples in Sec. 4 of
http://arxiv.org/abs/1210.8447

19. Dec 11, 2015

### Feeble Wonk

Thank you. This is a fascinating paper. I think this will address my confusion directly. It's very similar to a position I've read by Lee Smolin from the cosmological perspective. I'm hoping that I can just be a spectator now and watch while you professionals discuss this.

20. Dec 11, 2015

### zonde

Do I understand it correctly? The author of this paper gives an argument why universal wave function equipped with unitary evolution can not predict classical world (via decoherence induced branching).


text:
# Solve 1/P + 1/Q + 1/X = 1/(X + P + Q) - Mathematics

Sum

Solve 1/p + 1/q + 1/x = 1/(x + p + q)

#### Solution

1/p + 1/q + 1/x = 1/(x + p + q)

=> 1/p + 1/q + 1/x - 1/(x + p + 1) = 0

=> ((q + p)[1/(pq) + 1/(x^2 + px + qx)] = 0

=> (p + q) [(x^2 + px + qx + pq)/(pq(x^2 + px + qx))] = 0

=> x^2 + px + qx + pq = 0

=> x(x + p) + q(x + p) = 0

=> (x + p)(x + q) = 0

=> x = -p and x = -q

Is there an error in this question or solution?

#### APPEARS IN

Selina Concise Maths Class 10 ICSE


text:
# Sh:1068

• Kumar, A., & Shelah, S. (2017). A transversal of full outer measure. Adv. Math., 321, 475–485.
• Abstract:
We prove that for any set of real and equivalence relations on it such that every equivalence class is countable, there is a transversal (= a set of representations of the equivalence relations, i.e. having exactly one member in each equivalence class) with the same outer measure.
• published version (11p)
Bib entry
@article{Sh:1068,
author = {Kumar, Ashutosh and Shelah, Saharon},
title = {{A transversal of full outer measure}},
}