text:
I'm working on the C. elegans model organism, for which there exists an online biological information database: Wormbase.

I'm able to manually search for a transcript (say "K06C4.12") and this gets me the following URL: http://www.wormbase.org/species/c_elegans/cds/K06C4.12

Say I have this transcript ID: "F08G2.2". I guess that I can form a valid similar URL, as follows: http://www.wormbase.org/species/c_elegans/cds/F08G2.2

Indeed, it works as expected.

Now, I would like to go further and be able to download in an automated way what can be obtained by clicking on one of the "download" buttons in the "Sequence" section of the above pages. More specifically, I'm interested in the "spliced + UTR" and "conceptual translation" fasta sequences.

If I use the DOM inspector in Firefox (Shift-Ctrl-C), I can locate things like the following:

>K06C4.12 spliced + UTR
ATGGCCCCACCAAAGCCGTCTGCTAAAGGAGCAAAGAAAGCCGCCAAGACCGTCACCAAGCCAAAGGACGGAAAGAAGAG
ACGTCATGCCCGTAAAGAATCCTACTCCGTGTACATCTACCGTGTGCTCAAGCAGGTTCATCCAGACACTGGAGTCTCTT
CCAAGGCTATGTCCATCATGAACTCTTTCGTCAACGACGTCTTCGAGCGTATCGCTGCTGAGGCATCTCGTCTTGCTCAC
TACAACAAGAGATCCACCATCTCTTCTCGCGAAATTCAAACTGCCGTCCGTTTGATTCTTCCAGGAGAGCTTGCCAAGCA
CGCCGTCTCTGAGGGAACCAAGGCTGTCACCAAGTACACCTCTAGCAAGTAA

K06C4.12conceptual translation.fasta

>conceptual translation
MAPPKPSAKGAKKAAKTVTKPKDGKKRRHARKESYSVYIYRVLKQVHPDTGVSSKAMSIMNSFVNDVFERIAAEASRLAH
YNKRSTISSREIQTAVRLILPGELAKHAVSEGTKAVTKYTSSK*

view conceptual translation (123 aa)

>conceptual translation

MAPPKPSAKGAKKAAKTVTKPKDGKKRRHARKESYSVYIYRVLKQVHPDTGVSSKAMSIMNSFVNDVFERIAAEASRLAH

YNKRSTISSREIQTAVRLILPGELAKHAVSEGTKAVTKYTSSK*

I don't find these if I simply download the page with wget and search in the source code, so they must be dynamically generated.

How could I proceed to get the sequences?

The documentation about Wormbases's REST API is not very detailed, so I don't understand how to form an URL that would give me the desired sequences.

• Did you try to download the whole database and search there? (I am not sure if you need large amounts of data or not ...)
– llrs
Nov 24 '17 at 13:28
• I don't know where to download it, neither how it is structured. I don't need a large amount of data, so I hope querying the site should be OK.
– bli
Nov 24 '17 at 13:32
• To download it they said it is here. There should be a ftp site, so you could try that too.
– llrs
Nov 24 '17 at 13:33
• Indeed, they even provide a scary shell script (first time I see a "goto" instruction in a shell script) to install the database: ftp.wormbase.org/pub/wormbase/releases/… I'm not sure I'm ready to delve into this for the moment...
– bli
Nov 24 '17 at 13:45

Further inspecting the source code of the downloaded page, I found the following:

<span class="ui-icon ui-icon-close module-close" wname="sequences"

Sequences

This suggested me to have a look at http://www.wormbase.org/rest/widget/transcript/K06C4.12/sequences and I obtained a page containing the elements found in the DOM inspector.

This can be parsed, for instance using the bs4 python module:

# To get the page
import requests
# To parse the page content
from bs4 import BeautifulSoup
# To get this into Biopython
from io import StringIO
from Bio import SeqIO

rest_template = "http://www.wormbase.org/rest/widget/transcript/{accession}/sequences"
rest = rest_template.format(accession="K06C4.12")
# This did not work with "html.parser"
soup = BeautifulSoup(requests.get(rest).text, "html5lib")

def generate_fasta(soup):
name = None
for div in soup.find_all("div"):
if "ui-helper-hidden" in div["class"]:
if div["id"] == "content":
description_fields = fasta.description.split()
if len(description_fields) == 4:
fasta.id = "_".join(description_fields[:2])
if name is None:
name = fasta.name
elif len(description_fields) == 2:
# This assumes that the "conceptual_translation"
# is after the "unspliced + UTR" or "spliced + UTR"
# Otherwise, one would need to pass
# the accession as function parameter
fasta.id = f"{name}_translation"
else:
fasta.id = "_".join(description_fields)
yield (fasta.id, fasta)

records_dict = dict(generate_fasta(soup))
print(records_dict['K06C4.12_spliced'])
print(records_dict['K06C4.12_translation'])

Output:

ID: K06C4.12_spliced
Name: K06C4.12
Description: K06C4.12 spliced + UTR
Number of features: 0
Seq('ATGGCCCCACCAAAGCCGTCTGCTAAAGGAGCAAAGAAAGCCGCCAAGACCGTC...TAA', SingleLetterAlphabet())
ID: K06C4.12_translation
Name: conceptual
Description: conceptual translation
Number of features: 0
Seq('MAPPKPSAKGAKKAAKTVTKPKDGKKRRHARKESYSVYIYRVLKQVHPDTGVSS...SK*', SingleLetterAlphabet())


text:
This week we are going to take a look at scientific notation. You have probably seen it before, but you may not know what it is or how to use it. It is very common in math and science, where there large numbers or fractions that need to be rewritten as something shorter!

Scientific notation was first used around 500 BC in China and Greece. Since then, it has become one of the most prevalent ways to rewrite equations.

There are two parts to scientific notation. The exponent part goes after the decimal place and the pre-exponent part comes before the decimal place.

The exponent can get really big and even bigger than 100. When this happens, the equation is rewrote so that the 0s and 1s are replaced with a new symbol called e (for exponential). These new symbols are then put into the original expression along with the number 10 to create the final product.

The products of these changes are longer and narrower, which makes them easier to understand and work with. For example, the above expression could be rewitten as 5e6.

This way of rewriting equations is helpful because now instead of having a long, confusing expression, you have a short, easy to read one.

## Examples of scientific notation

There are many ways to use scientific notation in writing. Some examples include when you want to say that something is very large or very small, when you want to express a decimal as an exponent, and when you need to represent a value that can’t be expressed using standard numbers.

When you see a poof of text with lots of weird looking numbers after it, that’s usually because it’s called scientific notation. It was originally designed for mathematics, where there are rules about how to convert from normal (or conventional) numbers to exponentials and back again.

That way, mathematicians don’t have to worry about making mistakes when they multiply two numbers together or take their logarithm. They know what to do automatically!

But converting between exponential and regular numbers isn’t too difficult, so most people these days just leave all those pretty looking expressions to hang out in math and get used by students who learn calculus.

## Relationship between scientific and traditional notation

Traditional symbolism like arrows, circles, squares, and so on are used to describe the relationships in equations. These symbols are referred to as signs or markers.

The word equation comes from the Latin word for equal, addo, which is why we have the words “equate” and “equal.” An equals sign (or plus-sign symbol) is used to show that two things are the same. For example, if you take one cup of milk and pour it into another glass, then the amount in each doesn’t change, but the amount in the second glass will be twice as much because there is more liquid being poured in!

That’s what an equation means! In math, writing out steps can make concepts clear, so mathematicians use this method often. But when communicating mathematics to non-mathematicians, using pictures instead is better since they are intuitive and easier to understand.

There are several different styles of graphical representation of mathematical equations. Two common ones are exponent diagrams and fraction bar charts. Both convey the same information – how many times something is multiplied by something else.

With exponents, just remember that whatever element you put in front of the other one increases it by that factor.

## How to work with scientific notation

Many mathematical formulas require using either very large or very small numbers. These are called scientific notations, as they originate from science!

Very large numbers are known as exponentials. Exponents mean that there is an integer number times a power of another integer. For example, e^(2x) means 2 times the natural logarithm of whatever comes after the ^ symbol (in this case, the value of x).

The natural logaritiml of 9 is 1 (one), so 9^1 would be 9. Similarly, the natural logarithm of 0 is -infinity (-$\infty$), so 0^-infinty would be 0.

These types of expressions are common when solving equations. When doing so, you must have both very large and very small numbers present in your expression.

## What is a number in scientific notation?

A number in scientific notations is just like any other numbers you’ve learned so far, except that it uses a prefix that tells us something about the number it is.

The prefixes tell us more information about the number, so we can determine what the number means. For example, if the prefix was ‘x’ then the number would be one times whatever item follows the x. So, 1x means use this product once.

If the prefix was ‘y’ then the number would be two times whatever item follows the y. So, 2y would mean to double the amount for the next time or formula.

And if the prefix was ‘e’ then the number would be multiplied by itself (expanded) the number following it. So, e10 would be ten times itself (multiplied). An expression with an exponent of 10 is called a power of 10.

Numbers with no special prefix are simply normal numbers. The only difference between them and other number types is how they are written!

Question: How many times does twenty go into three?

Answer: Twenty goes into three twice, because there are two instances of twenty. Make sense? If yes, then here comes some math!

Twenty is a factor of three. This means that twenty is contained within three. So, twenty will always be a part of three things, or it will be included in three different items.

## How do I convert between scientific and traditional notation?

The way to read general expressions in science is by using either traditional or scientific notations. Traditional notation uses parentheses to denote how many times an expression is multiplied, while scientific notation multiplies quantifiers (parts of an expression) together.

For example, the equation e=mc2 means that energy equals the product of mass and the square of the speed of light.

## What are some tips for working with scientific notation?

There is no universal agreement as to what rules apply when using this notational system. Some people use an order of operations that goes like this: multiply, add, subtract, find product. Others go in the opposite direction: add, subtract, multiply, divide.

Some say you can’t mix exponents within one rule, but others do! It really depends on how the equation is set up and how it is being solved.

So, which method is correct? I would recommend using either the multiplication or division first, then adding the other two steps if needed. This way, your polynomial will be balanced and converted correctly.

## When should I use scientific notation?

Using scientific notations is very popular these days! There are several good reasons to use them, but one of the biggest is efficiency. By limiting yourself to only specific numbers in place of longer ones, your calculations run more quickly.

Most people learn basic arithmetic like adding, subtracting, multiplying, and dividing before moving onto higher math. For example, there’s an algorithm that calculates the area of a circle given a radius.

The way this works is by taking the square root of the diameter, which is also called the radian measure, and then using that as the radius for another circle. The two circles are related because the radian measurement is just a way to calculate circumference, so they have the same formula.

That means we can relate the first circle to the second easily! The area of the second circle is simply.5 * (radian)2 * (radius) = 0.05 * (square root of 2) * (radius squared).

Keep in mind that you can take either sqrt(a+b) or sqrt(ab) to find the radian measurement.

## What is a larger number in scientific notation?

A larger numbers are called exponents. Exponentiation, or raising a number to a power, is when you take a normal number and increase it by a given amount. For example, say there is a boat with an average speed of 5 miles per hour. Then someone comes along and says they can go this boat at twice the speed! They would have to double the speed (2x=4mph) to get the same result.

This is what exponentiation is. When we talk about exponential growth, we are referring to that situation where something is multiplied by itself over and over again. In other words, whatever thing you start with keeps getting bigger and bigger very quickly.

For instance, if you had one apple, you could make a million apples now by putting just one apple into each new piece. This is how our parents probably grew up; they never knew why their family didn’t have enough food, but as kids they always said “we were hungry today, but we weren’t hungry yesterday.

format: plain


text:
# Controllability

Controllability is an important property of a control system, and the controllability property plays a crucial role in many control problems, such as stabilization of unstable systems by feedback, or optimal control.

Controllability and observability are dual aspects of the same problem.

Roughly, the concept of controllability denotes the ability to move a system around in its entire configuration space using only certain admissible manipulations. The exact definition varies slightly within the framework or the type of models applied.

The following are examples of variations of controllability notions which have been introduced in the systems and control literature:

• State controllability
• Output controllability
• Controllability in the behavioural framework

## State controllability

The state of a deterministic system, which is the set of values of all the system's state variables (those variables characterized by dynamic equations), completely describes the system at any given time. In particular, no information on the past of a system is needed to help in predicting the future, if the states at the present time are known and all current and future values of the control variables (those whose values can be chosen) are known.

Complete state controllability (or simply controllability if no other context is given) describes the ability of an external input (the vector of control variables) to move the internal state of a system from any initial state to any other final state in a finite time interval.[1]:737

Note that controllability does not mean that once a state is reached, that state can be maintained, but merely that that (or any) state can be reached.

## Continuous linear systems

Consider the continuous linear system [note 1]

$\dot{\mathbf{x}}(t) = A(t) \mathbf{x}(t) + B(t) \mathbf{u}(t)$
$\mathbf{y}(t) = C(t) \mathbf{x}(t) + D(t) \mathbf{u}(t).$

There exists a control $u$ from state $x_0$ at time $t_0$ to state $x_1$ at time $t_1 > t_0$ if and only if $x_1 - \phi(t_0,t_1)x_0$ is in the column space of

$W(t_0,t_1) = \int_{t_0}^{t_1} \phi(t_0,t)B(t)B(t)^{T}\phi(t_0,t)^{T} dt$

where $\phi$ is the state-transition matrix, and $W(t_0,t_1)$ is the Controllability Gramian.

In fact, if $\eta_0$ is a solution to $W(t_0,t_1)\eta = x_1 - \phi(t_0,t_1)x_0$ then a control given by $u(t) = -B(t)^{T}\phi(t_0,t)^{T}\eta_0$ would make the desired transfer.

Note that the matrix $W$ defined as above has the following properties:

• $W(t_0,t_1)$ is symmetric
• $W(t_0,t_1)$ is positive semidefinite for $t_1 \geq t_0$
• $W(t_0,t_1)$ satisfies the linear matrix differential equation
$\frac{d}{dt}W(t,t_1) = A(t)W(t,t_1)+W(t,t_1)A(t)^{T}-B(t)B(t)^{T}, \; W(t_1,t_1) = 0$
• $W(t_0,t_1)$ satisfies the equation
$W(t_0,t_1) = W(t_0,t) + \phi(t_0,t)W(t,t_1)\phi(t_0,t)^{T}$[2]

### Continuous linear time-invariant (LTI) systems

Consider the continuous linear time-invariant system

$\dot{\mathbf{x}}(t) = A \mathbf{x}(t) + B \mathbf{u}(t)$
$\mathbf{y}(t) = C \mathbf{x}(t) + D \mathbf{u}(t)$

where

$\mathbf{x}$ is the $n \times 1$ "state vector",
$\mathbf{y}$ is the $m \times 1$ "output vector",
$\mathbf{u}$ is the $r \times 1$ "input (or control) vector",
$A$ is the $n \times n$ "state matrix",
$B$ is the $n \times r$ "input matrix",
$C$ is the $m \times n$ "output matrix",
$D$ is the $m \times r$ "feedthrough (or feedforward) matrix".

The $n \times nr$ controllability matrix is given by

$R = \begin{bmatrix}B & AB & A^{2}B & ...& A^{n-1}B\end{bmatrix}$

The system is controllable if the controllability matrix has full row rank (i.e. $\operatorname{rank}(R)=n$).

## Discrete linear time-invariant (LTI) systems

For a discrete-time linear state-space system (i.e. time variable $k\in\mathbb{Z}$) the state equation is

$\textbf{x}(k+1) = A\textbf{x}(k) + B\textbf{u}(k)$

where $A$ is an $n \times n$ matrix and $B$ is a $n \times r$ matrix (i.e. $\mathbf{u}$ is $r$ inputs collected in a $r \times 1$ vector). The test for controllability is that the $n \times nr$ matrix

$\mathcal{C} = \begin{bmatrix}B & AB & A^{2}B & \cdots & A^{n-1}B\end{bmatrix}$

has full row rank (i.e., $\operatorname{rank}(\mathcal C) = n$). That is, if the system is controllable, $\mathcal C$ will have $n$ columns that are linearly independent; if $n$ columns of $\mathcal C$ are linearly independent, each of the $n$ states is reachable by giving the system proper inputs through the variable $u(k)$.

### Derivation

Given the state $\textbf{x}(0)$ at an initial time, arbitrarily denoted as k=0, the state equation gives $\textbf{x}(1) = A\textbf{x}(0) + B\textbf{u}(0),$ then $\textbf{x}(2) = A\textbf{x}(1) + B\textbf{u}(1)= A^2\textbf{x}(0)+AB\textbf{u}(0)+B\textbf{u}(1),$ and so on with repeated back-substitutions of the state variable, eventually yielding

$\textbf{x}(n)=B\textbf{u}(n-1) + AB\textbf{u}(n-2) + \cdots + A^{n-1}B\textbf{u}(0) + A^n\textbf{x}(0)$

or equivalently

$\textbf{x}(n)-A^n\textbf{x}(0)= [B \, \, AB \, \, \cdots \, \, A^{n-1}B] [\textbf{u}^T(n-1) \, \, \textbf{u}^T(n-2) \, \, \cdots \, \, \textbf{u}^T(0)]^T.$

Imposing any desired value of the state vector $\textbf{x}(n)$ on the left side, this can always be solved for the stacked vector of control vectors if and only if the matrix of matrices at the beginning of the right side has full row rank.

### Example

For example, consider the case when $n=2$ and $r=1$ (i.e. only one control input). Thus, $B$ and $A B$ are $2 \times 1$ vectors. If $\begin{bmatrix}B & AB\end{bmatrix}$ has rank 2 (full rank), and so $B$ and $AB$ are linearly independent and span the entire plane. If the rank is 1, then $B$ and $AB$ are collinear and do not span the plane.

Assume that the initial state is zero.

At time $k=0$: $x(1) = A\textbf{x}(0) + B\textbf{u}(0) = B\textbf{u}(0)$

At time $k=1$: $x(2) = A\textbf{x}(1) + B\textbf{u}(1) = AB\textbf{u}(0) + B\textbf{u}(1)$

At time $k=0$ all of the reachable states are on the line formed by the vector $B$. At time $k=1$ all of the reachable states are linear combinations of $AB$ and $B$. If the system is controllable then these two vectors can span the entire plane and can be done so for time $k=2$. The assumption made that the initial state is zero is merely for convenience. Clearly if all states can be reached from the origin then any state can be reached from another state (merely a shift in coordinates).

This example holds for all positive $n$, but the case of $n=2$ is easier to visualize.

### Analogy for example of n = 2

Consider an analogy to the previous example system. You are sitting in your car on an infinite, flat plane and facing north. The goal is to reach any point in the plane by driving a distance in a straight line, come to a full stop, turn, and driving another distance, again, in a straight line. If your car has no steering then you can only drive straight, which means you can only drive on a line (in this case the north-south line since you started facing north). The lack of steering case would be analogous to when the rank of $C$ is 1 (the two distances you drove are on the same line).

Now, if your car did have steering then you could easily drive to any point in the plane and this would be the analogous case to when the rank of $C$ is 2.

If you change this example to $n=3$ then the analogy would be flying in space to reach any position in 3D space (ignoring the orientation of the aircraft). You are allowed to:

• fly in a straight line
• turn left or right by any amount (Yaw)
• direct the plane upwards or downwards by any amount (Pitch)

Although the 3-dimensional case is harder to visualize, the concept of controllability is still analogous.

## Nonlinear systems

Nonlinear systems in the control-affine form

$\dot{\mathbf{x}} = \mathbf{f(x)} + \sum_{i=1}^m \mathbf{g}_i(\mathbf{x})u_i$

is locally accessible about $x_0$ if the accessibility distribution $R$ spans $n$ space, when $n$ equals the rank of $x$ and R is given by:[3]

$R = \begin{bmatrix} \mathbf{g}_1 & \cdots & \mathbf{g}_m & [\mathrm{ad}^k_{\mathbf{g}_i}\mathbf{\mathbf{g}_j}] & \cdots & [\mathrm{ad}^k_{\mathbf{f}}\mathbf{\mathbf{g}_i}] \end{bmatrix}.$

Here, $[\mathrm{ad}^k_{\mathbf{f}}\mathbf{\mathbf{g}}]$ is the repeated Lie bracket operation defined by

$[\mathrm{ad}^k_{\mathbf{f}}\mathbf{\mathbf{g}}] = \begin{bmatrix} \mathbf{f} & \cdots & j & \cdots & \mathbf{[\mathbf{f}, \mathbf{g}]} \end{bmatrix}.$

The controllability matrix for linear systems in the previous section can in fact be derived from this equation.

## Output controllability

Output controllability is the related notion for the output of the system (denoted y in the previous equations); the output controllability describes the ability of an external input to move the output from any initial condition to any final condition in a finite time interval. It is not necessary that there is any relationship between state controllability and output controllability. In particular:

• A controllable system is not necessarily output controllable. For example, if matrix D = 0 and matrix C does not have full row rank, then some positions of the output are masked by the limiting structure of the output matrix. Moreover, even though the system can be moved to any state in finite time, there may be some outputs that are inaccessible by all states. A trivial numerical example uses D=0 and a C matrix with at least one row of zeros; thus, the system is not able to produce a non-zero output along that dimension.
• An output controllable system is not necessarily state controllable. For example, if the dimension of the state space is greater than the dimension of the output, then there will be a set of possible state configurations for each individual output. That is, the system can have significant zero dynamics, which are trajectories of the system that are not observable from the output. Consequently, being able to drive an output to a particular position in finite time says nothing about the state configuration of the system.

For a linear continuous-time system, like the example above, described by matrices $A$, $B$, $C$, and $D$, the $m \times (n+1)r$ output controllability matrix

$\begin{bmatrix} CB & CAB & CA^2B & \cdots & CA^{n-1}B & D\end{bmatrix}$

has full row rank (i.e. rank $m$) if and only if the system is output controllable.[1]:742 This result is known as Kalman's criterion of controllability.[citation needed]

## Controllability under input constraints

In systems with limited control authority, it is often no longer possible to move any initial state to any final state inside the controllable subspace. This phenomenon is caused by constraints on the input that could be inherent to the system (e.g. due to saturating actuator) or imposed on the system for other reasons (e.g. due to safety-related concerns). The controllability of systems with input and state constraints is studied in the context of reachability[4] and viability theory.[5]

## Controllability in the behavioural framework

In the so-called behavioral system theoretic approach due to Willems (see people in systems and control), models considered do not directly define an input–output structure. In this framework systems are described by admissible trajectories of a collection of variables, some of which might be interpreted as inputs or outputs.

A system is then defined to be controllable in this setting, if any past part of a behavior (trajectory of the external veriables) can be concatenated with any future trajectory of the behavior in such a way that the concatenation is contained in the behavior, i.e. is part of the admissible system behavior.[6]:151

## Stabilizability

A slightly weaker notion than controllability is that of stabilizability. A system is said to be stabilizable when all uncontrollable state variables can be made to have stable dynamics. Thus, even though some of the state variables cannot be controlled (as determined by the controllability test above) all the state variables will still remain bounded during the system's behavior.[7]

## Notes

1. ^ A linear time-invariant system behaves the same but with the coefficients being constant in time.

## References

1. ^ a b Katsuhiko Ogata (1997). Modern Control Engineering (3rd ed.). Upper Saddle River, NJ: Prentice-Hall. ISBN 0-13-227307-1.
2. ^ Brockett, Roger W. (1970). Finite Dimensional Linear Systems. John Wiley & Sons. ISBN 978-0-471-10585-5.
3. ^ Isidori, Alberto (1989). Nonlinear Control Systems, p. 92–3. Springer-Verlag, London. ISBN 3-540-19916-0.
4. ^ Claire J. Tomlin, Ian Mitchell, Alexandre M. Bayen and Meeko Oishi (2003). "Computational Techniques for the Verification of Hybrid Systems" (PDF). Proceedings of the IEEE. Retrieved 2012-03-04.
5. ^ Jean-Pierre Aubin (1991). Viability Theory. Birkhauser. ISBN 0-8176-3571-8.
6. ^ Jan Polderman, Jan Willems (1998). Introduction to Mathematical Systems Theory: A Behavioral Approach (1st ed.). New York: Springer Verlag. ISBN 0-387-98266-3.
7. ^ Brian D.O. Anderson; John B. Moore (1990). Optimal Control: Linear Quadratic Methods. Englewood Cliffs, NJ: Prentice Hall. ISBN 978-0-13-638560-8.


text:
is there any bound on the absolute number of algebraic integer in terms of its degree?

If Z is a sum of t distinct roots of unity and |Z| is a rational integer, can someone find a bound on |Z| in terms of k=deg(Q(Z):Q))?

Clearly we need to have distinct roots of unity otherwise this won't work!

Correction: Let assume that Z is not rational itself otherwise obviously it's wrong. Here I hope to extend the proof of Kronecker thm! I have "Z is a sum of t distinct roots of unity and |Z| is a rational integer" I conjecture that either Z is rational or a root of unity!

-
Not sure what you are asking but you might want to check: Loxton, J. H. On two problems of R. W. Robinson about sums of roots of unity. Acta Arith. 26 (1974/75), 159–174. –  Felipe Voloch Oct 9 '12 at 2:04
Even with the correction, all you have to do is multiply Will Sawin's sum by, say, a nonreal cube root of 1, $\omega$. Then $Z=-n\omega$ is not rational, $|Z|=n$ is a rational integer, and $k=2$. –  Gerry Myerson Oct 10 '12 at 4:06
@Gerry: sorry I edited my answer to fix it before seeing your identical comment. –  Will Sawin Oct 14 '12 at 17:20

If you multiply all the primitive roots of unity for the first $n$ primes by $-i$, then add them, you get $in$, whose degree is $2$ and whose absolute value is $n$, rational and unbounded.

-
Dear Will, Doesn't the question ask that $|Z|$ (where $Z$ is the sum of roots of unity in question) be an integer? Regards, –  Emerton Oct 9 '12 at 2:25
I edited it to a more elegant solution that deals with that issue. –  Will Sawin Oct 9 '12 at 3:17
In fact, this construction allows you to write any sum of roots of unity as a sum of distinct roots of unity. –  Will Sawin Oct 14 '12 at 22:46


text:
# Denna webbshop är stängd för köp.Klicka här för mer information

• Författare:

## David B Cook

• Förlag:

### World Scientific Europe Ltd

• Bandtyp: Inbunden
• Språk: Engelska
• Utgiven: 20180530
• Antal sidor: 196
•  ISBN10: 1786344904
•  ISBN13: 9781786344908

# Schrodinger's Mechanics: Interpretation

(Inbunden)
av

## David B Cook

##### Beskrivning:

The interpretation of quantum mechanics has been in dispute for nearly a century with no sign of a resolution. Using a careful examination of the relationship between the final form of classical particle mechanics (the Hamilton-Jacobi Equation) and Schrödinger's mechanics, this book presents a coherent way of addressing the problems and paradoxes that emerge through conventional interpretations.Schrödinger's Mechanics critiques the popular way of giving physical interpretation to the various terms in perturbation theory and other technologies and places an emphasis on development of the theory and not on an axiomatic approach. When this interpretation is made, the extension of Schrödinger's mechanics in relation to other areas, including spin, relativity and fields, is investigated and new conclusions are reached.


text:
## arXiv Analytics

### arXiv:1907.05414 [math.PR]AbstractReferencesReviewsResources

#### Variational principle for weakly dependent random fields

Published 2019-07-11Version 1

Using an alternative notion of entropy introduced by Datta, the max-entropy, we present a new simplified framework to study the minimizers of the specific free energy for random fields which are weakly dependent in the sense of Lewis, Pfister, and Sullivan. The framework is then applied to derive the variational principle for the loop $O(n)$ model and the Ising model in a random percolation environment in the nonmagnetic phase, and we explain how to extend the variational principle to similar models. To demonstrate the generality of the framework, we indicate how to naturally fit into it the variational principle for models with an absolutely summable interaction potential, and for the random-cluster model.


text:
# 47

Recently Luke Muelhauser posed the question, “Can noise have power?”, which basically asks whether randomization can ever be useful, or whether for every randomized algorithm there is a better deterministic algorithm. This question was posed in response to a debate between Eliezer Yudkowsky and Scott Aaronson, in which Eliezer contends that randomness (or, as he calls it, noise) can't ever be helpful, and Scott takes the opposing view. My goal in this essay is to present my own views on this question, as I feel many of them have not yet been brought up in discussion.

I'll spare the reader some suspense and say that I basically agree with Scott. I also don't think – as some others have suggested – that this debate can be chalked up to a dispute about the meaning of words. I really do think that Scott is getting at something important in the points he makes, which may be underappreciated by those without a background in a field such as learning theory or game theory.

Before I start, I'd like to point out that this is really a debate about Bayesianism in disguise. Suppose that you're a Bayesian, and you have a posterior distribution over the world, and a utility function, and you are contemplating two actions A and B, with expected utilities U(A) and U(B). Then randomly picking between A and B will have expected utility $\frac{U(A)+U(B)}{2}$, and so in particular at least one of A and B must have higher expected utility than randomizing between them. One can extend this argument to show that, for a Bayesian, the best strategy is always deterministic. Scott in fact acknowledges this point, although in slightly different language:

“Randomness provably never helps in average-case complexity (i.e., where you fix the probability distribution over inputs) -- since given any ensemble of strategies, by convexity there must be at least one deterministic strategy in the ensemble that does at least as well as the average.” -Scott Aaronson

I think this point is pretty iron-clad and I certainly don't wish to argue against it. Instead, I'd like to present several examples of scenarios where I will argue that randomness clearly is useful and necessary, and use this to argue that, at least in these scenarios, one should abandon a fully Bayesian stance. At the meta level, this essay is therefore an argument in favor of maintaining multiple paradigms (in the Kuhnian sense) with which to solve problems.

I will make four separate arguments, paralleling the four main ways in which one might argue for the adoption or dismissal of a paradigm:

1. Randomization is an appropriate tool in many concrete decision-making problems (game theory and Nash equilibria, indivisible goods, randomized controlled trials).

2. Worst case analyses (which typically lead to randomization) are often important from an engineering design perspective (modularity of software).

3. Random algorithms have important theoretical properties not shared by deterministic algorithms (P vs. BPP).

4. Thinking in terms of randomized constructions has solved many problems that would have been difficult or impossible without this perspective (probabilistic method, sampling algorithms).

1. Concrete usefulness

Two people are playing rock-paper-scissors. After a while one of them starts to lose on average. What should they do? Clearly randomizing will make them stop losing so they should probably do that. I want to contrast this with one of Eliezer's points, which, roughly, is, “clearly you should randomize if you're up against some adversarial superintelligence, but in this case we should label it as 'adversarial superintelligence' rather than worst case”. Many have taken this as evidence that Scott and Eliezer are really just arguing about the definition of words. I find it difficult to support this conclusion in light of the above rock-paper-scissors example: you don't need to be playing an adversarial superintelligence, you just need to be playing anyone who is better than you at rock-paper-scissors, which is a reasonably common occurrence.

A related situation shows up in many trade- and bargaining-related situations. For instances, suppose that I have a good that I value at $800, you value the same good at$1,000, but you only have $500 available to spend. We can still make a trade by agreeing that I will give you the good with probability 50%, in exchange for you paying me$450, leading us both to profit by $50 in expectation. This illustrates how randomness can be used to improve market liquidity. You might argue that we could have made the same trade without randomization by finding an event that we both agree has 50% subjective probability mass. But even if you are willing to do that, it is easy to see why another agent may not wish to --- they may not wish to divulge information about their beliefs, they may not trust you to accurately divulge your own beliefs, they may not even represent their beliefs as probabilities in the first place, etc. Since agents with these properties are fairly natural, it seems necessary to randomize unless one is willing to give up on profitable trading opportunities. As yet another example of concrete usefulness, let us consider randomized controlled trials --- in other words, using randomness in an experimental design to ensure that we have a representative subset of a population in order to infer a particular causal mechanism. When it is possible to do so, randomization is an incredibly useful aspect of experimental design, freeing us to make strong inferences about the data that would be much more difficult or impossible to make with the same level of confidence without using randomization. For example, say that we want to know whether a job training program improves participants' earnings. If we separate our sample into a treatment and control group using randomization, and we find a real difference in earnings between the two, there is a strong case that the job training program is the reason for the difference in earnings. If we use any other method to separate the two groups, we run the risk that our method for separating treatment from control was correlated with some other factor that can explain a difference in earnings. If we follow the “randomness never helps” mantra here, then presumably we should carefully model all of the possible confounding effects in order to make inferences about the main effect --- but often the purpose of experimentation is to better understand phenomena about which we currently know little, so that it is unreasonable to think that we can actually accurately model every possible confound in this way. In this situation, “randomness never helps” requires us to pretend to know the answer even in situations where we clearly do not, which seems to me to be problematic. This is despite the fact that there doesn't seem to be any "superintelligence" - or even intelligent adversary - involved in such situations. 2. Design properties As noted in the introduction, if we know the probability distribution of the environment, then the optimal strategy is always deterministic (for now, I ignore the issue of whether finding this optimal strategy is feasible, since we are talking about whether randomness helps "in principle"). On the other hand, if we do not know the distribution of the environment, there can be good reasons to randomize, since this can help to "smooth out" potential unexpected structure (examples include randomized quicksort as well as certain load-balancing algorithms). In other words, if we want to argue that randomness shouldn't help, the most obvious alternative to using randomization would be to reason about the probability distribution of the environment, which seems to match what Eliezer has in mind, as far as I can tell. For instance, Eliezer says: I certainly don't say "it's not hard work", and the environmental probability distribution should not look like the probability distribution you have over your random numbers - it should contain correlations and structure. But once you know what your probability distribution is, then you should do your work relative to that, rather than assuming "worst case". Optimizing for the worst case in environments that aren't actually adversarial, makes even less sense than assuming the environment is as random and unstructured as thermal noise. Note that reasoning about the probability distribution of the environment requires assuming a particular distribution in the first place. I want to argue that this is often bad from a software design perspective. First, software is supposed to be modular. By this, I mean that software should be broken into components that: 1. Have a clearly specified and verifiable input-output contract. 2. Are re-usable in a wide variety of situations (for the most part, if the input-output contract can only be satisfied in one instance ever then it isn't a very useful contract). Pretty much everyone who works on software agrees with this. However, requiring that the inputs to a piece of software follow some probability distribution is the opposite of being modular. Why is this? First, it is impossible to verify from a single input whether the input contract is satisfied, since the contract is now over the distribution of inputs, rather than a single input. Second, requiring a certain distribution over inputs limits the software to the cases where the inputs actually follow that distribution (or a sufficiently similar distribution), thus greatly limiting the scope of applicability. Even worse, such a distributional requirement propagates backward through the entire system: if A calls B, which calls C, and C requires its inputs to follow a certain probability distribution, then B's contract will probably also have to call for a certain probability distribution over inputs (unless all inputs to C from B are generated without reference to B's inputs). On the other hand, software operating under worst-case analyses will (by definition) work regardless of input, and will not generate assumptions that propagate back to the rest of the system. Therefore it is important to be willing to adopt a worst-case perspective (or at the very least, something weaker than assuming a full distribution over inputs), at least in this instance. If one wants to argue that the randomness in this scenario is only useful practically but that something deterministic would "in principle" be better, then they would have to also argue that modularity of software is useful practically but something non-modular would "in principle" be better. 3. Theoretical Properties The P vs. BPP question is the question of whether, whenever there is a polynomial-time randomized algorithm that behaves correctly on every individual input with probability 99.9999999999%, there is also a polynomial-time deterministic algorithm that behaves correctly on every input. If randomness was never useful, we would expect the answer to this question to be straightforwardly yes --- after all, whatever the random algorithm was doing, we should be able to improve upon it, and since a deterministic algorithm is either right or wrong, the only way to improve upon 99.9999999999% is to go all the way up to 100%. In reality, however, this is a long-standing open problem in complexity theory that is generally believed to be true, but which is far from proven. This point was already raised by Scott in his original thread, but did not seem to gain much traction. Several people were suspicious of the fact that the randomized algorithm only had to work almost all of the time on each input, while the deterministic algorithm had to work all of the time on each input. I have several responses to this. The first, somewhat flippant, response, is that we could of course only require that the deterministic algorithm work almost all of the time for each input, as well; but a deterministic algorithm can only get the answer right or wrong, so “almost always” and “always” amount to the same thing for a determistic algorithm. Secondly, let us consider what it should mean for an algorithm to work “for all practical purposes”. If I have a randomized algorithm, and its probability of failure is so small as to be negligible compared to, say, cosmic rays hitting the computer and corrupting the output, then this seems to meet the bar of “for all practical purposes” (thus motivating the definition of BPP). On the other hand, what should “all practical purposes” mean for a deterministic algorithm? Some might argue that if, on average across inputs to the algorithm, the output is very likely to be correct, then that should satisfy the requirement --- but this raises the question, what average should we use? An addition algorithm that thinks that 1+1=3 but is otherwise correct should surely not meet the “all practical purposes” bar, so the uniform distribution over inputs certainly shouldn't be used. So, perhaps I should be figuring out what I expect my input distribution to look like, and measure according to that? But now, you see, we have added baggage that wasn't in the original problem definition. All I said is that I had some abstract algorithmic problem that I wanted to solve for (some suitable definition of) all practical purposes. You've now told me that in order to do this, I need to figure out what the input distribution to the algorithm is going to be. But this is pretty unreasonable! Can you imagine an undergraduate computer science textbook full of algorithms that would work, but only if its inputs followed such-and-such a distribution? (I'm not claiming that such a guarantee is never useful, only disputing the notion that it should be raised up as the gold standard of algorithms on the same level as P and BPP.) I don't think anyone would learn much from such a textbook, and computer science would be in a pretty woeful state if this was all we could offer. I therefore contend that the only really reasonable notion of deterministically solving a problem for all practical purposes is that it should always work correctly on all possible inputs, thus leading to the complexity class P. This is all a long-winded way of saying that the P vs. BPP question really is the right way to formally ask whether randomness can help from an algorithmic standpoint. I'd also like to specially examine what happens if we use a Solomonoff prior (assigning probability $2^{-k}$ to programs of length k). If there is any program of length L that solves the problem correctly, then any deterministic algorithm of length M that is incorrect with probability much less than $2^{-M-L}$ with respect to the Solomonoff prior must always work (since we can take the program that runs the two algorithms against each other until it finds a discrepancy, then uses that as the input). Therefore, “almost always” with respect to Solomonoff is more or less the same as “always”, so we haven't really done anything different than redefine P in a very roundabout way. Crucially, this means that even if we are fully Bayesian, under a Solomonoff prior the question of whether every randomized polynomial-time algorithm has a deterministic counterpart still boils down to the unresolved and deep P vs. BPP question. [Addendum: it looks like Peter de Blanc and Wei Dai have made this last point already, see here and here.) 4. Usefulness in problem-solving There are important combinatorial properties, such as expansion properties of graphs, which are satisfied by random graphs but which we don't have any deterministic constructions for at all (or in some cases, we do, but they came much later and were harder to come by). Many important problems in graph theory, Ramsey theory, etc. were solved by considering random combinatorial objects (this was one of the great triumphs of Paul Erdos) and thinking in purely deterministic terms seems very unlikely to have solved these problems. Taking a random/pseudorandom perspective has even led to some of the modern triumphs of mathematics such as the Green-Tao theorem (there are arbitrarily long arithmetic progressions) and is fundamental to theoretical cryptography, game theory, convex analysis, and combinatorial approximation algorithms. If we abandon the use of random constructions, then we also abandon some of the most important work in modern computer science, economics, and applied mathematics, which seems unacceptable to me. On the algorithmic side, even if you are Bayesian, you want to somehow represent your posterior distribution, and one of the simplest ways to do so is by random samples. An alternative way is by a probability density function, but under extremely widely held complexity-theoretic assumptions there are many probability distributions that can be efficiently represented by samples but whose probability density function cannot be efficiently written down. More practically, the view of representing distributions by random samples has led to many important algorithms such as sequential Monte Carlo (SMC) and Markov Chain Monte Carlo (MCMC), which form some of the cornerstones of modern statistical inference (MCMC in particular has been called one of the 10 greatest algorithms of the 20th century). Again, abandoning this work on philosophical grounds seems unacceptable. Of course, it is theoretically possible that all of the above gains will eventually be reproduced without using randomness. However, it seems important to note that many of these are among the most beautiful and elegant results in their respective fields. I would object to any characterization of them as "hacks" that come down to a poorly-understood, lucky, but useful result. • The obstacles to "derandomizing" such things appear to be quite deep, as extremely intelligent people have been working on them for decades without success. If one wants to say that a randomization-free, Bayesian approach is "correct even if not practical," this becomes true for a very expansive definition of "practical," and one must concede that being a Bayesian is often completely impractical even when a great deal of time and resources are available for reasoning. • Thinking in terms of randomness is extremely useful in arriving at these deep and important results, and one of the main purposes of a paradigm is to provide modes of thought that lead to such results. I would therefore argue that randomization should be a fundamental lens (among many) with which to approach problems. Conclusion I have given several examples of how randomness is useful in both commonplace and fundamental ways. A possible objection is that yes, randomness might be useful from a pragmatic perspective, but it is still the case that in theory we can always do better without randomness. I don't think that such an objection answers the above arguments, except perhaps in some very weak sense. We saw how, in rock-paper-scissors, randomness is always going to be useful for one of the two players (unless both players are equally strong); we saw how using randomness is necessary to make certain profitable trades; and we saw how, even if we adopt a Solomonoff prior, the question of whether or not deterministic algorithms can compete with randomized algorithms is still open (and the likely resolution will rely upon constructing sufficiently pseudorandom numbers, rather than reasoning carefully about the distribution of inputs). Even if we discard all pragmatic concerns whatsoever, the above examples at the very least show that randomization is useful in a very deep and fundamental way. And importantly, at least from my perspective, we have also seen how randomization has been essential in resolving important problems across a variety of fields. This, if nothing else, is a compelling case that it should be adopted as a feature in one's worldview, rather than discarded on philosophical grounds. Acknowledgements Thanks to Holden Karnofsky for reading an early version of this essay and suggesting the randomized controlled trials example. Thanks also to Sindy Li for suggesting the idea of using randomness to improve market liquidity. # 47 80 comments, sorted by Highlighting new comments since New Comment Some comments are truncated due to high volume. Change truncation settings My economics PhD dissertation related to this. I proposed that if asymmetric information is a cause of parties failing to settle a lawsuit they could both improve their position using lotteries. Consider a simple example: I know that if we went to trial I would win at least$1 million, but trial would cost us both 200k. You don't accept my settlement offer of $1 million because you think I'm lying about the strength of my case. So I propose the following, we flip a coin and if the coin comes up heads we settle for$1 million, whereas if it comes up tails we go to trial and if I fail to win at least \$1 million I pay you a big penalty.

But then my dissertation concluded by saying that our failure to observe litigants using such lotteries is evidence against asymmetric information being a cause of parties failure to settle lawsuits.

5[anonymous]7yMy inner Yudkowsky says “or maybe it just hasn't occurred to them”.
9James_Miller7yBut I published my result in a prestigious journal in 1997 and told lots of high status people about it, and still no lotteries.
2roystgnr7yDid you ever ask anyone in a position to use a lottery why they wouldn't? "People aren't trying my idea" is evidence that it's a bad idea, but weak evidence, preferably replaced by "People say they aren't trying my idea because X" or "People aren't trying my idea but can't articulate why not" when possible.
4James_Miller7yI asked judge Richard Posner (one of my dissertation advisers) if he would be willing to use lotteries as a judge and he said no, it would get him impeached.
4Gust6yInteresting idea. Brazilian law explicitly admits lottery as a form of settling, but I'm not sure if that example with a penalty for not winning a lawsuit would be admissible.
2Jiro7yThe legal system is based on the legal fiction that the judge can infallibly make a decision. If the judge makes a decision in a way which is guaranteed to be fallible in a certain percentage of cases, he violates this assumption, even if the guaranteed fallibility from randomness is less than his normal fallibility when not using randomness.

Replace "adversarial superintelligence" with "adversarial game", and I think you'll get more agreement among the participants. There are plenty of cases where a "mixed strategy" is optimal. Note that this is not noise, and true randomness isn't necessary - it only needs to be unpredictable by the opponent, not necessarily random.

Where you don't have an opponent (or at least one that makes predictions), I'm with Eliezer: noise never helps at a fundamental level.

I do believe that randomness has a place in thinking about prob... (read more)

9Cyan7yTo my thinking, this is essentially equivalent to conjecturing that P = BPP [http://www.openproblemgarden.org/op/p_vs_bpp], which is plausible but still might be false. ETA: Didn't read the post before replying to the parent (saw it in the sidebar). Now I see that a good quarter of the post is about P = BPP. Egg on my face!
7EHeller7yThere is another case where noise helps- threshold effects. If you have as signal below a threshold, a bit of noise can push the signal up into the detectable region.
7Cyan7yDo you mean stochastic resonance [http://en.wikipedia.org/wiki/Stochastic_resonance]? If so, good example! (If not, it's still a good example -- it's just my good example. ;-)
5[anonymous]7yAlso, dithering [http://en.wikipedia.org/wiki/Dither].
6jsteinhardt7ySeveral of my examples did not have opponents. To list three: the bargaining example, the randomized controlled trials example, and P vs. BPP.

Thanks for writing this! This debate was bugging me too; I don't have the background to dive into it in detail as in this post but the fact that Eliezer was implicitly taking a pretty strong stance on P vs. BPP bugged me ever since I read the original LW post. This is also a great example of why I think LW needs more discussion of topics like computational complexity.

Many important problems in graph theory, Ramsey theory, etc. were solved by considering random combinatorial objects (this was one of the great triumphs of Paul Erdos) and thinking in purely deterministic terms seems very unlikely to have solved these problems.

From a Bayesian perspective, a probability is a cognitive object representing the known evidence about a proposition, flattened into a number. It wouldn't make sense to draw conclusions about e.g. the existence of certain graphs, just because we in particular are uncertain about the structure of s... (read more)

It seems that in the rock-scissors-paper example the opponent is quite literally an adversarial superintelligence. They are more intelligent than you (at this game), and since they are playing against you, they are adversarial. The RCT example also has a lot of actors with different conflicts of interests, especially money- and career-wise, and some can come pretty close to adversarial.

6[anonymous]7y"adversarial superintelligence" sounds like something you don't have to worry about facing pre-singularity. "someone who's better than you at rock-paper-scissors" sounds rather more mundane. Using the former term makes the situation look irrelevant by sneaking in connotations.

Requiring that the inputs to a piece of software follow some probability distribution is the opposite of being modular.

What? There is very little software that doesn't require inputs to follow some probability distribution. When provided with input that doesn't match that (often very narrow) distribution programs will throw it away, give up, or have problems.

You seem to have put a lot more thought into your other points, could you expand upon this a little more?

7jsteinhardt7yLet me try to express it more clearly here: I agree that it is both reasonable and common for programs to require that their inputs satisfy certain properties (or in other words, for the inputs to lie within a certain set). But this is different than requiring that the inputs be drawn from a certain probability distribution (in other words, requiring 5% of the inputs to be 0000, 7% to be 0001, 6% to be 0010, etc.). This latter requirement makes the program very non-modular because invoking a method in one area of the program alters the ways in which I am allowed to invoke the method in other areas of the program (because I have to make sure that the total fraction of inputs that are 0000 remains 5% across the program as a whole). Does this make more sense or is it still unclear? Thanks for the feedback.
1Lumifer7yWell, I don't know of a single piece of software which requires that its inputs come from specific probability distributions in addition to satisfying some set of properties. In fact, wouldn't that software object have to maintain some running counts to even be able to estimate from which distribution do its inputs come?
5jsteinhardt7yThis is in some sense my point. If you want to be so hardcore Bayesian that you even look down on randomized algorithms in software, then presumably the alternative is to form a subjective probability distribution over the inputs to your algorithm (or perhaps there is some other obvious alternative that I'm missing). But I don't know of many pieces of code that require their inputs to conform to such a subjective probability distribution; rather, the code should work for ALL inputs (i.e. do a worst-case rather than average-case analysis, which will in some cases call for the use of randomness). I take this to be evidence that the "never use randomness" view would call for software that most engineers would consider poorly-designed, and as such is an unproductive view from the perspective of designing good software.
3trist7yAny software that uses randomness requires you to meet a probability distribution over its inputs, namely that the random input needs to be random. I assume that you're not claiming that this breaks modularity, as you advocate the use of randomness in algorithms. Why?
1jsteinhardt7yBased on the discusssions with you and Lumifer, I updated the original text of that section substantially. Let me know if it's clearer now what I mean. EDIT: Also, thanks for the feedback so far!
3trist7yThe probability distribution part is better, though I still don't see how software that uses randomness doesn't fall under that (likewise: compression, image recognition, signal processing, and decision making algorithms).
1jsteinhardt7yIf the software generates its own randomness (for instance, in JAVA you can do this by creating a new instance of a Random object and calling nextInt(), etc.) then I don't see how this breaks modularity. Compression algorithms like bzip don't promise to make things smaller as part of their contract, they only promise to be lossless. To the extent that a user relies on them becoming smaller consistently, this can lead to trouble, for instance if I expect by inputs of size 10 kilobytes to compress down to 2 kilobytes, and then many of the inputs in a row stay at 10 kilobytes, I can get unexpected load that could create issues. I don't know of many image recognition algorithms that are integrated into a large system without a human in the loop, except perhaps Google Image Search which arguably has a human (the end-user) that filters the results at the end. This is precisely due to their non-modularity: they fail in unexpected and difficult-to-characterize ways, such that any attempt to enforce a contract or abstraction would be probably misguided at this point. The best they can currently promise is "we correctly implement the fast Fourier transform" and leave it up to the programmer to decide whether an FFT is what is merited in a given case. ETA: Another way to put the bzip example which might fit more with your language, is that yes the guarantee of bzip is that it is lossless and that it will make your data smaller as long as the data fit the properties that bzip attempts to exploit, and if that is what we want the contract to be then I would agree that bzip is non-modular.
3V_V7ynitpick: Java default PRNG is a linear congruential generator. It's not as bad as the infamous RANDU [http://en.wikipedia.org/wiki/RANDU], but I won't suggest to use it for anything that needs more than a hundred or so pseudo-random bits.
1IlyaShpitser7ybzip is a great source of random bits. :)
1Lumifer7yDo such people exist? I don't know anyone who fits such a description. Well, of course. But again, has anyone been arguing against that? The original dispute was talking about abstract optimality and whether particular solutions exist. No one suggested that in real-world practice at the current level of technology rand() Call Considered Harmful.
1Lumifer7yWell, let's put it this way. If you can implement a non-randomized solution that works within the specified constraints (time, etc.), it is preferable to a probabilistic one. The real question is what happens when you can't. I doubt that Eliezer would refuse to implement a probabilistic solution on the grounds that it's not pure enough and so no solution at all is better than a version tainted by its contact with the RNG. An exception might be when you need to be absolutely sure what the software does or does not do and any randomness introduces an unacceptable risk. But such a case is very rare.
1jsteinhardt7ySure, I agree with this. But he still seems to think that in principle it is always possible to improve over a randomized algorithm. But doing so requires having some knowledge of the distribution over the environment, and that would break modularity. Whether or not Eliezer himself is basing this argument on Bayesian grounds, it certainly seems to be the case that many commenters are, e.g.: Will_Pearson [http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/oww]: DonGeddis [http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/ox0]: And some comments by Eliezer: Eliezer_Yudkowsky [http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/t6a]: Eliezer_Yudkowsky [http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/t6c]:

Another way of swapping around the question is to ask under what circumstances Jacob Steinhardt would refuse to use a PRNG rather than an RNG because the PRNG wasn't random enough, and whether there's any instance of such that doesn't involve an intelligent adversary (or that ancient crude PRNG with bad distributional properties that everyone always cites when this topic comes up, i.e., has that happened more recently with an OK-appearing PRNG).

Obviously I don't intend to take a stance on the math-qua-math question of P vs. BPP. But to the extent that someone has to assert that an algorithm's good BPP-related properties only work for an RNG rather than a PRNG, and there's no intelligent adversary of any kind involved in the system, I have to question whether this could reasonably happen in real life. Having written that sentence it doesn't feel very clear to me. What I'm trying to point at generally is that unless I have an intelligent adversary I don't want my understanding of a piece of code to depend on whether a particular zero bit is "deterministic" or "random". I want my understanding to say that the code has just the same effect once the zero is gener... (read more)

1V_V7yIf your question is "Is there a known practical case not involving an intelligent adversarial environment where the use of a cryptographic PRNG or even a true RNG rather than a non-cryptographic PRNG is warranted?" Then the answer is no. In fact, this is the reason why it is conjectured that P = BPP. However, given the rest of your comment, it seems that you are referring to how we understand the theoretical properties of algorithms: If we are talking about understanding the theoretical properties of many useful randomized algorithms, I'd say that we can't "screen off" randomness. Even if the algorithm is implemented using a PRNG with a constant seed, and is thus fully deterministic at the level of what actually runs on the machine, when we reason about its theoretical properties, whether it is a formal analysis or a pre-formal intuitive analysis, we need to abstract away the PRNG and assume that the algorithm has access to true randomness. As it was already pointed out, if we were perfectly rational Bayesian agents, then, we would always be able to include the PRNG into our analysis: For instance, given a machine learning problem, a Bayesian agent would model it as a subjective probability distribution, and it may conclude that for that particular distribution the optimal algorithm is an implementation of Random Forest algorithm with the Mersenne Twister PRNG initialized with seed 42. For a slightly different subjective distribution, the optimal algorithm may be an implementation of a Neural Network trained with Error Backpropagation with weights initialized by a Linear Congruentlial PRNG with seed 12345. For another slightly different subjective distribution, the optimal algorithm may be some entirely different deterministic algorithm. In practice, we can't reason this way. Therefore we assume true randomness in order to say meaningful things about many practically useful algorithms.
1redlizard7yA more involved post about those Bad Confused Thoughts and the deep Bayesian issue underlying it would be really interesting, when and if you ever have time for it.
3Lumifer7yin principle are the key words. As the old saying goes, in theory there is no difference between theory and practice but in practice there is. You are using the word "modularity" in a sense weird to me. From my perspective, in the software context "modularity" refers to interactions of pieces of software, not inputs or distributions over the environment.
3jsteinhardt7yBased on the discusssions with you and trist, I updated the original text of that section substantially. Let me know if it's clearer now what I mean. Also, thanks for the feedback so far!
5itaibn07yI think an example of what jsteinhardt is referring to would be quicksort. It can take an arbitrary list as an argument, but for many perversely ordered inputs it takes Omega(n^2). However, it does have an efficient average-case complexity of O(n log n). In other words, if the input is sampled from the uniform distribution over permutations the algorithm is guaranteed to finish in O(n log n) time. Many of the other examples that were considered are similar, in that the algorithm doesn't give an error when given an input outside of the expected distribution, but rather silently works less effectively
1Lumifer7yQuicksort as a piece of software does not require any particular distribution. It is perfectly happy to work with "perversely ordered inputs". A human running quicksort with certain expectations about its performance might require a particular distribution, but that's not a characteristic of software. Recall that the original claim was that expecting a particular distribution breaks modularity.
6Nornagest7yIf a particular function was advertised as having O(n log(n)) running time, and it turns out that it actually runs in O(n^2) on a certain class of inputs, I would not feel it to be behaving in a very modular way. From an abstraction perspective, breaking time or resource assumptions on an input can be as bad or worse than rejecting it. Granted, it's pretty easy to get quicksort to behave on all but the most pathological data sets.
2Lumifer7yIf you have been misled with respect to the behavior of an algorithm on a specific class of inputs, what does it have to do with modularirty?? Modularity refers to the independence of a piece of software from the particulars of other pieces of software. It has nothing to do with performance metrics.
3VAuroch7yWhen picking which algorithms to use for particular pieces of your task, performance metrics are a significant factor. If you are told that your algorithm for Section A of the task runs in O(n log n) time and you know that the Section B portion of the program, which requires a stream on inputs from Section A, will run in O(n (log n)^2), you can assume that Section B will be constantly supplied with inputs and the program can busy-wait for the brief portions where it does not have input. If it turns out that for your specific input distribution Section A is taking O(n^2), that's a critical distinction with far-reaching effects on the design of the rest of your algorithm. It's somewhat harder to provide an example for non-networked, non-parallelized code, but I can do so on request.
1Lumifer7yI understand all that, but I'm reading it as "you should not make mistakes about the expected performance of your code and your application architecture should reflect your use cases." There are many ways to screw up a software system.
1DSimon7yI think this may be a distinction without a difference; modularity can also be defined as human expectations about software, namely that the software will be relatively easy to hook into a larger system.
1Lumifer7yI don't find this a useful definition, but YMMV, de gustibus, and all that...
0trist7ySo you're differentiating between properties where the probability of [0 1 2 3] is 1-ɛ while >3 is ɛ and probability distributions where the probability of 0 is 0.01, 1 is 0.003, etc? Got it. The only algorithms that I can think of that require the latter are those that require uniformly random input. I don't think those violate modularity though, as any are of the program that interfaces with that module must provide independently random input (which would be the straightforward way to meet that requirement with an arbitrary distribution). There's a difference between requiring and being optimized for though, and there are lots of algorithms that are optimized for particular inputs. Sort algorithms are a excellent example, if most of your lists are almost already sorted, there algorithms that are cheaper on average, but might take a long time with a number of rare orderings.

I'd also like to specially examine what happens if we use a Solomonoff prior (assigning probability 2^-k to programs of length k). If there is any program of length L that solves the problem correctly, then any deterministic algorithm of length M that is incorrect with probability much less than 2^-(M+L) with respect to the Solomonoff prior must always work (since we can take the program that runs the two algorithms against each other until it finds a discrepancy, then uses that as the input). Therefore, “almost always” with respect to Solomonoff is more

The randomized control trial is a great example where a superintelligence actually could do better by using a non-random strategy. Ideally, an AI could take its whole prior into account and do a value of information calculation. Even if it had no useful prior, that would just mean that any method of choosing is equally "random" under the the AI's knowledge.

5V_V7yAI != perfectly rational agent
1johnswentworth7yIdeally = perfectly rational agent
3V_V7yso why did you mention an 'AI'?
3alex_zag_al5yBayesian adaptive clinical trial designs place subjects in treatment groups based on a posterior distribution. (Clinical trials accrue patients gradually, so you don't have to assign the patients using the prior: you assign new patients using the posterior conditioned on observations of the current patients.) These adaptive trials are, as you conjecture, much more efficient than traditional randomized trials. Example: I-SPY 2 [http://www.ncbi.nlm.nih.gov/pubmed/19440188]. Assigns patients to treatments based on their "biomarkers" (biological measurements made on the patients) and the posterior derived from previous patients. When I heard one of the authors explain adaptive trials in a talk, he said they were based on multi-armed bandit theory [https://en.wikipedia.org/wiki/Multi-armed_bandit], with a utility function that combines accuracy of results with welfare of the patients in the trial. However, unlike in conventional multi-armed bandit theory, the trial design still makes random decisions! The trials are still sort of randomized: "adaptively randomized," with patients having a higher chance of being assigned to certain groups than others, based on the current posterior distribution.
1IlyaShpitser5yI don't understand this comment at all.
0PhilGoetz5yJWW suggests that an AI could partition trial subjects into control and experimental groups such that expected number of events in both was equal, and presumably also such that cases involving assumptions were distributed equally, to minimize the impact of assumptions. For instance, an AI doing a study of responses to an artificial sweetener could do some calculations to estimate the impact of each gene on sugar metabolism, then partition subjects so as to balance their allele frequencies for those genes. (A more extreme interpretation would be that the AI is partitioning subjects and performing the experiment not in a way designed to test a single hypothesis, but to maximize total information extracted from the experiment. This would be optimal, but a radical departure from how we do science. Actually, now that I think of it, I wrote a grant proposal suggesting this 7 years ago. My idea was that molecular biology must now be done by interposing a layer of abstraction via computational intelligence in between the scientist and the data, so that the scientist is framing hypotheses not about individual genes or proteins, but about causes, effects, or systems. It was not well-received.) There's another comment somewhere countering this idea by noting that this almost requires omniscience; the method one uses to balance out one bias may introduce another.
1IlyaShpitser5yThis doesn't require omniscience, or AI: people do this now (based on info they have). If you have more info, we know how to use it (there is theory). Why are we talking about AI, this is a math problem. -------------------------------------------------------------------------------- Slightly harshly worded suggestion (not to you specifically): maybe more reading, less invocation of robo-Jesus in vain.
2PhilGoetz5yEliezer claims that randomness is always bad; many other people claim that one way randomness is good is that it is unbiased. Partitioning subjects into experimental conditions must be unbiased. Using an algorithm and knowing that its biases are orthogonal to the phenomenon being investigated requires omniscience. Besides, if you knew in advance what was relevant, you wouldn't need to do the experiment. That is what the comment means. The use of the term "AI" is just to show that the claim is that no real-world agent can be smart enough to do unbiased partitioning in all cases, not just that we're not smart enough to do it. In practice, a possibly biased but intelligent partitioning is better when the sample size is small.
3IlyaShpitser5yWe know what property we want (that randomization will give you), good balance in relevant covariates between two groups. I can use a deterministic algorithm for this, and in fact people do, e.g. matching algorithms. Another thing people do is try all possible assignments (see: permutation tests for the null). Discussion of AI and omniscience is a complete red herring, you don't need that to show that you don't need randomness for this. We aren't randomizing for the sake of randomizing, we are doing it because we want some property that we can directly target deterministically. -------------------------------------------------------------------------------- I don't think EY can possibly know enough math to make his claim go through, I think this is an "intellectual marketing" claim. People do this a lot, if we are talking about your claim, you won the game.
1PhilGoetz5yIf you sort all the subjects on one criteria, it may be correlated in an unexpected way with another criteria you're unaware of. Suppose you want to study whether licorice causes left-handedness in a population from Tonawanda, NY. So you get a list of addresses from Tonawanda New York, sort them by address, and go down the list throwing them alternately into control and experimental group. Then you mail the experimental group free licorice for a ten years. Voila, after 10 years there are more left-handers in the experimental group. But even and odd addresses are on opposite sides of the street. And it so happens that in Tonawanda, NY, the screen doors on the front of every house are hinged on the west side, regardless of which way the house faces, because the west wind is so strong it would rip the door off its hinges otherwise. So people on the north side of the street, who are mostly in your experimental group, open the door with their left hand, getting a lot of exercise from this (the wind is very strong), while people on the south side open the screen door with their right hand. It seems unlikely to me that many hidden correlations would survive alternating picks from a sorted list like this rigged example, but if the sample size is large enough, you'd still be better off randomizing than following any deterministic algorithm, because "every other item from a list sorted on X" has low Kolmogorov complexity and can be replicated by an unknown correlate of your observable variable by chance.
3Vaniver5yThis is perhaps a useful place to illustrate the "randomness hath no power" argument: randomness is unbiased in expectation but we actually expect the absolute amount of biasedness for a randomly selected assignment to be nonzero. When biasing factors are known ahead of time, we do better by controlling for it directly (with, say, a paired assignment).
0johnswentworth5yExactly! This is a math problem! And it becomes a very complicated math problem very quickly as the prior information gets interesting. There's nothing magical about an AI; it can't figure out anything a human couldn't figure out in principle. The difference is the "superintelligence" bit: a superintelligent AI could efficiently use much more complicated prior information for experiment design.
0EHeller5yI don't understand the improvement you think is possible here. In a lot of cases, the math isn't the problem, the theory is known. The difficulty is usually finding a large enough sample size,etc.
0EHeller5yThere is a lot of statistical literature on optimal experimental design, and it's used all the time. Years ago at Intel, we spent a lot of time on optimal design of quality control measurements, and I have no doubt a lot of industrial scientists in other companies spend their time thinking about such things. The problem is, information is a model dependent concept (derivatives of log-likelihood depend on the likelihood), so if your prior isn't fairly strong, there isn't a lot of improvement to be had. A lot of science is exploratory, trying to optimize the experimental design is premature. Either way, this isn't stuff you need an AI for at all, it's stuff people talk about and think about now, today, using computer assisted human intellect.
0Lumifer5yDesigning experiments to get more information than just evidence for a single hypothesis is old hat [https://en.wikipedia.org/wiki/Design_of_experiments].
0johnswentworth5yI'm going to commit a social faux pas and respond to my own comment, because multiple subthreads are all saying essentially the same thing: this is just math, the theory is known, humans can already do it (often with some help from computers to get through the math). As I've read it, one of the major takeaways of lesswrong is that AI is not magical. If humans cannot possibly figure out the theory, neither can an AI. If humans cannot possibly do the math (possibly with some help from a computer), neither can an AI. Anything an AI can do, a human can also do in principle. They differ only in the degree: AIs will eventually be able to do much more complicated math, solve much more complicated problems, and self-improve much faster and more reliably. So if you look at my original suggestion and think "that's nothing special, a human can do that in theory" then you're completely correct. Things humans can do IN THEORY are EXACTLY the things with which an AI can help.

Something funny has happened to the font of this post, making it difficult for me to read.

3jsteinhardt7yShould be fixed now, I think.
1jsteinhardt7yIt seems to have to do with the fact that the font is sometimes Times, and sometimes whatever the default font is. I don't see an easy way of fixing this that doesn't involve me changing lots of HTML tags by hand. If anyone knows how to fix it easily then please let me know.
2Lumifer7yUm, load your post into a text editor and do a global search-and-replace..?
1Cyan7yI've had some success in the past cut-n-pasting text into Notepad (which automatically strips all formatting) and then back into the edit window.
2jsteinhardt7yWon't that kill things like bullets, links, images, etc.?
3Cyan7yYeah, I guess so. Still might be worth it relative to changing lots of HTML tags by hand. By my (in all likelihood, none too accurate) count, you have four links, three latex expressions (only one of which needs to be latex), two numbered lists, one bullet list, and a bunch of section titles. Not too bad to do by hand...

Instead, I'd like to present several examples of scenarios where I will argue that randomness clearly is useful and necessary, and use this to argue that, at least in these scenarios, one should abandon a fully Bayesian stance.

Sure. But, in order to check the communication channel, which of those examples do you think I (as one of the people in the previous post arguing for seeing the debate as one over word-interpretation) would disagree with either your content or your presentation? How about Eliezer?

I think I should once again push the point that t... (read more)

1jsteinhardt7yWhich field are we talking about? What people? The weighted majority algorithm (the topic of the post that started this all) is one of the cornerstones of statistical learning theory. I would guess that pretty much everyone who knows statistical learning theory well already knows that pure strategies are optimal given complete (probabilistic) knowledge of the environment.
1Lumifer7yAssuming the existence of closed-form solutions which is not a given. If your environment is sufficiently complex, you may not be able discover the optimal pure strategy in reasonable time.
3jsteinhardt7yI mean yes, I did just write a quarter of my post on this topic :).

Apparently the easiest way to construct an expander graph is via randomness. Deterministic constructions are very difficult.

This illustrates how randomness can be used to improve market liquidity.

While an interesting idea, I believe most people just call this "gambling".

Requiring that the inputs to a piece of software follow some probability distribution is the opposite of being modular.

I don't understand this. It's perfectly fine for a module (an object) to declare what kind of inputs it will accept. Modularity in software basically means "write to the declared interface and treat internals as a black box" and I don't see why requiring a particular s... (read more)

2DSimon7yI'm not sure what you're driving at here. A gambling system where everybody has a net expected gain is still a good use of randomness.
-1Lumifer7ySo what's the difference between a market where you can buy a chance to get some asset for cash, and a casino where you can buy a chance to get some asset for cash? You have non-coerced humans doing voluntary transactions in both of them. Don't forget that you still need a counterparty in the markets.
1jsteinhardt7yAttempted a clearer explanation in this [http://lesswrong.com/lw/kd4/the_power_of_noise/b0a6] comment. Any feedback you have would be great!


text:
# Place square bracket around a cross-reference

Is possible to instruct LaTeX to surround all cross-references with square brackets? For example when I write a command like some text \ref{some label} then when compiled I would like the output to be some text [12] without manually placing square brackets in the LaTeX input, as though I had entered some text [\ref{some label}].

• Welcome to TeX.SE. Do you want all cross-references to be surrounded by square brackets? Or just selected ones? And, do you load any cross-referencing packages (such as as varioref, hyperref, and cleveref)? – Mico Jul 3 '18 at 5:58

The \ref command is a low-level LaTeX instruction, and it is modified heavily by various packages, including hyperref and cleveref. I therefore wouldn't modify \ref directly, mainly to avoid all kinds of unpredictable conflicts with packages that a user may load. Instead, I suggest setting up a new macro called, say, \sbref (short for "square brackets \ref, I suppose) that invokes \ref:
\newcommand\sbref[1]{[\ref{#1}]}

If all cross-references generated by \ref in your document should be surrounded by square brackets, simply perform a global search and replace operation in your text editor, changing \ref to \sbref.


text:
A car starts from rest and moves along the $x-axis$ with constant acceleration $5\ ms^{-2}$ for 8 seconds. If it then continues with constant velocity, what distance will the car cover in 12 seconds since it started from the rest?

Complete Python Prime Pack for 2023

9 Courses     2 eBooks

Artificial Intelligence & Machine Learning Prime Pack

6 Courses     1 eBooks

Java Prime Pack 2023

8 Courses     2 eBooks

Given: A car starts from rest and moves along the x-axis with a constant acceleration of 5 m/sfor 8 seconds

To find:  what distance will the car cover in 12 seconds since it started from rest

Solution:

We know that,

$s\ =\ ut\ +\ \frac{1}{2} at^{2}$

But given that, the initial velocity is zero. Therefore, the distance covered by the car in first 8 seconds is

=> $s\ =\ 0\ +\ \frac{1}{2} \times 5\ \times 8^{2}$

=> $s\ =\ 160\ metres$

Therefore, the velocity attained at the end of 8 seconds is:

from, $v\ =\ u\ +\ at$

=> $v=\ 0\ +\ 5\ \times \ 8\ =\ 40\ m/s$

Therefore, since the car has moved with constant velocity from 8th second to 12 second, the distance traveled is

$s\ =\ v\ \times \ t\ =\ 40\ \times \ 4\ =\ 160\ m$.

Therefore, the total distance traveled is $160\ +\ 160\ =\ 320m$

Updated on 10-Oct-2022 13:28:42


text:
# [Seminar] Dr.Anil Kumar ( IOP Bhubaneswar, SINP Kolkata, HBNI Mumbai, India), 28th May, 2021

We are delighted to notice that there we will have a seminar talk about neutrino physics with Dr.Anil Kumar, from IOP Bhubaneswar, SINP Kolkata, HBNI Mumbai, India

Title : Exploring Oscillation Dip and Valley, NSI, and Earth’s Core using Atmospheric Neutrinos at INO-ICAL Detector
Speaker : Dr.Anil Kumar
Date & time : 10:00 AM 28th, May, 2021
The 50 kt magnetized Iron Calorimeter (ICAL) detector at the proposed India-based Neutrino Observatory (INO) aims to detect atmospheric neutrinos and antineutrinos separately in the multi-GeV range of energies and over a wide range of baselines. By utilizing its charge identification capability, ICAL can efficiently distinguish $\mu^-$ and $\mu^+$ events and hence, the parent particles $\nu_\mu$ and $\bar{\nu}_\mu$. We demonstrate for the first time that the oscillation dip and valley can be reconstructed using the ratio of upward-going and downward-going reconstructed muon events at ICAL separately for $\mu^-$ and $\mu^+$. The location of dip and the alignment of valley is used to constrain the atmospheric mass-squared difference in two separate ways. Further, we propose a new approach to explore the neutral-current non-standard neutrino interaction (NSI) parameter $\varepsilon_{\mu\tau}$ in atmospheric neutrino experiments using oscillation dips and valleys. We show that non-zero $\varepsilon_{\mu\tau}$ shifts the oscillation dip locations and bends the oscillation valley. Thanks to the charge identification capability of ICAL, the opposite shifts in the locations of oscillation dips as well as the contrast in the curvatures of oscillation valleys for $\mu^-$ and $\mu^+$ is used to constrain $|\varepsilon_{\mu\tau}|$. Next, we explore the possibility of utilizing neutrino oscillations in the presence of matter to extract information about the internal structure of Earth complementary to seismic studies. Using good directional resolution, ICAL would be able to observe 331 $\mu^-$ and 146 $\mu^+$ core-passing events with 500 kt$\cdot$yr exposure. With this exposure, we show for the first time that the presence of Earth’s core can be independently confirmed at ICAL with a median $\Delta \chi^2$ of 7.45 (4.83) assuming normal (inverted) mass ordering by ruling out the simple two-layered mantle-crust profile in theory while generating the prospective data with the PREM profile.


text:
×

# Can we generalize it?

Solve :$$\displaystyle \frac{1}{a_1}+\frac{1}{a_2}+\cdots+\frac{1}{a_n}=1$$ for positive integers $$(a_1,a_2,a_3,\cdots,a_n)$$ .

There could be a remarkable proof/method.

8 months, 2 weeks ago

Sort by:

Sylvester's sequence! · 8 months ago

What about 1/(n)(n+1)? Or for generalization, Leibniz's triangle? · 7 months, 3 weeks ago

Yes I've tried that and thanks for mentioning too , but I think those are not only solutions. Indeed suppose we are to generalize it for 5 variables & Leibniz's triangle gives us 5 different values only. But there exists many more pairs than that · 7 months, 3 weeks ago


text:
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Data Format in R, You’ll learn about data formats and why reformatting data can help you enhance your data analysis in this tutorial.

Data is typically acquired from a variety of sources and by a variety of persons, and it is kept in a variety of formats.

Data formatting is the process of transforming data into a standardized format that allows you to make meaningful comparisons.

Data formatting is an important aspect of dataset cleansing since it guarantees that data is consistent and easy to understand.

Let’s take an example of data set containing Cities, Bangalore, Bengaluru, Bnglr all are the different expressions be used to symbolize Bangalore City.

In the majority of cases, you’ll want to consider them all as a single unit, or format, to make statistical analysis easier later on.

Customer Segmentation K Means Cluster »

## Data Format in R

As discussed in one of our old posts, the same dataset will utilize here also.

library(tidyverse)
library(dplyr)
library(ggplot2)
head(data)

There is a column called “FlightDate” in the Airline dataset. The “FlightDate” field is formatted as “year-month-day,” with 2003 as the year, 03 as March, and 28 as the day.

The “FlightDate” field can be separated into three columns: “year,” “month,” and “day.”

Reformatting the date in tidyverse is as simple as typing one line of code. You can do the same while utilizing different packages also but here we are concentrating only on tidyverse package.

Because one of our old posts discussed the important “packages for data science” contains tidyverse.

Cluster Meaning-Cluster or area sampling in a nutshell »

This example reformats the column with the separate() function, separating the date and renaming the three new columns “year,” “month,” and “date.”

data1<-data %>% separate (FlightDate,sep="-", into=c("year","month", "day"))
head(data1)

The data type may be wrongly determined for a variety of reasons, including when importing a dataset into R or processing a variable.

For example, the allocated data type for the flight date is “character,” despite the fact that the desired data type is numeric.

str(data1)

It’s critical to investigate the column’s data type and convert it to the correct data type for further analysis; otherwise, the models you later construct may act strangely, and valid data may be interpreted as missing data.

KNN Algorithm Machine Learning » Classification & Regression »

The sapply() function in R can be used to verify the data type of each column in a dataset to determine column data types.

sapply(data1,typeof)

If this gives the wrong conversion then you can make use of mutate function.

data2<-data1 %>%
select(year, month, day) %>%
mutate_all(type.convert) %>%
mutate_if(is.character,as.numeric)
str(data2)

You learned in this tutorial that reformatting data is a method of bringing information into a common standard of expression, which allows you to make meaningful comparisons.

Principal component analysis (PCA) in R »

The post How to do Data Format in R appeared first on finnstats.


text:
# Question: How to write a procedure to determine if values are length of a triangle?

February 19 2013 by
false
Maple 16

0

Q3) Write a procedure, isTriangle, to determine if any three given
 numbers represent the lengths of the sides of a triangle.       The procedure should be able to deal with any input.I have done the following and get an error

isTriangle:=proc(a,b,c::nonnegative and numeric)if (a^2+b^2)=c^2 then print (a,b,c "represent the sides of a triangle")end if;end proc;Error, unexpected string
when stated 'unexpected string', the cursor is moved to the
word triangle between the l and e

is there something wrong with my procedure?
many thanks


text:
# Definition:Simultaneous Linear Equations/Matrix Representation

## Definition

A system of simultaneous linear equations can be expressed as:

$\mathbf A \mathbf x = \mathbf b$

where:

$\mathbf A = \begin {bmatrix} \alpha_{1 1} & \alpha_{1 2} & \cdots & \alpha_{1 n} \\ \alpha_{2 1} & \alpha_{2 2} & \cdots & \alpha_{2 n} \\ \vdots & \vdots & \ddots & \vdots \\ \alpha_{m 1} & \alpha_{m 2} & \cdots & \alpha_{m n} \\ \end {bmatrix}$, $\mathbf x = \begin {bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}$, $\mathbf b = \begin {bmatrix} \beta_1 \\ \beta_2 \\ \vdots \\ \beta_m \end {bmatrix}$

are matrices.

### Matrix of Coefficients

The matrix $\mathbf A$ is known as the matrix of coeffficients of the system.

### Augmented Matrix

Let $\begin {bmatrix} \mathbf A & \mathbf b \end {bmatrix}$ be the block matrix formed from $\mathbf A$ and $\mathbf b$.

Then $\begin {bmatrix} \mathbf A & \mathbf b \end {bmatrix}$ is known as the augmented matrix of the system.


text:
# Partially ordered set of compatible topologies

Let $X\neq \emptyset$ be a set and let ${\cal J} \subseteq {\cal P}(X)\setminus\{\emptyset\}$ be a collection of non-empty subsets of $X$. We say that a topology $\tau$ on $X$ is ${\cal J}$-compatible if for every $J\in {\cal J}$ there is a continuous surjective map $f:X\to J$, where $J$ carries the subset topology inherited from $(X,\tau)$.

Let $\text{Cptb}(X, {\cal J})$ denote the collection of ${\cal J}$-compatible topologies, ordered by $\subseteq$.

Note that ${\cal P}(X)$ is always the greatest element of $\text{Cptb}(X,{\cal J})$, and the indiscrete topology $\{\emptyset, X\}$ is always the smallest element.

If $\tau\in \text{Cptb}(X, {\cal J})$ and $\tau\neq{\cal P}(X)$, is there a minimal element in $\text{Cptb}(X, {\cal J})$ properly containing $\tau$ (= "above" $\tau$, in the poset we are considering)?

• No, if $X$ is finite and the topology $\tau$ itself is minimal in $Cptb(X,\mathcal J)$. – Taras Banakh Sep 1 '16 at 14:53
• Thanks @TarasBanakh for your remark. I forgot to exclude $\tau = {\cal P}(X)$, the discrete topology. In the case that $X$ is finite, $\text{Cptb}(X,{\cal J})$ is finite, so if $\tau \neq {\cal P}(X)$, then $\tau$ is not the greatest element of the poset $\text{Cptb}(X,{\cal J})$, therefore there is a minimal element in $\text{Cptb}(X,{\cal J})$ above $\tau$. – Dominic van der Zypen Sep 2 '16 at 13:48
• Setting $\mathcal J = \{X\}$ makes $\mathrm{Cptb}(X,\mathcal J)$ just the lattice of topologies on $X$, and then the answer to your question is known to be negative. Do you want to put further restrictions on $\mathcal J$? – Will Brian Sep 2 '16 at 14:01
• Thanks Will! That's already sufficient for me to see that there are no minimal elements as in the question in general. If you can copy your remark into an answer I'll accept it. - What I'm really keen to see is whether $\text{Cptb}(X,{\cal J})$ is a lattice, see my more recent question on that. – Dominic van der Zypen Sep 3 '16 at 7:56
• OK @DominicvanderZypen, I've done that now. – Will Brian Sep 6 '16 at 14:47

If we put $\mathcal J = \{X\}$, then $\mathrm{Cptb}(X,\mathcal J)$ is just the lattice of topologies on $X$. In this case, the answer to your question is known: if $X$ is infinite, then not every topology on $X$ has a "minimal" strict refinement.
For example, let $p \in X$ and let $\tau$ be the one-point compactification of $X - \{p\}$: that is, neighborhoods of $p$ are sets of the form $X - F$, where $F$ is a finite subset of $X$ not containing $p$, and every other point is isolated. I claim that $\tau$ has no minimal proper refinement. To see this, suppose $\tau'$ properly refines $\tau$. Then there is some infinite closed $Y \subseteq X - \{p\}$. Split $Y$ into two disjoint infinite sets $Y_0$ and $Y_1$, and define $\sigma$ to be the refinement of $\tau$ obtained by declaring $Y_0$ closed (in other words, $\sigma$ is the topology with subbasis $\tau \cup \{X - Y_0\}$). It is not hard to see that $\sigma$ is strictly between $\tau$ and $\tau'$.


text:
# Fourier Series. Writing a partial sum as an integral.

## Homework Statement

Given: View attachment 56653, show that this can be written as: View attachment 56651.

## Homework Equations

Hint: View attachment 56652

## The Attempt at a Solution

Quite confused by this question. For f'N I have:

$\frac{4}{∏}$Ʃ$\frac{cos((2n -1)x)}{(2n - 1)^{2}}$ but i'm not sure where to go from here.

I have read about the Dirichlet kernel, which is what I think I require, but I am rather lost.

Any help is appreciated.

Cheers.

## The Attempt at a Solution

#### Attachments

• hint.png
4.8 KB · Views: 371
• integral.png
3.1 KB · Views: 324
• partial sum.png
3.1 KB · Views: 330