text:
## Pre-school mathematics

29.03.2012 20:02

Recently the following question has been circling the Internet and also ended up in my inbox a few times. It's yet another one of those silly exercises that ask you to continue a sequence based on a number of examples. They are a pet peeve of mine and I'll explain why I hate them here once and for all instead of replying to everyone that sent a copy my way.

The logic of this specimen is particularly broken, even putting aside the questionable statement about pre-school children being able to solve it in mere minutes. It's obvious that whoever wrote that is not familiar with the meaning of the equals sign, as 8809 isn't equal to 6. So strictly reading, this exercise gives you 21 false statements. From that it can be simply inferred that the author is looking for another false statement, so ??? can be replaced with any number not equal to 2581.

With that out of the way, let's look at what was probably meant with this notation. You are given N=21 pairs of values (xi, yi) such that:

f(x_i) = y_i \qquad i\in[0 \dots N-1]

And asked to find yN for a given xN:

f(x_N) = y_N

From a purely mathematical standpoint, this problem has an infinite number of solutions. To demonstrate, assume that f is a function that satisfies the condition set above. In that case:

f(x) + g(x)

where

g(x_i) = 0 \qquad i \in [0 \dots N-1]

also satisfies this condition. Note that g is a function that can have an arbitrary value at xN and hence this new solution will give a different value of yN.

Just to make this a little bit more practical, here's an example of a polynomial that goes through N points:

f(x) = \sum_{i=0}^{N-1} (x - a_i) \prod_{j=0; j \neq i}^{N-1} (x - x_j)
a_i = x_i - \frac{y_i}{\prod_{j=0;j \neq i}^{N-1}(x_i - x_j)}

And here is how its graph looks like if you plug in the given 21 values:

At x=2581 it gives the value of around 5.8·1073.

Note that using the formula above and N=22, you can get a polynomial that goes through the required 21 points plus the 22nd that you can choose to your liking.

This is just one example. You could construct functions that meet the requirements from trigonometric or exponential functions in a similar way.

Sure, so this is probably not the solution the author of this question was looking for. You could say that I'm overcomplicating things and that the rule that pre-school children would come up with, based on the shape of the digits the author arbitrary chose to represent these numbers in, is nicer. But there is no such thing as niceness in mathematics (see interesting number paradox).

This is the big difference between mathematics and physics. Our universe has physical laws that for some reason can often be represented in certain mathematical forms. If a liter of water has a mass of one kilogram and two liters a mass of two kilograms it's quite likely that three liters will have a mass of three kilograms and so on. But this is purely based on our experience and beliefs about our environment and not on any purely mathematical basis. Such a prediction, just like the one above and many like it, when stripped of any physical context, does not make any sense.

Posted by | Categories: Ideas | Comments »

## Cutting styrofoam the analog way

14.03.2012 23:47

My dad is an avid model airplane builder and he is currently building a new electrically-propelled glider of his own design. This particular airplane will have styrofoam-core wings and two weekends ago he was trying out a new way of making them.

Such wing profiles are traditionally cut out from blocks of styrofoam with a hot-wire cutter that is pulled by hand over plywood templates. However the width of the cut depends on the speed of the wire as it travels through the material and since you can't manually pull the wire at a perfectly constant speed the wings end up full of little pits and grooves. Of course, nowadays you can get computer controlled cutters that control the hot wire with servos and can cut out any shape with perfect steadiness and without the need for templates.

My dad however went for another approach: he made a purely mechanical device that pulls the wire over the templates with constant speed. The interesting bit here is that the two ends of the wire usually have to travel different lengths through the styrofoam in the same amount of time, depending on the wing taper ratio. He achieved this with an adjustable system of levers, pulleys and ropes that would make for a nice high-school mechanics class demonstration.

The whole thing is powered by gravity and an occasional nudge by hand. In fact, for perfect straight cuts just the weight of the hot-wire cutter is sufficient without additional mechanisms.

As you can see from the (long) video below, it takes one to two minutes to make one cut (and you need several per wing, depending on the number of segments), so it's not the fastest thing around. But it makes up for it with perfectly smooth cuts and if you only make a few wings per year it's perfectly sufficient.

(Click to watch Cutting styrofoam the analog way video)

Posted by | Categories: Life | Comments »

## Reverse engineering Energycount 3000

12.03.2012 20:52

A while ago Gašper gave me this Energycount 3000 kit from Voltcraft for logging household electrical energy usage, with a wish that he would like to access its measurements from a computer. All this time it's been mostly gathering dust on my desk, but last week I've found some time to give it a closer look and made a few discoveries that are worth reporting.

The box contains two sensors that can be connected between a wall socket and a plug and a tiny battery-powered remote control for reading out the data. The short instruction leaflet explains that the sensors broadcast their measurements through radio every 5 seconds. With a push of the SCAN button you can set the remote control into a 6 second listening mode which catches the transmission and displays it on the small LCD screen.

The remote also has some calculation functions, like predicting the next electricity bill, but is otherwise nothing more than a remote display for the sensors, which apparently do all the logging. This makes sense: the remote control is limited by its batteries and as much functionality as possible is pushed to the wall plug where there is abundant power. The radio link is also obviously unidirectional. The sensors transmit their periodic reports and the remote receives them. All user interaction with the sensor is done through a push button on the sensor itself.

To get the data from the sensor it appears that all I would have to do is to eavesdrop on the transmission. Unfortunately, the box says this device operates on the 868 MHz ISM band, meaning that my 433 MHz receiver was useless. I could get a 868 MHz receiver module for it, but I suspected that these devices use something more complicated than on-off keying, so I looked for other possibilities.

Here is how sensors and the remote look from inside:

As you can see each device has two integrated circuits under a blob of epoxy. I'm guessing one is a microcontroller and the other obviously some kind of integrated ISM band transmitter or receiver. In both the sensor and the remote control they are connected with 5 copper traces. Having recently encountered and worked with chips like the CC1101, my first guess was something like that. I figured the five traces would carry a digital bus like SPI or I2C, so I soldered some wires to the tiny traces on the remote (destroying one of the termination capacitors in the process) and hooked it up to a logic analyzer.

Unfortunately, what I saw there didn't look at all like a synchronous transmission. Out of 5 lines, 2 seem not to carry any useful signals (all I saw there were some transients that looked like glitches). The remaining 3 might act as a digital bus immediately after the microcontroller wakes up the radio chip, but otherwise look like some pulse-width modulated signals for the majority of the 6 seconds when the radio is turned on.

This was a sort of a dead-end until I came across chips like the TH81112. These are ISM band receivers that can be used to receive ASK or FSK transmissions, but are much simpler that system-on-chip products like CC1101. They merely contain a tuner, intermediate frequency and a phase detector and therefore rely on the microcontroller or some other logic to do full FSK demodulation. In hindsight, it makes sense for Energycount to use something like this. It's a relatively cheap product and an expensive general-purpose transceiver like the CC1101 would probably be far too expensive for it.

Update: Gašper found a thread that has some interesting insights, particularly regarding the communication between the radio chip and the CPU. It's possible I was looking at the signals at a completely wrong time scale.

But at this point I didn't bother to mess further with the original receiver. I started up GNU Radio and Fun Cube Dongle Pro and tried to catch the transmissions with that. It turns out fishing out the correct channel in the 868 MHz ISM band is a challenge in itself if you're only limited to the 90 kHz bandwidth of the Fun Cube Dongle. But luckily having the transmitter and receiver close by meant that I was able to see the transmission burn through even when the receiver wasn't tuned exactly to the correct frequency. So after some bisection I found the transmission at 868.388 MHz.

The double peaks in the spectrogram gave me more confidence that this is indeed frequency-keying with 20 kHz deviation and with GNU Radio's FM demodulator block the bits in the packets became clearly visible.

Actually, it's impressive how easy software defined radio makes tasks like this. Throwing blocks around in the GNU Radio Companion in a few minutes is something that would otherwise take you weeks with a soldering iron. Next time I'm doing something similar I'll most likely skip the whole logic analyzer part and just skip right to sniffing the radio waves.

To conclude, this demodulated signal is now something that can be piped to the capture process from my AM433 project and it will hopefully produce binary data. But of course, getting useful information from a binary blob is a whole new matter and that will come in a follow-up post.

Posted by | Categories: Analog | Comments »

## z80dasm 1.1.3

06.03.2012 19:42

Last week Eric Smith found a bug in z80dasm, the disassembler for the Zilog Z80 microprocessor I put together a few years ago when I was researching Galaksija's ROM. It turns out a corner case in relative addressing where the offset would wrap around the 16-bit address space boundary of the CPU wasn't handled correctly. For such cases the code would create labels with excessively long names which overflowed some internal fixed-length string buffers, leading to stack corruption.

Hence a new release of z80dasm after almost 4 years. You can download the source tarball from https://www.tablix.org/~avian/z80dasm/.

I also used this opportunity to move the code from my old CVS repository to git, so you can now also clone the repository with:

\$ git clone https://www.tablix.org/~avian/git/z80dasm.git

As for binary packages, Eric is packaging z80dasm for Fedora. I've put together updated packages for Debian and will do my best to get them into the Debian Unstable as soon as possible (as Debian is now three releases of z80dasm behind).

Posted by | Categories: Code | Comments »

## Equestrian encounter

03.03.2012 17:38

One of the more unusual sights at the 28C3 last year in Berlin was a circle of hackers of all ages and genders sitting on the floor around a large flat panel monitor. They were watching a cartoon show with colorful ponies, while behind the screen you could see the dark basement with flags of various groups and hackerspaces hanging from the ceiling and a disarray of electronic devices strewn on tables. Maybe not surprisingly, this gathering also happened to be almost immediately below the no photography sign, which means that I was unable to find any evidence of it.

I have heard about this show now and then in the usual background noise of the web even before the congress. However this encounter made me somewhat curious and, having never actually overgrown watching an occasional cartoon, I decided to see what all this is about. For scientific reasons, of course.

Image by Chromamancer

I am talking about the latest remake of My Little Pony. It's an animated TV series that was created to advertise a line of Hasbro toys for girls, but in a funny turn of events actually got a sizable following of the opposite gender and in another age group. I guess something that would be unlikely to happen without the internet and the pseudo-anonymous discussions it enables.

After watching some of the show, I can say that I can understand to some degree why it has attracted such a broad audience. They say that a sign of good content for children is that it is also worth watching for adults. And in this particular example the latter certainly appears to be true.

You can often read that the show is well drawn and animated. For sure that is one part of the attraction. Remember my thoughts about the Avatar movie? I guess modern artists have become increasingly good at triggering emotional responses in their audience. The combination of human and animal visual traits in a character allows you to trigger more image recognition circuits than what is possible with only a human face. There's actually a word for that: superstimulus and it's quite amusing to see the parallels between songbirds falling for a red stick with white bands and people looking at what would be pretty deformed body shapes in nature.

Another thing that is often mentioned is the lack of cynicism in the show. Not surprisingly for a show that also carries an educational mark, the characters always find out that it's better to work for the greater good than for only your personal benefit. I can certainly see the appeal in this. It's nice to loose yourself in such fantasy after spending day after day in society that is increasingly focused on grabbing as much of the pie for yourself as possible and there isn't much space for laughter, truth, generosity and loyalty.

However, if this is one of the causes for the popularity among grown-ups, it's somewhat ironic if you consider why the show has been created in the first place: to increase profits of a multinational company that turns around billions of dollars each year. And in fact Hasbro has controlled the show from the start, making sure that it features things they can sell as toys.

But if you put these arguments aside, the creators of the show managed to paint a pretty consistent picture of a world where things are operated with hooves and mouths instead of fingers and three races divide their work to cover for each others shortcomings. This has made it possible for fans of the show to build upon it with their own original stories and, not having to stick to the original constraints, some of them are just as, if not more amusing than the original itself. So it's not all in the visual appearance either.

To sum it up, once you survive through the first view of the pink sugary overkill of the brand logo, it's an enjoyable show with memorable songs and a surprising lack of elements that could be described as fitting for a stereotypical little girl. If it makes more people think about their actions from the perspective of broader society, so much better. And if marketing departments can miss their target audience that much, I guess that's also a sign to be optimistic and might mean we have some time left before they figure out how to control our every thought.

Posted by | Categories: Life | Comments »

format: plain


text:
# Typed Arrays from String Arrays for Dataset Operation

## Preamble¶

In [2]:
:dep ndarray-csv = {version = "0.4.1"}
:dep ndarray = {version = "0.13.0"}
:dep darn = {version = "0.1.15"}
:dep ureq = {version = "0.11.4"}
:dep plotly = {version = "0.4.0"}
extern crate csv;

use std::io::prelude::*;
use std::fs::*;
use ndarray::prelude::*;
use std::str::FromStr;
use plotly::{Plot, Scatter, Layout};
use plotly::common::{Mode, Title};
use plotly::layout::{Axis};

## Introduction¶

In this section, we're going to move from a raw dataset stored in a single string array (ndarray::Array2<String>) to multiple arrays of the desired type. This will enable us to use the appropriate operations for our different types of data. We will demonstrate our approach using the Iris Flower dataset.

Before we move onto moving parts of our Iris Flower dataset into different typed arrays, we need to load it into our raw string array.

In [3]:
let file_name = "Iris.csv";

let res = ureq::get("https://shahinrostami.com/datasets/Iris.csv").call().into_string()?;

let mut file = File::create(file_name)?;
file.write_all(res.as_bytes());
let mut rdr = csv::Reader::from_path(file_name)?;
remove_file(file_name)?;

let data: Array2<String> = rdr.deserialize_array2_dynamic().unwrap();
let mut headers : Vec<String> = Vec::new();

for element in rdr.headers()?.into_iter() {
};

Let's display some rows from the string array to see if it;s loaded as expected.

In [4]:
darn::show_frame(&data, Some(&headers));
Out[4]:
Id SepalLengthCm SepalWidthCm PetalLengthCm PetalWidthCm Species
"1" "5.1" "3.5" "1.4" "0.2" "Iris-setosa"
"2" "4.9" "3.0" "1.4" "0.2" "Iris-setosa"
"3" "4.7" "3.2" "1.3" "0.2" "Iris-setosa"
"4" "4.6" "3.1" "1.5" "0.2" "Iris-setosa"
"5" "5.0" "3.6" "1.4" "0.2" "Iris-setosa"
... ... ... ... ... ...
"146" "6.7" "3.0" "5.2" "2.3" "Iris-virginica"
"147" "6.3" "2.5" "5.0" "1.9" "Iris-virginica"
"148" "6.5" "3.0" "5.2" "2.0" "Iris-virginica"
"149" "6.2" "3.4" "5.4" "2.3" "Iris-virginica"
"150" "5.9" "3.0" "5.1" "1.8" "Iris-virginica"

Without digging deeper, it looks like we have the correct number of columns and rows.

## Raw Dataset Dimensions¶

Once the data is loaded we may want to determine the number of samples (rows) and features (columns) in our dataset. We can get this information using the shape() function.

In [5]:
&data.shape()
Out[5]:
[150, 6]

We can see that it's returned an array, where the first element indicates the number of rows and the second element indicates the number of columns. If you have prior knowledge of the dataset then this may be a good indicator as to whether your dataset has loaded correctly. This information will be useful later when we're initialising a new array.

## Deciding on Data Types¶

Our dataset is currently loaded into a homogeneous array of strings, ndarray::Array2<String>. This data type has allowed us to load all our data in from a CSV file without prior knowledge of the suitable data types. However, it now means that we cannot apply pertinent operations depending on the feature. For example, we aren't able to easily determine any central tendencies for the SepealLenghCm column, or convert our units from centimetres to something else. All we can do right now is operate on these values as strings.

Let's see what happens if we try to operate on the data in its current form, e.g. if we want to find the mean average value of each column.

In [21]:
data.mean_axis(Axis(0)).unwrap()
data.mean_axis(Axis(0)).unwrap()
^^^^^^^^^ the trait num_traits::identities::Zero is not implemented for std::string::String
the trait bound std::string::String: num_traits::identities::Zero is not satisfied
data.mean_axis(Axis(0)).unwrap()
^^^^^^^^^ the trait num_traits::cast::FromPrimitive is not implemented for std::string::String
the trait bound std::string::String: num_traits::cast::FromPrimitive is not satisfied
data.mean_axis(Axis(0)).unwrap()
^^^^^^^^^ no implementation for std::string::String / std::string::String
cannot divide std::string::String by std::string::String

As expected, Rust is complaining about our data types. Let's look again at a summary of the dataset and make some decisions about our desired data types.

In [7]:
darn::show_frame(&data, Some(&headers));
Out[7]:
Id SepalLengthCm SepalWidthCm PetalLengthCm PetalWidthCm Species
"1" "5.1" "3.5" "1.4" "0.2" "Iris-setosa"
"2" "4.9" "3.0" "1.4" "0.2" "Iris-setosa"
"3" "4.7" "3.2" "1.3" "0.2" "Iris-setosa"
"4" "4.6" "3.1" "1.5" "0.2" "Iris-setosa"
"5" "5.0" "3.6" "1.4" "0.2" "Iris-setosa"
... ... ... ... ... ...
"146" "6.7" "3.0" "5.2" "2.3" "Iris-virginica"
"147" "6.3" "2.5" "5.0" "1.9" "Iris-virginica"
"148" "6.5" "3.0" "5.2" "2.0" "Iris-virginica"
"149" "6.2" "3.4" "5.4" "2.3" "Iris-virginica"
"150" "5.9" "3.0" "5.1" "1.8" "Iris-virginica"

We can see that we have six columns, the names of which we have stored in our headers vector.

In [8]:
&headers
Out[8]:
["Id", "SepalLengthCm", "SepalWidthCm", "PetalLengthCm", "PetalWidthCm", "Species"]

Let's go through them one-by-one and decide which data type will support our desired operations.

• Id. This is an identifier that came from the original CSV file. We don't have much use for this column in our upcoming analyses, so in this case, we're going to drop this column.
• SepalLengthCm, SepalWidthCm, PetalLengthCm, and PetalWidthCm. This is the multivariate data that describes each flower sample with regards to the length and width of the sepals and petals. These are numerical values with fractional parts, so we may want to store them in a floating-point data type, e.g. f32.
• Species. This column contains the true species of the flower samples. These are categorical values, so we may wish to convert them to numerical (integer) values, e.g. u32, or keep them as strings. We'll continue with the String type for now to keep things simple.

## Moving Data to Typed Arrays¶

Once we've decided on what data types we want to employ we can move onto creating our typed arrays. This involves converting values from String to the desired type, and moving our data over to the new and typed arrays.

### The Id Column¶

We've decided that we don't need this column, so it requires no action.

### The SepalLengthCm, SepalWidthCm, PetalLengthCm, and PetalWidthCm Columns¶

We've decided that we want these columns to have a data type of f32, so we need to convert and move them into a new homogeneous array, this time of ndarray::Array2<f32>. We're going to achieve this by using std::str::FromStr which gives us access to the from_str() function that allows us to parse values from strings.

Let's demonstrate this approach on the first of the four columns, SepalLengthCm. We'll dump the column to an output cell to see the before and after.

In [9]:
data.column(1)
Out[9]:
["5.1", "4.9", "4.7", "4.6", "5.0", "5.4", "4.6", "5.0", "4.4", "4.9", "5.4", "4.8", "4.8", "4.3", "5.8", "5.7", "5.4", "5.1", "5.7", "5.1", "5.4", "5.1", "4.6", "5.1", "4.8", "5.0", "5.0", "5.2", "5.2", "4.7", "4.8", "5.4", "5.2", "5.5", "4.9", "5.0", "5.5", "4.9", "4.4", "5.1", "5.0", "4.5", "4.4", "5.0", "5.1", "4.8", "5.1", "4.6", "5.3", "5.0", "7.0", "6.4", "6.9", "5.5", "6.5", "5.7", "6.3", "4.9", "6.6", "5.2", "5.0", "5.9", "6.0", "6.1", "5.6", "6.7", "5.6", "5.8", "6.2", "5.6", "5.9", "6.1", "6.3", "6.1", "6.4", "6.6", "6.8", "6.7", "6.0", "5.7", "5.5", "5.5", "5.8", "6.0", "5.4", "6.0", "6.7", "6.3", "5.6", "5.5", "5.5", "6.1", "5.8", "5.0", "5.6", "5.7", "5.7", "6.2", "5.1", "5.7", "6.3", "5.8", "7.1", "6.3", "6.5", "7.6", "4.9", "7.3", "6.7", "7.2", "6.5", "6.4", "6.8", "5.7", "5.8", "6.4", "6.5", "7.7", "7.7", "6.0", "6.9", "5.6", "7.7", "6.3", "6.7", "7.2", "6.2", "6.1", "6.4", "7.2", "7.4", "7.9", "6.4", "6.3", "6.1", "7.7", "6.3", "6.4", "6.0", "6.9", "6.7", "6.9", "5.8", "6.8", "6.7", "6.7", "6.3", "6.5", "6.2", "5.9"], shape=[150], strides=[6], layout=Custom (0x0), const ndim=1

It's clear from the output of data.column(1) that every element is a string. Now let's use mapv() to go through every element of data.column(1), parse each value to f32, and return the new array.

In [10]:
data.column(1).mapv(|elem| f32::from_str(&elem).unwrap())
Out[10]:
[5.1, 4.9, 4.7, 4.6, 5.0, 5.4, 4.6, 5.0, 4.4, 4.9, 5.4, 4.8, 4.8, 4.3, 5.8, 5.7, 5.4, 5.1, 5.7, 5.1, 5.4, 5.1, 4.6, 5.1, 4.8, 5.0, 5.0, 5.2, 5.2, 4.7, 4.8, 5.4, 5.2, 5.5, 4.9, 5.0, 5.5, 4.9, 4.4, 5.1, 5.0, 4.5, 4.4, 5.0, 5.1, 4.8, 5.1, 4.6, 5.3, 5.0, 7.0, 6.4, 6.9, 5.5, 6.5, 5.7, 6.3, 4.9, 6.6, 5.2, 5.0, 5.9, 6.0, 6.1, 5.6, 6.7, 5.6, 5.8, 6.2, 5.6, 5.9, 6.1, 6.3, 6.1, 6.4, 6.6, 6.8, 6.7, 6.0, 5.7, 5.5, 5.5, 5.8, 6.0, 5.4, 6.0, 6.7, 6.3, 5.6, 5.5, 5.5, 6.1, 5.8, 5.0, 5.6, 5.7, 5.7, 6.2, 5.1, 5.7, 6.3, 5.8, 7.1, 6.3, 6.5, 7.6, 4.9, 7.3, 6.7, 7.2, 6.5, 6.4, 6.8, 5.7, 5.8, 6.4, 6.5, 7.7, 7.7, 6.0, 6.9, 5.6, 7.7, 6.3, 6.7, 7.2, 6.2, 6.1, 6.4, 7.2, 7.4, 7.9, 6.4, 6.3, 6.1, 7.7, 6.3, 6.4, 6.0, 6.9, 6.7, 6.9, 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 5.9], shape=[150], strides=[1], layout=CF (0x3), const ndim=1

Looking at the output of our operations we can see that we were successful in parsing our string values into numerical ones. Let's now use this approach to create a new array named data_features of type ndarray::Array2<f32>. We'll need to convert our one-dimensional arrays into two-dimensional column arrays using insert_axis(Axis(1)) as we stack them.

In [11]:
let features: Array2::<f32> =
ndarray::stack![Axis(1),
data.column(1)
.mapv(|elem| f32::from_str(&elem).unwrap())
.insert_axis(Axis(1)),
data.column(2)
.mapv(|elem| f32::from_str(&elem).unwrap())
.insert_axis(Axis(1)),
data.column(3)
.mapv(|elem| f32::from_str(&elem).unwrap())
.insert_axis(Axis(1)),
data.column(4)
.mapv(|elem| f32::from_str(&elem).unwrap())
.insert_axis(Axis(1))];

If we don't want to copy and paste the same line of code multiple times (which we don't) we can use a loop instead. First we need to create an array of integers that identify the column indices of the features that we want to convert.

In [12]:
let selected_features = [1, 2, 3, 4];

We can now iterate through the array of column indices, named selected_features, and stack each converted column into a new array of type Array2::<f32>.

In [13]:
let mut features: Array2::<f32> =  Array2::<f32>::zeros((data.shape()[0],0));

for &f in selected_features.iter() {
features = ndarray::stack![Axis(1), features,
data.column(f as usize)
.mapv(|elem| f32::from_str(&elem).unwrap())
.insert_axis(Axis(1))];
};

Our headers vector (which describes 6 columns with 6 elements) doesn't broadcast onto our new features array (4 columns), so we'll create a new headers vector, feature_headers.

In [14]:
let feature_headers = headers[1..5].to_vec();

We now have our floating-point typed features (SepalLengthCm, SepalWidthCm, PetalLengthCm, and PetalWidthCm) in a single array. Let's see how the data looks in its new form.

In [15]:
darn::show_frame(&features, Some(&feature_headers));
Out[15]:
SepalLengthCm SepalWidthCm PetalLengthCm PetalWidthCm
5.1 3.5 1.4 0.2
4.9 3.0 1.4 0.2
4.7 3.2 1.3 0.2
4.6 3.1 1.5 0.2
5.0 3.6 1.4 0.2
... ... ... ...
6.7 3.0 5.2 2.3
6.3 2.5 5.0 1.9
6.5 3.0 5.2 2.0
6.2 3.4 5.4 2.3
5.9 3.0 5.1 1.8

We're only seeing 10 samples array summary, so let's plot our features to get a better idea.

In [23]:
let layout = Layout::new()
.xaxis(Axis::new().title(Title::new("Length (cm)")))
.yaxis(Axis::new().title(Title::new("Width (cm)")));

let sepal = Scatter::new(features.column(0).to_vec(), features.column(1).to_vec())
.name("Sepal")
.mode(Mode::Markers);
let petal = Scatter::new(features.column(2).to_vec(), features.column(3).to_vec())
.name("Petal")
.mode(Mode::Markers);

let mut plot = Plot::new();

plot.set_layout(layout);

darn::show_plot(plot);

Let's also check to see that we can now calculate the mean average per column.

In [17]:
features.mean_axis(Axis(0)).unwrap()
Out[17]:
[5.8433347, 3.054, 3.7586665, 1.1986669], shape=[4], strides=[1], layout=CF (0x3), const ndim=1

It appears to be operating as expected. You could validate the results by running the same operation in different software.

### The Species Column¶

We've decided to keep these as strings. Let's index the Species column and store it in its own array named labels.

In [18]:
let labels: Array1::<String> = data.column(5).to_owned();

Finally, we'll check the first 10 elements of our new labels array.

In [19]:
labels.slice(s![0..10])
Out[19]:
["Iris-setosa", "Iris-setosa", "Iris-setosa", "Iris-setosa", "Iris-setosa", "Iris-setosa", "Iris-setosa", "Iris-setosa", "Iris-setosa", "Iris-setosa"], shape=[10], strides=[1], layout=CF (0x3), const ndim=1

We could also use the itertools crate to apply some better presentation to this output.

In [20]:
:dep itertools = {version = "0.9.0"}
extern crate itertools;
use itertools::Itertools;

labels.slice(s![0..10]).iter().format("\n")
Out[20]:
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"
"Iris-setosa"

## Conclusion¶

In this section, we've demonstrated how to get parts of our raw string array into multiple arrays of various type. With this approach, we can now start operating on our data using appropriate operators for our analyses.

Support this work

You can access this notebook and more by getting the e-book on Data Analysis with Rust Notebooks.


text:
# Given the following, how do you calculate the mole fractions of water and alcohol in commercial vodka?

## Commercial vodka is typically sold at a concentration of 40.0% alcohol by volume (ABV), which is equivalent to 31.2% by mass ethanol($C {H}_{3} C {H}_{2} O H$) in water. The vapor pressure of pure water and ethanol at 20 °C are 17.5 torr and 44.6 torr respectively.

Jun 13, 2017

Well, let's take a $100 \cdot m L$ volume of vodka..........

$\chi \text{EtOH} = 0.151$; $\chi \text{H"_2"O} = 0.849$.

#### Explanation:

And this represents a $31.2 \cdot g$ mass of $E t O H$, and a $68.8 \cdot g$ mass of water......

And the $\text{mole fraction}$ ${\chi}_{\text{EtOH"="Moles of EtOH"/"Moles of EtOH+moles of water}}$

$\text{Moles of EtOH} = \frac{31.2 \cdot g}{46.07 \cdot g \cdot m o {l}^{-} 1} = 0.677 \cdot m o l$.

$\text{Moles of water} = \frac{68.8 \cdot g}{18.01 \cdot g \cdot m o {l}^{-} 1} = 3.82 \cdot m o l$.

$\chi \text{EtOH} = \frac{0.677 \cdot m o l}{0.677 \cdot m o l + 3.82 \cdot m o l} = 0.151$

Now of course, in a binary solution, ${\chi}_{\text{other component}} = 1 - 0.151$, but we might as well go thru the motions......

$\chi \text{H"_2"O} = \frac{3.82 \cdot m o l}{0.677 \cdot m o l + 3.82 \cdot m o l} = 0.849$.

And $\chi \text{EtOH"+chi"H"_2} O = 1$ as required.......

The vapour pressure of each component in solution will be proportional to their mole fractions.......and the vapour pressure of the solution will be this sum.........

${P}_{\text{EtOH"=0.151xx44.6*"Torr"=6.7*"Torr}}$

${P}_{\text{water"=0.849xx17.5*"Torr"=14.9*"Torr}}$

${P}_{\text{solution"=P_"EtOH"+P_"water"=(14.9+6.7)*"Torr}}$

$= 21.6 \cdot \text{Torr}$

As is typical of the vapour of such solutions; by comparison to the vapour pressures of the pure solvents, the vapour is ENRICHED with respect to the MORE VOLATILE component, which here is ethanol.


text:
Package website: release | development

The optmatch package implements the optimal full matching algorithm for bipartite matching problems. Given a matrix describing the distances between two groups (where one group is represented by row entries, and the other by column entries), the algorithm finds a matching between units that minimizes the average within grouped distances. This algorithm is a popular choice for covariate balancing applications (e.g. propensity score matching), but it also can be useful for design stage applications such as blocking. For more on the application and its implementation, see:

Hansen, B.B. and Klopfer, S.O. (2006) Optimal full matching and
related designs via network flows, JCGS 15 609-627.

optmatch is available on CRAN:

> install.packages("optmatch")
> library("optmatch")

## Choice of solvers

There are two different packages implementing the actual solver which can be used.

• The default, starting in 0.10.0, is the LEMON graph library’s Min Cost Flow solver, implemented in the rlemon package.
• In previous versions, the default was the RELAX-IV solver, which is now implemented in the rrelaxiv package.

Users wishing to utilize the RELAX-IV solver must install rrelaxiv separately, see that page for details. Once installed, RELAX-IV becomes the default solver.

The LEMON solver has four separate algorithms implemented, Cycle Cancelling (the default), Network Simplex, Cost Scaling, and Capacity Scaling. Each has its own trade-offs and performance quirks. See help(fullmatch) for details of how to choose which is being used.

## Using optmatch

In addition to the optimal full matching algorithm, the package contains useful functions for generating distance specifications, combining and editing distance specifications, and summarizing and displaying matches. This walk through shows how to use these tools in your matching workflow.

### Simulated data

Before we start, let’s generate some simulated data. We will have two groups, the “treated” and “control” groups. Without our knowledge, nature assigned units from a pool into one of these two groups. The probability of being a treated unit depends on some covariates. In the vector Z, let a 1 denote treated units and 0 denote control units

set.seed(20120111) # set this to get the exact same answers as I do
n <- 26 # chosen so we can divide the alphabet in half
W <- data.frame(w1 = rbeta(n, 4, 2), w2 = rbinom(n, 1, p = .33))

# nature assigns to treatment
tmp <- numeric(n)
tmp[sample(1:n, prob = W$w1^(1 + W$w2), size = n/2)] <- 1
W$z <- tmp # for convenience, let's give the treated units capital letter names tmp <- character(n) tmp[W$z == 1] <- LETTERS[1:(n/2)]
tmp[W$z == 0] <- letters[(26 - n/2 + 1):26] rownames(W) <- tmp As we can see with a simple table and plot, these groups are not balanced on the covariates, as they would be (in expectation) with a randomly assigned treatment. table(W$w2, W$z) library(lattice) ; densityplot(W$w1, groups = W$z) The next steps use the covariates to pair up similar treated and control units. For more on assessing the amount and severity of imbalance between groups on observed covariates, see the RItools R package. ### Setting up distances These two groups are different, but how different are individual treated units from individual control units? In answering this question, we will produce several distance specifications: matrices of treated units (rows) by control units (columns) with entries denoting distances. optmatch provides several ways of generating these matrices so that you don’t have to do it by hand. Let’s begin with a simple Euclidean distance on the space defined by W: distances <- list() distances$euclid <- match_on(z ~ w1 + w2, data = W, method = "euclidean")

The method argument tells the match_on function how to compute the distances over the space defined by the formula. The default method extends the simple Euclidean distance by rescaling the distances by the covariance of the variables, the Mahalanobis distance:

distances$mahal <- match_on(z ~ w1 + w2, data = W) You can write additional distance computation functions. See the documentation for match_on for more details on how to create these functions. To create distances, we could also try regressing the treatment indicator on the covariates and computing the difference distance for each treated and control pair. To make this process easier, match_on has methods for glm objects (and for big data problems, bigglm objects): propensity.model <- glm(z ~ w1 + w2, data = W, family = binomial()) distances$propensity <- match_on(propensity.model)

The glm method is a wrapper around the numeric method for match_on. The numeric method takes a vector of scores (for example, the linear prediction for each unit from the model) and a vector indicating treatment status (z) for each unit. This method returns the absolute difference between each treated and control pair on their scores (additionally, the glm method rescales the data before invoking the numeric method). If you wish to fit a “caliper” to your distance matrix, a hard limit on allowed distances between treated and control units, you can pass a caliper argument, a scalar numeric value. Any treated and control pair that is larger than the caliper value will be replaced by Inf, an unmatchable value. The caliper argument also applies to glm method. Calipers are covered in more detail in the next section.

The final convenience method of match_on is using an arbitrary function. This function is probably most useful for advanced users of optmatch. See the documentation of the match_on function for more details on how to write your own arbitrary computation functions.

### Combining and editing distances

We have created several representations of the matching problem, using Euclidean distance, Mahalanobis distance, the estimated propensity score, and an arbitrary function. We can combine these distances into single metric using standard arithmetic functions:

distances$all <- with(distances, euclid + mahal + propensity) You may find it convenient to work in smaller pieces at first and then stitch the results together into a bigger distance. The rbind and cbind functions let us add additional treated and control entries to a distance specification for each of the existing control and treated units, respectively. For example, we might want to combine a Mahalanobis score for units n through s with a propensity score for units t through z: W.n.to.s <- W[c(LETTERS[1:13], letters[14:19]),] W.t.to.z <- W[c(LETTERS[1:13], letters[20:26]),] mahal.n.to.s <- match_on(z ~ w1 + w2, data = W.n.to.s) ps.t.to.z <- match_on(glm(z ~ w1 + w2, data = W.t.to.z, family = binomial())) distances$combined <- cbind(mahal.n.to.s, ps.t.to.z)

The exactMatch function creates “stratified” matching problems, in which there are subgroups that are completely separate. Such matching problems are often much easier to solve than problems where a treated unit could be connected to any control unit.

There is another method for creating reduced matching problems. The caliper function compares each entry in an existing distance specification and disallows any that are larger than a specified value. For example, we can trim our previous combined distance to anything smaller than the median value:

distances$median.caliper <- caliper(distances$all, median(distances$all)) distances$all.trimmed <- with(distances, all + median.caliper)

Like the exactMatch function, the results of caliper used the sparse matrix representation mentioned above, so can be very efficient for large, sparse problems. As noted previously, if using the glm or numeric methods of match_on, passing the caliper’s width in the caliper argument can be more efficient.

### Speeding up computation

In addition to the space advantages of only storing the finite entries in a sparse matrix, the results of exactMatch and caliper can be used to speed up computation of new distances. The match_on function that we saw earlier has an argument called within that helps filter the resulting computation to only the finite entries in the within matrix. Since exactMatch and caliper use finite entries denote valid pairs, they make excellent sources of the within argument.

Instead of creating the entire Euclidean distance matrix and then filtering out cross-strata matches, we use the results of exactMatch to compute only the interesting cases:

tmp <- exactMatch(z ~ w2, data = W)
distances$exact <- match_on(z ~ w1, data = W, within = tmp) Users of previous versions of optmatch may notice that the within argument is similar to the old structure.formula argument. Like within, structure.formula focused distance on within strata pairs. Unlike structure.formula, the within argument allows using any distance specification as an argument, including those created with caliper. For example, here is the Mahalanobis distance computed only for units that differ by less than one on the propensity score. distances$mahal.trimmed <- match_on(z ~ w1 + w2, data = W,
within = match_on(propensity.model, caliper = 1))

### Generating the match

Now that we have generated several distances specifications, let’s put them to use. Here is the simplest way to evaluate all distances specifications:

matches <- lapply(distances, function(x) { fullmatch(x, data = W) })

The result of the matching process is a named factor, where the names correspond to the units (both treated and control) and the levels of the factors are the matched groups. Including the data argument is highly recommended. This argument will make sure that the result of fullmatch will be in the same order as the original data.frame that was used to build the distance specification. This will make appending the results of fullmatch on to the original data.frame much more convenient.

The fullmatch function as several arguments for fine tuning the allowed ratio of treatment to control units in a match, and how much of the pool to throw away as unmatchable. One common pattern for these arguments are pairs: one treated to one control unit. Not every distance specification is amendable to this pattern (e.g. when there are more treated units than control units in exactMatch created stratum). However, it can be done with the Mahalanobis distance matrix we created earlier:

mahal.match <- pairmatch(distances\$mahal, data = W)

Like fullmatch, pairmatch also allows fine tuning the ratio of matches to allow larger groupings. It is can be helpful as it computes what percentage of the group to throw away, giving better odds of successfully finding a matching solution.

Once one has generated a match, you may wish to view the results. The results of calls to fullmatch or pairmatch produce optmatch objects (specialized factors). This object has a special option to the print method which groups the units by factor level:

print(mahal.match, grouped = T)

If you wish to join the match factor back to the original data.frame:

W.matched <- cbind(W, matches = mahal.match)

Make sure to include the data argument to fullmatch or pairmatch, otherwise results are not guaranteed to be in the same order as your original data.frame or matrix.

## Using a development version of Optmatch

This section will help you get the latest development version of optmatch and start using the latest features. Before starting, you should know which branch you wish to install. Currently, the “master” branch is the main code base. Additional features are added in their own branches. A list of branches is available at (the optmatch project page)[https://github.com/markmfredrickson/optmatch].

### Installing a development version

You may need additional compilers as distributed by CRAN: OS X, Windows.

We recommend using dev_mode from the devtools package to install in-development version of the package so that you can keep the current CRAN version as the primary package. Activating dev_mode creates a secondary library of packages which can only be accessed while in dev_mode. Packages normally installed can still be used, but if different versions are installed normally and in dev_mode, the dev_mode version takes precedent if in dev_mode.

Install and load the devtools package:

> install.packages("devtools")
> library("devtools")

Activate dev_mode:

> dev_mode()
d>

Note that the prompt changes from > to d> to let you know you’re in dev_mode. Now choose the development branch you want to use. To install master:

d> install_github("markmfredrickson/optmatch")

Either way, the package is then loaded in the usual fashion, provided you’re still in dev_mode:

 d> library(optmatch)

Once you’ve done this you can disable dev_mode as follows

d> dev_mode()
>

The development version of the package remains loaded.

Note that if you load the package – ie, enter library(optmatch) (when the package hasn’t already been loaded otherwise) – while not in dev_mode, then you’ll get whatever version of the package may be installed in your library tree, not this development version.

If you want to switch between versions of optmatch, we suggest re-starting R.

## Developing for optmatch

You may use RStudio to develop for optmatch, by opening the optmatch.Rproj file. We suggest you ensure all required dependencies are installed by running

devtools::install_deps(dependencies = TRUE)

We prefer changes that include unit tests demonstrating the problem or showing how the new feature should be added. The test suite uses the testthat package to write and run tests. (Please ensure you have the latest version of testthat (or at least v0.11.0), as older versions stored the tests in a different directory, and may not test properly.) See the tests/testthat directory for examples. You can run the test suite via Build -> Test Package.

New features should include inline Roxygen documentation. You can generate all .Rd documents from the Roxygen code using Build -> Document.

Finally, you can use Build -> Build and Reload or Build -> Clean and Rebuild to load an updated version of optmatch in your current RStudio session. Alternatively, to install the developed version permanently, use Build -> Build Binary Version, followed by

install.packages("../optmatch_VERSION.tgz", repo=NULL)

You can revert back to the current CRAN version by

remove.packages("optmatch")
install.packages("optmatch")

Note: If you are building for release on CRAN, you need to ensure vignettes are compacted. This should be enabled automatically in the .Rproj file, but if not see this stackoverflow answer for some concerns about dealing with this with RStudio.

If you prefer not to use RStudio, you can develop using Make.

• make test: Run the full test suite.
• make document: Update all documentation from Roxygen inline comments.
• make interactive: Start up an interactive session with optmatch loaded. (make interactive-emacs will start the session inside emacs.)
• make check: Run R CMD check on the package
• make build: Build a binary package.
• make vignette: Builds any vignettes in vignettes/ directory
• make clean: Removes files built by make vignette, make document or make check. Should not be generally necessary, but can be useful for debugging.
• make release: Starts an interactive R session to submit a release to CRAN.


text:
# Easy lambdas

R already has anonymous functions. Just write function(x) do_someting and you’re done. This post is for all of us who are not satisified with this solution. For one, it’s exhausting to write to long function(x) instead of, say, L(x), but that’s not all of it! Take the following example, a calculation of the maximum likelihood estimate of a gamma distribution: set.seed(313) x = rgamma(100, 2, 4) nlm(f = function(p) -mean(dgamma(x, shape = p[1], rate = p[2], log = TRUE)), p = c(1, 1))\$estimate ## [1] 2.

# Handling side effects with the .H function

The Problem of Side Effects When I am writing $$\mathtt{R}$$ code, I often do stuff in the body of my script that creates undesired side effects. # n and x constants I wish to use later n = 100 x = pi^2/6 # lots of code # ... # I suddenly wish to plot something n = 1:1000 x = 0.1 plot(n, pnorm(sqrt(n)*x), type = "l") Notice that n and x have been rewritten.

#### Jonas Moss

Meta-contrarian statistician who wants to save science.

Ph.d. student in statistics, Universitas Osloensis

Oslo in Norway


text:
# Analysis of the $\frac {3}{2}^{\pm }$ Pentaquark States in the Diquark Model with QCD Sum Rules

Zhang, Jun-Xia (Department of Physics, North China Electric Power University, Baoding 071003, P.R. China) ; Wang, Zhi-Gang (Department of Physics, North China Electric Power University, Baoding 071003, P.R. China) ; Di, Zun-Yan (Department of Physics, North China Electric Power University, Baoding 071003, P.R. China)

07 December 2017

Abstract: In this article, we construct the scalar--diquark--axialvector--diquark--antiquark-type interpolating currents, and study the masses and pole res-idues of the $J^{P}=\frac{3}{2}^{\pm}$ hidden-charmed pentaquark states with the QCD sum rules. In calculations, we use the formula $\mu=\sqrt{M^2_{P}-(2{\mathbb{M}}_c)^2}$ to determine the energy scales of the QCD spectral densities. We obtain the masses of the hidden-charm pentaquark states with the strangeness $S=-1$ and $S=-2$, which can be confronted with the experimental data in the future.

Published in: Acta Physica Polonica B 48 (2017) 2013-2030


text:
# Gordon decomposition an right current

1. Sep 21, 2007

### OOO

Gordon decomposition an "right" current

I've got a question about the Dirac equation and the Gordon decomposition of the dirac current. The Gordon decomposition is

$$\bar \psi \gamma^\mu \psi = - \frac{1}{m}\Im (\bar \psi \partial^\mu \psi) - \frac{1}{2m}\partial_\mu ( \bar \psi \sigma^{\mu\nu} \psi)$$

where h=c=1. As is well known the divergence of the left hand side vanishes if we simply use the dirac equation and its adjoint. But the divergence of the second term on the right hand side (the so called spin current) vanishes as well because it is antisymmetric in $$(\mu,\nu)$$. As a consequence the first current on the rhs (the so called convective) must also be conserved.

So in total we have three conserved currents, namely

$$j^\mu = \bar \psi \gamma^\mu \psi$$

$$j^\mu_{conv} = - \frac{1}{m}\Im (\bar \psi \partial^\mu \psi)$$

$$j^\mu_{spin} = -\frac{1}{2m} \partial_\mu (\bar \psi \sigma^{\mu\nu} \psi)$$

Firstly: that seems a bit much to me. Especially considering the fact that charge conservation results from gauge invariance I would expect that there is exactly one conserved current as there is exactly one gauge generator for electromagnetism.

Secondly: how do I know that it is really the first (i.e. dirac) current which enters the Maxwell equations as its sources ? Of course we could look at the well known Lagrangian of Dirac+Maxwell and say "oh it works, why bother". But couldn't we assume that it is actually the convective current or the spin current or even an unspecified linear combination of the three which acts as the source of the electromagnetic field ?

2. Sep 21, 2007

### reilly

The Gordon Decomposition(GD) splits the canonical current into that due to a moving point charge -- like e v, e charge, v speed -- and due to a point magnetic moment -- the sigma mu nu term --.Why is the gamma mu term the right one? -- assume no anomalous moments. History, NR limit, what else could it be? gauge invariance all are important, but Dirac's genius is the main reason gamma mu is the right current for Maxwell....

A sure fire way to see what's right and what's not is: compute the electron scattering crosssection from a fixed positive charge with an arbitrary mixture of convective and magnetic currents. Check against some data -- switch to proton and alpha particles a la Rutherford. Back scattering is the big deal -- how is back scattering related to the convective/magnetic ratio? Also think about the anomalous magnetic moment of, say, the electron --what's with anomalous.
Good questions

Regards,
Reilly Atkinson

3. Sep 21, 2007

### OOO

Thank you Reilly,

I'm not sure that I have understood what you are trying to tell me. It sounds a bit shaky to me that you quote Dirac's genius as a reason. Of course one needs genius to find something new that also works. But 80 years later I suppose there should be some sound explanation "why" it worked so well. The phrase "no anomalous moments" urges one to think like: ah, okay, everything is normal - so I will believe it too.

Checking against the data is certainly what we should always do, yes. But I am asking myself if there is some deeper reason why the ratio between both currents is precisely 1 for the electron. Maybe it's just that I haven't grasped what you were saying about back scattering and so on.

4. Sep 21, 2007

### nrqed

I am not sure but at first sight it seems to me that maybe parity is a key issue. If you want parity to be a valid symmetry, the currents must be combined in just the right way.
But I may be wrong, I haven't looked at this in a very long time.

5. Sep 21, 2007

### Hans de Vries

There are two conserved currents, so the sum is also conserved.....

Both are sources of the electromagnetic field. Long distance fields are dominated by
the first because the charge related fields fall of by 1/r^2 while the spin related fields
fall of by 1/r^3.

In QED the spin current plays the most important role. The spin current, which is
the curl of the spin density, also causes, according to Stokes' law, effective global
electric currents. This is just the same as what happens in magnetic materials:

If the spin density stays equal then the "little circular currents" cancel each other,
but there is an effective global current circling at the edge of the magnetic material
where there is a gradient. The same happens with the electron's wave-function.

See for instance: Sakurai, chapter 3.5: "Gordon decomposition of the vector current"

Regards, Hans

6. Sep 21, 2007

### OOO

...and so does any linear combination...

Thank you for pointing this out to me. Indeed the spin current looks like the kind of trick one does when deriving the Maxwell equations in matter. I haven't seen it from this angle yet. But it sounds perfectly natural.

But then I am wondering if there is probably some way to reexpress the Dirac equation by absorbing magnetization and polarization into the electromagnetic field tensor in the usual way, i.e.: F+S=G where F=(E,B); S=(P,M); G=(D,H). Does this make sense to you ?

I will take a look at Sakurai. I haven't touched this book many times because it is mainly about nonrelativistic QM, at least as far as I remember.

7. Sep 21, 2007

### OOO

Now I have looked it up. But I'm afraid I haven't found anything about the Gordon decomposition in Sakurai 3.5. It is just the usual angular momentum algebra. We are talking about the chapter named "Eigenvalues and Eigenstates of Angular Momentum", right ?

Edit: Sorry, I see, there is another Book by Sakurai. I didn't know. So you are referring to "Advanced Quantum Mechanics". This will probably take me some time to get a copy of.

Last edited: Sep 21, 2007
8. Sep 22, 2007

### Hans de Vries

I was indeed referring to "Advanced Quantum Mechanics". This is a classic. There should
be books like these which handle the Dirac equation in the modern chiral representation.
The chiral form is assumed to be a better representation of the underlying physics since
the successes of the electroweak theory. It also maps perfectly to the Poincaré group of
rotations, boosts and translations.

Regards, Hans

9. Sep 24, 2007

### OOO

I have taken a look on Sakurai's second volume and I can see a bit more clearly now that the Dirac current is enforced by gyromagnetic ratio of the electron. I would have been happier if I had found some statement about "why" the gyromagnetic ratio is 2 (neglecting the higher order terms) for the electron. But maybe I have to swallow that as an experimental fact.

As to my first question, I'll repeat it once more: how can it be that there is only a gauge group with one generator, i.e. U(1), but there are three separately conserved currents: dirac current, convective current, spin current ?

10. Sep 24, 2007

### Demystifier

The Noether current is derived from the symmetry of the action, given the equations of motion. It is unique. The conservation of other currents is a pure consequence of the equation of motion, so they are not unique.

BTW, for some additional currents see also Eq. (11) in
http://xxx.lanl.gov/abs/hep-th/0702060

11. Sep 24, 2007

### OOO

Thank you for the answer, Demystifier
I don't understand this. If the equations of motion (the Dirac equation) are unique, why should a current that is a consequence of these equations not be unique ?

12. Sep 24, 2007

### OOO

For those who are interested: I have found a paper that deals with exactly these matters

http://arxiv.org/pdf/quant-ph/0305175

Unfortunately the author admits coming to no definite conclusion as well. At least he is able to show that there is an infinite number of conserved currents, each being constructed by yet another application of the Dirac operator.

Demystifier, I think I understand now what you meant by saying "not unique as a consequence of the equation of motion". From the above paper it looks like this could also be rephrased in terms of the non-uniqueness of the Lagrangian (I guess similar to the case of the energy-momentum tensor of the gauge field, which is also not unique).

13. Sep 24, 2007

### Hans de Vries

Well, the probability density of the Dirac equation can equally well be interpreted as charge density AND as a spin density. These three have always the same ratio. There are no area's in where there is more charge and less spin for example. So, I think you can say there is only one conserved current with multiple consequences.

Regards, Hans

14. Sep 24, 2007

### OOO

You mean that $$\psi^\dagger \psi$$ can be interpreted as a spin density ? Where can I read about this ? I'm not that much an attentive reader of textbooks, but I remember Ryder (and I think also Bjorken-Drell) talking about the difficulties with several definitions of "spin". So what kind of spin density is $$\psi^\dagger \psi$$ ?

I assume you mean ratio in the sense of an unspecified ratio resulting from arbitrary normalization, not a fixed definite ratio, don't you ?

15. Sep 24, 2007

### Hans de Vries

That would be $$\bar{\psi}\psi$$ or rather $$\bar{\psi}\gamma^0\psi$$. They are the same in the particle's restframe and otherwise relate as m and E. There is a special operation which returns you the spin density by its spin components.

If some area represents 1% of the probability then it also represents 1% of the charge and 1% of the spin/magnetic moment. For molecular modeling you need to assign such a charge and magnetic moment to the area (and all other area) to compute the fields which are seen by other electrons and the protons in the nuclei.

Regards, Hans

Last edited: Sep 24, 2007
16. Sep 24, 2007

### OOO

I'm afraid I haven't understood where this comes from. But anyway, some things are clearer for me now.

Thanks a lot for all your answers.

Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook


text:
# Revolving Door (Middle Levels) Conjecture (1970s)

Originator(s): I. Havel

Definitions: With [n]={1,...,n}, let B(n,k) denote the graph whose vertices are the k-sets and (n-k)-sets in [n], with vertices adjacent when one contains the other.

Conjecture: There is a cycle through the k-sets and (k+1)-sets in [2k+1] by alternately adding and deleting one element. More generally, Havel [Ha] conjectured that B(n,k) is Hamiltonian whenever 1 < k < n/2.

Background/motivation: The special case k=(n-1)/2 was popular at the Banff conferences in 1981 and 1984 and was misattributed to many people. It was widely known as the "Erdos Revolving Door Conjecture".

Comments/Partial results: The special case k=(n-1)/2 has been verified for k < 15. Let f(k) denote the fraction of the vertices in the longest cycle in B(2k+1,k); the conjecture states that f(k)=1. Dejter and Quintana [DQ1] proved that if f(k)=1, then f(k+1) > 3/4. Felsner and Trotter [FT] proved that f(k)>1/4 for all k. Savage and Winkler [SW] improved this to f(k) > .839. Shields and Savage [SS] proved that f(k) > .86. Johnson [J] proved that f(k)=1-o(1).

Another approach seeks cycles of restricted forms by using an auxiliary quotient graph on cyclic classes of binary (2k+1)-tuples; Dejter [D] showed that spanning paths in this graph lift to spanning cycles in B(2k+1,k). Also, a spanning cycle in B(2k+1,k) must consist of two 1-factors in the graph of possible edges; examples of 1-factors in this graph are discussed in [DKS, DSW, KT].

For the general conjecture, Hurlbert [Hu] proved that B(n,k) is Hamiltonian whenever n > ck2+k if k is large enough. Other extensions of the middle-levels problem are discussed by Dejter and Quintana [DQ2].

References:
[D] I.J. Dejter, Hamilton cycles and quotients of bipartite graphs. Graph theory with applications to algorithms and computer science (Kalamazoo, Mich., 1984), 189--199, Wiley-Intersci. Publ., Wiley, New York, 1985.
[DCQ] I.J. Dejter, J. Cordova, J.A. Quintana, Two Hamilton cycles in bipartite reflective Kneser graphs. Proceedings of the First Japan Conference on Graph Theory and Applications (Hakone, 1986). Discrete Math. 72 (1988), no. 1-3, 63--70.
[DQ1] I.J. Dejter, J. Quintana, Long cycles in revolving door graphs. Eighteenth Southeastern International Conference on Combinatorics, Graph Theory, and Computing (Boca Raton, Fla., 1987). Congr. Numer. 60 (1987), 163--168.
[DQ2] I.J. Dejter, J. Quintana, On an extension of a conjecture of I. Ha'vel. Graph theory, combinatorics, and applications, Vol. 1 (Kalamazoo, MI, 1988), 327--341, Wiley-Intersci. Publ., Wiley, New York, 1991.
[DKS] D.A. Duffus, H.A. Kierstead, H.S. Snevily, An explicit $1$-factorization in the middle of the Boolean lattice. J. Combin. Theory Ser. A 65 (1994), no. 2, 334--342.
[DSW] D.A. Duffus, B. Sands, R. Woodrow, Lexicographic matchings cannot form Hamiltonian cycles. Order 5 (1988), no. 2, 149--161.
[FT] S. Felsner, W.T. Trotter, Colorings of diagrams of interval orders and \alpha-sequences of sets, Discrete Math. 144 (1995) 23--31.
[Ha] I. Havel, Semipaths in directed cubes, in: M. Fiedler (Ed.), Graphs and other Combinatorial Topics, Teubner-Texte Math., Teubner, Leipzig, 1983, pp. 101--108.
[Hu] G. Hurlbert, The antipodal layers problem. Discrete Math. 128 (1994), no. 1-3, 237--245.
[J] J.R. Johnson, Long cycles in the middle two layers of the discrete cube. J. Combin. Theory Ser. A 105 (2004), no. 2, 255--271.
[KT] H.A. Kierstead, W.T. Trotter, Explicit matchings in the middle levels of the Boolean lattice. Order 5 (1988), no. 2, 163--171.
[SS] C.D. Savage, I. Shields, A Hamilton path heuristic with applications to the middle two levels problem, Congr. Numer. 140 (1999) 161--178.
[SW] C.D. Savage, P. Winkler, Monotone Gray codes and the middle levels problem, J. Combin. Theory Ser. A 70 (1995) 230--248.

#### Index Page; Glossary

Posted 2005?; Last update 8/31/07


text:
Difference between revisions of "Perron-Stieltjes integral"

A generalization of the Perron integral for a function of one real variable. A finite function $f$ is said to be integrable in the sense of Perron–Stieltjes with respect to a finite function $G$ on $[a,b]$ if on $[a,b]$ there exists a major function $M$ and a minor function $m$ for $f$ with respect to $G$ on $[a,b]$ having $M(a)=m(a)=0$ and such that at each point $x\in[a,b]$,

$$M(x+\beta)-M(x-\alpha)\geq f(x)(G(x+\beta)-G(x-\alpha))$$

$$n(x+\beta)-m(x-\alpha)\leq f(x)(G(x+\beta)-G(x-\alpha))$$

for all sufficiently small $\alpha\geq0$ and $\beta\geq0$, while the greatest lower bound of the numbers $M(b)$, where $M$ is any such major function of $f$ with respect to $G$, and the least upper bound of the numbers $m(b)$, where $m$ is any such minor function of $f$ with respect to $G$, coincide. Their common value is called the Perron–Stieltjes integral of $f$ with respect to $G$ on $[a,b]$ and is denoted by

$$(P-S)\int\limits_a^bf(x)dG(x).$$

This generalization of the Perron integral was introduced by A.J. Ward [1].

References

 [1] A.J. Ward, "The Perron–Stieltjes integral" Math. Z. , 41 (1936) pp. 578–604 [2] S. Saks, "Theory of the integral" , Hafner (1952) (Translated from French) [3] I.A. Vinogradova, V.A. Skvortsov, "Generalized integrals and Fourier series" Itogi Nauk. Mat. Anal. 1970 (1971) pp. 65–107 (In Russian)

A major function $U$ of a function $f$ on $[a,b]$ with respect to a function $G$ on $[a,b]$ is a function $U$ such that for each $x\in[a,b]$ there is an $\epsilon>0$ such that $U(d)-U(c)\geq f(x)(G(d)-G(c))$ for all $c\leq x\leq d$ with $|d-c|<\epsilon$. A minor function is defined similarly, but with the inequality sign reversed. Thus, a suitable lower derivative of $U$ with respect to $G$ majorizes $f$. More generally one considers additive interval functions $U$ and $G$ satisfying the above property, cf. [2] for details. If $G$ is an ordinary function on $[a,b]$, then its associated additive interval function, denoted by the same letter, is $G([c,d])=G(d)-G(c)$. A major function of $f$, without any specification of a $G$, is one with respect to the identify function $x\mapsto x$, $x\in[a,b]$.


text:
# tokenize — Tokenizer for Python source¶

Source code: Lib/tokenize.py

The tokenize module provides a lexical scanner for Python source code, implemented in Python. The scanner in this module returns comments as tokens as well, making it useful for implementing “pretty-printers,” including colorizers for on-screen displays.

To simplify token stream handling, all operator and delimiter tokens and Ellipsis are returned using the generic OP token type. The exact type can be determined by checking the exact_type property on the named tuple returned from tokenize.tokenize().

## Tokenizing Input¶

The primary entry point is a generator:

tokenize.tokenize(readline)

The tokenize() generator requires one argument, readline, which must be a callable object which provides the same interface as the io.IOBase.readline() method of file objects. Each call to the function should return one line of input as bytes.

The generator produces 5-tuples with these members: the token type; the token string; a 2-tuple (srow, scol) of ints specifying the row and column where the token begins in the source; a 2-tuple (erow, ecol) of ints specifying the row and column where the token ends in the source; and the line on which the token was found. The line passed (the last tuple item) is the logical line; continuation lines are included. The 5 tuple is returned as a named tuple with the field names: type string start end line.

The returned named tuple has an additional property named exact_type that contains the exact operator type for OP tokens. For all other token types exact_type equals the named tuple type field.

Changed in version 3.1: Added support for named tuples.

Changed in version 3.3: Added support for exact_type.

tokenize() determines the source encoding of the file by looking for a UTF-8 BOM or encoding cookie, according to PEP 263.

All constants from the token module are also exported from tokenize.

Another function is provided to reverse the tokenization process. This is useful for creating tools that tokenize a script, modify the token stream, and write back the modified script.

tokenize.untokenize(iterable)

Converts tokens back into Python source code. The iterable must return sequences with at least two elements, the token type and the token string. Any additional sequence elements are ignored.

The reconstructed script is returned as a single string. The result is guaranteed to tokenize back to match the input so that the conversion is lossless and round-trips are assured. The guarantee applies only to the token type and token string as the spacing between tokens (column positions) may change.

It returns bytes, encoded using the ENCODING token, which is the first token sequence output by tokenize().

tokenize() needs to detect the encoding of source files it tokenizes. The function it uses to do this is available:

tokenize.detect_encoding(readline)

The detect_encoding() function is used to detect the encoding that should be used to decode a Python source file. It requires one argument, readline, in the same way as the tokenize() generator.

It will call readline a maximum of twice, and return the encoding used (as a string) and a list of any lines (not decoded from bytes) it has read in.

It detects the encoding from the presence of a UTF-8 BOM or an encoding cookie as specified in PEP 263. If both a BOM and a cookie are present, but disagree, a SyntaxError will be raised. Note that if the BOM is found, 'utf-8-sig' will be returned as an encoding.

If no encoding is specified, then the default of 'utf-8' will be returned.

Use open() to open Python source files: it uses detect_encoding() to detect the file encoding.

tokenize.open(filename)

Open a file in read only mode using the encoding detected by detect_encoding().

New in version 3.2.

exception tokenize.TokenError

Raised when either a docstring or expression that may be split over several lines is not completed anywhere in the file, for example:

"""Beginning of
docstring

or:

[1,
2,
3

Note that unclosed single-quoted strings do not cause an error to be raised. They are tokenized as ERRORTOKEN, followed by the tokenization of their contents.

## Command-Line Usage¶

New in version 3.3.

The tokenize module can be executed as a script from the command line. It is as simple as:

python -m tokenize [-e] [filename.py]

The following options are accepted:

-h, --help

show this help message and exit

-e, --exact

display token names using the exact type

If filename.py is specified its contents are tokenized to stdout. Otherwise, tokenization is performed on stdin.

## Examples¶

Example of a script rewriter that transforms float literals into Decimal objects:

from tokenize import tokenize, untokenize, NUMBER, STRING, NAME, OP
from io import BytesIO

def decistmt(s):
"""Substitute Decimals for floats in a string of statements.

>>> from decimal import Decimal
>>> s = 'print(+21.3e-5*-.1234/81.7)'
>>> decistmt(s)
"print (+Decimal ('21.3e-5')*-Decimal ('.1234')/Decimal ('81.7'))"

The format of the exponent is inherited from the platform C library.
Known cases are "e-007" (Windows) and "e-07" (not Windows).  Since
we're only showing 12 digits, and the 13th isn't close to 5, the
rest of the output should be platform-independent.

>>> exec(s)  #doctest: +ELLIPSIS
-3.21716034272e-0...7

Output from calculations with Decimal should be identical across all
platforms.

>>> exec(decistmt(s))
-3.217160342717258261933904529E-7
"""
result = []
g = tokenize(BytesIO(s.encode('utf-8')).readline)  # tokenize the string
for toknum, tokval, _, _, _ in g:
if toknum == NUMBER and '.' in tokval:  # replace NUMBER tokens
result.extend([
(NAME, 'Decimal'),
(OP, '('),
(STRING, repr(tokval)),
(OP, ')')
])
else:
result.append((toknum, tokval))
return untokenize(result).decode('utf-8')

Example of tokenizing from the command line. The script:

def say_hello():
print("Hello, World!")

say_hello()

will be tokenized to the following output where the first column is the range of the line/column coordinates where the token is found, the second column is the name of the token, and the final column is the value of the token (if any)

$python -m tokenize hello.py 0,0-0,0: ENCODING 'utf-8' 1,0-1,3: NAME 'def' 1,4-1,13: NAME 'say_hello' 1,13-1,14: OP '(' 1,14-1,15: OP ')' 1,15-1,16: OP ':' 1,16-1,17: NEWLINE '\n' 2,0-2,4: INDENT ' ' 2,4-2,9: NAME 'print' 2,9-2,10: OP '(' 2,10-2,25: STRING '"Hello, World!"' 2,25-2,26: OP ')' 2,26-2,27: NEWLINE '\n' 3,0-3,1: NL '\n' 4,0-4,0: DEDENT '' 4,0-4,9: NAME 'say_hello' 4,9-4,10: OP '(' 4,10-4,11: OP ')' 4,11-4,12: NEWLINE '\n' 5,0-5,0: ENDMARKER ''  The exact token type names can be displayed using the -e option: $ python -m tokenize -e hello.py
0,0-0,0:            ENCODING       'utf-8'
1,0-1,3:            NAME           'def'
1,4-1,13:           NAME           'say_hello'
1,13-1,14:          LPAR           '('
1,14-1,15:          RPAR           ')'
1,15-1,16:          COLON          ':'
1,16-1,17:          NEWLINE        '\n'
2,0-2,4:            INDENT         '    '
2,4-2,9:            NAME           'print'
2,9-2,10:           LPAR           '('
2,10-2,25:          STRING         '"Hello, World!"'
2,25-2,26:          RPAR           ')'
2,26-2,27:          NEWLINE        '\n'
3,0-3,1:            NL             '\n'
4,0-4,0:            DEDENT         ''
4,0-4,9:            NAME           'say_hello'
4,9-4,10:           LPAR           '('
4,10-4,11:          RPAR           ')'
4,11-4,12:          NEWLINE        '\n'
5,0-5,0:            ENDMARKER      ''


text:
My Math Forum Polynomial Question - Nobody Can Figure it Out

 Algebra Pre-Algebra and Basic Algebra Math Forum

 October 21st, 2008, 06:43 PM #1 Newbie   Joined: Oct 2008 Posts: 1 Thanks: 0 Polynomial Question - Nobody Can Figure it Out Write 4(x^2) - 9(y^2) +4(x^3) + 6(x^2)(y) as a product of two non-constant polynomials with integral coefficients. This is just a simple problem that has stumped nearly everyone in my class. Can anyone help us out? Thanks. DISREGARD. IT WAS FIGURED OUT.
 October 22nd, 2008, 12:01 AM #2 Global Moderator   Joined: Dec 2006 Posts: 20,472 Thanks: 2039 $4x^2\,-\,9y^2\,+\,4x^3\,+\,6x^2y\,=\,(2x\,-\,3y)(2x\,+\,3y)\,+\,2x^2(2x\,+\,3y)\,=\,(2x^2\,+\ ,2x\,-\,3y)(2x\,+\,3y)$

 Tags figure, polynomial, question

 Thread Tools Display Modes Linear Mode

 Similar Threads Thread Thread Starter Forum Replies Last Post Nmbvc0 Algebra 7 March 6th, 2013 07:49 AM r-soy Physics 4 November 10th, 2011 05:15 AM indiegod Advanced Statistics 5 February 24th, 2010 07:07 AM indiegod Probability and Statistics 1 February 23rd, 2010 03:11 PM Nmbvc0 Calculus 0 December 31st, 1969 04:00 PM

 Contact - Home - Forums - Cryptocurrency Forum - Top


text:
# Aylien’s new natural-language add-on lets you analyze (and summarize) text in Google Drive

A natural language processing startup called Aylien released a Google Chrome add-on earlier this week that lets users analyze their text right from a spreadsheet in Google Drive. Aylien launched earlier this year, proudly touting its API-based approach to analyzing text and its focus on delivering usable products rather than cutting-edge NLP research.

I tested out the add-on, and it’s both easy and useful. It’s capable of standard tasks such as sentiment analysis, entity extraction, content extraction and classification (into preset categories), but also of summarizing long pieces of text and of breaking apart media URLs into separate fields for title, author, text, and images/media.

The free trial allows users to process 1,000 rows, which should be plenty to figure out whether it will work. Additional credits start at $10 for 1,000 rows and go up to$200 for 200,000 rows.

Sentiment analysis is the first thing most people think of when they hear NLP, but it’s usually spotty (Aylien is no exception there), so I think the other capabilities are actually more useful — especially for media companies. I put the Aylien add-on to work first on a 2013 feature I wrote about Hadoop, because it was long and included lots of names and topics.

Here’s a word cloud I created from the results, without filtering anything out. Not too shabby considering who I spoke with and what you might expect to find in a story about Hadoop.

Next — and this might have been unfair because that Hadoop article is roughly 4,000 words — I tested out the summarization feature. Here’s the 246-word summary it came up with, largely, apparently, by collecting some topic sentences. Not perfect, but one could argue that’s a fair version of the Hadoop journey up to a point.

Raymie Stata, founder and CEO of Hadoop startup VertiCloud (and former Yahoo CTO), calls MapReduce “a fantastic kind of abstraction” over the distributed computing methods and algorithms most search companies were already using:


text:
On the stationary Boltzmann equation
Séminaire Équations aux dérivées partielles (Polytechnique) (2001-2002), Talk no. 1, 11 p.

For stationary kinetic equations, entropy dissipation can sometimes be used in existence proofs similarly to entropy in the time dependent situation. Recent results in this spirit obtained in collaboration with A. Nouri, are here presented for the nonlinear stationary Boltzmann equation in bounded domains of $I\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}{R}^{n}$ with given indata and diffuse reflection on the boundary.

@article{SEDP_2001-2002____A1_0,
author = {Arkeryd, Leif},
title = {On the stationary Boltzmann equation},
journal = {S\'eminaire \'Equations aux d\'eriv\'ees partielles (Polytechnique)},
publisher = {Centre de math\'ematiques Laurent Schwartz, \'Ecole polytechnique},
year = {2001-2002},
note = {talk:1},
language = {en},
url = {http://www.numdam.org/item/SEDP_2001-2002____A1_0}
}

Arkeryd, Leif. On the stationary Boltzmann equation. Séminaire Équations aux dérivées partielles (Polytechnique) (2001-2002), Talk no. 1, 11 p. http://www.numdam.org/item/SEDP_2001-2002____A1_0/

[1] Arkeryd, L., Cercignani, C., ’On the convergence of solutions of the Enskog equation to solutions of the Boltzmann equation’, Comm. Part. Diff. Eqns. 14, 1989, 1071-1089. | Zbl 0688.76053

[2] A rkeryd, L., Nouri, A., ’The stationary Boltzmann equation in the slab with given weighted mass for hard and soft forces’, Ann. Scuola Norm. Sup. Pisa Cl. Sci. 27, 1998, 533-556. | Numdam | Zbl 0936.76076

[3] Arkeryd, L., Nouri, A., ’On the stationary Povzner equation in three space variables’, J. Math. Kyoto Univ. 39, 1999, 115-153. | Zbl 1010.35022

[4] Arkeryd, L., Nouri, A., ’${L}^{1}$ solutions to the stationary Boltzmann equation in a slab’, Ann. Fac. Sci. Toulouse Math. 9, 2000, 375-413. | Numdam | Zbl 0991.45005

[5] Arkeryd, L., Nouri, A., ’The stationary Boltzmann equation in $I\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}{R}^{n}$ with given indata’, to appear in Ann. Scuola Norm. Sup. di Pisa. | Zbl 02217248

[6] Cercignani, C., Illner, R., Pulvirenti, M., ’The mathematical theory of dilute gases’, Springer -Verlag, Berlin, 1994. | Zbl 0813.76001

[7] DiPerna, R. J., Lions, P. L., ’On the Cauchy problem for Boltzmann equations: Global existence and weak stability’, Ann. Math. 130, 1989, 321-366. | Zbl 0698.45010

[8] DiPerna, R. J., Lions, P. L., Meyer, Y., ’${L}^{p}$ regularity of velocity averages’, Anal. Non Lin. 8, 1991, 271-287. | Numdam | Zbl 0763.35014

[9] Grad, H., ’High frequency sound recording according to Boltzmann equation’, SIAM J. Appl. Math. 14, 1966, 935-955. | Zbl 0163.23203

[10] Guiraud, J. P., ’Problème aux limites intérieur pour l’équation de Boltzmann en régime stationaire, faiblement non linéaire’, J. Méc. Théor. Appl. 11, 1972, 183-231. | Zbl 0245.76061

[11] Heintz, A., in preparation.

[12] Maslova, N., ’Non linear evolution equations, Kinetic approach’, Series on Advances in Mathematics for Applied Sciences, Vol 10, World Scientific, 1993. | Zbl 0846.76002

[13] Panferov, V., ’On the existence of stationary solutions to the Povzner equation in a bounded domain’, 2000, submitted.

[14] Ukai, S., Asano, K., ’Steady solutions of the Boltzmann equation for a gas flow past an obstacle; I existence’, Arch. Rat. Mech. Anal. 84, 1983, 249-291. | Zbl 0538.76070