text:
# Lesson 9

Using Technology for Statistics

These materials, when encountered before Algebra 1, Unit 1, Lesson 9 support success in that lesson.

## 9.1: Estimation: Stack of Books (5 minutes)

### Warm-up

The purpose of an Estimation warm-up is to practice the skill of estimating a reasonable answer based on experience and known information, and also help students develop a deeper understanding of the meaning of standard units of measure. It gives students a low-stakes opportunity to share a mathematical claim and the thinking behind it (MP3). Asking yourself, “Does this make sense?” is a component of making sense of problems (MP1), and making an estimate or a range of reasonable answers with incomplete information is a part of modeling with mathematics (MP4).

### Launch

The first few times using this routine, ask students to explain the difference between a guess and an estimate. The goal is to understand that a guess is an answer without evidence, whereas an estimate is based on reasoning using the available information.

Display the image for all to see. Ask students to silently think of a number they are sure is too low, a number they are sure is too high, and a number that is about right, and write these down. Tell the students that their answers should be in relation to the person’s height. Then, write a short explanation for the reasoning behind their estimate.

### Student Facing

How tall is the stack of books?

1. Record an estimate that is:
too low  about right   too high

### Student Response

For access, consult one of our IM Certified Partners.

### Activity Synthesis

Ask a few students to share their estimate and their reasoning. If a student is reluctant to commit to an estimate, ask for a range of values. Display these for all to see in an ordered list or on a number line. Add the least and greatest estimate to the display by asking, “Is anyone’s estimate less than $$\underline{\hspace{.5in}}$$? Is anyone’s estimate greater than $$\underline{\hspace{.5in}}$$?” If time allows, ask students, “Based on this discussion, does anyone want to revise their estimate?”

Then, reveal the actual value and add it to the display. Tell students that the person’s height is 6 feet or 72 inches, so the stack of books is about 24 inches or 2 feet. (For estimation activities in following lessons, the same person will appear, so it is important that students know he is about 6 feet tall.)

Ask students how accurate their estimates were, as a class. Was the actual value inside their range of values? Was it toward the middle? How variable were their estimates? What were the sources of the error? Consider developing a method to record a snapshot of the estimates and the actual value so students can track their progress as estimators over time.

## 9.2: Spreadsheet Statistics (25 minutes)

### Activity

In this activity, students use their understanding of spreadsheets to compute useful statistics from data sets. In the associated Algebra 1 lesson, students will learn alternative methods for computing these values.

### Launch

If students need a reminder about how to use a spreadsheet as a calculator, demonstrate how to do basic calculations such as =A1+A2, =SUM(A1:A10), =(A1+A2)/A3, and =COUNT(A1:A50) or =LENGTH(A1:A50) in GeoGebra.

### Student Facing

Here is a list of the number of pages for fiction books on a shelf.

• 15
• 243
• 426
• 175
• 347
• 186
• 236
• 394
• 170
• 412
• 242
• 479
• 185
• 254
• 186
• 278
• 277
• 278
• 486
• 207
• 378
• 251
• 458
• 360
• 440
• 181
• 349
• 482
• 382
1. Describe a method to find the mean of these values. (Do not calculate the value yet.)
2. Input the data into a spreadsheet in the same column. In another column, write a spreadsheet formula that will compute the value of the mean for the values in the column. What is the mean number of pages in these books?
3. The first book on the list with 15 pages was a recording error and should have been 158 pages. You need to compute the new mean using 158 instead of 15. Would you rather compute the new mean by hand or use a computer? Explain your reasoning.
4. What is the new mean number of pages in these books with the updated value of 158 pages for the first book in the list?
5. Another book is found and added to the shelf. This new book has 519 pages. How would you update your spreadsheet formula to include this new value?

### Student Response

For access, consult one of our IM Certified Partners.

### Activity Synthesis

Select students to share their responses and methods for finding the mean from a list of values in a spreadsheet. The purpose of the discussion is to point out the advantage of using a spreadsheet to calculate statistics, especially for very large sets of data. For further discussion, consider asking these questions:

• "Which part of the process for finding the mean for these values took the most time?" (Typing the values into the spreadsheet and making sure they are correct.)
• "Often large data sets are collected electronically and automatically added to a spreadsheet. When there are thousands of data values in a spreadsheet, what other advantages do you think might be helpful when using technology to compute the mean?" (It may be difficult to understand the data when there are so many values, so using the spreadsheet to calculate the mean can help in understanding what is typical. It would definitely be faster to use the spreadsheet to calculate the mean than doing it by hand.)
• "What other statistics would be useful and easier to calculate using a spreadsheet?" (If the data set is large or the numbers are difficult to understand at a glance, the median, quartiles, interquartile range, mean absolute deviation, maximum, and minimum should all be easy for a computer to find from a list.)

## 9.3: Which Data Display? (10 minutes)

### Activity

In the associated Algebra 1 lesson, students will learn how to use technology to create data displays. In this lesson, students examine data displays to select an appropriate one for the data given.

### Student Facing

For each set of data, select the data display that is most informative, then explain your reasoning.

The total area of 50 U.S. states in thousands of square kilometers.

The population of the 50 U.S. states in thousands of people.

The population density of 50 U.S. states in people per square mile.

### Student Response

For access, consult one of our IM Certified Partners.

### Activity Synthesis

Select students to share their responses and reasonings. The purpose of the discussion is to examine the strengths and weaknesses of each of the types of data display as well as thinking about the interval size for histograms and axis marks for ease of reading.

The goal of the lesson is for students to have an introduction to navigating technology to calculate statistics and arrange a data display to be informative. Students should understand how to input data into the spreadsheet, create graphical representations, and calculate some statistics. To discuss what they learned, ask,

• “What are some things that technology allowed you to do that will make computing statistics easier in the future?” (After putting the data into the spreadsheet, it is relatively simple to compute the appropriate value, and it can update automatically with new data or after changing a value.)
• "What are some considerations when arranging a data display?" (The scale of the axes, which display type best shows the distribution, and the interval size for histograms are all important to consider when displaying data to be informative.)


text:
blob: a78e0c17a86146e7d23c172ea8c18a8c76e15c36 [file] [log] [blame]
 /*This file is prepared for Doxygen automatic documentation generation.*/ /*! \file ********************************************************************* * * \brief System-specific implementation of the \ref __write function used by the standard library. * * - Compiler: IAR EWAVR32 * - Supported devices: All AVR32 devices with a USART module can be used. * - AppNote: * * \author Atmel Corporation (Now Microchip): * https://www.microchip.com \n * Support and FAQ: https://www.microchip.com/support * ******************************************************************************/ /* Copyright (c) 2007, Atmel Corporation All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * 1. Redistributions of source code must retain the above copyright notice, * this list of conditions and the following disclaimer. * * 2. Redistributions in binary form must reproduce the above copyright notice, * this list of conditions and the following disclaimer in the documentation * and/or other materials provided with the distribution. * * 3. The name of ATMEL may not be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY ATMEL AS IS'' AND ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE EXPRESSLY AND * SPECIFICALLY DISCLAIMED. IN NO EVENT SHALL ATMEL BE LIABLE FOR ANY DIRECT, * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include #include "usart.h" _STD_BEGIN #pragma module_name = "?__write" //! Pointer to the base of the USART module instance to use for stdio. __no_init volatile avr32_usart_t *volatile stdio_usart_base; /*! \brief Writes a number of bytes, at most \a size, from the memory area * pointed to by \a buffer. * * If \a buffer is zero then \ref __write performs flushing of internal buffers, * if any. In this case, \a handle can be \c -1 to indicate that all handles * should be flushed. * * \param handle File handle to write to. * \param buffer Pointer to buffer to read bytes to write from. * \param size Number of bytes to write. * * \return The number of bytes written, or \c _LLIO_ERROR on failure. */ size_t __write(int handle, const uint8_t *buffer, size_t size) { size_t nChars = 0; if (buffer == 0) { // This means that we should flush internal buffers. return 0; } // This implementation only writes to stdout and stderr. // For all other file handles, it returns failure. if (handle != _LLIO_STDOUT && handle != _LLIO_STDERR) { return _LLIO_ERROR; } for (; size != 0; --size) { if (usart_putchar(stdio_usart_base, *buffer++) < 0) { return _LLIO_ERROR; } ++nChars; } return nChars; } _STD_END

format: plain


text:
### Fall 2017

The Fall 2017 SASMS will be held on Friday, November 10 at 4:30 in MC 5417. If you would like to give a talk, you may do so by clicking "Sign Up" above.

### Talks

Time
Speaker
Talk Title
4:30
Stephen Wen
The Mathematical Foundations of Economics
5:00
Letian Chen
Minimal Surfaces
5:30
Shouzhen Gu
Physics
6:00
Pure Math Club
Dinner
6:30
Zouhaier Ferchiou
7:00
Infinite pigeons.
7:30
Felix Bauckholt
A slightly more motivated look at impartial games
8:00
Sean Harrap
Probabilistically Checkable Proofs of Proximity
8:30
Alexandru Gatea
Block Characters of $S_{/infty}$
9:00
Jarry Gu
How to get a 🅱Math degree in Pure Mathematics in 1980

### Abstracts

#### The Mathematical Foundations of Economics

We will cover the foundations of choice theory to give a flavour of the study of economics and the mathematical reasoning that goes into it.

#### Minimal Surfaces

I will derive the minimal surface equation using variational method and then discuss the generalization to manifolds. If I have time I will discuss some further results of CM. I won't.

#### Physics

I will show how quantum mechanics is only a slight adjustment from classical mechanics.

#### Dinner

Hello, I'm dinner. I'd like to introduce you to me <3

#### How NOT to add fractions

Remember how to add fractions ? Reduce them to common denominators, then add numerators ? That's too complicated for me so I do point-wise addition. For this talk, I will be trying to convince you that this is not completely useless. (something something Riemann Hypothesis something something)

#### Infinite pigeons.

There are some similarities in the proofs from infinite graphs and real analysis. Also pigeons. See functional analysis assignment.

#### A slightly more motivated look at impartial games

You've been bamboozled! Due to a lack of preparation I'll just scrape together random cool stuff without any motivation at all.

#### Probabilistically Checkable Proofs of Proximity

A brief introduction to some semi-recent (as new as 10 years old) research on the P vs NP problem from a probabilistic perspective.

#### Block Characters of $S_{/infty}$

I will construct the set of extremal normalized block characters of $S_{\infty}$ from block characters of $S_n$ (where a block character is just a character depending only on the number of disjoint cycles of each permutation g).

#### How to get a 🅱Math degree in Pure Mathematics in 1980

It's a boring talk… You may leave early as you want


text:
I want to start exploring numerical integrators for differential equations and how the accuracy of various methods impacts the quantities measures from the resulting simulations. Many of the differential equations that arise in physics do not have known analytical solutions; numerical integration is the only way of simulating these systems.

My particular background is in molecular dynamics (MD). MD has its own unique requirements with a wide array of techniques used to evaluate the quality of a particular integrator. It can be easier, however, to evaluate integrators using a simple model with a known analytical solution before studying more complex models. As such, I’ll be using it to experiment with integrators and analyses I find in papers. Here, I want to describe the harmonic oscillator and some of its properties.

The potential energy function of the harmonic oscillator is given by:

where $U$ is the potential energy, $x$ is the position, $m$ is the mass, $\omega$ is the angular frequency.

We will simulate the harmonic oscillator in the microcanonical ensemble. This is the formulation seen in a first-year physics class where the total energy (potential plus kinetic) of the system is conserved. The dynamics are captured in the following second-order ordinary differential equation, otherwise known as Newton’s equations of motion:

The ordinary differential equation has two initial conditions: $x(0)$ and $v(0)$.

The analytical solution of the ODE is given by:

where $A$ is the amplitude of the oscillations, $t$ is the time, and $\phi$ is the phase.

We can verify the solution by taking its derivatives

and substituting back into the ODE:

Using the equations for the position $x(t)$ and and velocity $\frac{dx}{dt}$ and our initial conditions ($x(0)$ and $v(0)$, we can derive a system of equations for $A$ and $\phi$ and solve:

Note that there are multiple ways to solve for $\phi$ and $A$. This particular solution assumes that $x(0), \cos(\phi) \neq 0$.

We now have what we need to simulate a system with $m = 1, \omega = 2 \pi$ and initial conditions $x(0) = 1, v(0) = 0$. I placed the code in a GitHub repository I’ll be using for these experiments. I simulated the system for 10 seconds using a timestep of 0.01 seconds.

The resulting phase diagram looks like this:

The phase diagram will become an important part of our error analyses. Since we have an analytical solution, we can compute the errors in the positions and velocities resulting from the numerical integration. Beyond local and global error analysis, MD requires that integrators are symplectic, or volume preserving in phase space, in order to ensure conservation of energy. The phase diagrams will give us a nice visual way of confirming that property.

I also computed the potential and kinetic energies at each timestep. The kinetic energy is given as

and the total energy is:

for some fixed $E_T$.

If we plot the kinetic, potential, and total energy, we can see that the total energy (black) is conserved, but is transformed transformed between kinetic (cyan) and potential (magenta) energy over time:

In MD, it is also common to evaluate integrators by measuring the error in the energies. I’ll explore that further in a future blog post.

For now, I’ve walked through the basic harmonic oscillator model. I’ve discussed some of the physical choices made for the dynamics of the model. And lastly, I briefly explored two properties of the model.


text:
# charge density in a wire

If we have a infinitely thin and infinitely long straight wire on the $z$-Axis with given current $I(t)$, how can I compute the charge density?

I figured out that the current-density is given by $\vec{j}(\vec{r},t)=I(t)\delta(x)\delta(y)\vec{e_z}$.

But how can I compute the charge-density? I thought about the continuity equation, but did not understand the term $\nabla.\vec{j}=\nabla.(I(t)\delta(x)\delta(y)\vec{e_z})$=?? On the other side I tried to find a "direct" relationship between the charge density $\rho$ and the current $I(t)$ by $\rho=\frac{dQ}{dz}=I\frac{dt}{dz}$. But this seems absolutely wrong. Can you give me an advice?

-

The charge density will depend on the resistivity of the wire and on the potentials at the ends of the wire. The following is relevant: http://www.astrophysik.uni-kiel.de/~hhaertel/PUB/voltage_IRL.pdf

-
Thank you for your help! Lets say, it has no resistivity(infinitely thin) and no ends (since it is infinitely long). Maybe one can write analogously $\rho(r,t)=Q(t)\delta(x)\delta(y)$?? ps: I think the answer must depend only on I(t) like for the curren-density –  Braten Jan 4 at 13:25
@Braten: Not sure I agree. Let us imagine for a moment that the wire is extremely long, but still finite (an infinite wire just makes no sense). The wire can have arbitrary total charge, which can have a static configuration (with a (linear) charge density constant everywhere but near the ends). The current is zero for static configurations for any total charge. Therefore, current doesn't determine the charge (density) uniquely. You should define the potentials (with respect to infinity) at the ends (or at least at one end, if you know resistivity, the length, and the current). –  akhmeteli Jan 5 at 16:51
2. There might be only the moving charges. If they all have speed $v$, and charge density $\lambda$ coulombs per meter of wire length, the current will be $i = \lambda v$, or: $$\lambda = i/v$$


text:
# Methods of Calculating Weights in WACC

From the perspective of an investment analyst, the weights used in the calculation of WACC should be calculated on the basis of target capital structure that the firm expects to achieve. However, since it’s difficult to find out the company’s target capital structure, the analysts will use the current capital structure of the company as a proxy for their target capital structure. In doing so, they will current capital structure based on the market values of each component.

Let’s say the market values of a firm’s capital are: Debt outstanding – $4 million, Preferred -$1 million, and Equity - $5 million. Total capital is$10 million.

With this information we can calculate the weight of each component

Wd = 4/10 = 0.40

Wp = 1/10 = 0.10

We = 5/10 = 0.50

The analyst may also use any other information available to him while determining the weights. For example, if he notices that the firm has been consistently issuing more debt year after year, he may use a higher weight for the debt component. He the analyst believes that the firm’s current capital structure is not a good indicator of its target capital structure then he can also use the average capital structure of the industry.


text:
Octane has a molar mass of 114 grams per mole. What is the boiling point of octane at standard pressure?

Well, it's higher than $\text{n-hexane}$, and $\text{n-heptane}$........
See this site for the boiling point. But why should the boiling point of $\text{n-octane}$ be greater than that of $\text{n-hexane}$ and $\text{n-heptane}$? What intermolecular forces operate?


text:
## Exploiting past proof experience

• We are going to present two methods that allow to exploit previous expe-rience in the area of automated deduction. The first method adapts (learns)the parameters of a heuristic employed for controlling the application of infer-ence rules in order to find a known proof with as little redundant search effortas possible. Adaptation is accomplished by a genetic algorithm. A heuristiclearned that way can then be profitably used to solve similar problems. Thesecond method attempts to re-enact a known proof in a flexible manner in orderto solve an unknown problem whose proof is believed to lie in (close) vicinity.The experimental results obtained with an equational theorem prover show thatthese methods not only allow for impressive speed-ups, but also make it possibleto handle problems that were out of reach before.

$Rev: 13581$


text:
# Find grid cells in 2D grid based on object size and rotation

I am working on a game that uses a grid system that allows the player to place buildings in the world. Each cell in the grid is 64x64. All buildings will be a multiple of 64x64, so everything will evenly fit in the the grid.

Below shows an example of a building that is 128x64 and placed at grid position 64x64. The client sends the server the "base" cell, which in this case is 1,1 (@64x64), and knows the building that is being build is 128x64 so it sets both cells to the "occupied" state which means a building is placed in those 2 grids.

The problem I am having is allowing the client to rotate the building either 90, 180, or 270 degrees.

For example, if the client wanted to place the same building, but rotated 90 degrees, it would look like this:

Basically what I am asking is: Given the client sends to server the location to build 64x64, and sends 90 for the rotation, how can I (on the server) figure out which cells are going to be build on?

Currently I am using this code to get the grids, but it doesn't allow for rotation since I have no idea.

https://pastebin.com/UaaUa7jZ

Any help would be appreciated.

• Please include your code in the question itself, not in an external link that can rot. – DMGregory Mar 23 '20 at 2:42

I have previously implemented something similar for determining placement of items within a 2D grid and hope that sharing my ideas with you will provide some guidance.

It helps to normalise the coordinates for the items being placed into an offset around the origin as well as considering rotations as a fixed step rather than an angle (which you could convert to/from later if needed).

As an example, consider the following shape rotated in 4 different orientations:

The footprint of the object is defined by the offsets from the origin where 0,0 is the "base" cell as you described. To calculate the neighbouring cells for a given rotation, we can use a simple formula to convert from the original, non rotated footprint to the footprint of the rotated object.

For the original rotation (rot = 0), we simply do nothing and the footprint coordinates remain as is.

For each subsequent rotation we can follow these rules:

Rot = 1: First invert the x coordinate, then flip both the x and y coordinates.
Rot = 2: Invert both the x and y coordinates
Rot = 3: First invert the y coordinate, then flip both the x and y coordinates.

Here is an example for the footprints of the above shapes with the appropriate rotation:

To work through an example of this, given the footprint with rot = 1, we would convert the coordinates as such:

Rot = 1: First invert the x coordinate, then flip both the x and y coordinates.

0, 0 -> 0, -0
0, 1 -> 1, -0
1, 1 -> 1, -1

Hopefully this should be enough information to infer the remaining rotations with the aforementioned formulas.

Once you have calculated the new footprint coordinates for your given rotation, these can then be added to your world grid coordinates to determine which neighbouring cells need to be checked in relation to your base cell.

The server has to have the same knowledge of building types that the client has, and needs to know the current rotation and dimension(s).

To keep this information concise in your packets sent up to server, you can use 1 byte for the direction (this is really just an enum type that both client and server code have access to):

0=0 degrees
1=90 degrees
2=180 degrees
3=270 degrees

You can actually fit these 4 values into 2 bits i.e. 2^2=4, rather than 1 byte or 8 bits. But keeping it as a byte keeps it simple to start with.

Building shape could be either just length (say 1 unit wide x n units long), or length and width, both of which get sent to the server - you can use 1 byte for each of width and height, and later on, use fewer bits to save on bandwidth.

So your simplified packet layout might be e.g.:

[1 byte rotation factor|1 byte width| 1 byte height| other stuff you need to send]

When this arrives on the server, you multiple the rotation factor by 90 degrees or the equivalent radians, then figure out the direction the body length is going to go, and then check whether this collides with existing buildings on the server.

I would suggest for a more compressed layout (optimise only once the basic logic is working):

[2 bits rotation factor|3 bits width|3 bits height] totalling 8 bits or 1 byte

• It looks to me like OP could also use some help translating a packet like "Building A (3x2) at position 13, 4, rotated 90 degrees" into the corresponding tile coordinates that the footprint of the building will cover in that orientation. – DMGregory Mar 26 '20 at 14:04
• @DMGregory The assumption is that part is trivial. Do you not agree? Simple matrix transposition is one way to do this. OP - let us know if this is what has you confused. – Engineer Mar 26 '20 at 14:11
• I'd agree it's trivial for someone with your experience, but by my reading, I don't think OP is finding it so easy, so showing even a quick example of that could be a big help to them. – DMGregory Mar 26 '20 at 14:12
• Let's see if they come back and respond, in which case I will make good on your suggestion. – Engineer Mar 26 '20 at 14:14


text:
# Difference between revisions of "Precision Pendulum Assembly: Software management"

This page contains the instruction set and most software utilities used in the remote experiment controller and the way it is connected to the internet.

# Raspberry software

## Software management

The Raspberry Pi computer interfaces the experiment micro-controller and the e-lab servers. Without it, no remote access is possible. It's, therefore, essential it is properly setup and configured to ensure high reliability and safety.

### Operating System Installation

Should you start with a blank SD card, the first task is to install the Operating System (OS). There are several OSes supporting the Raspberry Pi (RPi) available and, in theory, any of those supporting the needed software can be used. However the use of Raspbian is highly recommended. It's the official OS from the Raspberry Pi Foundation for all RPi models, it has a wide selection of supported software available on the official repositories and tutorials and guides abound, online and in other mediums.

Instructions below this point assume that Raspbian is being used. Other OSes may have different configuration methods.

When creating a new SD card, the latest version of the OS should be installed. This ensures that the system is going to run with all the available security and performance fixes available. The latest version of Raspbian can be downloaded from this link. Three versions are available:

• The lite version contains the minimal amount of software needed to boot the RPi. No extra software is included. This implies the lack of a Graphical User Interface (GUI), meaning that, if connected to a display, the RPi will show only a command line (as pictured).
• The desktop version contains the same software as the lite version plus the software needed to use a GUI. When connected to a display, the RPi will show a GUI (pictured).
• The desktop and recommended software version contains the same software as the desktop version plus some pre-selected software and utilities.

The same software is available for installation on all versions, meaning that one can start with a lite version and later install all the software included in the desktop and recommended software version. If the RPi is being prepared to run solely as a server for the experiment, it's recommended that the lite or desktop version is chosen, as having an OS with only the needed software is safer, improves performance and maintainability. In particular, if you plan to use the RPi without a display, the lite version is of special interest as it doesn't package the GUI, which is only useful if you use a display.

The Raspberry Pi Foundation provides installation instructions for the Raspbian OS here.

### After Installation

After installing the OS, the SD card is now ready to be used to boot the RPi. However, before inserting the SD card on the RPi, one needs to consider how to interface it the RPi. It's possible to do so through two different methods, as follows:

#### Using the RPi through a keyboard, mouse and display

When using a keyboard, mouse and display, the RPi works just like any other computer. Since it's possible to interface with the RPi through the keyboard, mouse and display, the SD card can be inserted into the RPi and the boot process can be started by connecting the power adapter to the RPi. This is a simpler procedure, however the extra hardware needed makes it more expensive. Do not connect the RPi to the internet before completing the OS configuration, as described below. When successfully setup, you should see either a prompt showing:

pi@raspberry:~ $ or the usual graphical user interface of the OS, depending on the version of Raspbian chosen. Proceed to the RPi Operating System Configuration section. #### Connecting to the RPi through a ssh connection If a keyboard, mouse and display are not available to use and interact with the RPi, it's still possible to interact with it by setting up the SSH service before inserting the SD card into the RPi. The SSH service allows connections with a RPi via a network interface, through which commands can be sent. This makes use of the keyboard, mouse and display of a second computer, e.g. a already setup laptop or desktop computer. To do so, the Raspberry Pi and the computer used need their network setting to be adjusted. To enable the SSH service on the RPi externally: • Insert the SD card into a computer, e.g. the one used to install the OS on the card. • After installing the OS, the card will show on computers as being composed of two drives. A smaller drive of a few hundred MBs and a larger drive of a few GBs (depending on the size of the SD card). • On the smaller "drive", create an empty file called ssh. This enables the SSH service during the next boot of the RPi only. This setting will be made permanent later. • On the bigger "drive", edit the file "/etc/dhcpcd.conf". On the end of the file, there should be a section of text similar to this: # It is possible to fall back to a static IP if DHCP fails: # define static profile profile static_eth0 static ip_address=123.123.123.123/24 #static routers=192.168.1.1 #static domain_name_servers=192.168.1.1 # fallback to static profile on eth0 interface eth0 fallback static_eth0  Edit the file so it mimics the text shown above. The line: static ip_address=123.123.123.123/24  Defines the fallback IP address of the Raspberry Pi when it receives no IP offers through DHCP. In general, this happens when no connection is available on the network or it has been connected to a common laptop or desktop computer. Other fallback IP's other than 123.123.123.123 can be chosen. This one is suggested as it's easy to remember. • The SD card can now be safely unmounted from the computer and inserted into the Raspberry Pi and booted Next, the computer that will connect to the RPi must have it's own Ethernet connection configured so it connects to the same subnet as the RPi. This means that, following the configuration shown above, its' IP should be configured to be 123.123.123.xxx, where the last number can be any from 001 to 254, and a netmask of 255.255.255.000. The exact procedure to follow to accomplish this is highly dependent on the OS installed on the computer. Be sure to proceed in a way appropriate for your specific OS. If your computer uses the Ethernet port for day-to-day internet connection, it's possible that some specific configuration is attributed to it to make this connection possible on your network. Be sure to preserve the original network settings, by either taking screenshots of the appropriate dialogs or taking notes, of your Ethernet adapter so you can rollback any changes made. Below are some examples for common OSes. ##### Configuring the Ethernet Adapter on Windows 10/8.1/7 and connecting to the RPi • Microsoft provides instructions on how to manually configure the Ethernet adapter. • Follow the procedure for manually configuring IPv4 settings. When prompted, fill the IP Address field with an appropriate address according to the configuration selected for the RPi. Assuming the example configuration shown on this page was selected, this field can be filled with 123.123.123.124, and the field Subnet prefix length/Subnetmask can be filled either with 255.255.255.0 or 24. Settings regarding the Gateway and DNS fields and IPv6 can be ignored. • If not done so before, the RPi can be booted and connected to the computer throught Ethernet. This is accomplished by connecting one end of an Ethernet cable to the Ethernet port of the computer and the other end of the cable to the Ethernet port of the RPi • The network symbol on the Windows tray should now show the Ethernet connection. Possibly stating it is an unknown network. • Windows does not install a SSH client by default. To connect to the RPi via SSH one must be set up. Although others can be used, putty is an open source, free software SSH client for Windows. It can be installed or used as standalone, meaning no install necessary. The following steps will follow the use of putty. Be aware that although the general configuration also applies to other software, specific instructions may differ. • Fill the field Host Name (or IP Address) with the configured IP of the RPi. If the suggested configuration is being followed, this should be filled with 123.123.123.123. On the radio buttons below, select SSH. If you wish to save this configuration so that it's not needed to remember or rewrite it every time, write any memorable name on the field Saved Sessions and click Save. You can then retrieve this settings by selecting the name from the list and selecting Load. • A warning can appear stating that the host, the computer you're connecting to, is new and unknown. You'll be asked if the new computer can be trusted and the connection completed. Since this a fresh install of a computer you own and have setup by yourself, you can safely answer yes. When prompted, input the password. If this is not the first time following this guide, it's possible that the warning states that the host, the computer you're connecting to, has changed. This is expected as you can now be applying theses steps to a different RPi. Accept the warning by selecting Yes and the connection should be completed. • When prompted for the user and password, input the default user and password of the Raspian OS which can be found here. • Note that in this case pi and 123.123.123.123 are used since we're connecting to the RPi as user pi and it's directly connected to the computer with the configured IP of 123.123.123.123. After creating a new user and deleting the default one and connecting the RPi to regular network, this will change. • After successfully inputting the default password, a prompt will be shown stating: pi@raspberry:~$

You have logged in successfully and can now proceed to the RPi Operating System Configuration

• If there was an error stating
ssh: connect to host xxx.xxx.xxx.xxx port 22: No route to host

Where xxx.xxx.xxx.xxx is the IP used in the ssh command, the Ethernet adapters have been misconfigured. Please, revisit the configuration of the adapters. For a successful connection both the computer and the RPi must be configured to be in the same subnet. If you haven't followed the IP configuration suggested here, please note it is known to provide a functional set up.

##### Configuring the Ethernet Adapter on Linux using Network Manager and connecting to the RPi
• Linux isn't a monolithic OS. What is generally know as Linux is a set of several distributions, also know as distros, that build an OS on the same basis, the Linux kernel. Depending on the distro used, the specific procedure to follow will differ. The following procedure applies to distros that have Network Manager setup to manage connectivity. This is the case for most of the popular distros, such as Ubuntu and its flavours, Linux Mint, Fedora, etc.
• Right click on the Network Manager icon and select edit connections
• Click on the + sign on the bottom left of the window,select Ethernet on the dialog shown and click create. By creating a new configuration, the settings used for regular internet access are preserved, making it easier to roll back the changes.
• The name field can be filled with any name, although it's recommended to write an easy to remember one, such as Raspberry or Ethernet RPi.
• Select the IPv4 Settings tab and, on the Method dropdown menu, select Manual.
• Click Add next to the Adresses table and, when prompted, fill the Address field with an appropriate address according to the configuration selected for the RPi. Assuming the example configuration shown on this page was selected, this field can be filled with 123.123.123.124, and the neighboring field Netmask can be filled either with 255.255.255.0 or 24.
• The rest of the fields can be safely ignored. Click Save to save the new connection settings. The new settings will now appear on the list with the name selected.
• If not done so before, the RPi can be booted and connected to the computer throught Ethernet. This is accomplished by connecting one end of an Ethernet cable to the Ethernet port of the computer and the other end of the cable to the Ethernet port of the RPi
• The Network Manager icon should now show a spinning symbol, as it tries to connect to the RPi. Click on the Network Manager symbol and on the menu select the Ethernet connection with the name chosen before. The connection should now be set up.
• Most Linux distros include SSH client software on the default OS install, making it unnecessary to install any additional software. SSH commands follow the syntax ssh user@ip or ssh user@hostname. The default user and password of the Raspian OS can be found here. To connect to the RPi open a terminal on the computer, and write
ssh pi@RPI ADDRESS

RPI ADDRESS should be replaced by the IP address selected for the RPi on the previous section. Assuming the configuration suggested was followed, the command should be

ssh pi@123.123.123.123

Note that in this case pi and 123.123.123.123 are used since we're connecting to the RPi as user pi and it's directly connected to the computer with the configured IP of 123.123.123.123. After creating a new user and deleting the default one and connecting the RPi to regular network, this will change.

• A warning can appear stating that the host, the computer you're connecting to, is new and unknown. You'll be asked if the new computer can be trusted and the connection completed. Since this a fresh install of a computer you own and have setup by yourself, you can safely answer yes. When prompted, input the password. If this is not the first time following this guide, it's possible that the warning states that the host, the computer you're connecting to, has changed. This is expected as you can now be applying theses steps to a different RPi. Edit the file stated on the warning and delete the line starting with the IP of the RPi. If the example configuration is being followed, this will be the line starting with 123.123.123.123. Attempt to reconnect by using the SSH command again. You should know see the warning stating this is a new, unknown computer.
• After successfully inputting the default password, a prompt will be shown stating:
pi@raspberry:~ $ You have logged in successfully and can now proceed to the RPi Operating System Configuration • If there was an error stating ssh: connect to host xxx.xxx.xxx.xxx port 22: No route to host Where xxx.xxx.xxx.xxx is the IP used in the ssh command, the Ethernet adapters have been misconfigured. Please, revisit the configuration of the adapters. For a successful connection both the computer and the RPi must be configured to be in the same subnet. If you haven't followed the IP configuration suggested here, please note it is known to provide a functional set up. ##### Configuring the Ethernet Adapter on MacOS Catalina/Mojave/High Sierra and connecting to the RPi • Apple provides instructions on how to manually configure the Ethernet adapter. • Follow the procedure for manually configuring IPv4 settings. When prompted, fill the IP Address field with an appropriate address according to the configuration selected for the RPi. Assuming the example configuration shown on this page was selected, this field can be filled with 123.123.123.124, and the field Subnetmask can be filled either with 255.255.255.0 or 24. Settings regarding the Gateway and DNS fields and IPv6 can be ignored. • If not done so before, the RPi can be booted and connected to the computer throught Ethernet. This is accomplished by connecting one end of an Ethernet cable to the Ethernet port of the computer and the other end of the cable to the Ethernet port of the RPi • The network symbol should now show the connection as being completed. • MacOS includes a SSH client on the default OS install, making it unnecessary to install any additional software. SSH commands follow the syntax ssh user@ip or ssh user@hostname. The default user and password of the Raspian OS can be found here. To connect to the RPi open a terminal on the computer, and write ssh pi@RPI ADDRESS  RPI ADDRESS should be replaced by the IP address selected for the RPi on the previous section. Assuming the configuration suggested was followed, the command should be ssh pi@123.123.123.123  Note that in this case pi and 123.123.123.123 are used since we're connecting to the RPi as user pi and it's directly connected to the computer with the configured IP of 123.123.123.123. After creating a new user and deleting the default one and connecting the RPi to regular network, this will change. • A warning can appear stating that the host, the computer you're connecting to, is new and unknown. You'll be asked if the new computer can be trusted and the connection completed. Since this a fresh install of a computer you own and have setup by yourself, you can safely answer yes. When prompted, input the password. If this is not the first time following this guide, it's possible that the warning states that the host, the computer you're connecting to, has changed. This is expected as you can now be applying theses steps to a different RPi. Edit the file stated on the warning and delete the line starting with the IP of the RPi. If the example configuration is being followed, this will be the line starting with 123.123.123.123. Attempt to reconnect by using the SSH command again. You should know see the warning stating this is a new, unknown computer. • After successfully inputting the default password, a prompt will be shown stating: pi@raspberry:~$

You have logged in successfully and can now proceed to the RPi Operating System Configuration

• If there was an error stating

ssh: connect to host xxx.xxx.xxx.xxx port 22: No route to host Where xxx.xxx.xxx.xxx is the IP used in the ssh command, the Ethernet adapters have been misconfigured. Please, revisit the configuration of the adapters. For a successful connection both the computer and the RPi must be configured to be in the same subnet. If you haven't followed the IP configuration suggested here, please note it is known to provide a functional set up.

### RPi Operating System Configuration

After installation, the OS needs to be configured. The following instructions are to be followed on the terminal prompt. If you are using a keyboard and display connected to the RPi and installed an OS with GUI, please open a terminal emulator.

First, proceed with general system configuration by loading the raspi-config utilty, executing the command:

sudo raspi-config

When prompted input user's pi password. The menu that shows can be navigated using arrow keys for selection, escape key for moving backwards on the menus and enter to select the highlighted entry.

• The 1 Change User Password entry can be ignored as the pi user will be deleted later
• On the 2 Network Options entry, it's possible for the Hostname to be set and a Wi-fi connection to be configured.

The Hostname is the name assigned to the RPi and has no special meaning. It's helpful for easy recognition of the computer based only on name.

The Wi-fi entry allows the configuration of a Wi-fi network by entering the SSID (network name) and network password. This dialog only works for WEP, WPA and WPA2 schemes. WPA-entreprise must be manualy setup through the wpa_supplicant.

Ignore the remaining entries unless certain about the changes being performed.

• On the 3 Boot Options, it's the Desktop / CLI entry selects whether the RPi boots into a GUI environment or into a command prompt, the Wait for Network at Boot if it waits for network connection before completing the boot process and the Splash Screen if it shows an image or debugging text during boot.

Be aware that the Desktop / CLI entry only produces effects if you previously installed a version of Raspbian with GUI or have meanwhile installed a GUI on the lite version.

These settings are safe to setup according to users wishes.

• On the 4 Localisation Options entry, the Change Locale entry allows setting up the RPi so it's diplayed language, and other localisation elements, such as time and date format, name of days of the wiki, currency, etc, are according the RPi user's location, the Change Timezone entry allows setting the local timezone, so that the RPi shows the correct time and date, the Change Keyboard Layout entry allows the user to set up it's keyboard so that the keys typed in the physical keyboard match the ones that appear on screen, and the Change Wi-fi Country sets up the RPi so it's communicating on the right Wi-fi frequencies according to the local laws.

The Change Keyboard Layout entry is only important if the RPi is used with a keyboard directly connected to it. Users connecting through SSH do no need to configure this setting.

The Change Wi-fi Country entry must be configured if a Wi-fi connection is to be setup. Failure to do so may result in a non-working connection, as the RPi is trying to communicate in different frequencies as the rest of the local Wi-fi devices, and regulatory issues for the user for transmitting in prohibited frequencies.

• On the 5 Interfacing Options, the entries allow enabling/disabling the various listed services. All should be configured to disabled except for the SSH and/or VNC services, the Serial interface and Remote GPIO access.

Be aware that the VNC service will only work if a GUI is installed on the system. If not, only the SSH' service can be enabled.

Enabling the SSH service makes it unnecessary to create the ssh file on disk as described before. It will be available on every boot until disabled again.

On the Serial entry, please select No when prompted if the login shell should be available over serial and Yes when prompted if the serial port hardware should be enabled.

• Please ignore the remaining options, except for the Expand Filesystem option available on the Advanced Options entry.
• Exit the utility by selecting Finish and, when prompted, allow the RPi to reboot by selecting yes. If connected via SSH, the connection will be automatically terminated.
• Allow the RPi to finish rebooting. Do not power it off by disconnecting the power cable.
• When the green led has stopped flashing, the reboot is complete. Login into the RPi as before.

Before connecting the RPi to the internet, a new user must be to created and the the default user must be deleted. Since all fresh Raspbian OS installs have the same default user and password, that everyone can find on the internet, keeping them available on the OS is a very serious security vulnerability. Anybody that knows this is free to try to login into your RPi and, if successful, take control of the system.

The following commands create an user named newton. Other usernames can, and should be, chosen when configuring the RPi. The username newton is being used as an example.

• Create user newton
sudo useradd newton

When prompted to input user pi's password, input the same password used to login. This will start the user creation process where a new password for the new user is going to be asked for. Choose a new password, different from the default pi user password. When choosing the new password please remember that:

Strong passwords have at least 8 characters, a mixture of both uppercase and lowercase letters, a mixture of letters and numbers and include at least one special character, e.g., ! @ # ? ].

A strong password is hard to guess, but it should be easy to remember. A password that has to be written down is not strong, no matter how many of the above characteristics are employed.

Do not choose passwords that are composed of any word that can be found in a dictionary, in any language (e.g., airplane or aeroplano), a dictionary word with some letters simply replaced by numbers (e.g., a1rplan3 or aer0plan0), a repeated character or a series of characters (e.g., AAAAA or 12345), a keyboard series of characters (e.g., qwerty or qazwsx), anything that’s written down and stored somewhere near your computer.

Your username and password are one of the main mechanisms of defense against unauthorized access and tampering. Having easy to guess usernames, weak passwords or having them known to the public severely impacts the security of your devices and network.

• Add the new user to the system groups

Start by identifying the groups the original user pi is a member of by executing the command:

id

The command will output three lists in the format number(text). Keep the output on screen. Write, but don't execute, the following command:

sudo usermod -a -G

From the previous command output, select the list after groups=, except for the first element, by clicking and dragging with the mouse over the text. Copy it to the command line by clicking the middle mouse button after selecting the text. Edit the current command by using the arrows on the keyboard to move the cursor and removing the numbers and parenthesis but keeping the text and commas. Add the name of the user being created to the end of the command. The final command should have the same format as this:

sudo usermod -a -G group1,group2,group3,group4 newton

Confirm that the user was successfully added to the system groups by executing:

id newton

In the output, after groups=, the same groups as above should appear. Do not proceed before they do.

• Log out from the RPi and login using the new user
• Delete the pi user account

The default user pi still exist in the system. Leaving this account active is a security vulnerability. Delete it by executing:

sudo userdel -remove-home pi

• In order to configure the network where the RPi will be connected, it's often useful to know the MAC address of the network adapter. Execute the following command:
ifconfig

If planning to connect the experiment via Ethernet cable, copy the ether field on the eht0 entry of the output. If connecting via wireless connection, copy the ether field on the wlan0 entry of the output.

• If planning to use a VPN connection to manage remotely deployed experiments, now is also a good time to send the needed files to the RPi. Doing so ensures the needed secrecy of the keys, as they are sent through the cable that connects your computer and the RPi. If using the RPi with keyboard and display, consider shutting it down and writing the files directly on the sd card by inserting it on a computer. VPN configuration is highly dependent on your setup, so it should be completed on case by case basis.
• If not done so, shutdown the RPi. Wait for the green led on the RPi to stop blinking before disconnecting the power source. Move it to its permanent location, connecting it to experiment's electronics and network infrastructure.
• Turn on the RPi by connecting the power source. It should obtain a valid IP adress from your network. Network and firewall configuration is highly dependent on your specific network infrastructure. If the above instructions were followed, the RPi is configured to accept DHCP IP lease offers. In order for the hardware server and the video streaming to work, the RPi must be able to communicate
• Login into it. Update the software on the RPi by running the following commands in succession:
sudo apt-get update

The last command can take a while to complete. While it works, DO NOT DISCONNECT THE RPi FROM THE POWER SOURCE. IF YOU DO, YOU'LL MOST LIKELY CORRUPT THE SD CARD AND YOU'LL HAVE TO START ANEW.

• Reboot the RPi to ensure the latest software is running. Log into it again.
• Install the needed software for the hardware server and the video streaming.

For the hardware server, the following software should be installed:

sudo apt-get install openjdk-8-jre-headless librxtx-java

For the video streaming, various software could be used. The recommend way is to use GStreamer and its libraries that support video hardware enconding on the RPi. It can be installed by the following command:

sudo apt-get install gstreamer1.0-tools gstreamer1.0-plugins-ugly gstreamer1.0-plugins-good gstreamer1.0-plugins-base gstreamer1.0-plugins-bad\
gstreamer1.0-omx-rpi gstreamer1.0-omx-rpi-config gstreamer1.0-omx-generic gstreamer1.0-omx-generic-config

If needed, also install the software for the VPN connection. For example, the openvpn client would be installed by the command:

sudo apt-get install openvpn

Now that the VPN is install, complete the configuration of the VPN with the files transfered to the RPi on the last step.

Please note that the software manager will suggest extra packages, mostly dependencies of the ones explicited ordered to install. Accept any extra suggested packages.

### Network tools and utilities

The connectivity of the computer were the hardware server will be installed can be tested and verify by installing some tools such as nmap:

sudo apt-get install nmap

The nmap program is used to check for the port access testing to elab.ist.utl.pt, elab1.ist.utl.pt and elabmc.ist.utl.pt:

nmap -v -A elab.ist.utl.pt

Ports 443, 80, 8080, 22, and [9000..9010] are expected to be open. elabmc should have added the UDP ports for streaming (see last section).

We suggest to use the cron utility in case a regular reboot is required. To avoid simultaneous registration requests at the same time a delay of 5 minutes is added according to the experiment order. Nano is the elected editor.

sudo crontab -e

and the following line is added to reboot every 4 am plus 35 minutes (position 7 on the table)

0 4   *   *   *    /sbin/shutdown -r +35

Contab -l lists the jobs to be run at a particular time.

sudo crontab -l

### Install a OpenVPN certificate

If your certificate becomes invalid to allow your system to login on elab OpenVPN you can retrieve a recent one by issuing the following command:

echo "get ca.crt" | sftp -P2222 wpa@elab1.ist.utl.pt:/dist

Then you need to move it to the OpenVPN directory which as admin privileges:

sudo mv ./ca.crt /etc/openvpn/privnet

Finally just reboot the system to take effect.

## Autonomous codes

### Hardware Server

Now that you have the Raspberry Pi and the Pendulum all set up, you are ready to connect it to the World Pendulum Alliance network.

In order to do that you will need to deploy and run an application on the Raspberry. This application is called REC Hardware Server, it is a Java application and that is why it is necessary to have the JDK installed.

Once you run the REC Hardware Server, it will automatically try to contact your Pendulum where it is expecting to find its ID String (e.g. "WP_UESC_IOS"). If it finds the expected ID, it will then connect with the Pendulum and your pendulum should come alive at elab public page. Otherwise it will output an error message, in which case we advise you to contact the project management team to help with the debugging.

The REC Hardware Server specific for your pendulum will be provided by the project coordination in a .ZIP file, that will be made available at a specific folder at elab1 server.

To deploy a new version of the REC Hardware Server for a specific pendulum you should open a terminal (e.g. Putty) and access the machine where the HardwareServer will be deployed - in this case, the RPi - and follow these steps:

• Validate if the "rec-deployment" directory exists, if not create it by typing:
mkdir ${home_directory}/rec-deployment  ex: mkdir /home/wpa/rec-deployment  • List HardwareServer Zip files: echo "ls *.zip" | sftp -P2222 wpa@elab1.ist.utl.pt:/dist • Once you identified the Hardware Server for your pendulum, copy the hardwareServer Zip file to your machine (Raspberry): echo "get${hardwareServerZipFile} ~/rec-reployment/" | sftp -P2222 wpa@elab1.ist.utl.pt:/dist
ex. echo "get wpunicvraiHarwareServer_21-02-2020.zip ~/rec-reployment/" | sftp -P2222 wpa@elab1.ist.utl.pt:/dist
• Go to rec-deployment directory:
cd /home/wpa/rec-deployment
• Unzip HardwareServer's zip file to rec-deployment directory
unzip ${hardwareServerZipFile} -d${hardwareServerAliasName}
ex: unzip wpuesciosHardwareServer_24-01-2020.zip -d wpuescios
• Change to directory of experience:
cd ${hardwareServerAliasName} ex: cd wpuescios • Start HardwareServer: ./Start${AliasName}Driver.sh
ex: ./StartwpuesciosDriver.sh

NOTE

The HardwareServer should only be deployed after the complete calibration procedure described on the next page and executed with the minicom or other terminal program.

### Video Streaming

The video streaming can be done by using the picamera, the RPi proprietary camera, or a common USB camera. The following steps were tested on a USB camera. Furthermore, they were designed to work with a streaming server that receives the video from the RPi and resends it to all connected clients.

• Start the streaming on the RPi. This is done with the command:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=320,height=240,framerate=10/1 ! clockoverlay time-format="%x - %X" \
! videoconvert ! omxh264enc ! video/x-h264,profile=baseline ! h264parse ! mpegtsmux ! rtpmp2tpay ! udpsink host=HOST port=PORT async=false

Although it is almost certainly on /dev/video0, it's possible that the camera does not show on the RPi system on /dev/video0. Verify that it is by running:

ls /dev

On the output, the entry of the format videoX, where X is a number, is the logical location of the camera on the system. Edit the command above so that the number X is the same as the one listed on the output of the last command.

It's possible to increase the resolution of the video and the framerate by altering the width, height and framerate fields on the example command above. The example values shown above are very conservative but ensure that video streaming is possible even in the toughest network conditions by generating a very low bitrate stream. Note that any width, height and framerate asked for needs to be supported by the video camera, otherwise the command will return with an error. Edit the HOST and PORT fields so they point to your particular video streaming server.

Once it's verified that the video is running, it can be made permanent by slightly altering the command to:

nohup gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=320,height=240,framerate=10/1 ! clockoverlay time-format="%x - %X" \
! videoconvert ! omxh264enc ! video/x-h264,profile=baseline ! h264parse ! mpegtsmux ! rtpmp2tpay ! udpsink host=HOST port=PORT async=false &

This will run until the RPi is shutdown or the camera is disconnected.

### Specific Instructions for the World Pendulum Alliance Project

A List of TCP ports is deemed necessary to be open for the correct internet connection between the apparatus' computer and the central server.

For the main partner pendulums, the HOST parameter of the streaming command should be set as instructed on the training sessions. The PORT should be set as indicated on Port listing page. For each secondary pendulum, the same process may be followed. However, partners must set up independent streaming infrastructure for secondary pendulums video streaming.

In order to make the needed information available for users to be able to connect to the stream, SDP files must be made available. These can then be independently opened on video player software, like VLC, or seen through the pendulum client software. The file can be made by adapting the template below, replacing the EXPERIMENT NAME and PORT by the appropriate fields, as shown on the Port listing page. The HOST field should be set to the same value as described on the training sessions.

v=0
o=- 16251185917537979632 16251185917537979632 IN IP4 EXPERIMENT NAME
s=EXPERIMENT NAME
i=N/A
c=IN IP4 HOST
t=0 0
a=tool:vlc 3.0.8
a=recvonly
a=charset:UTF-8
m=video PORT RTP/AVP 33
b=RR:0
a=rtpmap:33 MP2T/90000

The SDP file should be created with the same name as indicated on the Port listing page. This file must then be transferred to the server responsible for making it available to the web as instructed on the training sessions.

In case ttyS0 is not being used, a symbolic linker ca be used if needed:

ln -s /dev/ttyAMA0 /dev/ttyS0

### Automatically Start on RPi Boot

To make the programs start automatically as the RPi boots, edit the file /etc/rc.local by executing

sudo nano /etc/rc.local

Assuming the hardware server has been placed on the folder indicated on the previous section, adding the following lines to the file, before the exit 0 and after the fi, will make the RPi start the hardware server and the video streaming automatically as it boots without further intervention.

sleep 120
su USER -c "/home/USER/rec-deployment/exp_name/expDaemon.sh start &"
su USER -c "(gst-launch-1.0 v4l2src device=/dev/video0 ! \
video/x-raw,width=320,height=240,framerate=10/1 ! \
clockoverlay time-format=\"%x - %X\" ! \
videoconvert ! \
omxh264enc ! \
video/x-h264,profile=baseline ! h264parse ! \
mpegtsmux ! rtpmp2tpay ! \
udpsink host=HOST port=PORT async=false ) &"

Note that the HOST and PORT fields need to be replaced as exampled above. The USER field should be changed to match the user account created, typically wpa as previously instructed.

In case ttyS0 is not being used, a symbolic linker ca be used if needed:

ln -s /dev/ttyAMA0 /dev/ttyS0

# dsPIC firmware

## Program dsPIC with Raspberry Pi

With the help of a programmer software, the dspic can be programmed using the Raspberry Pi. See this page for instructions.


text:
# Random polygons in plane convex sets

-
John Pardon, Princeton University
Fine Hall 401

Consider picking $N$ random points in a convex set $K$ and forming their convex hull $K_N$. Recently, there have been a number of results concerning the asymptotic behavior of random variables such as the area and number of vertices of $K_N$. These are, however, all limited to two special cases: 1) $K$ is a polygon and 2) $K$ is "smooth". I will discuss work which obtains uniform bounds over the family of all convex sets $K4$. These results include central limit theorems for the area and number of vertices of $K_N$.


text:
# Monitoring wetlands in the Okavango¶

## Background¶

The Wetlands Insight Tool (WIT) provides insights into a wetland’s seasonal and interannual dynamics. The WIT is a spatiotemporal summary of an wetland that combines multiple datasets derived from the Landsat archive held within DE Africa. Fractional cover, WOfS, and Landsat surface reflectance data are retrieved from DE Africa’s ODC and combined to produce a stack plot describing the percentage of a wetland polygon as vegetation fractional cover, open water, and wet vegetation through time.

Detailed Explanation: The code calculates the Tasselled-Cap Wetness (TCW, or just ‘wetness’) from surface reflectance and takes the maximum Fractional cover fraction per pixel, masking Fractional cover with TCW, and masking TCW with open water. For each pixel inside or overlapping the polygon describing the wetland, WIT calculates the dominant fractional cover type. WIT selects the largest percentage value for each pixel as the dominant fractional cover type. Fractional cover was masked using WOfS and TCW to remove areas of water and wet vegetation from areas where fractional cover is calculated. This is necessary as the fractional cover algorithm erroneously classifies water as green vegetation (PV). The resulting output is a stacked plot of open water, wet vegetation, photosynthetic vegetation, non-photosynthetic vegetation, and bare soil for the wetland polygon through time.

## Description¶

This notebook will run the Wetlands Insight Tool for the area encompassed by a polygon

2. Run the Wetlands Insight Tool

3. Plot the results as a stacked line plot

### Getting started¶

To run this analysis, run all the cells in the notebook, starting with the “Load packages” cell. ***

[1]:

import datacube
import pandas as pd
import seaborn as sns
import geopandas as gpd
import matplotlib.pyplot as plt
from datacube.utils import geometry

from deafrica_tools.wetlands import WIT_drill

from deafrica_tools.plotting import map_shapefile

/usr/local/lib/python3.8/dist-packages/geopandas/_compat.py:111: UserWarning: The Shapely GEOS version (3.8.0-CAPI-1.13.1 ) is incompatible with the GEOS version PyGEOS was compiled with (3.10.1-CAPI-1.16.0). Conversions between both will be slow.
warnings.warn(
/usr/local/lib/python3.8/dist-packages/dask/dataframe/utils.py:367: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
_numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
/usr/local/lib/python3.8/dist-packages/dask/dataframe/utils.py:367: FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
_numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)
/usr/local/lib/python3.8/dist-packages/dask/dataframe/utils.py:367: FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
_numeric_index_types = (pd.Int64Index, pd.Float64Index, pd.UInt64Index)

[2]:

create_local_dask_cluster()

### Cluster

• Workers: 1
• Cores: 4
• Memory: 28.14 GB

### Analysis parameters¶

The following cell sets important parameters for the analysis:

• shp_path: a location path to the vector file (e.g. folder/input.shp).

• time_range: The date range to analyse (e.g. ('2015', '2019'))

[3]:

shp_path = 'data/lake_ngami.geojson'

time_range = ('2000-12' , '2021-08')

We will also ensure the polygon is in WGS84 coordinates (epsg=4326) using the to_crs() method to make sure it can index the datacube correctly.

[4]:

gdf = gpd.read_file(shp_path).to_crs('epsg:4326')

## Plot the polygon¶

This cell will plot the polygon over a basemap using the deafrica_tools.plotting function map_shapefile

[5]:

#plot the shapefile over a basemap
gdf['id']=0
map_shapefile(gdf, attribute='id', fillOpacity=0.1)

## Run the Wetlands Insight Tool¶

Even for small areas, this code can take a long time to run so you will need to be patient. This is because the tool loads three seperate datasets (Landsat SR, Fractional Cover, and WOfS), and calculates a tasselled cap index on the fly.

If the region is large and/or the time-span is long, the cell below will take a long time to run. Use the Dask Dashboard to monitor the progress.

[6]:

export_name='results/lake_ngami_WIT.csv'

[7]:

df = WIT_drill(
gdf=gdf,
time=time_range,
min_gooddata=0.0,
TCW_threshold=-0.035,
resample_frequency="Q-DEC",
export_csv=export_name,
verbose=False,
)

[7]:

wofs_area_percent wet_percent green_veg_percent dry_veg_percent bare_soil_percent
2000-12-31 0.00 0.11 1.14 31.66 67.09
2001-03-31 0.00 0.48 73.46 6.21 19.85
2001-06-30 0.00 0.00 8.48 76.54 14.98
2001-09-30 0.57 0.11 0.00 83.45 15.87
2001-12-31 0.53 0.31 33.02 46.90 19.23

### Plot the results¶

If you would like to export the plot as a .png file, then set the variable below to True, and give the plot a name (e.g. the wetland’s name)

[8]:

export_plot = True
out_filename = 'results/Lake_Ngami_WIT.png'

[9]:

fontsize = 17

# generate plot

#set up color palette
pal = [sns.xkcd_rgb["cobalt blue"],
sns.xkcd_rgb["neon blue"],
sns.xkcd_rgb["grass"],
sns.xkcd_rgb["beige"],
sns.xkcd_rgb["brown"]]

#make a stacked area plot
plt.clf()
fig=plt.figure(figsize = (22,6))
plt.stackplot(df.index,
df.wofs_area_percent,
df.wet_percent,
df.green_veg_percent,
df.dry_veg_percent,
df.bare_soil_percent,
labels=['open water',
'wet',
'green veg',
'dry veg',
'bare soil',
], colors=pal, alpha = 0.6)

#set axis limits to the min and max
plt.axis(xmin = df.index[0], xmax = df.index[-1], ymin = 0, ymax = 100)
plt.tick_params(labelsize=fontsize)
#add a legend and a tight plot box
plt.legend(loc='lower left', framealpha=0.6, fontsize=fontsize)
plt.title('Fractional Cover, Wetness, and Water', fontsize=fontsize)
plt.tight_layout();
if export_plot:
#save the figure
plt.savefig(out_filename);

<Figure size 432x288 with 0 Axes>

Contact: If you need assistance, please post a question on the Open Data Cube Slack channel or on the GIS Stack Exchange using the open-data-cube tag (you can view previously asked questions here). If you would like to report an issue with this notebook, you can file one on Github.

Compatible datacube version:

[10]:

print(datacube.__version__)

1.8.7

Last Tested:

[11]:

from datetime import datetime
datetime.today().strftime('%Y-%m-%d')

[11]:

'2022-06-21'


text:
## Elementary Technical Mathematics

Convert Numbers to improper fractions reduce the fraction multiply both sides by $\frac{8}{3}$ y=5


text:
# Best Practice to Python Create Directory Recursively – Python Tutorial

By | September 4, 2019

When we are copying, moving or saving files with python, one problem we must concern is: Does the file directory exist? In this tutorial, we will introduce you how to create directories recursively with python.

For example:

If you plan to move file: f:\test.pdf to f:\pdf\1\test.pdf. You must be sure directories pdf and 1 exist. If they do not exist, you will fail to move file.

As to directory f:\pdf\1\, how can we create directory pdf and 1?

## Python create directory recursively

Here we write an example code to create directories recursively in python.

def createDirs(d):
if not os.path.exists(d):
os.makedirs(d)

If directory pdf and 1 do not exist. createDirs() function can create them.

## How to use this function?

createDirs("f:\\pdf\\1")

Then directory pdf and 1 are created.


text:
### Still have math questions?

94. A forest fi e leaves behind an area of grass burned in an expanding circular pattern. If the radius of the circle of burning grass is increasing with time according to the formula $$r ( t ) = 2 t + 1$$ , express the area burned as a function of time, $$t$$ (minutes).
$$(4t^{2}+ 4t+ 1)\pi$$


text:
# Largest eigenvalues of sparse inhomogeneous Erdős-Rényi graphs

Friday, 21 April, 2017

## Published in:

arXiv:1704.02953

We consider inhomogeneous Erd\H{o}s-R\'enyi graphs. We suppose that the maximal mean degree d satisfies d \ll \log n. We characterize the asymptotic behavior of the n1−o(1) largest eigenvalues of the adjacency matrix and its centred version. We prove that these extreme eigenvalues are governed at first order by the largest degrees and, for the adjacency matrix, by the nonzero eigenvalues of the expectation matrix. Our results show that the extreme eigenvalues exhibit a novel behaviour which in particular rules out their convergence to a nondegenerate point process. Together with the companion paper [3], where we analyse the extreme eigenvalues in the complementary regime d \gg \log n, this establishes a crossover in the behaviour of the extreme eigenvalues around d \sim \log n. Our proof relies on a new tail estimate for the Poisson approximation of an inhomogeneous sum of independent Bernoulli random variables, as well as on an estimate on the operator norm of a pruned graph due to Le, Levina, and Vershynin.

## Author(s):

Florent Benaych-Georges
Charles Bordenave
Antti Knowles


text:
# Math Help - Whole numbers multiplied and divided by fractions without a whole number?

1. ## Whole numbers multiplied and divided by fractions without a whole number?

I want to know how for instance, how to multiply/divide the following:

7 over 8 divided by negative 2
negative 7 times one over five

I could just use a calculator figure it out, but I do not know how to times a whole number by a fraction or divide them - please help me understand?

2. Originally Posted by hoiguy1208
I want to know how for instance, how to multiply/divide the following:

7 over 8 divided by negative 2
negative 7 times one over five

I could just use a calculator figure it out, but I do not know how to times a whole number by a fraction or divide them - please help me understand?
$\displaystyle\frac{\left(\frac{7}{8}\right)}{-2}=\frac{7}{8}\left(-\frac{1}{2}\right)=-\frac{7(1)}{8(2)}$

$\displaystyle\ -7\left(\frac{1}{5}\right)=-\frac{7}{1}\left(\frac{1}{5}\right)=-\frac{7}{5}$


text:
Which of the following is true about centrosome

(A)membraneless organelle

(B)contains two centrioles

(C)surrounded by amorphous pericentriolar

(D)All of these


text:
## 5.22 Profinite spaces

Here is the definition.

Definition 5.22.1. A topological space is profinite if it is homeomorphic to a limit of a diagram of finite discrete spaces.

This is not the most convenient characterization of a profinite space.

Lemma 5.22.2. Let $X$ be a topological space. The following are equivalent

1. $X$ is a profinite space, and

2. $X$ is Hausdorff, quasi-compact, and totally disconnected.

If this is true, then $X$ is a cofiltered limit of finite discrete spaces.

Proof. Assume (1). Choose a diagram $i \mapsto X_ i$ of finite discrete spaces such that $X = \mathop{\mathrm{lim}}\nolimits X_ i$. As each $X_ i$ is Hausdorff and quasi-compact we find that $X$ is quasi-compact by Lemma 5.14.5. If $x, x' \in X$ are distinct points, then $x$ and $x'$ map to distinct points in some $X_ i$. Hence $x$ and $x'$ have disjoint open neighbourhoods, i.e., $X$ is Hausdorff. In exactly the same way we see that $X$ is totally disconnected.

Assume (2). Let $\mathcal{I}$ be the set of finite disjoint union decompositions $X = \coprod _{i \in I} U_ i$ with $U_ i$ nonempty open (and closed) for all $i \in I$. For each $I \in \mathcal{I}$ there is a continuous map $X \to I$ sending a point of $U_ i$ to $i$. We define a partial ordering: $I \leq I'$ for $I, I' \in \mathcal{I}$ if and only if the covering corresponding to $I'$ refines the covering corresponding to $I$. In this case we obtain a canonical map $I' \to I$. In other words we obtain an inverse system of finite discrete spaces over $\mathcal{I}$. The maps $X \to I$ fit together and we obtain a continuous map

$X \longrightarrow \mathop{\mathrm{lim}}\nolimits _{I \in \mathcal{I}} I$

We claim this map is a homeomorphism, which finishes the proof. (The final assertion follows too as the partially ordered set $\mathcal{I}$ is directed: given two disjoint union decompositions of $X$ we can find a third refining both.) Namely, the map is injective as $X$ is totally disconnected and hence $\{ x\}$ is the intersection of all open and closed subsets of $X$ containing $x$ (Lemma 5.12.11) and the map is surjective by Lemma 5.12.6. By Lemma 5.17.8 the map is a homeomorphism. $\square$

Proof. Let $i \mapsto X_ i$ be a diagram of profinite spaces over the index category $\mathcal{I}$. Let us use the characterization of profinite spaces in Lemma 5.22.2. In particular each $X_ i$ is Hausdorff, quasi-compact, and totally disconnected. By Lemma 5.14.1 the limit $X = \mathop{\mathrm{lim}}\nolimits X_ i$ exists. By Lemma 5.14.5 the limit $X$ is quasi-compact. Let $x, x' \in X$ be distinct points. Then there exists an $i$ such that $x$ and $x'$ have distinct images $x_ i$ and $x'_ i$ in $X_ i$ under the projection $X \to X_ i$. Then $x_ i$ and $x'_ i$ have disjoint open neighbourhoods in $X_ i$. Taking the inverse images of these opens we conclude that $X$ is Hausdorff. Similarly, $x_ i$ and $x'_ i$ are in distinct connected components of $X_ i$ whence necessarily $x$ and $x'$ must be in distinct connected components of $X$. Hence $X$ is totally disconnected. This finishes the proof. $\square$

Lemma 5.22.4. Let $X$ be a profinite space. Every open covering of $X$ has a refinement by a finite covering $X = \coprod U_ i$ with $U_ i$ open and closed.

Proof. Write $X = \mathop{\mathrm{lim}}\nolimits X_ i$ as a limit of an inverse system of finite discrete spaces over a directed set $I$ (Lemma 5.22.2). Denote $f_ i : X \to X_ i$ the projection. For every point $x = (x_ i) \in X$ a fundamental system of open neighbourhoods is the collection $f_ i^{-1}(\{ x_ i\} )$. Thus, as $X$ is quasi-compact, we may assume we have an open covering

$X = f_{i_1}^{-1}(\{ x_{i_1}\} ) \cup \ldots \cup f_{i_ n}^{-1}(\{ x_{i_ n}\} )$

Choose $i \in I$ with $i \geq i_ j$ for $j = 1, \ldots , n$ (this is possible as $I$ is a directed set). Then we see that the covering

$X = \coprod \nolimits _{t \in X_ i} f_ i^{-1}(\{ t\} )$

refines the given covering and is of the desired form. $\square$

Lemma 5.22.5. Let $X$ be a topological space. If $X$ is quasi-compact and every connected component of $X$ is the intersection of the open and closed subsets containing it, then $\pi _0(X)$ is a profinite space.

Proof. We will use Lemma 5.22.2 to prove this. Since $\pi _0(X)$ is the image of a quasi-compact space it is quasi-compact (Lemma 5.12.7). It is totally disconnected by construction (Lemma 5.7.9). Let $C, D \subset X$ be distinct connected components of $X$. Write $C = \bigcap U_\alpha$ as the intersection of the open and closed subsets of $X$ containing $C$. Any finite intersection of $U_\alpha$'s is another. Since $\bigcap U_\alpha \cap D = \emptyset$ we conclude that $U_\alpha \cap D = \emptyset$ for some $\alpha$ (use Lemmas 5.7.3, 5.12.3 and 5.12.6) Since $U_\alpha$ is open and closed, it is the union of the connected components it contains, i.e., $U_\alpha$ is the inverse image of some open and closed subset $V_\alpha \subset \pi _0(X)$. This proves that the points corresponding to $C$ and $D$ are contained in disjoint open subsets, i.e., $\pi _0(X)$ is Hausdorff. $\square$

Comment #5552 by Curious dilettante on

What is the condition of being cofiltered in Lemma 0ET8 required for? I attempted to carry out the proof and it appears that I did not use this condition. I mean, two different points in the limit are separated by a map to a space of the diagram and then you can lift the desired properties of separation ($T_2$ and totally disconnected) from that space to the limit, right?

Comment #5736 by on

Comment #6209 by Amnon Yekutieli on

The spaces in condition (2) are called Stone spaces in topology literature.

I think (based on recent investigations, and on reading the Scholze-Clausen condensed math notes) that "limit" and "cofiltered limits" should be replaced by "inverse limit of finite discrete spaces". In other words, there is an inverse system (in the most naive sense of directed sets) of finite discrete spaces $(X_{i})$ , $i \in I$ , and $X \cong \lim_{\leftarrow i} X_i$ .

Also, it seems that the set of partitions of a Stone space X into finite disjoint unions of closed (and open) sets is actually codirected (the opposite of directed set), by the relation of refinement. This codirected set accounts for all finite discrete quotients of X. Thus X is the inverse limit of this inverse system.

This is analogous to "open subgroups of finite index" in the case of profinite abelian groups.

Furtheremore, the caegory of Stone spaces (or profinite sets) seems to be equivalent to Pro(Set_{fin}), the cat of pro-objects of the cat of finite sets.

Amnon

Comment #6210 by on

@#6209 In the stacks project we use colimits and limits to distinguish between "projective limits" and "direct limits" and we do not use this terminology. See Section 4.14. Having said this, I think what you say in your second paragraph is the content ot Lemma 5.22.2. What you say in your third paragraph follows easily from Lemmas 5.22.4 and 5.22.2; we just have a different order of the arguments. We don't have the characterization of the category of profinite spaces you mention in your fifth paragraph. I may add this the next time I go through all the comments. Thanks!

Comment #6302 by Zhouhang MAO on

@#6209 Another point that Johan did not address: for any filtered ($\infty$-)category $\mathcal I$, there exists a functor $\mathcal J\to\mathcal I$ where $\mathcal J$ is a directed partially ordered set. See https://kerodon.net/tag/02QA

Comment #6349 by on

@#6302. See also Lemma 4.21.5. Overlap between Kerodon and the Stacks project!

Comment #7082 by Amnon Yekutieli on

In Thm 0.17 of the paper Rings of Bounded Continuous Functions there is a characterization of profinite spaces (aka Stone spaces) in terms of their rings of continuous real valued functions. This provides (see Cor 0.18 there) a conceptual proof that the Stone-Cech compatification of a discrete space is profinite, cf. Example 090C.

In your comment you can use Markdown and LaTeX style mathematics (enclose it like $\pi$). A preview option is available if you wish to see how it works out (just click on the eye in the toolbar).


text:
# Spin spherical harmonics

Not to be confused with Spin-weighted spherical harmonics.

In quantum mechanics, spin spherical harmonics are spinors that are eigenstates of the square of the angular momentum operator, and so are the natural spinorial analog of vector spherical harmonics.

They are given in matrix form by[1]

$Y_{(j-\frac{1}{2},\frac{1}{2})jm}=\left(\begin{array}{c} \sqrt{\frac{j+m}{2j}}Y_{j-\frac{1}{2},m-\frac{1}{2}}\\ \sqrt{\frac{j-m}{2j}}Y_{j-\frac{1}{2},m+\frac{1}{2}}\end{array}\right)$

$Y_{(j+\frac{1}{2},\frac{1}{2})jm}=\left(\begin{array}{c} -\sqrt{\frac{j-m+1}{2j+2}}Y_{j+\frac{1}{2},m-\frac{1}{2}}\\ \sqrt{\frac{j+m+1}{2j+2}}Y_{j+\frac{1}{2},m+\frac{1}{2}}\end{array}\right)$

## Notes

1. ^ Biedenharn, L. C.; Louck, J. D. (1981), Angular momentum in Quantum Physics: Theory and Application, Encyclopedia of Mathematics 8, Reading: Addison-Wesley, p. 283, ISBN 0-201-13507-8


text:
## minima, diffraction

A screen is placed 48.8 cm from a single slit, which is illuminatedwith light of wavelength 648 nm. If the distance between the secondand fourth minima in the diffraction pattern is 2.87 mm, what isthe width of the slit?

I'm having a hard time figuring out what information can begathered when it gives you the minima. Thanks in advance!


text:
# Asymptotic spherical shapes in some spectral optimization problems

created by mazzoleni on 06 Nov 2018

[BibTeX]

Preprint

Inserted: 6 nov 2018
Last Updated: 6 nov 2018

Year: 2018

Abstract:

We study the optimization of the positive principal eigenvalue of an indefinite weighted problem, associated with the Neumann Laplacian in a box $\Omega\subset\mathbb{R}^N$, which arises in the investigation of the survival threshold in population dynamics. When trying to minimize such eigenvalue with respect to the weight, one is lead to consider a shape optimization problem, which is known to admit no spherical optimal shapes (despite some previously stated conjectures). We investigate whether spherical shapes can be recovered in some singular perturbation limit. More precisely we show that, whenever the negative part of the weight diverges, the above shape optimization problem approaches in the limit the so called spectral drop problem, which involves the minimization of the first eigenvalue of the mixed Dirichlet-Neumann Laplacian. We prove that, for suitable choices of the box $\Omega$, the optimal shapes for this second problem are indeed spherical; moreover, for general $\Omega$, we show that small volume spectral drops are asymptotically spherical, with center at points of $\partial\Omega$ having large mean curvature.


text:
📝 Exercise M6.01#

The aim of this notebook is to investigate if we can tune the hyperparameters of a bagging regressor and evaluate the gain obtained.

We will load the California housing dataset and split it into a training and a testing set.

from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split

data, target = fetch_california_housing(as_frame=True, return_X_y=True)
target *= 100  # rescale the target in k\$
data_train, data_test, target_train, target_test = train_test_split(
data, target, random_state=0, test_size=0.5)

Note

If you want a deeper overview regarding this dataset, you can refer to the Appendix - Datasets description section at the end of this MOOC.

Create a BaggingRegressor and provide a DecisionTreeRegressor to its parameter base_estimator. Train the regressor and evaluate its generalization performance on the testing set using the mean absolute error.

# Write your code here.

Now, create a RandomizedSearchCV instance using the previous model and tune the important parameters of the bagging regressor. Find the best parameters and check if you are able to find a set of parameters that improve the default regressor still using the mean absolute error as a metric.

Tip

You can list the bagging regressor’s parameters using the get_params method.

# Write your code here.


text:
Home > Absolute Error > Relative And Absolute Error Examples

# Relative And Absolute Error Examples

## Contents

When you have estimated the error, you will know how many significant figures to use in reporting your result. Absolute error is positive. This simple equation tells you how far off you were in comparison to the overall measurement. Please select a newsletter. http://supercgis.com/absolute-error/relative-absolute-error-examples.html

Machines used in manufacturing often set tolerance intervals, or ranges in which product measurements will be tolerated or accepted before they are considered flawed. Weisstein, http://mathworld.wolfram.com/RelativeError.html. The goal of a good experiment is to reduce the systematic errors to a value smaller than the random errors. What are the absolute and relative errors of the approximation 3.14 to the value π? http://www.regentsprep.org/regents/math/algebra/am3/LError.htm

## Absolute Error Formula

Note, however that this doesn't make sense when giving percentages, as your error is not 10% of 2 feet. Any measurements within this range are "tolerated" or perceived as correct. If the absolute error was 3.52, is this significant?

No matter what the source of the uncertainty, to be labeled "random" an uncertainty must have the property that the fluctuations from some "true" value are equally likely to be positive The error is a smaller percentage of the total measurement.[8] 2ft20ft=.1feet{\displaystyle {\frac {2ft}{20ft}}=.1feet} .1∗100=10%{\displaystyle .1*100=10\%} Relative Error. 3 Calculate Relative Error all at once by turning the numerator (top of fraction) The smaller the unit, or fraction of a unit, on the measuring device, the more precisely the device can measure. Absolute Error Definition ISBN 81-297-0731-4 External links Weisstein, Eric W. "Percentage error".

The precision is said to be the same as the smallest fractional or decimal division on the scale of the measuring instrument. Relative Error Formula The relative error is the absolute error divided by the magnitude of the exact value. If you are measuring a football field and the absolute error is 1 cm, the error is virtually irrelevant. https://www.mathsisfun.com/measure/error-measurement.html The Relative Error is the Absolute Error divided by the actual measurement.

For example, you measure a length to be 3.4 cm. Relative Error Chemistry b.) The relative error in the length of the field is c.) The percentage error in the length of the field is 3. The relative error is usually more significant than the absolute error. Random errors Random errors arise from the fluctuations that are most easily observed by making multiple trials of a given measurement.

## Relative Error Formula

The actual length of this field is 500 feet. Another word for this variation - or uncertainty in measurement - is "error." This "error" is not the same as a "mistake." It does not mean that you got the wrong Absolute Error Formula This would be a conservative assumption, but it overestimates the uncertainty in the result. What Is Absolute Error Simply substitute the equation for Absolute Error in for the actual number.

The length of a table in the laboratory is not well defined after it has suffered years of use. his comment is here Secondly, relative error only makes sense when measured on a ratio scale, (i.e. Ways to Improve Accuracy in Measurement 1. In principle, you should by one means or another estimate the uncertainty in each measurement that you make. Absolute Error Formula Chemistry

To record this measurement as either 0.4 or 0.42819667 would imply that you only know it to 0.1 m in the first case or to 0.00000001 m in the second. You might also enjoy: Sign up There was an error. Greatest Possible Error: Because no measurement is exact, measurements are always made to the "nearest something", whether it is stated or not. this contact form Please try the request again.

Absolute Accuracy Error Example: 25.13 mL - 25.00 mL = +0.13 mL absolute error Relative Accuracy Error Example: (( 25.13 mL - 25.00 mL)/25.00 mL) x 100% = 0.52% Absolute Error And Relative Error In Numerical Analysis Relative values are ratios, and have no units. For this same case, when the temperature is given in Kelvin, the same 1° absolute error with the same true value of 275.15 K gives a relative error of 3.63×10−3 and

## Sometimes the quantity you measure is well defined but is subject to inherent random fluctuations.

c.) the percentage error in the measured length of the field Answer: a.) The absolute error in the length of the field is 8 feet. If v ≠ 0 , {\displaystyle v\neq 0,} the relative error is η = ϵ | v | = | v − v approx v | = | 1 − v For example, you measure a length to be 3.4 cm. Relative Error Definition Topic Index | Algebra Index | Regents Exam Prep Center Created by Donna Roberts

Error in Measurement Topic Index | Algebra Index |

You would find different lengths if you measured at different points on the table. This document contains brief discussions about how errors are reported, the kinds of errors that can occur, how to estimate random errors, and how to carry error estimates into calculated results. For example a meter stick should have been manufactured such that the millimeter markings are positioned much more accurately than one millimeter. navigate here In plain English: The absolute error is the difference between the measured value and the actual value. (The absolute error will have the same unit label as the measured quantity.) Relative

What is the maximum absolute and relative error of voltage? For example, if you're measuring something with a meter stick, the smallest unit marked on the meter stick is 1 millimeter (mm). Another example would be if you measured a beaker and read 5mL. Note: as before, the stated voltage is an approximation of the actual voltage.

To do so, simply subtract the measured value from the expected one. Thanks, You're in! The first gives how large the error is, while the second gives how large the error is relative to the correct value. This tells you what percentage of the final measurement you messed up by.

A low relative error is, of course, desirable. On occasion, the relative error by 100 and refer to as the percent relative error.


text:
Probability of exactly R runs of M consecutive success

I know how to compute the probability that 0 runs (or one or more) of M consecutive successes will occur in a sequence of N trials, each with probability P of success.

Is there an explicit formula (mass function) or a method of calculation (for example in Maple) to find the probability that exactly R runs of M consecutive successes will occur in a sequence of N trials, each trial with probability P of success?

I have the same question about the cumulative distribution function?

-
To clarify: in FFSSSF (S = success) how many "runs of M=2 consecutive successes" you'd count ? 2? 0? –  leonbloy Jan 25 '11 at 17:44
I count SSS 2 successes –  Jean-Pierre Jan 26 '11 at 2:20

Let's first count the probability that exactly $R$ runs of $M$ successes happen in $N$ trials, and the last trial fails. Denote by $p$ the probability of success, and by $q$ the probability of failure.
Each set of runs can be "tokenized" into parts of the form $S^kF$ (this is because the last trial fails). If $k < M$ then we count no runs, and otherwise we count $k - M + 1$ runs. We're going to form a bivariate generating function, with $x$ being the total number of trials, and $y$ being the number of runs. Each part contributes the following: $$P(x,y) = xq(1 + px + \cdots + (px)^{M-1} + (px)^My + (px)^{M+1}y^2 + \cdots).$$ We can get an explicit formula for $P(x,y)$ using geometric series identities: $$P(x,y) = xq\left(\frac{(px)^{M-1}-1}{px-1} + \frac{(px)^{M-1}}{1-pxy}\right).$$ The required probability is the coefficient of $x^N y^R$ in $1/(1-P(x,y))$.
Now let's do the same when the last trial succeeds. The last piece is then $P(x,y)/xq$, and so we want the coefficient of $x^N y^R$ in $(P(x,y)/xq)/(1-P(x,y))$. In total, the required probability is the coefficient of $x^N y^R$ in $$\frac{1+P(x,y)/xq}{1-P(x,y)}.$$
Seems like the two formulas for $P(x,y)$ don't agree. Shouldn't the two fractions in the explicit formula be $((px)^M-1)/(px-1)$ and $(px)^M y/(1-pxy)$? –  mjqxxxx Jan 26 '11 at 4:49