text:
# Yield-to-Maturity and its assumption

Reading about Yield-to-Maturity (YTM) I found out that two assumptions have to be made:

• the bond holder must keep the bond until maturity;
• coupons must be reinvested at the same YTM. Violating those hypotheses causes the bond holder incurring in two types of risks: the price risk and the reinvestment risk.

So I was wondering: since interest rates are ever-changing, will a bond holder be automatically subject to the reinvestment risk? It is hard for me envisaging that he can still reinvest his coupons at the same YTM.

I kindly ask you where my reasoning fails.

• Yes, the bond holder will have reinvestment risk because X could be anything on a coupon date. If you want no reinvestment risk you could buy a zero coupon bond.
– dm63
Apr 14, 2017 at 12:49

It's simpler to just think of the yield to maturity as the internal rate of return of the bond given the current price. It's like the discount rate you would apply to the final payout and coupons, such that the result is the market price.

A short paper by Forbes, Hatem, and Paul explains that yield to maturity ignores reinvestment. Strictly speaking, yield to maturity is an internal rate of return, not the return you would get at the horizon. The return you get at the horizon depends on the reinvestment policy. The return at the horizon only matches the yield to maturity if the coupons are invested at the same yield as the yield to maturity.

For instance, assume a \$1000 bond with \$50 annual payments and 2 years until maturity and a 10% yield to maturity. The current price is \$913.22. The sum of the return and the coupons is \$1100. Ignoring reinvestment, the return at the end of two years is 20% cumulatively (1100 / 913.22 - 1) or 9.75% annualized. However, if the \$50 from year 1 is re-invested at a 10% rate, then the investor would now have$1105, generating a 21% return cumulatively (1105 / 913.22 - 1) or 10% annualized.

Long story short, the yield to maturity is a bond's internal rate of return given its current price. It is only the return you would earn if you held the bond to maturity if you reinvest at that same rate.

Yes, its called yield to maturity because you hove to hold it till maturity.

For the reinvestment risk, suppose the day you get the coupon the coupon the day bond is trading at x%. Now with the money you get from the coupon, you can buy these bonds and realize x%. So the reinvestment risk is eliminated to the extent of the liquidity in the market.

• Fine. However, the probability that the percentage x% matches the YTM is low, isn't it? That's what I want to understand. Mar 15, 2017 at 11:41
• right, so you can invest at the current YTM which might not be equal to the yield at which was the bond was issued ! Mar 15, 2017 at 12:12

Just to add two things:

• in fixed income world bonds are typically quoted by YTM instead of prices but if we go back to prices, then an analogous question would be: what conditions need to hold so that an investor who bought a bond at price X will realize the rate of return (YTM) of Y%?

• I see YTM only in terms of a measure that allows me to compare similar bonds - the realized rate of return is a different thing

• also: YTM assumes a flat yield curve so this is quite far from reality

• Two things: 1) Most bond markets are actually quoted on a price basis and yields are derived from price, not the other way around. 2) If you plot subsequently realized bond return vs starting yield levels, they track each other extremely well, even if you're rolling your bonds monthly. So YTM is actually a very good expected return measure for default-free bonds. Apr 15, 2017 at 19:20

I think for not too large yield movements , you would earn the ytm by holding for bond duration (rather than till maturity) and reinvesting coupons in bond

• i think what i wrote is correct... it can be checked in a spreadsheet Apr 25, 2017 at 14:10


text:
# 'Textheight' is not a member of 'System.Windows.Forms.PictureBox'

I am converting an old VB6 project to .NET. In the VB6 Project .TextHeight and .TextWidth were used for a picture box object. After I migrated the project from VB6 to .NET the compiler threw the error 'Textheight' is not a member of 'System.Windows.Forms.PictureBox'

tmpX = tmpX - (p.TextWidth(tmpStr) / 2.0#)

and

tmpY = tmpY - (p.Textheight(tmpStr) / 2.0#)

What is the .NET equivalent for these?

-
I'm not much familiar with VB6 but I think that the title and the question itself has no sense, a picturebox control in VB.NET displays pictures, not text, it has not text properties. what object type if "p"? –  ElektroStudios Oct 21 '13 at 20:07
@ElektroStudios pictureboxs could do this in VB6, and the title is the error message that studio gives me after the migration –  nate Oct 21 '13 at 20:10
Graphics.MeasureString –  jac Oct 21 '13 at 20:10
I'd look into TextRenderer.MeasureText. –  Jason Tyler Oct 21 '13 at 20:12

       txtPrompt.Height = Convert.ToInt32(Me.CreateGraphics().MeasureString(txtPrompt.Text, txtPrompt.Font, txtPrompt.Width).Height + 15)

format: plain


text:
TraceClasses {methods} R Documentation

## Classes Used Internally to Control Tracing

### Description

The classes described here are used by the R function trace to create versions of functions and methods including browser calls, etc., and also to untrace the same objects.

### Usage

### Objects from the following classes are generated
### by calling trace() on an object from the corresponding
### class without the "WithTrace" in the name.

"functionWithTrace"
"MethodDefinitionWithTrace"
"MethodWithNextWithTrace"
"genericFunctionWithTrace"
"groupGenericFunctionWithTrace"

### the following is a virtual class extended by each of the
### classes above

"traceable"

### Objects from the Class

Objects will be created from these classes by calls to trace. (There is an initialize method for class "traceable", but you are unlikely to need it directly.)

### Slots

.Data:

The data part, which will be "function" for class "functionWithTrace", and similarly for the other classes.

original:

Object of the original class; e.g., "function" for class "functionWithTrace".

### Extends

Each of the classes extends the corresponding untraced class, from the data part; e.g., "functionWithTrace" extends "function". Each of the specific classes extends "traceable", directly, and class "VIRTUAL", by class "traceable".

### Methods

The point of the specific classes is that objects generated from them, by function trace(), remain callable or dispatchable, in addition to their new trace information.

function trace


text:
# Federated Reconstruction for Matrix Factorization

Stay organized with collections Save and categorize content based on your preferences.

This tutorial explores partially local federated learning, where some client parameters are never aggregated on the server. This is useful for models with user-specific parameters (e.g. matrix factorization models) and for training in communication-limited settings. We build on concepts introduced in the Federated Learning for Image Classification tutorial; as in that tutorial, we introduce high-level APIs in tff.learning for federated training and evaluation.

We begin by motivating partially local federated learning for matrix factorization. We describe Federated Reconstruction (paper, blog post), a practical algorithm for partially local federated learning at scale. We prepare the MovieLens 1M dataset, build a partially local model, and train and evaluate it.

!pip install --quiet --upgrade tensorflow-federated

import nest_asyncio
nest_asyncio.apply()

import collections
import functools
import io
import os
import requests
import zipfile
from typing import List, Optional, Tuple

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import tensorflow as tf
import tensorflow_federated as tff

np.random.seed(42)

## Background: Matrix Factorization

Matrix factorization has been a historically popular technique for learning recommendations and embedding representations for items based on user interactions. The canonical example is movie recommendation, where there are $$n$$ users and $$m$$ movies, and users have rated some movies. Given a user, we use their rating history and the ratings of similar users to predict the user's ratings for movies they haven't seen. If we have a model that can predict ratings, it's easy to recommend users new movies that they'll enjoy.

For this task, it's useful to represent users' ratings as an $$n \times m$$ matrix $$R$$:

This matrix is generally sparse, since users typically only see a small fraction of the movies in the dataset. The output of matrix factorization is two matrices: an $$n \times k$$ matrix $$U$$ representing $$k$$-dimensional user embeddings for each user, and an $$m \times k$$ matrix $$I$$ representing $$k$$-dimensional item embeddings for each item. The simplest training objective is to ensure that the dot product of user and item embeddings are predictive of observed ratings $$O$$:

$argmin_{U,I} \sum_{(u, i) \in O} (R_{ui} - U_u I_i^T)^2$

This is equivalent to minimizing the mean squared error between observed ratings and ratings predicted by taking the dot product of the corresponding user and item embeddings. Another way to interpret this is that this ensures that $$R \approx UI^T$$ for known ratings, hence "matrix factorization". If this is confusing, don't worry–we won't need to know the details of matrix factorization for the rest of the tutorial.

## Exploring MovieLens Data

Let's start by loading the MovieLens 1M data, which consists of 1,000,209 movie ratings from 6040 users on 3706 movies.

def download_movielens_data(dataset_path):
if dataset_path.startswith('http'):
r = requests.get(dataset_path)
z = zipfile.ZipFile(io.BytesIO(r.content))
z.extractall(path='/tmp')
else:
tf.io.gfile.makedirs('/tmp/ml-1m/')
for filename in ['ratings.dat', 'movies.dat', 'users.dat']:
tf.io.gfile.copy(
os.path.join(dataset_path, filename),
os.path.join('/tmp/ml-1m/', filename),
overwrite=True)

def load_movielens_data(
data_directory: str = "/tmp",
) -> Tuple[pd.DataFrame, pd.DataFrame]:
"""Loads pandas DataFrames for ratings, movies, users from data directory."""
# Load pandas DataFrames from data directory. Assuming data is formatted as
os.path.join(data_directory, "ml-1m", "ratings.dat"),
sep="::",
names=["UserID", "MovieID", "Rating", "Timestamp"], engine="python")
os.path.join(data_directory, "ml-1m", "movies.dat"),
sep="::",
names=["MovieID", "Title", "Genres"], engine="python")

# Create dictionaries mapping from old IDs to new (remapped) IDs for both
# MovieID and UserID. Use the movies and users present in ratings_df to
# determine the mapping, since movies and users without ratings are unneeded.
movie_mapping = {
old_movie: new_movie for new_movie, old_movie in enumerate(
ratings_df.MovieID.astype("category").cat.categories)
}
user_mapping = {
old_user: new_user for new_user, old_user in enumerate(
ratings_df.UserID.astype("category").cat.categories)
}

# Map each DataFrame consistently using the now-fixed mapping.
ratings_df.MovieID = ratings_df.MovieID.map(movie_mapping)
ratings_df.UserID = ratings_df.UserID.map(user_mapping)
movies_df.MovieID = movies_df.MovieID.map(movie_mapping)

# Remove nulls resulting from some movies being in movies_df but not
# ratings_df.
movies_df = movies_df[pd.notnull(movies_df.MovieID)]

return ratings_df, movies_df

Let's load and explore a couple Pandas DataFrames containing the rating and movie data.

ratings_df, movies_df = load_movielens_data()

We can see that each rating example has a rating from 1-5, a corresponding UserID, a corresponding MovieID, and a timestamp.

ratings_df.head()

Each movie has a title and potentially multiple genres.

movies_df.head()

It's always a good idea to understand basic statistics of the dataset:

print('Num users:', len(set(ratings_df.UserID)))
print('Num movies:', len(set(ratings_df.MovieID)))

Num users: 6040
Num movies: 3706

ratings = ratings_df.Rating.tolist()

plt.hist(ratings, bins=5)
plt.xticks([1, 2, 3, 4, 5])
plt.ylabel('Count')
plt.xlabel('Rating')
plt.show()

print('Average rating:', np.mean(ratings))
print('Median rating:', np.median(ratings))

Average rating: 3.581564453029317
Median rating: 4.0

We can also plot the most popular movie genres.

movie_genres_list = movies_df.Genres.tolist()
# Count the number of times each genre describes a movie.
genre_count = collections.defaultdict(int)
for genres in movie_genres_list:
curr_genres_list = genres.split('|')
for genre in curr_genres_list:
genre_count[genre] += 1
genre_name_list, genre_count_list = zip(*genre_count.items())

plt.figure(figsize=(11, 11))
plt.pie(genre_count_list, labels=genre_name_list)
plt.title('MovieLens Movie Genres')
plt.show()

This data is naturally partitioned into ratings from different users, so we'd expect some heterogeneity in data between clients. Below we display the most commonly rated movie genres for different users. We can observe significant differences between users.

def print_top_genres_for_user(ratings_df, movies_df, user_id):
"""Prints top movie genres for user with ID user_id."""
user_ratings_df = ratings_df[ratings_df.UserID == user_id]
movie_ids = user_ratings_df.MovieID

genre_count = collections.Counter()
for movie_id in movie_ids:
genres_string = movies_df[movies_df.MovieID == movie_id].Genres.tolist()[0]
for genre in genres_string.split('|'):
genre_count[genre] += 1

print(f'\nFor user {user_id}:')
for (genre, freq) in genre_count.most_common(5):
print(f'{genre} was rated {freq} times')

print_top_genres_for_user(ratings_df, movies_df, user_id=0)
print_top_genres_for_user(ratings_df, movies_df, user_id=10)
print_top_genres_for_user(ratings_df, movies_df, user_id=19)

For user 0:
Drama was rated 21 times
Children's was rated 20 times
Animation was rated 18 times
Musical was rated 14 times
Comedy was rated 14 times

For user 10:
Comedy was rated 84 times
Drama was rated 54 times
Romance was rated 22 times
Thriller was rated 18 times
Action was rated 9 times

For user 19:
Action was rated 17 times
Sci-Fi was rated 9 times
Thriller was rated 9 times
Drama was rated 6 times
Crime was rated 5 times

## Preprocessing MovieLens Data

We'll now prepare the MovieLens dataset as a list of tf.data.Datasets representing each user's data for use with TFF.

We implement two functions:

• create_tf_datasets: takes our ratings DataFrame and produces a list of user-split tf.data.Datasets.
• split_tf_datasets: takes a list of datasets and splits them into train/val/test by user, so the val/test sets contain only ratings from users unseen during training. Typically in standard centralized matrix factorization we actually split so that the val/test sets contain held-out ratings from seen users, since unseen users don't have user embeddings. In our case, we'll see later that the approach we use to enable matrix factorization in FL also enables quickly reconstructing user embeddings for unseen users.
def create_tf_datasets(ratings_df: pd.DataFrame,
batch_size: int = 1,
max_examples_per_user: Optional[int] = None,
max_clients: Optional[int] = None) -> List[tf.data.Dataset]:
"""Creates TF Datasets containing the movies and ratings for all users."""
num_users = len(set(ratings_df.UserID))
# Optionally limit to max_clients to speed up data loading.
if max_clients is not None:
num_users = min(num_users, max_clients)

def rating_batch_map_fn(rating_batch):
"""Maps a rating batch to an OrderedDict with tensor values."""
# Each example looks like: {x: movie_id, y: rating}.
# We won't need the UserID since each client will only look at their own
# data.
return collections.OrderedDict([
("x", tf.cast(rating_batch[:, 1:2], tf.int64)),
("y", tf.cast(rating_batch[:, 2:3], tf.float32))
])

tf_datasets = []
for user_id in range(num_users):
# Get subset of ratings_df belonging to a particular user.
user_ratings_df = ratings_df[ratings_df.UserID == user_id]

tf_dataset = tf.data.Dataset.from_tensor_slices(user_ratings_df)

# Define preprocessing operations.
tf_dataset = tf_dataset.take(max_examples_per_user).shuffle(
buffer_size=max_examples_per_user, seed=42).batch(batch_size).map(
rating_batch_map_fn,
num_parallel_calls=tf.data.experimental.AUTOTUNE)
tf_datasets.append(tf_dataset)

return tf_datasets

def split_tf_datasets(
tf_datasets: List[tf.data.Dataset],
train_fraction: float = 0.8,
val_fraction: float = 0.1,
) -> Tuple[List[tf.data.Dataset], List[tf.data.Dataset], List[tf.data.Dataset]]:
"""Splits a list of user TF datasets into train/val/test by user.
"""
np.random.seed(42)
np.random.shuffle(tf_datasets)

train_idx = int(len(tf_datasets) * train_fraction)
val_idx = int(len(tf_datasets) * (train_fraction + val_fraction))

# Note that the val and test data contains completely different users, not
# just unseen ratings from train users.
return (tf_datasets[:train_idx], tf_datasets[train_idx:val_idx],
tf_datasets[val_idx:])

# We limit the number of clients to speed up dataset creation. Feel free to pass
# max_clients=None to load all clients' data.
tf_datasets = create_tf_datasets(
ratings_df=ratings_df,
batch_size=5,
max_examples_per_user=300,
max_clients=2000)

# Split the ratings into training/val/test by client.
tf_train_datasets, tf_val_datasets, tf_test_datasets = split_tf_datasets(
tf_datasets,
train_fraction=0.8,
val_fraction=0.1)

As a quick check, we can print a batch of training data. We can see that each individual example contains a MovieID under the "x" key and a rating under the "y" key. Note that we won't need the UserID since each user only sees their own data.

print(next(iter(tf_train_datasets[0])))

OrderedDict([('x', <tf.Tensor: shape=(5, 1), dtype=int64, numpy=
array([[1907],
[2891],
[1574],
[2785],
[2775]])>), ('y', <tf.Tensor: shape=(5, 1), dtype=float32, numpy=
array([[3.],
[3.],
[3.],
[4.],
[3.]], dtype=float32)>)])

We can plot a histogram showing the number of ratings per user.

def count_examples(curr_count, batch):
return curr_count + tf.size(batch['x'])

num_examples_list = []
# Compute number of examples for every other user.
for i in range(0, len(tf_train_datasets), 2):
num_examples = tf_train_datasets[i].reduce(tf.constant(0), count_examples).numpy()
num_examples_list.append(num_examples)

plt.hist(num_examples_list, bins=10)
plt.ylabel('Count')
plt.xlabel('Number of Examples')
plt.show()

Now that we've loaded and explored the data, we'll discuss how to bring matrix factorization to federated learning. Along the way, we'll motivate partially local federated learning.

## Bringing Matrix Factorization to FL

While matrix factorization has been traditionally used in centralized settings, it's especially relevant in federated learning: user ratings may live on separate client devices, and we may want to learn embeddings and recommendations for users and items without centralizing the data. Since each user has a corresponding user embedding, it's natural to have each client store their user embedding–this scales much better than a central server storing all the user embeddings.

One proposal for bringing matrix factorization to FL goes as follows:

1. The server stores and sends the item matrix $$I$$ to sampled clients each round
2. Clients update the item matrix and their personal user embedding $$U_u$$ using SGD on the above objective
3. Updates to $$I$$ are aggregated on the server, updating the server copy of $$I$$ for the next round

This approach is partially local–that is, some client parameters are never aggregated by the server. Though this approach is appealing, it requires clients to maintain state across rounds, namely their user embeddings. Stateful federated algorithms are less appropriate for cross-device FL settings: in these settings the population size is often much larger than the number of clients that participate in each round, and a client usually participates at most once during the training process. Besides relying on state that may not be initialized, stateful algorithms can result in performance degradation in cross-device settings due to state getting stale when clients are infrequently sampled. Importantly, in the matrix factorization setting, a stateful algorithm leads to all unseen clients missing trained user embeddings, and in large-scale training the majority of users may be unseen. For more on the motivation for stateless algorithms in cross-device FL, see Wang et al. 2021 Sec. 3.1.1 and Reddi et al. 2020 Sec. 5.1.

Federated Reconstruction (Singhal et al. 2021) is a stateless alternative to the aforementioned approach. The key idea is that instead of storing user embeddings across rounds, clients reconstruct user embeddings when needed. When FedRecon is applied to matrix factorization, training proceeds as follows:

1. The server stores and sends the item matrix $$I$$ to sampled clients each round
2. Each client freezes $$I$$ and trains their user embedding $$U_u$$ using one or more steps of SGD (reconstruction)
3. Each client freezes $$U_u$$ and trains $$I$$ using one or more steps of SGD
4. Updates to $$I$$ are aggregated across users, updating the server copy of $$I$$ for the next round

This approach does not require clients to maintain state across rounds. The authors also show in the paper that this method leads to fast reconstruction of user embeddings for unseen clients (Sec. 4.2, Fig. 3, and Table 1), allowing the majority of clients who do not participate in training to have a trained model, enabling recommendations for these clients. See the Federated Reconstruction Google AI Blog post for more key results.

## Defining the Model

We'll next define the local matrix factorization model to be trained on client devices. This model will include the full item matrix $$I$$ and a single user embedding $$U_u$$ for client $$u$$. Note that clients will not need to store the full user matrix $$U$$.

We'll define the following:

class UserEmbedding(tf.keras.layers.Layer):
"""Keras layer representing an embedding for a single user, used below."""

def __init__(self, num_latent_factors, **kwargs):
super().__init__(**kwargs)
self.num_latent_factors = num_latent_factors

def build(self, input_shape):
shape=(1, self.num_latent_factors),
initializer='uniform',
dtype=tf.float32,
name='UserEmbeddingKernel')
super().build(input_shape)

def call(self, inputs):
return self.embedding

def compute_output_shape(self):
return (1, self.num_latent_factors)

def get_matrix_factorization_model(
num_items: int,
num_latent_factors: int) -> tff.learning.reconstruction.Model:
"""Defines a Keras matrix factorization model."""
# Layers with variables will be partitioned into global and local layers.
# We'll pass this to tff.learning.reconstruction.from_keras_model.
global_layers = []
local_layers = []

# Extract the item embedding.
item_input = tf.keras.layers.Input(shape=[1], name='Item')
item_embedding_layer = tf.keras.layers.Embedding(
num_items,
num_latent_factors,
name='ItemEmbedding')
global_layers.append(item_embedding_layer)
flat_item_vec = tf.keras.layers.Flatten(name='FlattenItems')(
item_embedding_layer(item_input))

# Extract the user embedding.
user_embedding_layer = UserEmbedding(
num_latent_factors,
name='UserEmbedding')
local_layers.append(user_embedding_layer)

# The item_input never gets used by the user embedding layer,
# but this allows the model to directly use the user embedding.
flat_user_vec = user_embedding_layer(item_input)

# Compute the dot product between the user embedding, and the item one.
pred = tf.keras.layers.Dot(
1, normalize=False, name='Dot')([flat_user_vec, flat_item_vec])

input_spec = collections.OrderedDict(
x=tf.TensorSpec(shape=[None, 1], dtype=tf.int64),
y=tf.TensorSpec(shape=[None, 1], dtype=tf.float32))

model = tf.keras.Model(inputs=item_input, outputs=pred)

return tff.learning.reconstruction.from_keras_model(
keras_model=model,
global_layers=global_layers,
local_layers=local_layers,
input_spec=input_spec)

Analagous to the interface for Federated Averaging, the interface for Federated Reconstruction expects a model_fn with no arguments that returns a tff.learning.reconstruction.Model.

# This will be used to produce our training process.
# User and item embeddings will be 50-dimensional.
model_fn = functools.partial(
get_matrix_factorization_model,
num_items=3706,
num_latent_factors=50)

We'll next define loss_fn and metrics_fn, where loss_fn is a no-argument function returning a Keras loss to use to train the model, and metrics_fn is a no-argument function returning a list of Keras metrics for evaluation. These are needed to build the training and evaluation computations.

We'll use Mean Squared Error as the loss, as mentioned above. For evaluation we'll use rating accuracy (when the model's predicted dot product is rounded to the nearest whole number, how often does it match the label rating?).

class RatingAccuracy(tf.keras.metrics.Mean):
"""Keras metric computing accuracy of reconstructed ratings."""

def __init__(self,
name: str = 'rating_accuracy',
**kwargs):
super().__init__(name=name, **kwargs)

def update_state(self,
y_true: tf.Tensor,
y_pred: tf.Tensor,
sample_weight: Optional[tf.Tensor] = None):
absolute_diffs = tf.abs(y_true - y_pred)
# A [batch_size, 1] tf.bool tensor indicating correctness within the
# threshold for each example in a batch. A 0.5 threshold corresponds
# to correctness when predictions are rounded to the nearest whole
# number.
example_accuracies = tf.less_equal(absolute_diffs, 0.5)
super().update_state(example_accuracies, sample_weight=sample_weight)

loss_fn = lambda: tf.keras.losses.MeanSquaredError()
metrics_fn = lambda: [RatingAccuracy()]

## Training and Evaluation

Now we have everything we need to define the training process. One important difference from the interface for Federated Averaging is that we now pass in a reconstruction_optimizer_fn, which will be used when reconstructing local parameters (in our case, user embeddings). It's generally reasonable to use SGD here, with a similar or slightly lower learning rate than the client optimizer learning rate. We provide a working configuration below. This hasn't been carefully tuned, so feel free to play around with different values.

Check out the documentation for more details and options.

# We'll use this by doing:
# state = training_process.initialize()
# state, metrics = training_process.next(state, federated_train_data)
training_process = tff.learning.reconstruction.build_training_process(
model_fn=model_fn,
loss_fn=loss_fn,
metrics_fn=metrics_fn,
server_optimizer_fn=lambda: tf.keras.optimizers.SGD(1.0),
client_optimizer_fn=lambda: tf.keras.optimizers.SGD(0.5),
reconstruction_optimizer_fn=lambda: tf.keras.optimizers.SGD(0.1))

We can also define a computation for evaluating our trained global model.

# We'll use this by doing:
# eval_metrics = evaluation_computation(state.model, tf_val_datasets)
# where state is the state from the training process above.
evaluation_computation = tff.learning.reconstruction.build_federated_evaluation(
model_fn,
loss_fn=loss_fn,
metrics_fn=metrics_fn,
reconstruction_optimizer_fn=functools.partial(
tf.keras.optimizers.SGD, 0.1))

We can initialize the training process state and examine it. Most importantly, we can see that this server state only stores item variables (currently randomly initialized) and not any user embeddings.

state = training_process.initialize()
print(state.model)
print('Item variables shape:', state.model.trainable[0].shape)

ModelWeights(trainable=[array([[-0.02840446,  0.01196523, -0.01864688, ...,  0.03020107,
0.00121176,  0.00146852],
[ 0.01330637,  0.04741272, -0.01487445, ..., -0.03352419,
0.0104811 ,  0.03506917],
[-0.04132779,  0.04883525, -0.04799002, ...,  0.00246904,
0.00586842,  0.01506213],
...,
[ 0.0216659 ,  0.00734354,  0.00471039, ...,  0.01596491,
-0.00220431, -0.01559857],
[-0.00319657, -0.01740328,  0.02808609, ..., -0.00501985,
-0.03850871, -0.03844522],
[ 0.03791947, -0.00035037,  0.04217024, ...,  0.00365371,
0.00283421,  0.00897921]], dtype=float32)], non_trainable=[])
Item variables shape: (3706, 50)

We can also try to evaluate our randomly initialized model on validation clients. Federated Reconstruction evaluation here involves the following:

1. The server sends the item matrix $$I$$ to sampled evaluation clients
2. Each client freezes $$I$$ and trains their user embedding $$U_u$$ using one or more steps of SGD (reconstruction)
3. Each client calculates loss and metrics using the server $$I$$ and reconstructed $$U_u$$ on an unseen portion of their local data
4. Losses and metrics are averaged across users to calculate overall loss and metrics

Note that steps 1 and 2 are the same as for training. This connection is important, since training the same way we evaluate leads to a form of meta-learning, or learning how to learn. In this case, the model is learning how to learn global variables (item matrix) that lead to performant reconstruction of local variables (user embeddings). For more on this, see Sec. 4.2 of the paper.

It's also important for steps 2 and 3 to be performed using disjoint portions of clients' local data, to ensure fair evaluation. By default, both the training process and evaluation computation use every other example for reconstruction and use the other half post-reconstruction. This behavior can be customized using the dataset_split_fn argument (we'll explore this further later).

# We shouldn't expect good evaluation results here, since we haven't trained
# yet!
eval_metrics = evaluation_computation(state.model, tf_val_datasets)
print('Initial Eval:', eval_metrics['eval'])

Initial Eval: OrderedDict([('loss', 14.340279), ('rating_accuracy', 0.0)])

We can next try running a round of training. To make things more realistic, we'll sample 50 clients per round randomly without replacement. We should still expect train metrics to be poor, since we're only doing one round of training.

federated_train_data = np.random.choice(tf_train_datasets, size=50, replace=False).tolist()
state, metrics = training_process.next(state, federated_train_data)
print(f'Train metrics:', metrics['train'])

Train metrics: OrderedDict([('rating_accuracy', 0.0), ('loss', 14.317455)])

Now let's set up a training loop to train over multiple rounds.

NUM_ROUNDS = 20

train_losses = []
train_accs = []

state = training_process.initialize()

# This may take a couple minutes to run.
for i in range(NUM_ROUNDS):
federated_train_data = np.random.choice(tf_train_datasets, size=50, replace=False).tolist()
state, metrics = training_process.next(state, federated_train_data)
print(f'Train round {i}:', metrics['train'])
train_losses.append(metrics['train']['loss'])
train_accs.append(metrics['train']['rating_accuracy'])

eval_metrics = evaluation_computation(state.model, tf_val_datasets)
print('Final Eval:', eval_metrics['eval'])

Train round 0: OrderedDict([('rating_accuracy', 0.0), ('loss', 14.7013445)])
Train round 1: OrderedDict([('rating_accuracy', 0.0), ('loss', 14.459233)])
Train round 2: OrderedDict([('rating_accuracy', 0.0), ('loss', 14.52466)])
Train round 3: OrderedDict([('rating_accuracy', 0.0), ('loss', 14.087793)])
Train round 4: OrderedDict([('rating_accuracy', 0.011243612), ('loss', 11.110232)])
Train round 5: OrderedDict([('rating_accuracy', 0.06366048), ('loss', 8.267054)])
Train round 6: OrderedDict([('rating_accuracy', 0.12331288), ('loss', 5.2693872)])
Train round 7: OrderedDict([('rating_accuracy', 0.14264487), ('loss', 5.1511016)])
Train round 8: OrderedDict([('rating_accuracy', 0.21046545), ('loss', 3.8246362)])
Train round 9: OrderedDict([('rating_accuracy', 0.21320973), ('loss', 3.303812)])
Train round 10: OrderedDict([('rating_accuracy', 0.21651311), ('loss', 3.4864292)])
Train round 11: OrderedDict([('rating_accuracy', 0.23476052), ('loss', 3.0105433)])
Train round 12: OrderedDict([('rating_accuracy', 0.21981856), ('loss', 3.1807854)])
Train round 13: OrderedDict([('rating_accuracy', 0.27683082), ('loss', 2.3382564)])
Train round 14: OrderedDict([('rating_accuracy', 0.26080742), ('loss', 2.7009728)])
Train round 15: OrderedDict([('rating_accuracy', 0.2733109), ('loss', 2.2993557)])
Train round 16: OrderedDict([('rating_accuracy', 0.29282996), ('loss', 2.5278995)])
Train round 17: OrderedDict([('rating_accuracy', 0.30204678), ('loss', 2.060092)])
Train round 18: OrderedDict([('rating_accuracy', 0.2940266), ('loss', 2.0976772)])
Train round 19: OrderedDict([('rating_accuracy', 0.3086304), ('loss', 2.0626144)])
Final Eval: OrderedDict([('loss', 1.9961331), ('rating_accuracy', 0.30322924)])

We can plot training loss and accuracy over rounds. The hyperparameters in this notebook have not been carefully tuned, so feel free to try different clients per round, learning rates, number of rounds, and total number of clients to improve these results.

plt.plot(range(NUM_ROUNDS), train_losses)
plt.ylabel('Train Loss')
plt.xlabel('Round')
plt.title('Train Loss')
plt.show()

plt.plot(range(NUM_ROUNDS), train_accs)
plt.ylabel('Train Accuracy')
plt.xlabel('Round')
plt.title('Train Accuracy')
plt.show()

Finally, we can calculate metrics on an unseen test set when we're finished tuning.

eval_metrics = evaluation_computation(state.model, tf_test_datasets)
print('Final Test:', eval_metrics['eval'])

Final Test: OrderedDict([('loss', 1.9566978), ('rating_accuracy', 0.30792442)])

## Further Explorations

Nice work on completing this notebook. We suggest the following exercises to explore partially local federated learning further, roughly ordered by increasing difficulty:

• Typical implementations of Federated Averaging take multiple local passes (epochs) over the data (in addition to taking one pass over the data across multiple batches). For Federated Reconstruction we may want to control the number of steps separately for reconstruction and post-reconstruction training. Passing the dataset_split_fn argument to the training and evaluation computation builders enables control of the number of steps and epochs over both reconstruction and post-reconstruction datasets. As an exercise, try performing 3 local epochs of reconstruction training, capped at 50 steps and 1 local epoch of post-reconstruction training, capped at 50 steps. Hint: you'll find tff.learning.reconstruction.build_dataset_split_fn helpful. Once you've done this, try tuning these hyperparameters and other related ones like learning rates and batch size to get better results.

• The default behavior of Federated Reconstruction training and evaluation is to split clients' local data in half for each of reconstruction and post-reconstruction. In cases where clients have very little local data, it can be reasonable to reuse data for reconstruction and post-reconstruction for the training process only (not for evaluation, this will lead to unfair evaluation). Try making this change for the training process, ensuring the dataset_split_fn for evaluation still keeps reconstruction and post-reconstruction data disjoint. Hint: tff.learning.reconstruction.simple_dataset_split_fn might be useful.

• Above, we produced a tff.learning.Model from a Keras model using tff.learning.reconstruction.from_keras_model. We can also implement a custom model using pure TensorFlow 2.0 by implementing the model interface. Try modifying get_matrix_factorization_model to build and return a class that extends tff.learning.reconstruction.Model, implementing its methods. Hint: the source code of tff.learning.reconstruction.from_keras_model provides an example of extending the tff.learning.reconstruction.Model class. Refer also to the custom model implementation in the EMNIST image classification tutorial for a similar exercise in extending a tff.learning.Model.

• In this tutorial, we've motivated partially local federated learning in the context of matrix factorization, where sending user embeddings to the server would trivially leak user preferences. We can also apply Federated Reconstruction in other settings as a way to train more personal models (since part of the model is completely local to each user) while reducing communication (since local parameters are not sent to the server). In general, using the interface presented here we can take any federated model that would typically be trained fully globally and instead partition its variables into global variables and local variables. The example explored in the Federated Reconstruction paper is personal next word prediction: here, each user has their own local set of word embeddings for out-of-vocabulary words, enabling the model to capture users' slang and achieve personalization without additional communication. As an exercise, try implementing (as either a Keras model or a custom TensorFlow 2.0 model) a different model for use with Federated Reconstruction. A suggestion: implement an EMNIST classification model with a personal user embedding, where the personal user embedding is concatenated to the CNN image features before the last Dense layer of the model. You can reuse much of the code from this tutorial (e.g. the UserEmbedding class) and the image classification tutorial.

If you're still looking for more on partially local federated learning, check out the Federated Reconstruction paper and open-source experiment code.

[{ "type": "thumb-down", "id": "missingTheInformationINeed", "label":"Missing the information I need" },{ "type": "thumb-down", "id": "tooComplicatedTooManySteps", "label":"Too complicated / too many steps" },{ "type": "thumb-down", "id": "outOfDate", "label":"Out of date" },{ "type": "thumb-down", "id": "samplesCodeIssue", "label":"Samples / code issue" },{ "type": "thumb-down", "id": "otherDown", "label":"Other" }]
[{ "type": "thumb-up", "id": "easyToUnderstand", "label":"Easy to understand" },{ "type": "thumb-up", "id": "solvedMyProblem", "label":"Solved my problem" },{ "type": "thumb-up", "id": "otherUp", "label":"Other" }]


text:
# 1. Create an artificial dataset¶

### 1.1 Fix random seed for reproducibility¶

In [2]:
seed = 7
np.random.seed(seed)

### 1.2 We need to generate two sets of data¶

In [3]:
# The first way is by using the already existing functions:
X,y = make_circles(n_samples = 3000, noise = 0.08, factor=0.3)
print(X.shape)
print(y.shape)
print(type(X))
print(type(y))

# Spliting data into train and test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

(3000, 2)
(3000,)
<class 'numpy.ndarray'>
<class 'numpy.ndarray'>

In [4]:
# The other way is by hand:
X0 = [X[i,:] for i in range(len(y)) if y[i] == 0] # returns the row where the corresponding label is 0
X1 = [X[i,:] for i in range(len(y)) if y[i] == 1] # returns the row where the corresponding label is 1

X0_np = np.array(X0) # convert it into an array
X1_np = np.array(X1)

X0_train = X0_np[:1000,:].T # picked the first 1000 samples in the second axis
X0_test = X0_np[1000:,:].T # picked the remaining 500 samples in the second axis

X1_train = X1_np[:1000,:].T  # picked the first 1000 samples in the second axis
X1_test = X1_np[1000:,:].T # picked the remaining 500 samples in the second axis

X_train = np.hstack([X0_train,X1_train]).T # all training examples
y_train=np.zeros((1,2000))
y_train[0, 1000:] = 1

X_test = np.hstack([X0_test,X1_test]).T # all test examples
y_test=np.zeros((1,1000))
y_test[0, 500:] = 1


text:
Synchronous Generator – Zero Power Factor Characteristics and Potier Triangle

The zero power factor characteristics (ZPFC) is the graph plotted between the armature terminal voltage per phase and the field current, when the machine being operating with constant rated armature current at synchronous speed and zero lagging power factor. The ZPFC is also known as Potier Characteristics after its originator.

In order to maintain the power factor very low, the alternator is loaded by means of reactors or by an under-excited synchronous motor. The shape of the ZPFC is very much like that of the open-circuit characteristic (O.C.C.) displaced downwards and to the right.

Phasor Diagram

Figure-1 shows the phasor diagram of the alternator corresponding to zero power factor lagging load.

Here, the terminal voltage per phase (V) is taken as the reference phasor. At zero power factor lagging, the armature current (𝐼𝑎) lags behind the voltage V by 90°. The voltage drop (𝐼𝑎𝑅𝑎) in the armature resistance is drawn parallel to the current (𝐼𝑎) and the drop (𝐼𝑎𝑋𝑎𝐿) in the leakage reactance perpendicular to (𝐼𝑎).

Hence, the generated voltage per phase is then,

$$\mathrm{𝐸_{𝑔} = 𝑉 + 𝐼_{𝑎}𝑅_{𝑎} + 𝐼_{𝑎}𝑋_{𝑎L} … (1)}$$

If

• $𝐹_{𝑎𝑟}$ = Armature reaction MMF (in phase with $𝐼_{𝑎}$)

• $𝐹_{𝑓}$ = Field MMF

• $𝐹_{𝑟}$ = Resultant MMF in the air gap

The three MMF phasors 𝐹𝑓 , 𝐹𝑟 and 𝐹𝑎𝑟 are in phase and their magnitudes are related by the following equation −

$$\mathrm{𝐹_{𝑓} = 𝐹_{𝑟} + 𝐹_{𝑎𝑟} … (2)}$$

If the armature resistance (𝑅𝑎) is neglected, the resulting phasor diagram will be as shown in Figure-2. From Figure-2, it can be seen that the terminal voltage per phase (V), the reactance voltage drop (𝐼𝑎𝑋𝑎𝐿) and the generated voltage (𝐸𝑔) are all in phase. Hence, the terminal voltage (V) is practically equal to the arithmetical difference between 𝐸𝑔 and 𝐼𝑎𝑋𝑎𝐿i.e.,

$$\mathrm{𝑉 = 𝐸_{𝑔} − 𝐼_{𝑎}𝑋_{𝑎𝐿} … (3)}$$

The arithmetical expressions given in eqns. (2) and (3) form the basis for the Potier triangle.

Also, Eqn. (2) can be transformed into its equivalent field current form by dividing both sides by the effective number of turns per pole (𝑇𝑓) on the rotor.Thus,

$$\mathrm{\frac{𝐹_{𝑓}}{𝑇_{𝑓}}=\frac{𝐹_{r}}{𝑇_{𝑓}}+\frac{𝐹_{𝑎𝑟}}{𝑇_{𝑓}}}$$

$$\mathrm{\Rightarrow\:𝐼_{𝑓} = 𝐼_{𝑟} + 𝐼_{𝑎𝑟} … (4)}$$

Potier Triangle

The O.C.C. and ZPFC are shown in Figure-3.

Consider a point b on the ZPFC corresponding to the rated terminal voltage (V) and a field current of (𝑂𝑀 = 𝐼𝑓).

If under this operating condition, the armature reaction MMF (𝐹𝑎𝑟) has a value expressed in equivalent field current of (𝐿𝑀 = 𝐼𝑎𝑟), then the equivalent field current of the resultant MMF (𝐹𝑟) would be (𝑂𝐿 = 𝐼𝑟).

This resultant field current (𝑂𝐿 = 𝐼𝑟) would result in a generated voltage (𝐿𝑐 = 𝐸𝑔) from the O.C.C. Since, for zero power factor lagging operation, the generated voltage is given by,

$$\mathrm{𝑬_{𝑔} = 𝑽 + 𝑰_{𝑎}𝑿_{𝑎𝐿} … (5)}$$

Here, the vertical distance ac must be equal to the leakage reactance voltage drop (𝐼𝑎𝑋𝑎𝐿) where 𝐼𝑎 is the rated armature current. Therefore, the armature leakage reactance is,

$$\mathrm{𝑋_{𝑎𝐿} =\frac{Voltage\:𝑎𝑐\:per \:phase}{Rated\:armature\:current}… (6)}$$

Now, the triangle formed by the vertices a, b and c is known as the Potier Triangle.

In the synchronous machine, the effect of the field leakage flux in combination with the armature leakage flux gives rise to an equivalent leakage reactance 𝑋𝑝, known as Potier Reactance. It is given by,

$$\mathrm{𝑋_{𝑝} =\frac{Voltage\:drop\:per\:phase (voltage\:𝑎𝑐)}{ZPF\:rated\:armature\:current\:per\: phase (𝐼_{𝑎})}… (7)}$$

For a cylindrical rotor synchronous machine, the Potier reactance (𝑋𝑝) is approximately equal to the armature leakage reactance (𝑋𝑎𝐿) while in a salient-pole machine, the 𝑋𝑝 may be as large as 3 times 𝑋𝑎𝐿.


text:
40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
 40m Log, Page 271 of 339 Not logged in
ID Date Author Type Category Subject
6302   Tue Feb 21 22:06:18 2012 jamieUpdateLSCbeatbox DFD installed in 1X2 rack

I have installed a proto version of the ALS beatbox delay-line frequency discriminator (DFD, formally known as MFD), in the 1X2 rack in the empty space above the RF generation box.

That empty space above the RF generation box had been intentionally left empty to provide needed ventilation airflow for the RF box, since it tends to get pretty hot.  I left 1U of space between the RF box and the beatbox, and so far the situation seems ok, ie. the RF box is not cooking the beatbox.  This is only a temporary arrangement, though, and we should be able to clean up the rack considerably once the beatbox is fully working.

For power I connected the beatbox to the two unused +/- 18 V Sorensen supplies in the OMC power rack next to the SP table.  I disconnected the OMC cable that was connected to those supplies originally.  Again, this is probably just temporary.

Right now the beatbox isn't fully functioning, but it should be enough to use for lock acquisition studies.  The beatbox is intended to have two multi-channel DFDs, one for each arm, each with coarse and fine outputs.  What's installed only has one DFD, but with both coarse and fine outputs.  It is also intended to have differential DAQ outputs for the mixer IF outputs, which are not installed in this version.

The intended design was also supposed to use a comparator in the initial amplification stages before the delay outputs.  The comparator was removed, though, since it was too slow and was limiting the bandwidth in the coarse channel.  I'll post an updated schematic tomorrow.

I made some initial noise measurements:  with a 21 MHz input, which corrseponds to a zero crossing for a minimal delay, the I output is at ~200 nVrms/\sqrt{Hz} at 5 Hz, falling down to ~30 nVrms about 100 Hz, after which it's mostly flat.  I'll make calibrated plots for all channels tomorrow.

The actual needed delay lines are installed/hooked up either.  Either Kiwamu will hook something up tonight, or I'll do it tomorrow.

6794   Mon Jun 11 21:50:08 2012 yutaUpdateGreen Lockingbeatbox looks OK

Summary:
We need I-Q frequency deiscriminator to control the arm length fine and continuously.
I checked the beatbox (LIGO-D1102241-v4; see elog #6302) and it was working.

What I did:
1. Measured some transferfunctions with a network analyzer (Aligent 4395A) and checked the cabling is correct.

2. Put 30 m/1.5 m delay line and checked I-Q outputs are actually orthogonal. I did this by sweeping the frequency of RF input to the beatbox. See attached picture. You can see nice circle on the oscilloscope.

Some measurement results:

- Gains of the transferfunctions(@ 10-100MHz) between;

RF in -> RF mon: -25 to -20 dB
RF in -> fine delay out: -50 to -40 dB
RF in -> coarse delay out: -50 to -40 dB
RF in -> LO of mixer RMS-1: ~ +4 dB  (RMS-1 needs +7 dB LO)

- 30m delay line(RG-142B/U) had -2 dB loss.

Note:
- RF input must be larger than about -3 dBm to get enough LO to the mixer. Otherwise, you won't get I-Q outputs.
- The comparator, whitening filter and differential DAQ outputs are not installed in the current beatbox.
- Current beatbox only has electronics for the one arm.
- The print on the board D1102241 says +15V and -15V, but they are actually opposite. Cabling is swapped in order to supply correct power to the ICs.

Attachment 1: CIMG1522.JPG
734   Thu Jul 24 11:49:07 2008 MashaSummaryAuxiliary lockingbelated weekly summary
I designed a high pass filter to whiten the spectrum from the Mach Zehnder to optimize the
input into the ADC. The swept sine response measurement and the effect of the filter on the
spectrum are attached. If I start using the digital system (it is currently down in Bridge),
I will decide if the filter needs to be improved/better matched to the ADC there.

I moved from the 40m to Rana's lab in bridge. I am making a new and improved Mach Zehnder
setup with a 50m fiber in one arm; currently the transmission through the fiber is 44%. I
am working out how to mode match the laser to the fiber to improve this number.
Attachment 1: filter_tr_function.pdf
Attachment 2: filtered_spectr0724.pdf
7342   Tue Sep 4 20:25:22 2012 jamieOmnistructureVACbetter in-air "lite" access connector needed

We really need something better to replace the access connector when we're at air.  This tin foil tunnel crap is dumb.  We can't do any locking in the evening after we've put on the light doors.  We need something that we can put in place of the access connector that allows us access to the OMC and IOO tables, while still allowing IMC locking, and can be left in place at night.

7343   Wed Sep 5 09:50:25 2012 SteveOmnistructureVACbetter in-air "lite" access connector needed

 Quote: We really need something better to replace the access connector when we're at air.  This tin foil tunnel crap is dumb.  We can't do any locking in the evening after we've put on the light doors.  We need something that we can put in place of the access connector that allows us access to the OMC and IOO tables, while still allowing IMC locking, and can be left in place at night.

It is in the shop. It will be ready for the next vent. Koji's dream comes through.

Attachment 1: IMG_1612.JPG
7344   Wed Sep 5 10:50:15 2012 jamieOmnistructureVACbetter in-air "lite" access connector needed

Quote:

 Quote: We really need something better to replace the access connector when we're at air.  This tin foil tunnel crap is dumb.  We can't do any locking in the evening after we've put on the light doors.  We need something that we can put in place of the access connector that allows us access to the OMC and IOO tables, while still allowing IMC locking, and can be left in place at night.

It is in the shop. It will be ready for the next vent. Koji's dream comes through.

Can we see the full design?  If we can't lock the mode cleaner with this thing on then it's really of no use.  We want it to be equivalent to the light doors, but allow us to keep the mode cleaner locked.  That's the most important aspect.

7345   Wed Sep 5 13:11:43 2012 jamieOmnistructureVACbetter in-air "lite" access connector needed

Quote:

Quote:

 Quote: We really need something better to replace the access connector when we're at air.  This tin foil tunnel crap is dumb.  We can't do any locking in the evening after we've put on the light doors.  We need something that we can put in place of the access connector that allows us access to the OMC and IOO tables, while still allowing IMC locking, and can be left in place at night.

It is in the shop. It will be ready for the next vent. Koji's dream comes through.

Can we see the full design?  If we can't lock the mode cleaner with this thing on then it's really of no use.  We want it to be equivalent to the light doors, but allow us to keep the mode cleaner locked.  That's the most important aspect.

It also needs to be wide enough that the MMT beam can go through, so that we can not only lock the MC, but also work on the rest of the IFO.

7952   Tue Jan 29 10:59:37 2013 lazy personFrogsGeneralbetter plan

I propose we work around this problem with giant flip-flops.  These are in the vein of the take-off-your-shoes-and-put-on-Crocs, without the taking off your shoes part.  They're a little annoying on the sticky mats, but otherwise great.  They are also super easy to put on and take off without hands, so there's no excuse for wearing them around the control room.

I propose we buy many pairs of the smalls in green (since we already have one green small...they are big on me, so should be just right for most people), and a few mediums in, say, blue, and a few larges in black, and then maybe a few extra larges in green for people with extraordinarily large feet (they only have 3 colors).  Then we can keep a few pairs of each by each door to the lab, and have no more tracking dirty control room filth into the lab.

2670   Fri Mar 12 17:08:22 2010 steveConfigurationVACbg-RGAscan at d18

RGA scan of rga-region only at day 18   This is the back ground of the rga with some calibration gas.

Attachment 1: bg-d18_20100312scan.png
11040   Mon Feb 16 21:52:51 2015 ranaHowToTreasurebig Dataviewer windows

Following this entry, I have made the same change in the controls account on rossa:

In the ~/.grace/gracerc file (create one if it doesn't exist), put in a line which reads:

PAGE LAYOUT FREE

Now we can scale our dataviewer live and playback plots by stretching the window with our mouse. The attached screenshot shows how I filled up one of the vertical monitors with a DV window for arm locking.

Attachment 1: bigDV.png
15680   Tue Nov 17 13:24:40 2020 ChubUpdateGeneralbig UPS on the way

Ordered 11/16 from CDW, on PO# S492940, the high voltage Tripp Lite SMART5000XFMRXL  for TP-1.  Should be arriving in about a week.

894   Thu Aug 28 19:02:25 2008 rana, josephb, robSummaryComputersbig boot
This afternoon Joe did something with an .ini file (look for his detailed elog entry) and the computers went bad.
RFM network screen not active - filter modules not working.

We went around and booted every machine as has been done before. The correct order for a memory corruption
fixing big boot is the following:

[1] RESET the RFM switches near the FB racks.
[2] Power cycle c1dcuepics.
[3] Power cycle all other crates with real time CPUs:
c1iscey, daqctrl, daqawg, c1susvme1, c1susvme2, c1sosvme, c1iovme, c1lsc, c1asc, & c1iscex
[4] Start up all FEs as described in Wiki.
[5] Burt restore everyone (losepics, iscepics, assepics, omcepics?)
13731   Thu Apr 5 13:46:42 2018 gautamUpdateSUSbig earthquake

Seems like there was a 5.3 magnitude EQ ~10km from us (though I didn't feel it). All watchdogs were tripped so our mirrors definitely felt it. ITMX is stuck (but all the other optics are damping fine). I tried the usual jiggling of DC bias voltage but ITMX still seems stuck. Probably a good sign that the magnet hasn't come off, but not ideal that I can't shake it free..

edit: after a bit more vigorous shaking, ITMX was freed. I had to move the bias slider by +/-10,000 cts, whereas initially I was trying +/-2000 cts. There is a tendency for the optic to get stuck again once it has been freed (while the optic's free swinging motion damps out), so I had to keep an eye out and as soon as the optic was freed, I re-engaged the damping servos to damp out the optic motion quickly.

Attachment 1: EQ_April52018.png
7923   Tue Jan 22 09:10:19 2013 SteveUpdatePEMbig foot is dirty

Please wipe, clean car wheels and wear booties entering the 40m lab.

Obviously this person has no idea about our clean room rules.

Attachment 1: IMG_1873.JPG
Attachment 2: IMG_1874.JPG
513   Tue Jun 3 10:19:45 2008 tobinConfigurationComputersbig machine
Several of us transported the big new awesome Sun box from Bridge over to
the 40m last week. If I recall correctly, it's a SunFire X4600 with
something like sixteen 64-bit AMD processor cores at 2.8 GHz. It sounds
like a jet engine when it starts up (before the cooling fans are throttled
back) and has four power supplies (each with its own connection
to the wall). It has slick removable hard disks and fan units too. Our
working name for it is "megatron".

Anyway. It came with two hard disks, one with Solaris 10 installed. I took
the other hard disk over to Alex, who copied a Realtime Linux installation
onto it. Alex says it boots and runs fine.

It remains for you guys to install the machine onto rails and install the
whole thing into a rack. Before it goes into service as a realtime control
machine, you might as well install Matlab on it and do some heavy-duty
computation.

3490   Mon Aug 30 22:45:49 2010 kiwamuUpdateCDSbinary outputs for the new CDS

{ Joe and Kiwamu }

Today we made some efforts to get the binary outputs (BOs) working.

They still are not working but the situation is getting better.

So far the BO cards were not recognized by any realtime codes when we ran the codes on the new front end machine C1SUS.

We put some printk commands in an initialization code like Yoichi did (see this entry) to confirm if the initialization of the BOs properly happens or not.

Then we found that we had to put the BO modules also in an IOP model file which controls all the ADCs and the DACs.

We put the BO modules in the IOP file and then BOs started being recognized by the IOP, however they still are not fully recognized by the realtime control process.

We continue this work...

[Some notes]

[front end code]

First of all we looked at the front end c-code c1sus.c living under /cvs/cds/calech/cds/advLigoRTS/src/fe/c1sus/.

It was okay because there was a proper BO statement like

CDS_CARDS cards_used[] = { {CON_32DO,0}, {CON_32DO,2}};

[initialization code]

There is an initialization code called map.c living under /cvs/cds/calech/cds/advLigoRTS/src/fe.

This code is complied when we do the make commands as described on the wiki.

Eventually the initialization code is executed only when the IOP starts up. This happens when we type startc1x02 at /cvs/cds/rtcs/caltech/c1/script/.

[printk statments]

We made a backup file named map_20100830.c.back for map.c. Then we added to map.c some pintk statements in a while loop which looks for available BOs.

After running the make commands for the IOP file and startc1x02, we basically can check the results of those printk statements by using dmesg.

We found that map.c was running correctly because  map.c went in the while loop 4 times which is exactly the same number as the BOs we put in the model file.

However the code failed to install the BOs each time.

[BO modules in IOP file]

Joe pointed out the failure in map.c was caused by lack of the BO modules in the IOP file c1x02.mdl.

Indeed putting the BO modules in the IOP fixed the problem.

Another thing we found at this time is that there is a maximum number of BOs we can put in a model file.

The maximum number is 4, which is not enough for us because we need to put 5 of them including a 16bit BIO and four 32bit BOs.

Anyway now the IOP can recognize the BO cards, this fact can be found if you look at the log file /cvs/cds/rtcds/caltech/c1/target/c1x02/logs/log.txt.

The log file saids "3 Contec 32ch PCIe DO cards found", which is a good sign.

[BO modules in realtime code]

Although the IOP started seeing the BO cards, the realtime code c1sus still didn't fully recognize the BO cards.

If we look at the log file log.txt at /cvs/cds/rtcds/caltech/c1/target/c1sus/logs/, there is an evidence that the code found some cards.

The log file saids

   Model 6 = 10

   Model 7 = 4

   Model 8 = 4

   Model 9 = 4

   Model 10 = 0.

It looks like these corresponds to the BO cards.

So the code found some cards, but doesn't know what they are.

We need few more debugging for the BOs...

6616   Mon May 7 21:05:38 2012 DenUpdateCDSbiquad filter form

I wanted to switch the implementation of IIR_FILTER from DIRECT FORM II to BIQUAD form in C1IOO and C1SUS models. I modified RCG file /opt/rtcds/rtscore/release/src/fe/controller.c by adding #define CORE_BIQUAD line:

#ifdef OVERSAMPLE #define CORE_BIQUAD       #if defined(CORE_BIQUAD)

C1IOO model compiled, installed and is running now. C1SUS model compiled, but during installation I've got an error:

controls@c1sus ~ 0rtcds install c1sus Installing system=c1sus site=caltech ifo=C1,c1 Installing /opt/rtcds/caltech/c1/chans/C1SUS.txt Installing /opt/rtcds/caltech/c1/target/c1sus/c1susepics Installing /opt/rtcds/caltech/c1/target/c1sus Installing start and stop scripts /opt/rtcds/caltech/c1/scripts/killc1sus Performing install-daq Updating testpoint.par config file /opt/rtcds/caltech/c1/target/gds/param/testpoint.par /opt/rtcds/rtscore/branches/branch-2.5/src/epics/util/updateTestpointPar.pl -par_file=/opt/rtcds/caltech/c1/target/gds/param/archive/testpoint_120507_205359.par -gds_node=21 -site_letter=C -system=c1sus -host=c1sus Installing GDS node 21 configuration file /opt/rtcds/caltech/c1/target/gds/param/tpchn_c1sus.par Installing auto-generated DAQ configuration file /opt/rtcds/caltech/c1/chans/daq/C1SUS.ini Installing EDCU ini file /opt/rtcds/caltech/c1/chans/daq/C1EDCU_SUS.ini Installing Epics MEDM screens Running post-build script ERROR: Could not find file: test.py Searched path: :/opt/rtcds/userapps/release/cds/c1/scripts:/opt/rtcds/userapps/release/cds/common/scripts:/opt/rtcds/userapps/release/isc/c1/scripts:/opt/rtcds/userapps/release/isc/common/scripts:/opt/rtcds/userapps/release/sus/c1/scripts:/opt/rtcds/userapps/release/sus/common/scripts:/opt/rtcds/userapps/release/psl/c1/scripts:/opt/rtcds/userapps/release/psl/common/scripts Exiting make: *** [install-c1sus] Error 1 Jamie, what is this test.py? 6622 Tue May 8 09:47:53 2012 JamieUpdateCDSbiquad filter form  Quote: I wanted to switch the implementation of IIR_FILTER from DIRECT FORM II to BIQUAD form in C1IOO and C1SUS models. I modified RCG file /opt/rtcds/rtscore/release/src/fe/controller.c by adding #define CORE_BIQUAD line: #ifdef OVERSAMPLE #define CORE_BIQUAD #if defined(CORE_BIQUAD) I am really not ok with anyone modifying controller.c. If we're going to be messing around with that we need to change procedure significantly. This is the code that runs all the models, and we don't currently have any way to track changes in the code. Did you change it back? If not, do so immediately and stop messing with it. Please consult with us first before embarking on these kinds of severe changes to our code. This is the kind of shit that other people have done that has bit us in the ass in the past. Futhermore, there is already a way to enable biquad filters in the new version with out modifying the RCG source. All you need to do is set biquad=1 in the cdsParameters block for you model. ### DO NOT MESS WITH CONTROLLER.C! 6624 Tue May 8 10:43:42 2012 DenUpdateCDSbiquad filter form Quote:  Quote: I wanted to switch the implementation of IIR_FILTER from DIRECT FORM II to BIQUAD form in C1IOO and C1SUS models. I modified RCG file /opt/rtcds/rtscore/release/src/fe/controller.c by adding #define CORE_BIQUAD line: #ifdef OVERSAMPLE #define CORE_BIQUAD #if defined(CORE_BIQUAD) I am really not ok with anyone modifying controller.c. If we're going to be messing around with that we need to change procedure significantly. This is the code that runs all the models, and we don't currently have any way to track changes in the code. Did you change it back? If not, do so immediately and stop messing with it. Please consult with us first before embarking on these kinds of severe changes to our code. This is the kind of shit that other people have done that has bit us in the ass in the past. Futhermore, there is already a way to enable biquad filters in the new version with out modifying the RCG source. All you need to do is set biquad=1 in the cdsParameters block for you model. ### DO NOT MESS WITH CONTROLLER.C! ok 7041 Thu Jul 26 17:39:49 2012 DenUpdatedigital noisebiquad key is working I've filtered a 1 Hz sin wave excitation with a notch filter inside c1sus and c1rfm models. The biquad key is switched on in the last one, c1sus uses DF2. The results are indeed different. Still I do not like huge (2n+1) harmonics in the output of the biquad filter, I do not get them in the simulations. They are absent in the time series as well. So this is not a psd-estimation effect. 7045 Fri Jul 27 14:30:49 2012 JamieUpdatedigital noisebiquad key is working What is "DQF"? Is that the biquad? And what is the difference between DF1 and DF2? Why don't you just write out the name, so it's more clear. 7050 Mon Jul 30 14:24:25 2012 DenUpdatedigital noisebiquad key is working  Quote: What is "DQF"? Is that the biquad? And what is the difference between DF1 and DF2? Why don't you just write out the name, so it's more clear. DQF - biquad form DF1 - direct form 1 DF2 - direct form 2 LNF - low-noise form The difference between them is described in Matt's slides G0900928-v1. I think, LNF coefficients are incorrect in the presentation 6748 Sun Jun 3 23:50:00 2012 DenUpdateCDSbiquad=1 From now all models calculate iir filters using biquad form. I've added biquad=1 to cdsParameters to all models except c1cal, built, installed and restarted them. 4597 Mon May 2 13:43:05 2011 steveFrogsPhotosbirthday boys .....Happy.... Birthday.... to.... Joseph... and... Jamie...Happy....Birthday..... to.... You............sing with us........Happy Birthday.....to you Attachment 1: P1070622.JPG 11972 Wed Feb 3 08:39:17 2016 SteveUpdateCDSblank daily summary pages Daily summary pages are blank today. Yesterday is ok 7545 Mon Oct 15 08:12:45 2012 SteveUpdateVACblank vac monitor VME crate blew its fuse and the fuse housing is hard to access. Attachment 1: blankvac.png 1184 Sun Dec 7 16:12:53 2008 ranaUpdateDAQbooted awg because it was red 12389 Tue Aug 9 19:35:49 2016 ranaUpdateSEIboth Guralp seismometers are functioning and being acquired After some cable swapping, we now have both Guralp seismometers running and the times series and spectra look similar to each other and motley healthy. Bean and I took a look at the whole situation today. Ben had nicely fixed the Dsub end of the EX cable (the EY one is still just a sad joke), After installing this newly fixed cable, we still saw no signals. There was some confusion in the control room about using the MED displays to diagnose seismometers: flickering MEDM values cannot be used for this. It would be like checking a pizza box temperature to determine if the pizza is any good. 1. Although the +/- 12V LEDs on the front panel are dim, we confirmed that the produce 11.94V even when loaded with a seismometer. So its a LED circuit problem not a power problem. 2. We were able to inject signals into the front panel with a breakout board and see them in DV for Input 1, but not Input 2. 3. After Ben left, I kept poking around and found that the Guralp chassis output gets broken out into 3x3 BNC cables before going to the PEM BNC panel (and then on to the PEM ADC). This is where the problem was. 4. The Input #3 BNC cables were connected to the long cables going to the 'GUR2' channels of the PEM. The Input#2 BNC cables were connected to some short BNC cables that were just hanging from the rack. So, somewhere during the debugging of the past N months, someone plugged this in wrong and didn't notice or forgot to switch it back. So all of the tests using DV or DTT or MEDM since that time have been invalid. Tomorrow, Lydia is going to change all of the labels and channel names. The new names will be EX & EY to prevent this kind of huge waste of time with channel name swapping. That means no more illegal names with the label maker, Steve. From the spectrum you can see that the EX seismometer (GUR2) is still not centered or at least its oscillating at 245 Hz for some reason. This should go away after some power cycling or recentering using the magic wand. I noticed some anomalies in the mechanical setups at the ends: 1. Some junk has been stored on top of the EX seismometer. Please never, even temporarily, store your power supplies, tools, or donuts on top of the vibration sensitive sensors. Just put it on the floor and improve your carma. 2. The EY seismometer has some fishy wires being fished between the can and the rubber seal. This is verboten. That seal must be flush to prevent pressure fluctuations and wires in there will ruin the smooth contact permanently. Temperature sensor wires must go through the grantie block feed-through or else its pointless. 3. The flimsy insulation on the EY seismo is waay toooo mickey mouse. Real thermal insulation should be done using the yellow foam that Jenne used for the seismo huddle test. This flimsy silvery stuff is OK for making hats and mittens and beer cozy's, but its not research grade foam. Attachment 1: 1goodday.png 6840 Wed Jun 20 18:09:23 2012 yutaUpdateLockingboth arms aligned, ITMX oplev centered [Jenne, Yuta] We aligned FPMI. I also centered ITMX oplev because the light was not hitting on QPD. Alignment procedure we took was; 1. Align Y arm to the Y end green(Y green trans to PSL is now 195 uW with Y end laser measured temperature 34.14 degC). 2. Aligned IR using PZT2 to Yarm(Now, TRY ~ 0.90). 3. Aligned ITMX monitoring AS spots. 4. Aligned X arm so that TRX maximize. 5. Fine adjusted both BS and X arm(Now, TRX ~ 0.82). Beam spot position on ETMX looks a little too high & left (from ETMXF camera), but we will leave it until ASS scripts is fixed. 8162 Mon Feb 25 21:25:14 2013 yutaUpdateAlignmentboth arms locked in IR [Jenne, Evan, Yuta] After Y alignment, X arm is aligned to IR and we got both arms locked in IR. There's some dift (input pointing?) and this made aligning both arms tough. I will elog about it later. Attached is ETMYF. ETMXF, ITMYF, ITMXF when both arms are locked by IR. Alignment Procedure: 1. Algined BS/ITMX to get MI fringe in AS. We got X arm flashing at this point. 2. Use BS/ITMX/ETMX to get TRX maximized, without losing good MI fringe in AS. We reached 0.75. 3. There was clipping of TRX beam at Xend optics. Since whole IFO alignment is started from Y green, this clipping is because of poor Y green pointing. But we needed clear TRX for aligning Xarm, so we re-arranged Xend TRX path to avoid clipping. X arm issues: - Beam motion at TRX is larger than TRY. Turning off clean table air didn't help. Maybe we need BS oplev on or ETMX coil gain balancing. - X green scatters into TRX PD and ETMXT camera. Fix them. Attachment 1: QUAD1_1045890588.png 12448 Fri Aug 26 17:48:14 2016 gautamUpdateSUSbounce mode coupling reduction We worked on reducing the bounce mode coupling into the sensor signals today. After some trial and error, essentially following the procedure I had put up in my previous elog, we think we were successful in reducing the coupling. We have now left the optic free swinging, so that we can collect some data and look at a spectrum with finer bandwidth. But as per the methodology we followed, we saw that the peak height corresponding to the bounce mode increased when we rotated the OSEM either side of its current position (except for the side OSEM, which we felt was in a good enough position to warrant not touching it and messing it up - of course only the spectrum will tell us if we are right or not. I also took some pictures with the camera with the IR filter removed, but we couldn't get any real information from these photos. I also checked with Jenne and Jamie who both suggested that they didn't have any metric with which they judged if the rotation of the OSEM was good enough or not. So we will wait to have a look at the spectrum from later tonight, and if it looks reasonable enough, I vote we move on. As Eric suggested, perhaps we can repalce the UL OSEM coil and see if that solves the apparent UL coil problem. Then we should move on to putting the arm cavity together. Addendum 11pm 26 Aug 2016: I've uploaded the spectra - looks like our tweaking has gained us a factor of ~2 on LL, LR and SD, and no significant improvement on UL and UR compared to yesterdays spectrum. Attachment 1: ETMY_BounceSpectra_26Aug2016_1.pdf 456 Sun Apr 27 18:11:58 2008 robDAQComputersbr40m? The testpoint manager (which runs on fb40m) crashed this afternoon. Upon re-starting it, I found there was a rogue dtt process on op440m and also a daqd daemon running on br40m. One or both of these caused the tpman to crash. br40m is the frame broadcaster, which is never used here as we don't run DMT. I killed the daqd process there. The way to find if there is a rogue process is to watch the output to the console from the tpman when you start it: Allocate new TP handle 56 by 131.215.113.203 Allocate new TP handle 57 by 131.215.113.203 Allocate new TP handle 58 by 131.215.113.203 Allocate new TP handle 59 by 131.215.113.203 Allocate new TP handle 60 by 131.215.113.203 Allocate new TP handle 61 by 131.215.113.203 Allocate new TP handle 62 by 131.215.113.203 Allocate new TP handle 63 by 131.215.113.203 Allocate new TP handle 64 by 131.215.113.203 Allocate new TP handle 65 by 131.215.113.203 Allocate new TP handle 66 by 131.215.113.203 Allocate new TP handle 67 by 131.215.113.203 Allocate new TP handle 68 by 131.215.113.203 If you see something like this, with a new TP handle being allocated every few seconds, you need to log in to the corresponding host and kill whatever process has run away. 457 Sun Apr 27 22:57:15 2008 ajwDAQComputersbr40m?  Quote: The testpoint manager (which runs on fb40m) crashed this afternoon. Upon re-starting it, I found there was a rogue dtt process on op440m and also a daqd daemon running on br40m. One or both of these caused the tpman to crash. br40m is the frame broadcaster, which is never used here as we don't run DMT. I killed the daqd process there. The way to find if there is a rogue process is to watch the output to the console from the tpman when you start it: Allocate new TP handle 56 by 131.215.113.203 Allocate new TP handle 57 by 131.215.113.203 Allocate new TP handle 58 by 131.215.113.203 Allocate new TP handle 59 by 131.215.113.203 Allocate new TP handle 60 by 131.215.113.203 Allocate new TP handle 61 by 131.215.113.203 Allocate new TP handle 62 by 131.215.113.203 Allocate new TP handle 63 by 131.215.113.203 Allocate new TP handle 64 by 131.215.113.203 Allocate new TP handle 65 by 131.215.113.203 Allocate new TP handle 66 by 131.215.113.203 Allocate new TP handle 67 by 131.215.113.203 Allocate new TP handle 68 by 131.215.113.203 If you see something like this, with a new TP handle being allocated every few seconds, you need to log in to the corresponding host and kill whatever process has run away. I *think* Alex is responsible for the daqd daemon running on br40m (he set up some new stuff recently, a data concentrator and broadcaster); I'll make sure he sees this post. 3832 Mon Nov 1 10:17:55 2010 steveUpdateGeneralbreakers relabelled Pedro of the electrical shop helped to retrace wires from the racks to the breakers. Many wrong labels were replaced. These labels are still representing the south arm as Y and the east arm as X. Racks: 1X1,2,3,4 and 1Y3,4,5,6,7 and 9 are connected through manual disconnect swithes to AC power breaker PC-1 on the west wall of IFO room 104. This panel PC-1 is connected to T1 isolation transformer in room 101 Panel L - # 38, 40, 42 Racks: 1Y1 & 2 psl and mc are connected through manual disconnect switches to AC power breaker PC-2 just west of the psl enclosure. This panel is connected to T2 isolation transformer in room 101 Panel L - # 32, 34, 36 Attachment 1: P1060970.JPG Attachment 2: P1060972.JPG Attachment 3: P1060975.JPG 2420 Tue Dec 15 21:39:34 2009 AlbertoUpdateABSLbrief summary of this afternoon's measurements I took measurements of the open loop gain of the AbsL PLL with the old Universal PDH Box. I Also measured the filter shape of both the new and the old PDH box. I'm going to plot the results in a nice form tomorrow morning. For who's interested, the PLL UGF was at 10KHz. I can't lock the PLL with the new PDH box. Measuring its filter's shape, as suggested by Koji, I found out that it differs from the old one. That despite the fact that the two boxes should share the same circuit schematic. O,r at least, that is what it looks like from the schematics in the DCC. I need to understand whether that is intentional and, if that was the case, what kind of use Rich Abbott designed it for. Tomorrow I'm going to post in the elog the filter's transfer functions too. Before leaving the lab I closed the auxiliary laser's mechanical shutter. 5978 Tue Nov 22 15:18:18 2011 kiwamuUpdateGreen Lockingbroad band noise depends on the gain of Y green PDH. and comaprator broken  Quote from #5970 Here is a plot of the latest noise : high frequency noise is still unknown. (The broad-band noise vs. gain of the Y end green PDH) Last night I was trying to identify the broad band noise which is white and dominant above 20 Hz (#5970). I found that the level of the noise depended on the servo gain of the Y end green PDH loop. Decreasing the servo gain lowers the noise level by a factor of 2 or so. This was quite repeatable. (I changed the gain knob of the PDH box from the minimum to a point where the servo starts oscillating) (Malfunction in the comparator) However I had to give up further investigations because the comparator signal suddenly became funny: sometimes it outputs signals and sometimes not. It seems the comparator circuit became broken for some reason. I will fix it. 5298 Wed Aug 24 16:13:36 2011 kiwamuUpdateSUSbroke UL magnet on ITMX I broke the UL magnet on ITMX The ITMX tower was shipped into the Bob's clean room to put the magnet back on. Since we found that all the magnets were relatively high (#5296) in the shadow sensors, we decided to slide the OSEM holder bar upward. During the work, I haven't made the OSEMs far enough from the magnets. So the magnets and OSEMs touched as I moved the holder. Then the UL magnets were broken off and fell into the UL coil. 10120 Wed Jul 2 11:47:26 2014 SteveUpdateVACbroken changeover regulator of N2 supply  Quote: This morning valve condition: V1, VM1, V4 and V5 valves were closed. IFO pressure rose to 1.3 mTorr It was caused by low N2 pressure. Our vacuum valves are moved-controlled by 60-70 PSI of nitrogen. When this supply drops below 50-60 PSI the interlock closes V1 valve. This is the minimum pressure required to move the large valves. It is our responsibility to check the N2 cylinder pressure supply. The vacuum valve configuration is back to VAC. NORMAL, CC1 4.8E-6 Torr PS: Bob says that the second cylinder was full this morning, but the auto-switch over did not happen. The TESCOM automatic changeover regulator [ model ACS 012-1011 ] manifold is most likely clogged. The new one will arrive 8-8-2014 This means that the IFO pressure may go up to a few mTorr when we change cylinder or V1 valve triggered because there is no nitrogen supply. 7656 Thu Nov 1 17:37:00 2012 SteveUpdateGeneralbronze bushing for 40m vac Suprema- SS clear edge mirror mount 2" diameter is modified for 40m vacuum use. One left and one right handed one. It's adjustment screw housing is bronze! It is not ideal for out gassing. It will be baked and scanned. If it passes we should use it. We may need these to bring out some pick-off beams. Attachment 1: IMG_1764.JPG 7660 Fri Nov 2 03:28:54 2012 ranaUpdateGeneralbronze bushing for 40m vac  Quote: Suprema- SS clear edge mirror mount 2" diameter is modified for 40m vacuum use. One left and one right handed one. It's adjustment screw housing is bronze! It is not ideal for out gassing. It will be baked and scanned. If it passes we should use it. We may need these to bring out some pick-off beams. I vote against it. We don't know about the grease inside the screw bushings - scans are not everything if adjusting the screw loosens up the grease. If we need more pick off mirrors lets just make some of the kind that we already use inside for the 2" optics. 10 Tue Oct 23 11:08:20 2007 steveOtherGeneralbrush fires There are big brush fires around LA 40 days plot show no effect in the 40m lab Attachment 1: brushfires.jpg 1381 Mon Mar 9 23:55:38 2009 OsamuDAQComputersbscteststand and kami1 outside martian This morning there was a confliction of tpman running on fb40m and kami1. Alex fixed it temporary but Rana suggested it was better to move both PCs outside martian. We moved both PCs physically to the control room and connected to general network with a local router. I believe it won't conflict anymore but if you guess these PC might have trouble please feel free to shutdown. Today's work summary: *connected expansion chassis to bscteststand *obtained signals on dataviewer, dtt for both realtime and past data on bscteststand with 64kHz timing signal Questions: Excitation channels are not shown, only "other" is shown. qts.mdl should run with 16kHz but 16kHz timing causes a slow speed on dataviewer and failing data aquisition on dtt. We are using 64kHz timing but is it really correct? 6830 Mon Jun 18 17:28:03 2012 yutaSummaryComputersbugs in CDS_PARTS/simLinkParts/Fcn Fcn module in CDS_PARTS is used to include a user defined function in a model. We should be able to use this by entering desired function, but I found some bugs. BUG1: Fcn doen't work without ";" If you put ";" after the function, we can compile. sin(u[1]); But if you put without ";", like sin(u[1]) you get the following error message when compiling. controls@c1ioo ~ 0 rtcds make c1gcv
### building c1gcv...
Cleaning c1gcv...
Done
Parsing the model c1gcv...
Done
Building EPICS sequencers...
Done
Building front-end Linux kernel module c1gcv...
echo 'Built on date' date >> target/c1gcvepics/README.making_changes
make[1]: Leaving directory /opt/rtcds/caltech/c1/rtbuild'

make[1]: Entering directory /opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv'
make -C /lib/modules/2.6.34.1/build SUBDIRS=/opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv modules
make[2]: Entering directory /usr/src/linux-2.6.34.1-cs'
CC [M]  /opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv/c1gcv.o
make[2]: Leaving directory /usr/src/linux-2.6.34.1-cs'
make[1]: Leaving directory /opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv'
/opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv/c1gcv.c: In function 'feCode':
/opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv/c1gcv.c:615: error: expected expression before ';' token
make[3]: *** [/opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv/c1gcv.o] Error 1
make[2]: *** [_module_/opt/rtcds/caltech/c1/rtbuild/src/fe/c1gcv] Error 2
make[1]: *** [default] Error 2
make: *** [c1gcv] Error 1

BUG2: sindeg doesn't work properly

sindeg should work as cosine with input in degrees.
I made a simple model to test this(below).

Output of the filter module C1:ALS-BEATY_FINE_PHASE goes to "PHASE_in"
sindeg of this goes to C1:ALS-BEATY_FINE_I_ERR
cosdeg of this goes to C1:ALS-BEATY_FINE_Q_ERR

If you sweep the phase input, you should get sin and cos, but you get the following.
cosdeg (C1:ALS-BEATY_FINE_Q_ERR) looks OK, but sindeg (C1:ALS-BEATY_FINE_I_ERR) looks funny. It looks like ~20000 is its period.

7942   Thu Jan 24 16:31:46 2013 SteveUpdatePEMbuilding exterier wall painted

The wood exteier walls, gutters and doors were painted at CES-Annex building #69

Attachment 1: IMG_1880.JPG
1958   Thu Aug 27 16:14:28 2009 steveSummaryOMCburned photodiode

Old -pre 6/2009  LLO DCPD 3 mm od GTRAN photodiode

Attachment 1: 20090827_173252.jpg
Attachment 2: 20090827_170802.jpg
264   Fri Jan 25 09:22:21 2008 steveUpdatePEMburned toast award goes to
DYM for collaborating with the enemy.

In order to minimize the number of ants visiting our lab we have to take out side the lab
all food left over and organic waste. If you are eating here do not expect someone else
to clean up after you.

4053   Tue Dec 14 11:24:35 2010 josephbUpdateCDSburt restore

I had updated the individual start scripts, but forgotten to update the rc.local file on the front ends to handle burt restores on reboot.

I went to the fb machine and into /diskless/root/etc/ and modified the rc.local file there.

Basically in the loop over systems, I added the following line:

/opt/epics-3.14.9-linux/base/bin/linux-x86/burtwb -f /opt/rtcds/caltech/c1/burt/autoburt/latest/${i}epics.snap -l /opt/rtcds/caltech/c1/burt/autoburt/logs/${i}epics.log.restore -v

The \${i} gets replaced with the system name in the loop (c1sus, c1mcs, c1rms, etc)`

3706   Wed Oct 13 17:04:34 2010 josephb, yutaUpdateCDSburt restore now working with filter settings

Previously, burt restoring was not setting filters on and off, which was required us to constantly go through all the filter banks and turn them on.  We figured it out that the autoBurt.req file doesn't seem to be setup to restore those values, so that snapshots made with the default .req file don't restore either.

So I went to each of the suspension directories (/opt/rtcds/caltech/c1/target/c1SYS/c1SYSepics/   where SYS is sus, mcs, or rms) and modified teh autoBurt.req files found there with the following incantation:

sed -i 's/RO $$.*SW[12]R.*$$/\1/' autoBurt.req

This removes the RO at the beginning of the lines which have SW1R or SW2R in them.  These are the channels which correspond to filter bank switches.  As far as I can tell, the RO means to leave it alone.  Unfortunately, I didn't see an obvious autoBurt file specification in the DCC or on google in the 2 minutes I took to look for it.

We need to talk to Alex to figure out how that autoBurt.req file is generated and get it so it doesn't default to not restoring filter bank settings.

4869   Thu Jun 23 22:00:22 2011 JamieUpdateSUSburt snapshot

I recorded a burt snapshot of these settings: /opt/rtcds/caltech/c1/burt/autoburt/snapshots/2011/Jun/23/21:40

1056   Fri Oct 17 21:41:09 2008 YoichiUpdateComputer Scripts / Programsburtwb missing on Solaris but installed on linux64
c1lsc stalled this evening, so I powercycled it.
After that, I tried to lock arms to confirm the computer is working.
Then I realized that the restore alignment buttons do not work from any control room computer.
I found that it was because burtwb command was missing. For Solaris, looks like there used to be /cvs/cds/epics/extensions/burtwb but now
there is no /cvs/cds/epics directory. I thought there were directories other than "caltech" in /cvs/cds/, weren't there ?
Right now, there is only /cvs/cds/caltech.
Anyway, I installed burt for 64bit linux computer (under /cvs/cds/caltech/apps/linux64/epics/extensions/).
At this moment the alignment save/restore works on allegra (and probably on rosalba), but not on op440m yet.
ELOG V3.1.3-


text:
# Difference between revisions of "2015 AMC 10A Problems/Problem 17"

## Problem

A line that passes through the origin intersects both the line $x = 1$ and the line $y=1+ \frac{\sqrt{3}}{3} x$. The three lines create an equilateral triangle. What is the perimeter of the triangle?

$\textbf{(A)}\ 2\sqrt{6} \qquad\textbf{(B)} \ 2+2\sqrt{3} \qquad\textbf{(C)} \ 6 \qquad\textbf{(D)} \ 3 + 2\sqrt{3} \qquad\textbf{(E)} \ 6 + \frac{\sqrt{3}}{3}$

## Solution

Since the triangle is equilateral and one of the sides is a vertical line, the other two sides will have opposite slopes. The slope of the other given line is $\frac{\sqrt{3}}{3}$ so the third must be $-\frac{\sqrt{3}}{3}$. Since this third line passes through the origin, its equation is simply $y = -\frac{\sqrt{3}}{3}x$. To find two vertices of the triangle, plug in $x=1$ to both the other equations.

$y = -\frac{\sqrt{3}}{3}$

$y = 1 + \frac{\sqrt{3}}{3}$

We now have the coordinates of two vertices. $(1, -\frac{\sqrt{3}}{3})$ and $(1, 1 + \frac{\sqrt{3}}{3})$. Apply the distance formula, $\sqrt{(x_1 - x_2)^2 + (y_1 - y_2)^2}$.

$\sqrt{(1-1)^2 + \left(-\frac{\sqrt{3}}{3} - \left(1 + \frac{\sqrt{3}}{3}\right)\right)^2}$

$\sqrt{\left(-\frac{\sqrt{3}}{3} - 1 - \frac{\sqrt{3}}{3}\right)^2}$

The length of one side is $1 + \frac{2\sqrt{3}}{3}$

The perimeter of the triangle is $3 * (1 + \frac{2\sqrt{3}}{3})$, so the answer is $\boxed{\textbf{(D) }3 + 2\sqrt{3}}$


text:
# Some lessons from the Maseno workshop

There are a number of lessons to be learned from the Maseno workshop around the tutor-web and Smileycoin.
First, it seems clear that the suitcase concept can be used in the Kenyan context. Using the laptop as a server completely changes the access speed and appropriate tablets can be found for about $65 in Kenya. This setup could be used directly in a location such as Maseno. Even for the extremely large classes it would be possible to give WiFi access to a single laptop server a central location. The WiFi access point actually tested during this trip had a limit of 15 simultaneous users. But allowing 15 people to download simultaneously would also be more than enough since the download of a set of questions takes less than a minute. One implementation for a class of 600-800 students could therefore be to provide a support room, with access to the laptop server, where students could come to log in and download the material. In a particularly constructive session the workshop participants also demonstrated many alternate possibilities and solutions to issues such as the prohibitive costs and logistics of providing laptops to very many Kenyan high schools as well as the problems associated with low-income high school students who will not be able to purchase tablet computers. Completing the most recent tutor-web developments has led to not only the concept of the laptop+tablets setup but also made the tutor-web software more mobile. In fact to ensure mobility a browser (usually Google Chrome) downloads a drill set which stays in the tablet’s memory and maintaining functionality even when losing the WiFi signal. This is not a function of running on a tablet or Android but a result of functionality provided by the JavaScript language used in the tutor-web development. One interesting aspect of Kenyan life is the common ownership of a mobile phone. Most families will have access to at least one such device even if they can not afford a tablet. Of course some students may own a laptop or mobile phone also. The group in the workshop had a number of devices typical of Kenyans, ranging from laptops to phones running Android or Windows. All of these devices could connect to the tutor-web and work in the appropriate manner. When using a smart phone, these connection can be either to a laptop server as used in the workshop or to the regular tutor-web server. In either case, once a link is established and drill questions are downloaded a student could continue to work on the drills even when offline. Quitting the browser or turning off the device is not a problem since the data are persistent in memory and reappear when restarting the browser. Now, smartphones are not allowed inside Kenyan high schools. But all-in-all, having the tutor-web running on a$65 tablet as well as on already existing mobile phones and laptops should give very many students access to educational material and methods which they have been sorely lacking, even without the laptop server. It may not be possible for all students to use this inside the classroom without the laptop server, but most should be able to access it as a part of a homework assignment.
The most important task of the day may therefore be simply to assist instructors and demonstrate how they can assign tutor-web homework.
A final suggestion during the workshop was to make the Kenyan national exams available as drills in the tutor-web system. This could be done also for primary school material and would make a very useful addition to the existing material.

Archive
2016 (2)
2015 (13)
2014 (4)


text:
# Math Help - Interest Problems

1. ## Interest Problems

I need to use the formula

PV=A/(1+i)^n

where:
PV is the present value
A is the final amount
i is the interest per compounding periods
n is the number of compunding periods

Now the question is:

An investment grows from $600 to$1200 in 9 years. If the interest was compounded monthly, what was the annual rate?

2. Originally Posted by VDestinV
I need to use the formula

PV=A/(1+i)^n

where:
PV is the present value
A is the final amount
i is the interest per compounding periods
n is the number of compunding periods

Now the question is:

An investment grows from $600 to$1200 in 9 years. If the interest was compounded monthly, what was the annual rate?
$A = P \left( 1 + \frac{i}{100} \right) ^{n}$

$1200 = 600 \left( 1 + \frac{i}{100} \right) ^{9 \times 12}$

$1200 = 600 \left( 1 + \frac{i}{100} \right) ^{108}$

$2 = \left( 1 + \frac{i}{100} \right) ^{108}$

$\sqrt[108]{2} = 1 + \frac{i}{100}$

$i = 100 \left( \sqrt[108]{2} - 1 \right)$

This gives you the monthly rate.

$12 \times i = 7.73$
That would be the annual rate.


text:
# fa.bianp.net

Cross-validation iterators in scikit-learn are simply generator objects, that is, Python objects that implement the __iter__ method and that for each call to this method return (or more precisely, yield) the indices or a boolean mask for the train and test set. Hence, implementing new cross-validation iterators that behave as the ones in scikit-learn is easy with this in mind. Here goes a small code snippet that implements a holdout cross-validator generator following the scikit-learn API.

import numpy as np
from sklearn.utils import check_random_state

class HoldOut:
"""
Hold-out cross-validator generator. In the hold-out, the
data is split only once into a train set and a test set.
Unlike in other cross-validation schemes, the hold-out
consists of only one iteration.

Parameters
----------
n : total number of samples
test_size : 0 < float < 1
Fraction of samples to use as test set. Must be a
number between 0 and 1.
random_state : int
Seed for the random number generator.
"""
def __init__(self, n, test_size=0.2, random_state=0):
self.n = n
self.test_size = test_size
self.random_state = random_state

def __iter__(self):
n_test = int(np.ceil(self.test_size * self.n))
n_train = self.n - n_test
rng = check_random_state(self.random_state)
permutation = rng.permutation(self.n)
ind_test = permutation[:n_test]
ind_train = permutation[n_test:n_test + n_train]
yield ind_train, ind_test

Contrary to other cross-validation schemes, holdout relies on a single split of the data. It is well known than in practice holdout performs much worse than KFold or LeaveOneOut schemes. However, holdout has the advantage that its theoretical properties are easier to derive. For examples of this see e.g. Section 8.7 of Theory of classification: a survey of some recent advances and the very recent The reusable holdout.