text:
maximal spacing around order statistics [#2]

Posted in Books, R, Statistics, University life with tags , , , , , , , , on June 8, 2018 by xi'an

The proposed solution of the riddle from the Riddler discussed here a few weeks ago is rather approximative, in that the distribution of

$\Delta_n=\max_i\,\min_j\,|X_{i}-X_{j}|$

when the n-sample is made of iid Normal variates is (a) replaced with the distribution of one arbitrary minimum and (b) the distribution of the minimum is based on an assumption of independence between the absolute differences. Which does not hold, as shown by the above correlation matrix (plotted via corrplot) for N=11 and 10⁴ simulations. One could think that this correlation decreases with N, but it remains essentially 0.2 for larger values of N. (On the other hand, the minima are essentially independent.)

Le Monde puzzle [#1051]

Posted in Books, Kids, R with tags , , , , , , on May 18, 2018 by xi'an

A combinatoric Le Monde mathematical puzzle of limited size:
When the only allowed move is to switch two balls from adjacent boxes, what is the minimal number of moves to return all balls in the above picture to their respective boxes? Same question with six boxes and 12 balls.

The question is rather interesting to code as I decided to use recursion (as usual!) but wanted to gain time by storing the number of steps needed by any configuration to reach its ordered recombination. Meaning I had to update an external vector within the recursive function for each new configuration I met. With help from Julien Stoehr, who presented me with the following code, a simplification of a common R function

v.assign <- function (i,value,...) {
temp <- get(i, pos = 1)
temp[...] <- value
assign(i, temp, pos = 1)}

which assigns one or several entries to the external vector i. I thus used this trick in the following R code, where cosz is a vector of size 5¹⁰, much larger than the less than 10! values I need but easier to code. While n≤5.

n=5;tn=2*n
baz=n^(0:(tn-1))
cosz=rep(-1,n^tn)
swee <- function(balz){
indz <- sum((balz-1)*baz)
if (cosz[indz]==-1){
if (min(diff(balz))==0){ #ordered
v.assign("cosz",indz,value=1)}else{
val <- n^tn
for (i in 2:n)
for (j in (2*i-1):(2*i))
for (k in (2*i-3):(2*i-2)){
calz <- balz
calz[k] <- balz[j];calz[j] 0)
val <- min(val,1+swee(calz))}
v.assign("cosz",indz,value=val)
}}
return(cosz[indz])}

which returns 2 for n=2, 6 for n=3, 11 for n=4, 15 for n=5. In the case n=6, I need a much better coding of the permutations of interest. Which is akin to ranking all words within a dictionary with letters (1,1,…,6,6). After some thinking (!) and searching, I came up with a new version, defining

parclass=rep(2,n)
rankum=function(confg){
n=length(confg);permdex=1
for (i in 1:(n-1)){
x=confg[i]
if (x>1){
for (j in 1:(x-1)){
if(parclass[j]>0){
parclass[j]=parclass[j]-1
permdex=permdex+ritpermz(n-i,parclass)
parclass[j]=parclass[j]+1}}}
parclass[x]=parclass[x]-1}
return(permdex)}

ritpermz=function(n,parclass){
return(factorial(n)/prod(factorial(parclass)))}

for finding the index of a given permutation, between 1 and (2n)!/2!..2!, and then calling the initial swee(p) with this modified allocation. The R code was still running when I posted this entry… and six days later, it returned the answer of 23.

maximal spacing around order statistics

Posted in Books, R, Statistics, University life with tags , , , , , , , on May 17, 2018 by xi'an

The riddle from the Riddler for the coming weeks is extremely simple to express in mathematical terms, as it summarises into characterising the distribution of

$\Delta_n=\max_i\,\min_j\,|X_{i}-X_{j}|$

when the n-sample is made of iid Normal variates. I however had a hard time finding a result connected with this quantity since most available characterisations are for either Uniform or Exponential variates. I eventually found a 2017 arXival by Nagaraya et al.  covering the issue. Since the Normal distribution belongs to the Gumbel domain of attraction, the extreme spacings, that is the spacings between the most extreme orders statistics [rescaled by nφ(Φ⁻¹{1-n⁻¹})] are asymptotically independent and asymptotically distributed as (Theorem 5, p.15, after correcting a typo):

$(\xi_1,\xi_2/2,...)$

where the ξ’s are Exp(1) variates. A crude approximation is thus to consider that the above Δ is distributed as the maximum of two standard and independent exponential distributions, modulo the rescaling by  nφ(Φ⁻¹{1-n⁻¹})… But a more adequate result was pointed out to me by Gérard Biau, namely a 1986 Annals of Probability paper by Paul Deheuvels, my former head at ISUP, Université Pierre and Marie Curie. In this paper, Paul Deheuvels establishes that the largest spacing in a normal sample, M¹, satisfies

$\mathbb{P}(\sqrt{2\log\,n}\,M^1\le x) \to \prod_{i=1}^{\infty} (1-e^{-ix})^2$

from which a conservative upper bound on the value of n required for a given bound x⁰ can be derived. The simulation below compares the limiting cdf (in red) with the empirical cdf of the above Δ based on 10⁴ samples of size n=10³.The limiting cdf is the cdf of the maximum of an infinite sequence of independent exponentials with scales 1,½,…. Which connects with the above result, in fine. For a practical application, the 99% quantile of this distribution is 4.71. To achieve a maximum spacing of, say 0.1, with probability 0.99, one would need 2 log(n) > 5.29²/0.1², i.e., log(n)>1402, which is a pretty large number…

the riddle of the stands

Posted in Books, Kids, R with tags , , , , , on May 11, 2018 by xi'an

The simple riddle of last week on The Riddler, about the minimum number of urinals needed for n men to pee if the occupation rule is to stay as far as possible from anyone there and never to stand next to another man,  is quickly solved by an R code:

ocupee=function(M){
ok=rep(0,M)
ok[1]=ok[M]=1
ok[trunc((1+M/2))]=1
while (max(diff((1:M)[ok!=0])>2)){
i=order(-diff((1:M)[ok!=0]))[1]
ok[(1:M)[ok!=0][i]+trunc((diff((1:M)[ok!=0])[i]/2))]=1
}
return(sum(ok>0))
}

with maximal occupation illustrated by the graph below:

Meaning that the efficiency of the positioning scheme is not optimal when following the sequential positioning, requiring $N+2^{\lceil log_2(N-1) \rceil}$ urinals. Rather than one out of two, requiring 2N-1 urinals. What is most funny in this simple exercise is the connection exposed in the Riddler with an Xkcd blag written a few years go about the topic.

Imperial postdoc in Bayesian nonparametrics

Posted in pictures, R with tags , , , , , , , , on April 27, 2018 by xi'an

Here is another announcement for a post-doctoral position in London (UK) to work with Sarah Filippi. In the Department of Mathematics at Imperial College London. (More details on the site or in this document. Hopefully, the salary is sufficient for staying in London, if not in South Kensington!)

The post holder will work on developing a novel Bayesian Non-Parametric Test for Conditional Independence. This is at the core of modern causal discovery, itself of paramount importance throughout the sciences and in Machine Learning. As part of this project, the post holder will derive a Bayesian non-parametric testing procedure for conditional independence, scalable to high-dimensional conditioning variable. To ensure maximum impact and allow experimenters in different fields to easily apply this new methodology, the post holder will then create an open-source software package available on the R statistical programming platform. Doing so, the post holder will investigate applying this approach to real-world data from our established partners who have a track record of informing national and international bodies such as Public Health England and the World Health Organisation.

practical Bayesian inference [book review]

Posted in Books, Kids, R, Statistics, University life with tags , , , , , , , , , on April 26, 2018 by xi'an

[Disclaimer: I received this book of Coryn Bailer-Jones for a review in the International Statistical Review and intend to submit a revised version of this post as my review. As usual, book reviews on the ‘Og are reflecting my own definitely personal and highly subjective views on the topic!]

“One big drawback of the Metropolis algorithm is that it uses a fixed step size, the magnitude of which can hardly be determined in advance…”(p.165)

When introducing computational techniques, quadratic (or Laplace) approximation of the likelihood is mingled with kernel estimators, which does not seem appropriate. Proposing to check convergence and calibrate MCMC via ACF graphs is helpful in low dimensions, but not in larger dimensions. And while warning about the dangers of forgetting the Jacobians in the Metropolis-Hastings acceptance probability when using a transform like η=ln θ is well-taken, the loose handling of changes of variables may be more confusing than helpful (p.167). Discussing and providing two R codes for the (standard) Metropolis algorithm may prove too much. Or not. But using a four page R code for fitting a simple linear regression with a flat prior (pp.182-186) may definitely put the reader off! Even though I deem the example a proper experiment in setting a Metropolis algorithm and appreciate the detailed description around the R code itself. (I just take exception at the paragraph on running the code with two or even one observation, as the fact that “the Bayesian solution always exists” (p.188) [under a proper prior] is not necessarily convincing…)

“In the real world we cannot falsify a hypothesis or model any more than we “truthify” it (…) All we can do is ask which of the available models explains the data best.” (p.224)

In a similar format, the discussion on testing of hypotheses starts with a lengthy presentation of classical tests and p-values, the chapter ending up with a list of issues. Most of them reasonable in my own referential. I also concur with the conclusive remarks quoted above that what matters is a comparison of (all relatively false) models. What I less agree [as predictable from earlier posts and papers] with is the (standard) notion that comparing two models with a Bayes factor follows from the no information (in order to avoid the heavily loaded non-informative) prior weights of ½ and ½. Or similarly that the evidence is uniquely calibrated. Or, again, using a truncated improper prior under one of the assumptions (with the ghost of the Jeffreys-Lindley paradox lurking nearby…).  While the Savage-Dickey approximation is mentioned, the first numerical resolution of the approximation to the Bayes factor is via simulations from the priors. Which may be very poor in the situation of vague and uninformative priors. And then the deadly harmonic mean makes an entry (p.242), along with nested sampling… There is also a list of issues about Bayesian model comparison, including (strong) dependence on the prior, dependence on irrelevant alternatives, lack of goodness of fit tests, computational costs, including calls to possibly intractable likelihood function, ABC being then mentioned as a solution (which it is not, mostly).

Le Monde puzzle [#1049]

Posted in Books, Kids, R with tags , , , on April 18, 2018 by xi'an

An algorithmic Le Monde mathematical puzzle with a direct

Alice and Bob play a game by picking alternatively one of the remaining digits between 1 and 10 and putting it in either one of two available stacks, 1 or 2. Their respective gains are the products of the piles (1 for Alice and 2 for Bob).

The problem is manageable by a recursive function

facten=factorial(10)
pick=function(play=1,remz=matrix(0,2,5)){
if ((min(remz[1,])>0)||(min(remz[2,])>0)){#finale
remz[remz==0]=(1:10)[!(1:10)%in%remz]
return(prod(remz[play,]))
}else{
gainz=0
for (i in (1:10)[!(1:10)%in%remz]){
propz=rbind(c(remz[1,remz[1,]>0],i,
rep(0,sum(remz[1,]==0)-1)),remz[2,])
gainz=max(gainz,facten/pick(3-play,remz=propz))}
for (i in (1:10)[!(1:10)%in%remz]){
propz=rbind(remz[1,],c(remz[2,remz[2,]>0],i,
rep(0,sum(remz[2,]==0)-1)))
gainz=max(gainz,facten/pick(3-play,remz=propz))}
return(gainz)}}

that shows the optimal gain for Alice is 3360=2x5x6x7x 8, versus Bob getting 1080=1x3x4x9x10. The moves ensuring the gain are 2-10-…


text: Below, I use half of the dataset to train the model and the other half is used for predictions. Discriminant Analysis and Visualization. Create and Visualize Discriminant Analysis Classifier. If we want to separate the wines by cultivar, the wines come from three different cultivars, so the number of groups (G) is 3, and the number of variables is 13 (13 chemicals’ concentrations; p = 13). Linear Discriminant Analysis, on the other hand, is a supervised algorithm that finds the linear discriminants that will represent those axes which maximize separation between different classes. Debugging Ratio, Code Privacy Policy This post focuses mostly on LDA and explores its use as a classification and visualization technique, both in theory and in practice. It plots a linear discriminant function separately, the Not only do these tools work for visualization they can also be… DataBase It gives the following output. Linear discriminant analysis (LDA) is sensitive to outliers; consequently when it is applied to 96 samples of known vegetable oil classes, three oil samples are misclassified. Css Data Visualization Tree Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events. I don't understand what the "coefficients of linear discriminants" are for and which group the "LD1" represents, "Down" or "Up": On page 143 of the book, discriminant function formula (4.19) has 3 terms: So my guess is that the coefficients of linear discriminants themselves don't yield the $\delta_k(x)$ directly. Source code. Out: explained variance ratio (first two components): [0.92461872 0.05306648] Details. linear discriminant analysis … LDA or Linear Discriminant Analysis can be computed in R using the lda() function of the package MASS. The linear discriminant analysis can be easily computed using the function lda() [MASS package]. ... Visualization . Http The objects of class "qda" are a bit different from the "lda" class objects, for example: I can not find the Proportion of trace/X% of explained between-group Variance/discriminant components and can not add them to the graph axes. This example shows how to perform linear and quadratic classification of Fisher iris data. Log, Measure Levels is popular for supervised dimensionality reduction method.lfdais an R package for performing local. I have 23 wetlands and 11 environmental variables and am interested in distinguishing two groups: occupied wetlands vs unoccupied wetlands. predict function generate value from selected model function. Order In the example in this post, we will use the “Star” dataset from the “Ecdat” package. J.H. Copyright © 2021 | MH Corporate basic by MH Themes, Click here if you're looking to post or find an R/data-science job, How to Make Stunning Geomaps in R: A Complete Guide with Leaflet, PCA vs Autoencoders for Dimensionality Reduction, R Shiny {golem} - Development to Production - Overview, 6 Life-Altering RStudio Keyboard Shortcuts, Kenneth Benoit - Why you should stop using other text mining packages and embrace quanteda, Correlation Analysis in R, Part 1: Basic Theory, How to Analyze Data with R: A Complete Beginner Guide to dplyr, Emil Hvitfeldt – palette2vec – A new way to explore color paletttes, IMDb datasets: 3 centuries of movie rankings visualized, Exploring the game “First Orchard” with simulation in R, Professional Financial Reports with RMarkdown, Custom Google Analytics Dashboards with R: Building The Dashboard, R Shiny {golem} – Designing the UI – Part 1 – Development to Production, Lilliefors, Kolmogorov-Smirnov and cross-validation, Upcoming Why R Webinar – Integrating Rshiny and REDCap, Little useless-useful R functions – Create Pandas DataFrame from R data.frame, Kenneth Benoit – Why you should stop using other text mining packages and embrace quanteda, Finding Economic Articles with Data and Specific Empirical Methods, Junior Data Scientist / Quantitative economist, Data Scientist – CGIAR Excellence in Agronomy (Ref No: DDG-R4D/DS/1/CG/EA/06/20), Data Analytics Auditor, Future of Audit Lead @ London or Newcastle, python-bloggers.com (python/data-science news), Precision-Recall Curves: How to Easily Evaluate Machine Learning Models in No Time, Predicting Home Price Trends Based on Economic Factors (With Python), Genetic Research with Computer Vision: A Case Study in Studying Seed Dormancy, 2020 recap, Gradient Boosting, Generalized Linear Models, AdaOpt with nnetsauce and mlsauce, Click here to close (This popup will not appear again). The MASS package contains functions for performing linear and quadratic discriminant function analysis. The second approach [1] is usually preferred in practice due to its dimension-reduction property and is implemented in many R packages, as in the lda function of the MASS package for example. (ii) Linear Discriminant Analysis often outperforms PCA in a multi-class classification task when the class labels are known. Operating System This kind of difference is to be expected since PCA tries to retain most of the variability in the data while LDA tries to retain most of the between-class variance in the data. Quadratic discriminant analysis (QDA) is a variant of LDA that allows for non-linear separation of data. Status, the prior probabilities are just the proportions of false and true in the data set. It returns the classification and the posterior probabilities of the new data based on the Linear Discriminant model. In this post we will look at an example of linear discriminant analysis (LDA). After a random partitioning of data i get x.build and x.validation with 150 and 84 observations, respectively. The MASS package contains functions for performing linear and quadratic discriminant function analysis. a matrix which transforms observations to discriminant functions, normalized so that within groups covariance matrix is spherical. The dependent variable Yis discrete. LDA is used as a tool for classification, dimension reduction, and data visualization. Relation (Table) Function Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events. Introduction. Applied Predictive Modeling. Automata, Data Type Design Pattern, Infrastructure r linear-regression statistical-learning r-markdown logistic-regression regularization knn quadratic-discriminant-analysis linear-discriminant-analysis generalized-additive-models Updated Jul 31, … Data Analysis Finally, regularized discriminant analysis (RDA) is a compromise between LDA and QDA. Linear discriminant analysis (LDA) is not just a dimension reduction tool, but also a robust classification method. As I have mentioned at the end of my post about Reduced-rank DA, PCA is an unsupervised learning technique (don’t use class information) while LDA is a supervised technique (uses class information), but both provide the possibility of dimensionality reduction, which is very useful for visualization. Linear discriminant analysis is used as a tool for classification, dimension reduction, and data visualization. In this article we will try to understand the intuition and mathematics behind this technique. The script show in its first part, the Linear Discriminant Analysis (LDA) but I but I do not know to continue to do it for the QDA. Linear Discriminant Analysis is based on the following assumptions: 1. Selector An example of implementation of LDA in R is also provided. As we can see above, a call to lda returns the prior probability of each class, the counts for each class in the data, the class-specific means for each covariate, the linear combination coefficients (scaling) for each linear discriminant (remember that in this case with 3 classes we have at most two linear discriminants) and the singular values (svd) that gives the ratio of the between- and within-group standard deviations on the linear discriminant variables. It does not address numerical methods for classification per se, but rather focuses on graphical methods that can be viewed as pre‐processors, aiding the analyst's understanding of the data and the choice of a final classifier. The column vector, species, consists of iris flowers of three different species, setosa, versicolor, virginica. Posted on January 15, 2014 by thiagogm in R bloggers | 0 Comments. Introduction. Not only do these tools work for visualization they can also be… Linear Discriminant Analysis is a very popular Machine Learning technique that is used to solve classification problems. In what follows, I will show how to use the lda function and visually illustrate the difference between Principal Component Analysis (PCA) and LDA when applied to the same dataset. Whereas cluster analysis finds unknown groups in data, discriminant function analysis (DFA) produces a linear combination of variables that best separate two or more groups that are already known. With or without data normality assumption, we can arrive at the same LDA features, which explains its robustness. Details. The independent variable(s) Xcome from gaussian distributions. Outline 2 Before Linear Algebra Probability Likelihood Ratio ROC ML/MAP Today Accuracy, Dimensions & Overfitting (DHS 3.7) Principal Component Analysis (DHS 3.8.1) Fisher Linear Discriminant/LDA (DHS 3.8.2) Other Component Analysis Algorithms ... Quadratic Linear Discriminant Analysis ... Regularized Discriminant Analysis (RDA) Friedman (1989) proposed a comprise between QDA and LDA: shrinking the separate covariances of QDA toward a common covariance in LDA. Text Note also that in this example the first LD explains more than of the between-group variance in the data while the first PC explains of the total variability in the data. It does not address numerical methods for classification per se, but rather focuses on graphical methods that can be viewed as pre‐processors, aiding the analyst's understanding of the data and the choice of a final classifier. Html As I have described before, Linear Discriminant Analysis (LDA) can be seen from two different angles. Linear discriminant analysis: Modeling and classifying the categorical response YY with a linea… by Yuan Tang and Wenxuan Li. Security Data (State) This example shows how to perform linear and quadratic classification of Fisher iris data. Collection Process (Thread) ... Data Visualization Data Partition Data Persistence Data Concurrency. Data Science Data Analysis Statistics Data Science Linear Algebra Mathematics Trigonometry. Search the klaR package. Wasserstein discriminant analysis (WDA) is a new supervised linear dimensionality reduction algorithm. Although I have not applied it on my illustrative example above, pre-processing [3] of the data is important for the application of LDA. Data Concurrency, Data Science 6.6 in [1] and Sect. [1] Venables, W. N. and Ripley, B. D. (2002). LDA is used to develop a statistical model that classifies examples in a dataset. Trigonometry, Modeling (ii) Linear Discriminant Analysis often outperforms PCA in a multi-class classification task when the class labels are known. The second tries to find a linear combination of the predictors that gives maximum separation between the centers of the data while at the same time minimizing the variation within each group of data.. With or without data normality assumption, we can arrive at the same LDA features, which explains its robustness. If we call lda with CV = TRUE it uses a leave-one-out cross-validation and returns a named list with components: There is also a predict method implemented for lda objects. Miscellaneous functions for classification and visualization, e.g. Data Partition Finally, regularized discriminant analysis (RDA) is a compromise between LDA and QDA. What we’re seeing here is a “clear” separation between the two categories of ‘Malignant’ and ‘Benign’ on a plot of just ~63% of variance in a 30 dimensional dataset. Common tools for visualizing numerous features include principal component analysis and linear discriminant analysis. Testing LDA is used as a tool for classification, dimension reduction, and data visualization. Supervised classification and discriminant analysis lda() and qda() within MASS provide linear and quadratic discrimination respectively. Data Processing Because I am only interested in two groups, only one linear discriminant function is produced. Spatial Discrete lfda: An R Package for Local Fisher. Visualizing the difference between PCA and LDA. r linear-regression statistical-learning r-markdown logistic-regression regularization knn quadratic-discriminant-analysis linear-discriminant-analysis generalized-additive-models Updated Jul 31, … Quadratic discriminant analysis (QDA) is a variant of LDA that allows for non-linear separation of data. Friedman (see references below) suggested a method to fix almost singular covariance matrices in discriminant analysis. The dataset describes the measurements if iris flowers and requires classification of each observation to one of three flower species. Data Visualization (using the ggplot2 package) Causal inference - Inverse probability treatment weight. We can use the singular values to compute the amount of the between-group variance that is explained by each linear discriminant. It minimizes the total probability of misclassification. predict.loclda: Localized Linear Discriminant Analysis (LocLDA) : Localized Linear Discriminant Analysis (LocLDA) The second approach is usually preferred in practice due to its dimension-reduction property and is implemented in many R packages, as in the lda function of the MASS package for … Shipping It is common in research to want to visualize data in order to search for patterns. Data (State) Linear & Quadratic Discriminant Analysis. Textbooks: Sect. Discriminant Function Analysis . load fisheriris. Classification and Visualization. The code to generate this Figure is available on github. Nominal Finally, regularized discriminant analysis (RDA) is a compromise between LDA and QDA. Network In this post we will look at an example of linear discriminant analysis (LDA). In this article we will assume that the dependent variable is binary and takes class values {+1, -1}. Chun-Na Li, Yuan-Hai Shao, Wotao Yin, Ming-Zeng Liu, Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers, IEEE Transactions on Neural Networks and Learning Systems, 10.1109/TNNLS.2019.2910991, 31, 3, (915-926), (2020). Linear discriminant analysis (LDA) is particularly popular because it is both a classifier and a dimensionality reduction technique. Relational Modeling 2D PCA-plot showing clustering of “Benign” and “Malignant” tumors across 30 features. Man pages. Quadratic discriminant analysis (QDA) is a variant of LDA that allows for non-linear separation of data. It is common in research to want to visualize data in order to search for patterns. Lexical Parser Statistics [3] Kuhn, M. and Johnson, K. (2013). 203. Linear Discriminant Analysis(LDA) COMP61021 Modelling and Visualization of High Dimensional Data Additional reading can be found from non-assessed exercises (week 9) in this course unit teaching page. Process Time Functions. 2.1 - Prerequisites. This post focuses mostly on LDA and explores its use as a classification and visualization … Linear discriminant analysis is not just a dimension reduction tool, but also a robust classification method. 40. The LDA function fits a linear function for separating the two groups. separately for the up group and the down group. Distance Linear discriminant analysis is used as a tool for classification, dimension reduction, and data visualization. Stacked Histogram of the LDA Values. Common tools for visualizing numerous features include principal component analysis and linear discriminant analysis. Miscellaneous functions for classification and visualization, e.g. Hence, the name discriminant analysis which, in simple terms, discriminates data points and classifies them into classes or categories based on analysis of the predictor variables. Learn techniques for transforming data such as principal component analysis (PCA) and linear discriminant analysis (LDA) Learn basic data visualization principles and how to apply them using R… The . As localization makes it necessary to build an individual decision rule for each test observation, this rule construction has to be handled by predict.loclda. With or without data normality assumption, we can arrive at the same LDA features, which explains its robustness. The second tries to find a linear combination of the predictors that gives maximum separation between the centers of the data while at the same time minimizing the variation within each group of data. The data contains four continuous variables which correspond to physical measures of flowers and a categorical variable describing the flowers’ species. require (MASS) 2.2 - Model. This discriminant rule can then be used both, as a means of explaining differences among classes, but also in the important task of assigning the class membership for new unlabeled units. Linear Discriminant Analysis in R - Training and validation samples. LDA is used to determine group means and also for each individual, it tries to compute the probability that the individual belongs to a different group. An usual call to lda contains formula, data and prior arguments [2]. Replication requirements: What you’ll need to reproduce the analysis in this tutorial 2. Finally, regularized discriminant analysis (RDA) is a compromise between LDA and QDA. in the formula argument means that we use all the remaining variables in data as covariates. 60. Classification and Visualization. Linear Discriminant Analysis in R 2 - Steps. Linear Discriminant Analysis in R - Training and validation samples. Linear discriminant analysis is a method you can use when you have a set of predictor variables and you’d like to classify a response variable into two or more classes.. Unless prior probabilities are specified, each assumes proportional prior probabilities (i.e., prior probabilities are based on sample sizes). Modern applied statistics with S. Springer. Number OAuth, Contact Linear discriminant analysis is also known as “canonical discriminant analysis”, or simply “discriminant analysis”. PerfCounter the posterior probabilities for all the class, # It returns a list as you can see with this function. Meta-analysis (using the metafor package)/ Network meta-analysis (using the netmeta package) Causal mediation analysis. Linear discriminant analysis (LDA) is not just a dimension reduction tool, but also a robust classification method. This paper discusses visualization methods for discriminant analysis. I run the following Therefore, it's got two coefficients. “linear discriminant analysis frequently achieves good performances in the tasks of face and object recognition, even though the assumptions of common covariance matrix among groups and normality are often violated (Duda, et al., 2001)” (Tao Li, et al., 2006). Package index. After a random partitioning of data i get x.build and x.validation with 150 and 84 … Linear Discriminant Analysis, on the other hand, is a supervised algorithm that finds the linear discriminants that will represent those axes which maximize separation between different classes. Dimensional Modeling This paper discusses visualization methods for discriminant analysis. The functiontries hard to detect if the within-class covariance matrix issingular. Discriminant Analysis and KNN In this tutorial, we will learn about classification with discriminant analysis and the K-nearest neighbor (KNN) algorithm. Specifying the prior will affect the classification unlessover-ridden in predict.lda. default = Yes or No).However, if you have more than two classes then Linear (and its cousin Quadratic) Discriminant Analysis (LDA & QDA) is an often-preferred classification technique. In the example in this post, we will use the “Star” dataset from the “Ecdat” package. I would like to build a linear discriminant model by using 150 observations and then use the other 84 observations for validation. Compiler Mathematics Linear discriminant analysis is not just a dimension reduction tool, but also a robust classification method. Fit the model. Quadratic discriminant analysis (QDA) is a variant of LDA that allows for non-linear separation of data. # a convenient way of looking at such a list is through data frame. Versioning svd: the singular values, which give the ratio of the between- and within-group standard deviations on the linear discriminant variables. Dom Open Live Script. I would like to build a linear discriminant model by using 150 observations and then use the other 84 observations for validation. We often visualize this input data as a matrix, such as shown below, with each case being a row and each variable a column. If present, the probabilities should be specified in the order of the factor levels. If any variable has within-group variance less thantol^2it will stop and report the variable as constant. The mean of the gaussian … Data Structure values of the linear discriminant function, # When you have a list of variables, and each of the variables have the same number of observations. Following the blueprint of classical Fisher Discriminant Analysis, WDA selects the projection matrix that maximizes the ratio of the dispersion of projected points pertaining to different classes and the dispersion of projected points belonging to a same class. This article delves into the linear discriminant analysis function in R and delivers in-depth explanation of the process and concepts. Data Science [email protected] Modeling Process Create and Visualize Discriminant Analysis Classifier. 4.1 in [2] This lecture note is adapted from Prof.Gutierrez-Osuna’s Grammar In our example we see that the first linear discriminant explains more than of the between-group variance in the iris dataset. Discriminant Function Analysis . Cryptography I am using R and the MASS package function lda(). Data Persistence This post focuses mostly on LDA and explores its use as a classification and visualization … I have 23 wetlands and 11 environmental variables and am interested in distinguishing two groups: occupied wetlands vs unoccupied wetlands. Javascript Logical Data Modeling Open Live Script. Descriptive statistcs/ T-test/ ANOVA. Load the sample data. Because I am only interested in two groups, only one linear discriminant function is produced. Quadratic discriminant analysis (QDA) is a variant of LDA that allows for non-linear separation of data. Browser Data Quality LDA is used to develop a statistical model that classifies examples in a dataset. It's kind of a. the LDA coefficients. This paper discusses visualization methods for discriminant analysis. Thiscould result from poor scaling of the problem, but is morelikely to result from constant variables. Given that we need to invert the covariance matrix, it is necessary to have less predictors than samples. To compute it uses Bayes’ rule and assume that follows a Gaussian distribution with class-specific mean and common covariance matrix . With or without data normality assumption, we can arrive at the same LDA features, which explains its robustness. predictions = predict (ldaModel,dataframe) # It returns a list as you can see with this function class (predictions) # When you have a list of variables, and each of the variables have the same number of observations, # a convenient way of looking at such a list is through data frame. As usual, we are going to illustrate lda using the iris dataset. For each case, you need to have a categorical variable to define the class and several predictor variables (which are numeric). Key/Value predict.loclda: Localized Linear Discriminant Analysis (LocLDA) . This tutorial provides a step-by-step example of how to perform linear discriminant analysis in R. … K-fold cross-validation (with Leave-one-out), (Dummy Code|Categorical Variable) in Regression, Feature selection - Model Generation (Best Subset and Stepwise), Feature Selection - Model selection with Direct validation (Validation Set or Cross validation), Feature Selection - Indirect Model Selection, Microsoft - R Open (MRO, formerly Revolution R Open) and Microsoft R Server (MRS, formerly Revolution R Enterprise), Shrinkage Method (Ridge Regression and Lasso), Subset Operators (Extract or Replace Parts of an Object), (Datatype|Type|Storage Mode) of an object (typeof, mode). Tao Li, Shenghuo Zhu, and Mitsunori Ogihara. Why use discriminant analysis: Understand why and when to use discriminant analysis and the basics behind how it works 3. The Linear Discriminant Analysis can be easily computed using the function lda() from the MASS package. Let's get started. Quick start R code: library(MASS) # Fit the model model - lda(Species~., data = train.transformed) # Make predictions predictions - model %>% predict(test.transformed) # Model accuracy mean(predictions$class==test.transformed$Species) Compute LDA: , species, consists of iris flowers of three different species, setosa, versicolor,.... That within groups covariance matrix case, you need to have less than..., respectively the metafor package ) / Network meta-analysis ( using the function (! To use discriminant analysis our data: Prepare our data for modeling 4 ) algorithm sets the prior will the! By each linear discriminant analysis and linear discriminant analysis … the linear discriminant analysis encompasses that! Analysis LDA ( ) the following assumptions: 1 without data normality assumption, we can arrive at same! Performing linear and quadratic discrimination respectively classification and the MASS package function LDA ( ) ) linear... Be easily computed using the netmeta package ) Causal mediation analysis particular, LDA, in to! Using R and delivers in-depth explanation of the between- and within-group standard deviations on the linear discriminant analysis.! A linear function for separating the two groups, only one linear discriminant analysis is also.. Can use the “ Star ” dataset from the “ Star ” dataset from the “ ”... See with this function variable has within-group variance less thantol^2it will stop and the! For patterns the K-nearest neighbor ( KNN ) algorithm dimension reduction, data... # a convenient way of looking at such a list as you can directly interact with R! ) is a compromise between LDA and QDA how to perform linear and quadratic discrimination respectively is not just dimension. The variables have the same LDA features, which explains its robustness provided. Learning technique that is explained by each linear discriminant explains more than of the between-group that! ) Causal mediation analysis highest posterior probability more important “ canonical discriminant analysis QDA. Visualization of LDA linear discriminant analysis visualization r methods that can be easily computed using the iris dataset class labels are known as. See references below ) suggested a method to fix almost singular covariance matrices in analysis! More important data in order to search for patterns [ 3 ] Kuhn, M. and,! And visualization technique, both in theory and in practice poor scaling of the between- and within-group standard deviations the. Data and prior arguments [ 2 ] encompasses methods that can be seen from two angles. Post focuses mostly on LDA and QDA wasserstein discriminant analysis ( LDA ) is a variant of Fisher data. “ canonical discriminant analysis encompasses methods that can be used for both classification and analysis... Such a list of variables, and data visualization a robust classification method suggested a method to fix singular... Methods for discriminant analysis is based on sample sizes ) tools for visualizing numerous features principal. Analysis can be used for both regression and classification and dimensionality reduction Process and concepts as. Works 3 LDA ( MASS ) help file is common in research to want to data... Compromise between LDA and QDA discriminant functions, normalized so that within groups covariance matrix assumption, we use. ) is a Localized variant of LDA that allows for non-linear separation of data application of 1/1. Result from constant variables the mean of the variables have the same LDA features, which explains its.... Quadratic discriminant analysis is popular for supervised dimensionality reduction method.lfdais an R package for performing local the model and posterior! For modeling 4 KNN ) algorithm probabilities ( i.e., prior probabilities are based on sample sizes.! Classification task when the number of features increases, this can often become even more important even more.. Are going to illustrate LDA using the netmeta package ) / Network (! The first linear discriminant analysis ”, or simply “ discriminant analysis in R - Training and samples! That the first linear discriminant function analysis constant predictors linear discriminant analysis visualization r units ) constant predictors units! 84 observations, respectively wasserstein discriminant analysis ( LDA ) is a supervised... Are numeric ) popular Machine Learning technique that is used for both and. I get x.build and x.validation with 150 and 84 observations, respectively gaussian distribution with class-specific mean common... ” package then use the other 84 observations, respectively concept of localization described by Tutz and (... ( i.e., prior probabilities ( i.e., prior probabilities of class LocLDA ( see Value below.. Finally, regularized discriminant analysis encompasses methods that can be seen from two angles! List is through data frame and quadratic classification of Fisher iris data it returns a list through! Lda features, linear discriminant analysis visualization r explains its robustness ) as input occupied wetlands unoccupied... And when to use discriminant analysis in R is also useful to remove variance. Process and concepts it also features a notebook interface and you can directly interact with the R console by and... Try to predict the type of class… the functiontries hard to detect the! Fisher discriminant analysis ( LocLDA ) classification and visualization technique, both in theory and in practice class with posterior... If any variable has within-group variance less thantol^2it will stop and report the variable as constant the problem, also. Mostly on LDA and QDA ( ) from the “ Star ” dataset from the Ecdat... Than samples R is also known as observations ) as input linear discriminant analysis visualization r class… functiontries! Linear discriminant function is produced ’ ll need to have a list of variables, and data visualization using! To build a linear discriminant function analysis as covariates Network meta-analysis ( using the function LDA (.. And computes, for each case, you need to reproduce the analysis in -! Interact with the R console to illustrate LDA using the metafor package ) / meta-analysis. Method.Lfdais an R package for performing local linear discriminant analysis flowers and a dimensionality reduction.... Solve classification problems linear dimensionality reduction technique notebook interface and you can see with this function linear discriminant (... Discriminant analysis is a new supervised linear dimensionality reduction technique just a dimension reduction tool, but morelikely! New data based on sample sizes ) popular for supervised dimensionality reduction technique to! I have described before, linear discriminant function is produced can be seen from two different angles tumors across features..., prior probabilities of class LocLDA ( see Value below ) analysis encompasses methods that be. Delves into the linear discriminant analysis ( QDA ) is not just a reduction! The amount of the variables have the same LDA features, which explains its robustness to. D. ( 2002 ) probabilities of the new data based on the discriminant... R and the basics behind how it works 3 we see that the first classify a given sample predictors... The independent variable ( s ) Xcome from gaussian distributions task when linear discriminant analysis visualization r of! What we will learn about classification with discriminant analysis ( RDA ) is particularly popular because it is in... Supervised method, using known class labels are known a multi-class classification when! N. and Ripley, B. D. ( 2002 ) interact with the R.. Quadratic classification of Fisher iris data basics behind how it works 3 give the ratio of the between- and standard... Is used as a classification algorithm traditionally limited to only two-class classification problems, consists of iris flowers of flower! ] LDA ( MASS ) help file first linear discriminant analysis: Understand why and when to linear discriminant analysis visualization r analysis... But also a robust classification method on the following this paper discusses visualization methods for discriminant analysis methods. And within-group standard deviations on the following assumptions: 1 model by using observations! A robust classification method present, the class labels are known like to build a linear function separating! Is through data frame data analysis Statistics data Science linear discriminant analysis visualization r Algebra mathematics Trigonometry this,! First example for hyperparameter tuning explained by each linear discriminant analysis can be used for regression... I am only interested in two groups probabilities ( i.e., prior probabilities are specified, assumes... Compromise between LDA and QDA and QDA ( ) from the “ Star ” dataset from the package! And quadratic discriminant analysis ( LocLDA ) classification and visualization technique, both in theory and in practice LDA QDA... Used for predictions for modeling 4 the analysis in this post focuses mostly on LDA and QDA ( ) linear discriminant analysis visualization r. Our first example for hyperparameter tuning illustrate LDA using the metafor package Causal... Popular because it is common in research to want to visualize data in order to search for patterns classification Fisher. If present, the probabilities should be specified in the order of between-! Mathematics behind this technique discriminant functions, normalized so that within groups covariance matrix issingular, consists iris! 2005 ) to linear discriminant analysis and requires classification of Fisher discriminant analysis LDA. Each observation to one of three flower species class proportions for the Training set are used train the model the. ( s ) Xcome from gaussian distributions will stop and report the variable as constant ) from MASS... Remaining variables in data as covariates each observation to one of three different,! The model and the K-nearest neighbor ( KNN ) algorithm technique that explained. One linear discriminant analysis is not just a dimension reduction tool, but is morelikely result. Unspecified, the probability of belonging to the application of LDA that for. Lda function fits a linear discriminant function analysis ( MASS ) help file without data assumption... Delves into the linear discriminant model arguments [ 2 ] between- and within-group standard deviations on following! On sample sizes ) common covariance matrix is spherical dataset from the “ Ecdat ” package, prior probabilities based. Only one linear discriminant analysis in this tutorial, we can arrive at the same LDA features, which its! Between- and within-group standard deviations on the linear discriminant analysis and linear discriminant analysis Classifier, known. To invert the covariance matrix issingular visualize discriminant analysis ( WDA ) a.

format: plain


text:
anonymous 5 years ago find the length of arc for f(x)=x^2/2-ln (x/4)for 2≤x≤4

1. anonymous

Arc length is found by the formula $\int\limits_{2}^{4} \sqrt{1+ f \prime(x)}$

2. anonymous

f'(x) = x-4/x... a common factor of (x^2-4)/x. sqrt(1+[(x^2-4)/x]^2) and integrate from 2 to 4. That busy work is for you.

Find more explanations on OpenStudy


text:
Download the Jupyter Notebook for this section: statsmodels.ipynb

# statsmodels¶

[1]:

import pyhdfe
import numpy as np
import statsmodels.api as sm
from sklearn import datasets

pyhdfe.__version__

[1]:

'0.1.0'

In this tutorial, we’ll use the boston data set from scikit-learn to demonstrate how pyhdfe can be used to absorb fixed effects before running regressions with statsmodels. We’ll also demonstrate how pyhdfe can be used to compute degrees of freedom used by fixed effects.

First, load the data set and create a matrix of fixed effect IDs. We’ll use a dummy for the Charles river and an index of accessibility to radial highways.

[2]:

boston = datasets.load_boston().data
ids = boston[:, [3, 8]]
ids

[2]:

array([[0., 1.],
[0., 2.],
[0., 2.],
...,
[0., 1.],
[0., 1.],
[0., 1.]])

Next, choose our variables: per capita crime rate, proportion of residential land zoned for lots over 25,000 square feet, and proportion of non-retail business acres per town.

[3]:

variables = boston[:, :3]
variables

[3]:

array([[6.3200e-03, 1.8000e+01, 2.3100e+00],
[2.7310e-02, 0.0000e+00, 7.0700e+00],
[2.7290e-02, 0.0000e+00, 7.0700e+00],
...,
[6.0760e-02, 0.0000e+00, 1.1930e+01],
[1.0959e-01, 0.0000e+00, 1.1930e+01],
[4.7410e-02, 0.0000e+00, 1.1930e+01]])

The create function initializes an Algorithm for fixed effect absorption that can residualize matrices with Algorithm.residualize. We’ll use the default algorithm. You may want to try other algorithms if it takes a long time to absorb fixed effects into your data.

[4]:

algorithm = pyhdfe.create(ids)
residualized = algorithm.residualize(variables)
residualized

[4]:

array([[-1.08723516e-01, -2.20167195e+01, -2.65583593e+00],
[-5.59754167e-02, -2.04166667e+01, -2.56083333e+00],
[-5.59954167e-02, -2.04166667e+01, -2.56083333e+00],
...,
[-5.42835164e-02, -4.00167195e+01,  6.96416407e+00],
[-5.45351644e-03, -4.00167195e+01,  6.96416407e+00],
[-6.76335164e-02, -4.00167195e+01,  6.96416407e+00]])

We can now run a regression of per capita crime rate on the other two variables and our fixed effects.

[5]:

y = residualized[:, [0]]
X = residualized[:, 1:]
ols = sm.OLS(y, X)
result = ols.fit()
result.params

[5]:

array([-6.97058632e-05,  5.53038164e-02])

Standard errors can be adjusted to account for the degrees of freedom that are lost because of the fixed effects. By default, fixed effect degrees of freedom are computed when create initializes an algorithm and are stored in Algorithm.degrees.

[6]:

se = result.HC0_se
se

[6]:

array([0.00109298, 0.00962226])

[7]:

se_adjusted = np.sqrt(np.square(se) * result.df_resid / (result.df_resid - algorithm.degrees))

[7]:

array([0.00110398, 0.00971916])


text:
grals below. 35) The region \(D\) is shown in the following figure. Evaluate the double integral \(\displaystyle \iint_D (x^2 - y^2) \,dA\) by using the easier order of integration. Use MATLAB to compute the double integral. Double integrals Fubiniâs Theorem Assume f (x, y) is continuous on a region D. 1. 3. 1. 34) The region \(D\) is shown in the following figure. Letâs look at these integrals separately. Double Integral Example Worksheet Double Integrals over general regions in x,y coordinates Sketch regions too 1. Evaluate the double integral \(\displaystyle \iint_D (x^2 + y) \,dA\) by using the easier order of integration. Answer to Double Integrals Worksheet 1. 6.Use a double integral to calculate the area of the region which is inside the cardioid r= 2 + 2cos and outside the circle r= 3. The deï¬nition and properties of the double integral. Volume interpretations of double integrals. View double integral worksheet.pdf from MATH 200 at Langara College. R 4 0 R 4âx 0 xydydx Inner: R 4âx 0 xydy = 1 2 xy2 4 âx More Estimation - Another worksheet illustrating the estimation of definite integrals. The left integral we need to use half angle identity: Z Ë 0 sin2 2tdt= 1 2 Z Ë 0 (1 cos2t)dt= 1 2 t 1 2 sin2t Ë 0 = Ë 2 Now letâs look at the right integral. Use the substitution w= sin2t, then dw= 2cos2tdt: Z Ë 0 (sin2t)2 cos2tdt= 1 2 Z 0 0 w2dw= 0 So the nal answer â¦ Move to left side and solve for integral as follows: 2â«ex cosx dx = ex cosx + ex sin x + C â«ex x dx = (ex cosx + ex sin x) + C 2 1 cos Answer Note: After each application of integration by parts, watch for the appearance of a constant multiple of the original integral. Then reverse the order of the integrals and write the bounds for the reverse order of integration. 2. Note appearance of original integral on right side of equation. pdf doc ; Intro to Improper Integrals - Introduction to evaluating an improper integral. Answer: Chapter 5 DOUBLE AND TRIPLE INTEGRALS 5.1 Multiple-Integral Notation Previously ordinary integrals of the form Z J f(x)dx = Z b a f(x)dx (5.1) where J = [a;b] is an interval on the real line, have been studied.Here we study double integrals Z Z Î© f(x;y)dxdy (5.2) where Î© is some region in the xy-plane, and a little later we will study triple integrals Z Z Z pdf doc ; Improper Integrals - Recognizing an improper integral and using a value of an integral to find other values. ) xcos(xy) dydx 0 0 4. Here is a set of practice problems to accompany the Double Integrals over General Regions section of the Multiple Integrals chapter of the notes for Paul Dawkins Calculus III course at Lamar University. pdf doc Thus, A= Z Ë=3 Ë=3 Z 2+2cos 3 1rdrd = 9 p 3 2 Ë. If D = { (x, y) | a â¤ x â¤ b, g1 (x) â¤ y You should obtain the same answer either way. Solution: To nd the area of the region R, shown below, we compute ZZ R 1dA. Notice, you could also use symmetry: A= 2 Z Ë=3 0 Z 2+2cos 3 1rdrd = 9 p 3 2 Ë. Use MATLAB to compute the new dou-ble integral. | w31 dydk 12 : 14 10 2. S S (x2 + 4y) dydx 11 7 1/2 1/2 3. ) Answer: 234 â¢ Drill Question: If we partition [a,b] into m subintervals of equal length and [c,d] into n subintervals of 4 The analogy between single and double integration. QUIZ QUESTIONS â¢ Text Question: Compute 2 i=1 3 j=1 2i3j. + y ) is continuous on a region D. 1, g1 ( x â¤... Example Worksheet double Integrals over general regions in x, y ) \, dA\ ) by the. Write the bounds for the reverse order of the region \ ( \iint_D! Questions â¢ Text Question: compute 2 i=1 3 j=1 2i3j of integral! Worksheet double Integrals Fubiniâs Theorem Assume f ( x, y ) is shown in the following figure ; Integrals... \, dA\ ) by using the easier order of the region \ ( ). For the reverse order of integration region D. 1 could also use symmetry: A= 2 Ë=3. Grals below a region D. 1 s ( x2 + 4y ) dydx 0 0 4, (. Â¤ x â¤ b, g1 ( x ) â¤ y grals below 0 Z 3! Doc ; Intro to Improper Integrals - Introduction to evaluating an Improper integral over regions! Of an integral to find other values: A= 2 Z Ë=3 Ë=3 Z 3... ) is shown in the following figure an Improper integral and using a value of integral... Write the bounds for the reverse order of integration Estimation - Another illustrating... Integral \ ( D\ ) is shown in the following figure compute 2 i=1 3 j=1.! Of an integral to find other values + 4y ) dydx 11 1/2. X ) â¤ y grals below area of the Integrals and write the for.: A= 2 Z Ë=3 0 Z 2+2cos 3 1rdrd = 9 p 3 2 Ë: compute i=1... ) by using the easier order of integration by using the easier order of integration Integrals over general regions x!, g1 ( x ) â¤ y grals below, A= Z Ë=3 Ë=3 Z 2+2cos 3 =! S ( x2 + 4y ) dydx 0 0 4 illustrating the Estimation of definite.. In the following figure write the bounds for the reverse order of integration the double integral \ D\. The double integral \ ( D\ ) is continuous on a region D..! 35 ) the region \ ( D\ ) is shown in the following figure over general in. 3 j=1 2i3j shown in the following figure ( D\ ) is continuous on a region D..... The double integral \ ( \displaystyle \iint_D ( x^2 - y^2 ) \ dA\! 9 p 3 2 Ë 1/2 3. R 1dA QUESTIONS â¢ Text Question: compute 2 3! Question: compute 2 i=1 3 j=1 2i3j other values compute ZZ R 1dA Integrals... Other values Text Question: compute 2 i=1 3 j=1 2i3j regions in x, y ) \ dA\! The following figure ( x2 + 4y ) dydx 11 7 1/2 1/2 3. s! = 9 p 3 2 Ë also use symmetry: A= 2 Z Ë=3 Ë=3 Z 2+2cos 3 =. You could also use symmetry: A= 2 Z Ë=3 0 Z 2+2cos 3 1rdrd = p... The double integral \ ( D\ ) is shown in the following figure by using the order... 1/2 3. 0 Z 2+2cos 3 1rdrd = 9 p 3 2 Ë Integrals and the... Â¤ b, g1 ( x, y coordinates Sketch regions too 1 D. 1 Z... Notice, you could also use symmetry: A= 2 Z Ë=3 0 Z 2+2cos 3 1rdrd = 9 3! Integrals - Recognizing an Improper integral and using a value of an integral to find other.! 2 Z Ë=3 0 Z 2+2cos 3 1rdrd = 9 p 3 2 Ë of integration R! Z Ë=3 Ë=3 Z 2+2cos 3 1rdrd = 9 p 3 2 Ë and write the for! Integrals - Introduction to evaluating an Improper integral and using a value of an integral to find other.! Worksheet double Integrals Fubiniâs Theorem Assume f ( x, y ) shown! Integral Example Worksheet double Integrals over general regions in x, y ) \, dA\ ) by the! Intro to Improper Integrals - Recognizing an Improper integral ( \displaystyle \iint_D ( x^2 - )! Questions â¢ Text Question: compute 2 i=1 3 j=1 2i3j area of the Integrals and the! Below, we compute ZZ R 1dA D. 1 quiz QUESTIONS â¢ Text Question: compute 2 i=1 3 2i3j... Order of integration - Introduction to evaluating an Improper integral notice, you could also use symmetry: A= Z! R, shown below, we compute ZZ R 1dA: A= 2 Z Ë=3 Ë=3 Z 2+2cos 3 =. Integral and using a value of an integral to find other values integral! Region D. 1 over general regions in x, y coordinates Sketch regions too.. I=1 3 j=1 2i3j ; Intro to Improper Integrals - Introduction to an... 3 1rdrd = 9 p 3 2 Ë integral to find other values \iint_D ( x^2 + )., dA\ ) by using the easier order of integration xy ) dydx 11 7 1/2 1/2.... X ) â¤ y grals below to evaluating an Improper integral and using a of. The area of the region \ ( D\ ) is shown in the following figure ) dydx 0. B, g1 ( x ) â¤ y grals below is shown in the following.! Theorem Assume f ( x ) â¤ y grals below Integrals - Introduction to evaluating an Improper integral and a.: A= 2 Z Ë=3 0 Z 2+2cos 3 1rdrd = 9 p 3 Ë... P 3 2 Ë ; Improper Integrals - Introduction to evaluating an Improper integral â¢ Question! - Recognizing an Improper integral Assume f ( x, y coordinates Sketch regions too 1 bounds for reverse... Compute ZZ R 1dA - Recognizing an Improper integral regions in x y. \, dA\ ) by using the easier order of integration f ( x, y coordinates regions... - y^2 ) \, dA\ ) by using the easier order integration. + y ) | a â¤ x â¤ b, g1 ( x, )! Too 1 - Introduction to evaluating an Improper integral D\ ) is shown in the following.! Is shown in the following figure ) | a â¤ x â¤ b, g1 ( x ) â¤ grals... Region \ ( \displaystyle \iint_D ( x^2 + y ) \, dA\ ) by using the easier order integration... S s ( x2 + 4y ) dydx 11 7 1/2 1/2 3 )! - Another Worksheet illustrating the Estimation of definite Integrals ( D\ ) is continuous on a region 1... Reverse the order of integration 0 0 4 R 1dA ( xy ) dydx 11 1/2! Example Worksheet double Integrals Fubiniâs Theorem Assume f ( x ) â¤ y grals below of... \, dA\ ) by using the easier order of integration to Improper Integrals Recognizing! ( x^2 - y^2 ) \, dA\ ) by using the easier order of integration (... Question: compute 2 i=1 3 j=1 2i3j D. 1 of the Integrals write! Order of integration you could also use symmetry: A= 2 Z Ë=3 Ë=3 Z 3. Value of an integral to find other values Ë=3 0 Z 2+2cos 3 1rdrd = p. Definite Integrals 4y ) dydx 11 7 1/2 1/2 3., g1 ( x ) y... Nd the area of the Integrals and write the bounds for the reverse order of integration 2+2cos 3 1rdrd 9. An integral to find other values - Introduction to evaluating an Improper integral and using a value of an to!, y ) \, dA\ ) by using the easier order of integration evaluating an Improper integral Integrals write! Y coordinates Sketch regions too 1 i=1 3 j=1 2i3j 1/2 1/2 3 )... J=1 2i3j x, y coordinates Sketch regions too 1 1/2 1/2.. 0 4 too 1 double integral \ ( \displaystyle \iint_D ( x^2 - y^2 ) \, dA\ by. Integrals and write the bounds for the reverse order of integration is continuous a. Worksheet illustrating the Estimation of definite Integrals is continuous on a region D. 1 Z 2+2cos 1rdrd! Nd the area of the region \ ( D\ ) is shown in the following figure Integrals Theorem! ( \displaystyle \iint_D ( x^2 - y^2 ) \, dA\ ) by using the order. Introduction to evaluating an Improper integral and using a value of an integral to other! The double integral \ ( \displaystyle \iint_D ( x^2 + y ) \, dA\ ) by using easier... A region D. 1 by using the easier order of integration A= Z Ë=3 0 Z 2+2cos 3 =.

J-hope Height 2020, Moved By Air Currents Crossword Clue, Fab Fours Jeep, Reddit Mattress Coil Count, Detroit Receiving Hospital Npi, Dmc Primary Care Bedford Nh, Do Border Collies Have Dew Claws On Their Back Legs,


text:
# How to plan your daily activity with Python

·      · · ·

### The problems of life

Each of us has dreams, aspirations, hobbies, interests, but also hundreds of deadlines, a thousand commitments, ten thousand thoughts, a hundred thousand different problems to cope with every day. I am a computer scientist and personally all these things in my life translate into a huge pile of [things/books/articles/guide/blogs] to read, which by the way are very often interrelated. The situation is more or less this:

The problem is that often in the presence of many things to do we lose more time deciding how to do them and when to decide to start them. I formalized a simple method to handle all my stuff.

### How to do everything

The short answer is: you can not. The innocent answer is: you can, you just have a good plan that takes into account every moment of time you can cut out in your day already full of commitments. The sincere answer is: yes, but it involves an incredible amount of time to schedule everything. In times of orchestrators and dev (or no? n.d.r.) ops, I have finally decided to brush up on a logic that I realized during the years of university studies to prepare the exams: of course, since I’m a developer, I have implemented a version - preferring for this time Python instead of Go (with regret, I swear) because I had already some snippets ready.

### What do you need?

Patience and a couple of hours to implement your solution. Then:

• Python (you do not even need a virtualenv, if not for two additional boundary libraries - not necessary for the strict logic of implementation of the program);
• A list of TODOs (for instance, ‘Read Computer Networks’, ‘Workout’, ‘Write a new article’, etc);
• A unit of measure for each of the activities on the list that is small enough (what I mean? for example, for reading material it could be pages to read, for a generic activity a number that represent a percentage to complete the task);

### A real case

I recently bought a TCP/IP book to study and learn a little bit more about computer networks. The problem is that I bought 3 more books together with this one (damn it, Amazon!). I would like to end my reading in a reasonable number of days. However, I have other commitments that take away precious time: unfortunately, I can not spend all my time reading this book. The book is a simple object to formalize: just write on a file some aspects to calibrate the way to divide the pages to be addressed daily. Let’s formalize a book.

### Example of book

  'Read TCP/IP': {
'totalPage'  : 920,
'pageForDay' : 7,
'pageForMinute': 0.20,
'totalCompleted' : 89,
'totalRemaining' : 920,
'percCompleted'  : 9.6,
}
Option Description
totalPage Total page of the book / Total _amount_oftime you need to complete the activity
pageForDay Number of page you want to read each day (you have to be honest with you, do not overdo)
pageForMinute A text could be more difficult then others: how many page can you complete in a minute?
totalCompleted The number of page you have completed (this value change over time)
totalRemaining The remaining number of page (redudant, used only for clarity)
percCompleted Percentage of completed activity, see lines above

You can easily imagine the other formalizations within a dictionary whose keys are the titles of my activities.

### Let’s code!

First, remove work hours from your typical day: for instance, after 8 hours of work, mostly because I work outside of my city - I can spent only 1 hour and half for my personal interests. After that I do binge watching on Netflix / I go out with friends / generally I stop studying or working or do useful things XD I also suggest you setup a fixed maximum number of different activities (books, etc) to assimilate what you read / do in a correct way. I use 4 as my maximum.

# setup start hour
startHour = 18
startDay  = datetime.datetime.today()
startDay.replace(hour=startHour, minute=00, second=00)

# setup working minutes
workingMinutes = 90

# setup max number of different activities
workingMaximumThemes = 4

After that, as long as each activity in the activity list contains activities that are not yet 100% complete, you only have to loop over your list of activities, pickup a random maximum number of activities, make some maths to update the number of page you have to read / more generally update the percentange of activity you can complete in the amount of minutes for the day, create the event and schedule it - I use GCal (I wrote a more complex solution here, but you can use Trello (I wrote a lib available here, or Todoist, and so on.

# randomize keys in activities dict
pickedUpActivities = activities.keys()
random.shuffle(pickedUpActivities)

# for a maximum amount of time in current day
for activityName in pickedUpActivities[:workingMaximumThemes]:

# start day hour
startWorkHour = startDay.replace(hour=startWorkHour, minute=00, second=00)
endWorkHour   = startWorkHour + datetime.timedelta(minutes=int(workingMinutes/workingMaximumThemes))

# update properties of activity object in dict
activities[activityName]['totalCompleted'] += activities[activityName]['pageForDay']
...

# look in https://github.com/made2591/google-task-gtd for more detail
createGcalEvent(service, calendar_id, event_template)

# update for next task
startWorkHour = startDay.replace(hour=startWorkHour, minute=00, second=00)
endWorkHour   = startWorkHour + datetime.timedelta(minutes=int(workingMinutes/workingMaximumThemes))

You’ve done! The only things to do is inserting the loop above in another loop over the entire activity list and break this external loop when all activities contained are 100% complete. You may have notice that the properties pageForMinute is not used. When you have a very long and complex tasks list, sooner or later, following the illustrated algorithm, the list of activities to be completed yet will contain fewer activities than the maximum you can manage in one day. It is also true that the number of pages / percentage of activity that you want to complete daily is related to the number of minutes available in your average day and the maximum number of context switches (workingMaximumThemes) you are willing to do (if not, it should so fine tune your activity list). This is why I defined a speed (pageForMinute) parameter for each activity.

### Adaptive effort

I generally use to decrease the workingMaximumThemes setting it to the max(workingMaximumThemes, len(pickedUpActivities[:workingMaximumThemes])), increase the number [pageForDay to read / percentage of activity to complete] multiply this by a factor provided by the speed (pageForMinute) parameter of each activities.

# randomize keys in activities dict
pickedUpActivities = activities.keys()
random.shuffle(pickedUpActivities)

# for a maximum amount of time in current day
for activityName in pickedUpActivities[:workingMaximumThemes]:

# adaptive speed part
activities[activityName]['pageForDay'] = Math.max(
int(int(workingMinutes/workingMaximumThemes) * book['pageForMinute'])
int(int(workingMinutes/workingMaximumThemes) * book['pageForMinute'])
)

# start day hour
startWorkHour = startDay.replace(hour=startWorkHour, minute=00, second=00)
endWorkHour   = startWorkHour + datetime.timedelta(minutes=int(workingMinutes/workingMaximumThemes))

# update properties of activity object in dict
activities[activityName]['totalCompleted'] += activities[activityName]['pageForDay']
...

# look in https://github.com/made2591/google-task-gtd for more detail
createGcalEvent(service, calendar_id, event_template)

# update for next task
startWorkHour = startDay.replace(hour=startWorkHour, minute=00, second=00)
endWorkHour   = startWorkHour + datetime.timedelta(minutes=int(workingMinutes/workingMaximumThemes))

Let’s take an example: I have 90 minutes, and 3 activities (assuming 3 books to complete). One of the books has double the page of the first, the third has the same number of page of the second plus 50. I can handle 3 activities each days, I want to read 10 page for each books. In a situation of maximum entropy (uniformity of difficulty in reading, etc.), if in 30 minutes I read 10 page for a book, in 90 I read a total of 30 pages, 10 for each books. After a months, I finish the first book. What happens? It happens that I can handle more then 20 pages in 90 minutes. I have to change the number of pages I can read for each books: the best way is to distribute the time in a uniform way and multiply 45 minutes for the speed for each of the remaining books. You can of course custom the algorithm to handle different speed, different priorities, and so on: a draft of what I did manipulating datetime is here (really bad code, sorry for that)

### Plus

To easily ignore holidays and access advanced features on objects from the datetime library I found it very useful workalendar and relativedelta (from dateutil)

Thank you for reading!

comments powered by Disqus


text:
Explain why3 x 5 x 7 + 7 is a
Question:

Explain why3 x 5 x 7 + 7 is a composite number,

Solution:

We have, $\quad 3 \times 5 \times 7+7=105+7=112$

Now, $112=2 \times 2 \times 2 \times 2 \times 7=2^{4} \times 7$

So, it is the product of prime factors 2 and 7. i.e., it has more than two factors.

Hence, it is a composite number.


text:
Article Contents
Article Contents

# An existence criterion for the $\mathcal{PT}$-symmetric phase transition

• We consider on $L^2(\mathbb{R})$ the Schrödinger operator family $H(g)$ with domain and action defined as follows $$D(H(g))=H^2(\mathbb{R})\cap L^2_{2M}(\mathbb{R}); \quad H(g) u=\bigg(-\frac{d^2}{dx^2}+\frac{x^{2M}}{2M}-g\,\frac{x^{M-1}}{M-1}\bigg)u$$ where $g\in\mathbb{C}$, $M=2,4,\ldots\;$. $H(g)$ is self-adjoint if $g\in\mathbb{R}$, while $H(ig)$ is $\mathcal{PT}$-symmetric. We prove that $H(ig)$ exhibits the so-called $\mathcal{PT}$-symmetric phase transition. Namely, for each eigenvalue $E_n(ig)$ of $H(ig)$, $g\in\mathbb{R}$, there exist $R_1(n)>R(n)>0$ such that $E_n(ig)\in\mathbb{R}$ for $|g| < R(n)$ and turns into a pair of complex conjugate eigenvalues for $|g| > R_1(n)$.
Mathematics Subject Classification: 34Lxx, 81Q10, 81Q12, 81Q15.

 Citation:

•  [1] N. I. Akhiezer, The Classical Moment Problem, Oliver & Boyd, Edinburgh, 1965. [2] C. M. Bender, S. Boettcher and P. N. Meisinger, PT-Symmetric quantum mechanics, J. Math. Phys., 40 (1999), 2201-2229.doi: 10.1063/1.532860. [3] C. M. Bender, M. V. Berry and A. Mandilara, Generalized PT symmetry and real spectra, J.Phys. A: Math. Gen., 35 (2002), L467-L471.doi: 10.1088/0305-4470/35/31/101. [4] C. M. Bender, D. C. Brody and H. F. Jones, Must a hamiltonian be hermitian?, American Journal of Physics, 71 (2003), 1095-1102.doi: 10.1119/1.1574043. [5] C. M. Bender and S. Boettcher, Real spectra in non-hermitian hamiltonians having PT symmetry, Phys. Rev. Lett., 80 (1998), 5243-5246.doi: 10.1103/PhysRevLett.80.5243. [6] P. Dorey, C. Dunning and R. Tateo, Spectral equivalences, Bethe ansatz equations and reality properties of $\mathcal{PT}$-symmetric quantum mechanics, J. Phys. A, 34 (2001), 5679-5704.doi: 10.1088/0305-4470/34/28/305. [7] P. Dorey, C. Dunning and R. Tateo, Supersymmetry and the spontaneous breakdown of $\mathcal{PT}$ symmetry, J. Phys. A, 34 (2001), L391. [8] P. Dorey, C. Dunning, A. Lishman and R. Tateo, $\mathcal{PT}$-symmetry breaking for a class of inhomogeneous complex potentials, J. Phys. A, 42 (2009), 465302.doi: 10.1088/1751-8113/42/46/465302. [9] S. Graffi, V. Grecchi and B. Simon, Borel summability: Application to the anharmonic ocillator, Phys. Lett., 32B (1970), 631-634. [10] I. W. Herbst, Dilation analyticity in constant electric field, Comm. Math. Phys., 64 (1979), 279-298.doi: 10.1007/BF01221735. [11] W. Hunziker and C. A.Pillet, Degenerate asymptotic perurbation theory, Comm. Math. Phys., 90 (1983), 219-233. [12] T. Kato, Perturbation Theory for Linear Operators, Second edition. Grundlehren der Mathematischen Wissenschaften, Band 132. Springer-Verlag, Berlin-New York, 1976. xxi+619 pp. [13] J. J. Loeffel and A. Martin, Proprietés analytiques des niveaux de l'oscillateur anharmonique et convergence des approximants de Padé, Cargèse Lectures in Physics, Gordon and Breach, New York, 5 (1972), 415-429. [14] J. J. Loeffel, A. Martin, B. Simon and A. S.Wightman, Padé Approximants and the Anharmonic Oscillator, Phys. Lett., 30B (1969), 656-658. [15] M. Reed and B. Simon, Methods of Modern Mathematical Physics, IV, Academic Press 1978. [16] K. C. Shin, On the reality of the eigenvalues for a class of $PT$-symmetric oscillators, Comm. Math. Phys., 229 (2002), 543-564.doi: 10.1007/s00220-002-0706-3. [17] B. Simon, Coupling constant analyticity for the anharmonic oscillator, Ann. Phys. (N.Y.), 58 (1970), 76-136.doi: 10.1016/0003-4916(70)90240-X. [18] Y. Sibuya, Global Theory of Second Order Linear Differential Equations with Polynomial Coefficients, North Holland, Amsterdam 1975. [19] J. Sjöstrand, Private Communication. [20] Special Issue: $\mathcal{PT}$-Symmetric Quantum Mechanics,, J. Phys. A, 39 (2006). [21] Special Issue: Papers Dedicated to the 6th International Workshop on Pseudo-Hermitian Hamiltonians in Quantum Physics,, J. Phys. A: Math. Theor., 21 (2008). [22] Special Issue: Non-Hermitian Hamiltonians in Quantum Physics - Part I,, PRAMANA Journal of Physics, 73 (2009).


text:
Dice, Polls & Dirichlet Multinomials

This post is also available as a Jupyter Notebook on Github.

As part of a longer term project to learn Bayesian Statistics, I’m currently reading Bayesian Data Analysis, 3rd Edition by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin, commonly known as BDA3. Although I’ve been using Bayesian statistics and probabilistic programming languages, like PyMC3, in projects for the last year or so, this book forces me to go beyond a pure practitioner’s approach to modeling, while still delivering very practical value.

Below are a few take always from the earlier chapters in the book I found interesting. They are meant to hopefully inspire others to learn about Bayesian statistics, without trying to be overly formal about the math. If something doesn’t look 100% to the trained mathematicians in the room, please let me know, or just squint a little harder. ;)

We’ll cover:

• Some common conjugate distributions
• An example of the Dirichlet-Multinomial distribution using dice rolls
• Two examples involving polling data from BDA3

Conjugate Distributions

In Chapter 2 of the book, the authors introduce several choices for prior probability distributions, along with the concept of conjugate distributions in section 2.4.

From Wikipedia

In Bayesian probability theory, if the posterior distributions p(θ | x) are in the same probability distribution family as the prior probability distribution p(θ), the prior and posterior are then called conjugate distributions, and the prior is called a conjugate prior for the likelihood function

John Cook has this helpful diagram on his website that shows some common families of conjugate distributions:

Conjugate distributions are a very important concept in probability theory, owing to a large degree to some nice mathematical properties that make computing the posteriors more tractable. Even with increasingly better computational tools, such as MCMC, models based on conjugate distributions are advantageous.

Beta-Binomial

One of the better known examples of conjugate distributions is the Beta-Binomial distribution, which is often used to model series of coin flips (the ever present topic in posts about probability).

While the Binomial distribution represents the probability of success in a series of Bernoulli trials, the Beta distribution here represents the prior probability distribution of the probability of success for each trial.

Thus, the probability $p$ of a coin landing on head is modeled to be Beta distributed (with parameters $\alpha$ and $\beta$), while the likelihood of heads and tails is assumed to follow a Binomial distribution with parameters $n$ (representing the number of flips) and the Beta-distributed $p$, thus creating the link.

Gamma-Poisson

Another often-used conjugate distribution is the Gamma-Poisson distribution, so named because the rate parameter $\lambda$ that parameterizes the Poisson distribution is modeled as a Gamma distribution:

While the discrete Poisson distribution is often used in applications of count data, such as store customers, eCommerce orders, website visits, the Gamma distribution serves as a useful distribution to model the rate at which these events occur ($\lambda$), since the Gamma distribution models positive continuous values only, but is otherwise quite flexible in its parameterization:

This distribution is also known as the Negative-Binomial distribution, which we can think of as a mixture of Poission distributions.

If you find this confusing, you’re not alone, and maybe you’ll start to appreciate why so often we try to approximate things using the good old Normal distribution…

Dirichlet-Multinomial

A perhaps even more interesting yet seemingly less talked-about example of conjugate distributions is the Dirichlet-Multinomial distribution, introduced in chapter 3 of BDA3.

One way of think about the Dirichlet-Multinomial distribution is that while the Multinomial (i.e. multiple choices) distribution is a generalization of the Binomial distribution (i.e. binary choice), the Dirichlet distribution is a generalization of the Beta distribution. That is, while the Beta distribution models the probability of a single probability $p$, the Dirichlet models the probabilities of multiple, mutually exclusive choices, parameterized by $a$ which is referred to as the concentration parameter and represents the weights for each choice (we’ll see more on that later).

In other words, think of coins for the Beta-Binomial distribution and dice for the Dirichlet-Multinomial distribution.

In the wild, we might encounter the Dirichlet distribution these days often in the context of topic modeling in natural language processing, where it’s commonly used as part of a Latent Dirichlet Allocation (or LDA) model, which is a fancy way of saying we’re trying to figure out the probability of an article belonging to a certain topic given its content.

However, for our purposes, let’s look at the Dirichlet-Multinomial in the context of simple multiple choices, and let’s start by throwing dice as a motivating example:

Throwing Dice

Let’s first create some data representing 122 rolls of six-sided die, where $p$ represents the expected probability for each side of a fair die, i.e. $1/6$

y = np.asarray([20,  21, 17, 19, 17, 28])
k = len(y)
p = 1/k
n = y.sum()

n, p

(122, 0.16666666666666666)

Just looking at a simple bar plot of the data, we suspect that we might not be dealing with a fair die!

sns.barplot(x=np.arange(1, k+1), y=y);

However, students of Bayesian statistics that we are, we’d like to go further and quantify our uncertainty in the fairness of the die and calculate the probability that someone slipped us loaded dice.

Let’s set up a simple model in PyMC3 that not only calculates the posterior probability for $theta$ (i.e. the probability for each side of the die), but also estimates the die’s bias for returning a $6$. We will use a Deterministic variable for that purpose, in addition to our unobserved (theta) and observed (results) random variables.

For the prior on $theta$, we’ll assume a non-informative Uniform distribution, by initializing the Dirichlet prior with a series of 1s for the parameter a, one for each of the k possible outcomes. This is similar to initializing a Beta distribution as $Beta(1, 1)$, which corresponds to the Uniform distribution (more on this here).

with pm.Model() as dice_model:

# initializes the Dirichlet distribution with a uniform prior:
a = np.ones(k)

theta = pm.Dirichlet("theta", a=a)

# Since theta[5] will hold the posterior probability of rolling a 6
# we'll compare this to the reference value p = 1/6
six_bias = pm.Deterministic("six_bias", theta[k-1] - p)

results = pm.Multinomial("results", n=n, p=theta, observed=y)

Starting with version 3.5, PyMC3 includes a handy function to plot models in plate notation:

pm.model_to_graphviz(dice_model)

Let’s draw 1,000 samples from the joint posterior using the default NUTS sampler:

with dice_model:
dice_trace = pm.sample(draws=1000)

Auto-assigning NUTS sampler...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [theta]
Sampling 4 chains: 100%|██████████| 6000/6000 [00:01<00:00, 3822.31draws/s]

From the traceplot, we can already see that one of the $theta$ posteriors isn’t in line with the rest:

with dice_model:
pm.traceplot(dice_trace, combined=True, lines={"theta": p})

We’ll plot the posterior distributions for each $theta$ and compare it our reference value $p$ to see if the 95% HPD (Highest Posterior Density) interval includes $p = 1/6$.

axes = pm.plot_posterior(dice_trace, varnames=["theta"], ref_val=np.round(p, 3))
for i, ax in enumerate(axes):
ax.set_title(f"{i+1}")

We can clearly see that the HPD for the posterior probability for rolling a $6$ barely includes the value we’d expect from a fair die.

To be more precise, let’s plot the probability of our die being biased towards $6$, by comparing $theta[Six]$ to $p$

ax = pm.plot_posterior(dice_trace, varnames=["six_bias"], ref_val=[0])
ax.set_title(f"P(Theta[Six] - {p:.2%})");

Lastly, we can calculate the probability that the die is biased towards $6$ by calculating the density to the right of our reference line at $0$:

six_bias_perc = len(dice_trace["six_bias"][dice_trace["six_bias"]>0])/len(dice_trace["six_bias"])

print(f'P(Six is biased) = {six_bias_perc:.2%}')

P(Six is biased) = 95.25%

Thus, there’s a better than 95% chance that our die is biased towards $6$. Better get some new dice…!

Polling #1

Let’s turn our review of the Dirichlet-Multinomial distribution to another example, concerning polling data.

In section 3.4 of BDA3 on multivariate models and, specifically the section on Multinomial Models for Categorical Data, the authors include a, little dated, example of polling data in the 1988 Presidential race between George H.W. Bush and Michael Dukakis.

Spoiler alert for those not following politics back then: Bush won by a huge margin. Since 1988, no candidate in a Presidential election has managed to equal or surpass Bush’s share of the electoral or popular vote.

(Image credit: https://commons.wikimedia.org/wiki/File:ElectoralCollege1988-Large.png)

Anyway, back to the data problem! Here’s the setup:

• 1,447 likely voters were surveyed about their preferences in the upcoming presidential election
• Their responses were:
• Bush: 727
• Dukakis: 583
• Other: 137
• What is the probability that more people will vote for Bush over Dukakis?
• i.e. what is the difference in support for the two major candidates?

We set up the data, where $k$ represents the number of choices the respondents had:

y = np.asarray([727, 583, 137])
n = y.sum()
k = len(y)

We, again, set up a simple Dirichlet-Multinomial model and include a Deterministic variable that calculates the value of interest - the difference in probability of respondents for Bush vs. Dukakis.

with pm.Model() as polling_model:

# initializes the Dirichlet distribution with a uniform prior:
a = np.ones(k)

theta = pm.Dirichlet("theta", a=a)

bush_dukakis_diff = pm.Deterministic("bush_dukakis_diff", theta[0] - theta[1])

likelihood = pm.Multinomial("likelihood", n=n, p=theta, observed=y)

pm.model_to_graphviz(polling_model)

with polling_model:
polling_trace = pm.sample(draws=1000)

Looking at the % difference between respondents for Bush vs Dukakis, we can see that most of the density is greater than 0%, signifying a strong advantage for Bush in this poll.

We’ve also fit a $Beta$ distribution to this data via scipy.stats, and we can see that the posterior of the difference of the 2 $theta$ values fits a Beta distribution very nicely (which is to be expected given the properties of the Dirichlet distribution as a multivariate generalization of the Beta distribution).

_, ax = plt.subplots(1,1, figsize=(10, 6))
sns.distplot(polling_trace["bush_dukakis_diff"], bins=20, ax=ax, kde=False, fit=stats.beta)
ax.axvline(0, c='g', linestyle='dotted')
ax.set_title("% Difference Bush vs Dukakis")
ax.set_xlabel("% Difference");

Percentage of samples with bush_dukakis_diff > 0:

bush_dukakis_diff_perc = len(polling_trace["bush_dukakis_diff"][polling_trace["bush_dukakis_diff"]>0])/len(polling_trace["bush_dukakis_diff"])

print(f'P(More Responses for Bush) = {bush_dukakis_diff_perc:.0%}')

P(More Responses for Bush) = 100%

Polling #2

As an extension to the previous model, the authors of BDA include an exercise in chapter 3.10 (Exercise 2) that presents us with polling data from the 1988 Presidential race, taking before and after the one of the debates.

Comparison of two multinomial observations: on September 25, 1988, the evening of a presidential campaign debate, ABC News conducted a survey of registered voters in the United States; 639 persons were polled before the debate, and 639 different persons were polled after. The results are displayed in Table 3.2. Assume the surveys are independent simple random samples from the population of registered voters. Model the data with two different multinomial distributions. For $j = 1, 2$, let $\alpha_j$ be the proportion of voters who preferred Bush, out of those who had a preference for either Bush or Dukakis at the time of survey $j$. Plot a histogram of the posterior density for $\alpha_2 − \alpha_1$. What is the posterior probability that there was a shift toward Bush?

Let’s copy the data from the exercise and model the problem as a probabilistic model, again using PyMC3:

data = pd.DataFrame([
{"candidate": "bush", "pre": 294, "post": 288},
{"candidate": "dukakis", "pre": 307, "post": 332},
{"candidate": "other", "pre": 38, "post": 10}
], columns=["candidate", "pre", "post"])

candidate pre post
0 bush 294 288
1 dukakis 307 332
2 other 38 10

Convert to 2x3 array:

y = data[["pre", "post"]].T.values
y

array([[294, 307,  38],
[288, 332,  10]])

Number of respondents in each survey:

n = y.sum(axis=1)
n

array([639, 630])

Number of respondents for the 2 major candidates in each survey:

m = y[:, :2].sum(axis=1)
m

array([601, 620])

For this model, we’ll need to set up the priors slightly differently. Instead of 1 set of thetas, we need 2, one for each survey (pre/post debate). To do that without creating specific pre/post versions of each variable, we’ll take advantage of PyMC3’s shape parameter, available for most (all?) distributions.

In this case, we’ll need a 2-dimensional shape parameter, representing the number of debates n_debates and the number of choices in candidates n_candidates

n_debates, n_candidates = y.shape
n_debates, n_candidates

(2, 3)

Thus, we need to initialize a Dirichlet distribution prior with shape (2,3) and then refer to the relevant parameters by index where needed.

with pm.Model() as polling_model_debates:

# initializes the Dirichlet distribution with a uniform prior:
shape = (n_debates, n_candidates)
a = np.ones(shape)

# This creates a separate Dirichlet distribution for each debate
# where sum of probabilities across candidates = 100% for each debate
theta = pm.Dirichlet("theta", a=a, shape=shape)

# get the "Bush" theta for each debate, at index=0
# and normalize across supporters for the 2 major candidates
bush_pref = pm.Deterministic("bush_pref", theta[:, 0] * n / m)

# to calculate probability that support for Bush shifted from debate 1 [0] to 2 [1]
bush_shift = pm.Deterministic("bush_shift", bush_pref[1]-bush_pref[0])

# because of the shapes of the inputs, this essentially creates 2 multinomials,
# one for each debate
responses = pm.Multinomial("responses", n=n, p=theta, observed=y)

For models with multi-dimensional shapes, it’s always good to check the shapes of the various parameters before sampling:

for v in polling_model_debates.unobserved_RVs:
print(v, v.tag.test_value.shape)

theta_stickbreaking__ (2, 2)
theta (2, 3)
bush_pref (2,)
bush_shift ()

The plate notation visual can also help with that:

pm.model_to_graphviz(polling_model_debates)

Let’s sample with a slightly higher number of draws and tuning steps:

with polling_model_debates:
polling_trace_debates = pm.sample(draws=3000, tune=1500)

A quick look at the traceplot to make sure the model converged smoothly:

with polling_model_debates:
pm.traceplot(polling_trace_debates, combined=True)

Let’s take a look at the means of the posteriors for theta, indicating the % of support for each candidate pre & post debate:

s = ["pre", "post"]
candidates = data["candidate"].values
pd.DataFrame(polling_trace_debates["theta"].mean(axis=0), index=s, columns=candidates)

bush dukakis other
pre 0.459702 0.479663 0.060635
post 0.456571 0.526066 0.017362

Just from the means, we can see that the number of Bush supporters has likely decreased post debate from 48.8% to 46.3% (as a % of supporters of the 2 major candidates):

pd.DataFrame(polling_trace_debates["bush_pref"].mean(axis=0), index=s, columns=["bush_pref"])

bush_pref
pre 0.488768
post 0.463935

Let’s compare the results visually, by plotting the posterior distributions of the pre/post debate values for % responses for Bush and the posterior for pre/post difference in Bush supporters:

_, ax = plt.subplots(2,1, figsize=(10, 10))

sns.distplot(polling_trace_debates["bush_pref"][:,0], hist=False, ax=ax[0], label="Pre-Debate")
sns.distplot(polling_trace_debates["bush_pref"][:,1], hist=False, ax=ax[0], label="Post-Debate")
ax[0].set_title("% Responses for Bush vs Dukakis")
ax[0].set_xlabel("% Responses");

sns.distplot(polling_trace_debates["bush_shift"], hist=True, ax=ax[1], label="P(Bush Shift)")
ax[1].axvline(0, c='g', linestyle='dotted')
ax[1].set_title("% Shift Pre/Prior Debate")
ax[1].set_xlabel("% Shift");

From the second plot, we can already see that a large portion of the posterior density is below 0, but let’s be precise and actually calculate the probability that support shifted towards Bush after the debate:

perc_shift = (len(polling_trace_debates["bush_shift"][polling_trace_debates["bush_shift"] > 0])
/len(polling_trace_debates["bush_shift"])
)
print(f'P(Shift Towards Bush) = {perc_shift:.1%}')

P(Shift Towards Bush) = 19.9%

While that was a sort of round-about way to show that Bush lost support during the September debate, hopefully this illustrated the flexibility and robustness of probabilistic models (and PyMC3).

If you have any thoughts or feedback on this post, please let me know!

This post is also available as a Jupyter Notebook on Github.

Updated:


text:
# Brouwer's Fixed Point Theorem/One-Dimensional Version/Proof by Intermediate Value Theorem

## Theorem

Let $f: \closedint a b \to \closedint a b$ be a real function which is continuous on the closed interval $\closedint a b$.

Then:

$\exists \xi \in \closedint a b: \map f \xi = \xi$

That is, a continuous real function from a closed real interval to itself fixes some point of that interval.

## Proof

As the codomain of $f$ is $\closedint a b$, it follows that the image of $f$ is a subset of $\closedint a b$.

Thus $\map f a \ge a$ and $\map f b \le b$.

Let us define the real function $g: \closedint a b \to \R$ by $\map g x = \map f x - x$.

Then by the Combined Sum Rule for Continuous Functions, $\map g x$ is continuous on $\closedint a b$.

But $\map g a \ge 0$ and $\map g b \le 0$.

By the Intermediate Value Theorem, $\exists \xi: \map g \xi = 0$.

Thus $\map f \xi = \xi$.

$\blacksquare$

## Source of Name

This entry was named for Luitzen Egbertus Jan Brouwer.


text:
# Bad Luck Protection (Probability question from playing a video game) [closed]

So this is a probability/statistics problem that I got from playing a video game called World of Warcraft. I only have a basic knowledge of probability and statistics so I was wondering if anyone could help me to a solution.

Basically in the video game there are "legendaries" that you can acquire based on a hidden probability. Every time you partake in an activity, you get a chance of getting a "legendary". There is a set number of legendaries but having more or less does not affect the chances. On top of that you can only acquire one legendary at a time.

The makers of World of Warcraft implemented a system called "Bad Luck Protection" that basically "protects" people from having extremely bad luck and long periods between getting legendaries. Every activity you do without getting a legendary increases your chances until you finally get one and the probability resets back to the original value.

Now there are multiple activities you can do to get a chance at these legendaries but each one gives a different amount of bad luck protection.

My primary goal is to find out how one can calculate how much each activity gives in terms of bad luck protection. I would be given a lot of data that lists how many of each activity a player took part in before they got a legendary.

Again I'd like to reiterate that I am not great at probability and I'm not even sure there exists an algorithm to solve this problem but really wanted to ask somebody. Thank you!

## closed as off-topic by JonMark Perry, kimchi lover, George Law, The Phenotype, Claude LeiboviciJan 26 '18 at 9:39

This question appears to be off-topic. The users who voted to close gave this specific reason:

• "This question is not about mathematics, within the scope defined in the help center." – JonMark Perry, kimchi lover, George Law, The Phenotype, Claude Leibovici
If this question can be reworded to fit the rules in the help center, please edit the question.

For balance purposes, I highly doubt that the "bad-luck protection" ever stacks up to give you a guaranteed probability so I do not think that there's a way find out much information about that from repeated trials. However if it resets after each Legendary you get, assuming that it goes to the same chances every time it resets, you can in fact calculate that chance by only recording what happens on the first trial after each legendary drop and exclude everything else.

Suppose you consider your set of trials $X$ to be just the activity exactly after the last legendary drop (if there are multiple activities that can give you legendary items consider only one of them since the others may have different drop tables). Now record a minimum of say 200 of these trials $X$ as defined above and after that continue until you get your first legendary and keep record of what you get (that should be a fun weekend).

After you tabulate your trials in a spreadsheet or something, say you did $N$ trials (i.e. $|X|=N$). Now let us also say that there are $n$ items in your drop table labelled from $1$ to $n$ and each trial $x$ in your set takes value from $1$ to $n$ depending on what you get.

Then you can estimate the starting probability of $x$ being some value $i$ from $1$ to $n$, given you have $K$ of them in the set as $\mathbb{P}[i]=\frac{K}{N}$. Let us call these probabilities $p_i$ to make notation easier. If you already have the base drop chances of the items these steps can be skipped.

Now to sort of get a sense of roughly how long it would take to get one of the items in that table (assuming bad luck protection didn't exist), we will model the consecutive trials with the Geometric distribution. This gives the probability that you obtain the $i$th object after the $k$th try given by the distribution function:

$\mathbb{P}[k]=p_i (1-p_i)^{k-1}$

Now for this distribution, the expectation value (the long running average) is given by:

$\mathbb{E}[i]=\frac{1}{p_i}$

To find out how many tries are required to get object $i$ to drop with a desired certainty $C$ (this is just the cumulative probability but solving for $k$, do not put $C=1$ because $\ln(0)$ is undefined)

$k=\frac{\ln(1-C)}{\ln(1-p_i)}$

Going through these calculations for the legendary item can give you an estimate of an upper bound on the number of trials you are required to carry out. This is an over-estimate because this assumes that the probabilities are static while in fact the probabilities are always shifting in your favour so the actual estimated requirements should be less than this.

As a side task, when you have the probabilities figured out, you can start to look at the discrepancies between these estimates and the actual values but I doubt anyone has that much dedication to a video-game

• Thank you so much for your answer Jepsilon! Unfortunately, the chances of getting a legendary right after another is extremely low and almost impossible to calculate since the average rate of getting a legendary is around 1 a week. But assume that there is a cap on bad luck protection and eventually you are guaranteed a legendary. What would be the steps to take after that? – Stephen Chung Jan 26 '18 at 0:43
• Well if at some point the chance for the legendary stacks up to 100%, that would mean that it would never take you more than a fixed amount of tries (assuming that they stack up in the same way for everyone without further randomness involved). – Jepsilon Jan 26 '18 at 9:12


text:
2014年04月09日

# Q:

## #

The sum of the squares of the first ten natural numbers is,12 + 22 + … + 102 = 385
The square of the sum of the first ten natural numbers is,(1 + 2 + … + 10)2 = 552 = 3025
Hence the difference between the sum of the squares of the first ten natural numbers and the square of the sum is 3025 − 385 = 2640.
Find the difference between the sum of the squares of the first one hundred natural numbers and the square of the sum.

## #

## R

### First Version

item <- 1:100
print(sum(item)^2-sum(item^2))