text:
## Fitting the Besag, York, and Mollie spatial autoregression model with discrete data

Rudy Banerjee writes:

I am trying to use the Besag, York & Mollie 1991 (BYM) model to study the sociology of crime and space/time plays a vital role. Since many of the variables and parameters are discrete in nature is it possible to develop a BYM that uses an Integer Auto-regressive (INAR) process instead of just an AR process?

I’ve seen INAR(1) modeling, even a spatial INAR or SINAR paper but they seem to be different that the way BYM is specified in the Bayes framework.

Does it even make sense to have a BYM that is INAR? I can think of discrete jumps in independent variables that affect the dependent variable in discrete jumps. (Also, do these models violate convexity requirements often required for statistical computing?)

1. To see how to fit this sort of model in a flexible way, see this Stan case study, Spatial Models in Stan: Intrinsic Auto-Regressive Models for Areal Data, from Mitzi Morris.

2. Rather than trying to get cute with your discrete modeling, I’d suggest a simple two-level approach, where you use an underlying continuous model (use whatever space-time process you want, BYM or whatever) and then you can have a discrete data model (for example, negative binomial, that is, overdispersed Poisson) on top of that.

## Stan development in RStudio

Check this out! RStudio now has special features for Stan:

– Improved, context-aware autocompletion for Stan files and chunks

– A document outline, which allows for easy navigation between Stan code blocks

– Inline diagnostics, which help to find issues while you develop your Stan model

– The ability to interrupt Stan parallel workers launched within the IDE

This is awesome—especially that last feature. Rstudio is my hero.

And don’t forget this: If you don’t currently have Stan on your computer, you can play with this demo version on the web, thanks to RStudio Cloud.

## David Brooks discovers Red State Blue State Rich State Poor State!

The New York Times columnist writes:

Our political conflict is primarily a rich, white civil war. It’s between privileged progressives and privileged conservatives. You could say that tribalism is the fruit of privilege. People with more stresses in their lives necessarily pay less attention to politics. . . .

I’ve had some differences with Brooks in the past, but when he agrees with me, I’m not gonna complain.

As David Park, Boris Shor, Joe Bafumi, Jeronimo Cortina, and I wrote ten years ago:

The cultural divide of the two Americas looms larger at high incomes . . . A theme throughout this book is that the cultural differences between states—the things that make red and blue America feel like different places—boil down to differences among richer people in these states.

And:

Consistent with Ronald Inglehart’s hypothesis of postmaterialism, survey data show social issues to be more important to higher-income voters. This can be viewed as a matter of psychology and economics, with social policy as a luxury that can be afforded once you have achieved some material comfort. Our results stand in contradiction to the commonly held idea that social issues distract lower-income voters from their natural economic concerns.

It took a few years, but it seems that our ideas have finally become conventional wisdom.

## The AAA tranche of subprime science, revisited

Tom Daula points us to this article, “Mortgage-Backed Securities and the Financial Crisis of 2008: A Post Mortem,” by Juan Ospina and Harald Uhlig. Not our usual topic at this blog, but then there’s this bit on page 11:

We break down the analysis by market segment defined by loan type (Prime, Alt-A, and Subprime). Table 5 shows the results and documents the third fact: the subprime AAA-rated RMBS did particularly well. AAA-rated Subprime Mortgage Backed Securities were the safest securities among the non-agency RMBS market. As of December 2013 the principal-weighted loss rates AAA-rated subprime securities were on average 0.42% [2.2% for Prime AAA same page]. We do not deny that even the seemingly small loss of 0.42% should be considered large for any given AAA security.

Nonetheless, we consider this to be a surprising fact given the conventional narrative for the causes of the financial crisis and its assignment of the considerable blame to the subprime market and its mortgage-backed securities. An example of this narrative is provided by Gelman and Loken (2014):

We have in mind an analogy with the notorious AAA-class bonds created during the mid-2000s that led to the subprime mortgage crisis. Lower-quality mortgages [that is, mortgages with high probability of default and, thus, high uncertainty] were packaged and transformed into financial instruments that were (in retrospect, falsely) characterized as low risk.

OK, our paper wasn’t actually about mortgages; it was about statistics. We were just using mortgages as an example. But if Ospina and Uhlig are correct, we were mistaken in using AAA-rated subprime mortgages as an example of a bad bet. Analogies are tricky things!

Overall, I think it fits your data collection/measurement theme, and how doing that well can provide novel insights. In that vein, they provide a lot of detail to replicate the results, in case folks disagree. There’s the technical appendix which (p.39) “serves as guide for replication and for understanding the contents of our database” as well as (p.7) a replication kit available from the authors. As to the latter, (p.15) footnote 15 guides the reader to where exactly where to look for the one bit of modeling in the paper (“For a detailed list of the covariates employed, refer to MBS Project/Replication/DefaultsAnalysis/Step7”).

## Toward better measurement in K-12 education research

Billy Buchanan, Director of Data, Research, and Accountability, Fayette County Public Schools, Lexington, Kentucky, expresses frustration with the disconnect between the large and important goals of education research, on one hand, and the gaps in measurement and statistical training, on the other.

Buchanan writes:

I don’t think that every classroom educator, instructional coach, principal, or central office administrator needs to be an expert on measurement. I do, however, think that if we are training individuals to be researchers (e.g., PhDs) we have a duty to make sure they are able to conduct the best possible research, understand the various caveats and limitations to their studies, and especially understand how measurement – as the foundational component of all research across all disciplines (yes, even qualitative research) – affects the inferences derived from their data. . . .

In essence, if a researcher wants to use an existing vetted measurement tool for research purposes, they should already have access to the technical documentation and can provide the information up front so as our staff reviews the request we can also evaluate whether or not the measurement tool is appropriate for the study. If a researcher wants to use their own measure, we want them to be prepared to provide sufficient information about the validity of their measurement tool so we can ensure that they publish valid results from their study; this also has an added benefit for the researcher by essentially motivating them to generate another paper – or two or three – from their single study.

He provides some links to resources, and then he continues:

I would like to encourage other K-12 school districts to join with us in requiring researchers to do the highest quality research possible. I, for one, at least feel that the students, families, and communities that we serve deserve nothing less than the best and believe if you feel the same that your organization would adopt similar practices. You can find our policies here: Fayette County Public Schools Research Requests. Or if you would like to use our documentation/information and/or contribute to what we currently have, you can submit an issue to our documentation’s GitHub Repository (https://github.com/fcps/research-request).

## He had a sudden cardiac arrest. How does this change the probability that he has a particular genetic condition?

Megan McArdle writes:

I have a friend with a probability problem I don’t know how to solve. He’s 37 and just keeled over with sudden cardiac arrest, and is trying to figure out how to assess the probability that he has a given condition as his doctors work through his case. He knows I’ve been sharply critical of doctors’ failures to properly assess the Type I/Type II tradeoff, so he reached out to me, but we quickly got into math questions above my pay grade, so I volunteered to ask if you would sketch out the correct statistical approach.

To be clear, he’s an engineer, so he’s not asking you to do the work for him! Just to sketch out in a few words how he might approach information gathering and setting up a problem like “given that you’ve had sudden cardiac arrest, what’s the likelihood that a result on a particular genetic test is a false positive?”

I agree that the conditional probability should change, given the knowledge that he had the cardiac arrest. Unfortunately it’s hard for me to be helpful here because there are too many moving parts: of course the probability of the heart attack conditional on having the condition or not, but also the relevance of the genetic test to his health condition. This is the kind of problem that is addressed in the medical decision making literature, but I don’t think I have anything useful to add here, beyond emphasizing that the calculation of any such probability is an intermediate step in this person’s goal of figuring out what he should do next regarding his heart condition.

I’m posting the question here, in case any of you can point to useful materials on this. In addition to the patient’s immediate steps in staying alive and healthy, this is a general statistical issue that has to be coming up in medical testing all the time, in that tests are often done in the context of something that happened to you, so maybe there is some general resource on this topic?

## Understanding Chicago’s homicide spike; comparisons to other cities

Michael Masinter writes:

As a longtime blog reader sufficiently wise not to post beyond my academic discipline, I hope you might take a look at what seems to me to be a highly controversial attempt to use regression analysis to blame the ACLU for the recent rise in homicides in Chicago. A summary appears here with followup here.

I [Masinter] am skeptical, but loathe to make the common mistake of assuming that my law degree makes me an expert in all disciplines. I’d be curious to know what you think of the claim.

The research article is called “What Caused the 2016 Chicago Homicide Spike? An Empirical Examination of the ‘ACLU Effect’ and the Role of Stop and Frisks in Preventing Gun Violence,” and is by Paul Cassell and Richard Fowles.

I asked Masinter what were the reasons for his skepticism, and he replied:

Two reasons, one observational, one methodological, and a general sense of skepticism.

Other cities have restricted the use of stop and frisk practices without a corresponding increase in homicide rates.

Quite a few variables appear to be collinear. The authors claim that Bayesian Model Averaging controls for that, but I lack the expertise to assess their claim.

More generally, their claim of causation appears a bit like post hoc propter hoc reasoning dressed up in quantitative analysis.

Perhaps I am wrong, but I have come to be skeptical of such large claims.

I asked Masinter if it would be ok to quote him, and he responded:

Yes, but please note my acknowledged lack of expertise. Too many of my law prof colleagues think our JD qualifies us as experts in all disciplines, a presumption magnified by our habit of advocacy.

I replied, “Hey, don’t they know that the only universal experts are M.D.’s and, sometimes, economists?”, and Masinter then pointed me to this wonderful article by Arthur Allen Leff from 1974:

With the publication of Richard A. Posner’s Economic Analysis of Law, that field of learning known as “Law and Economics” has reached a stage of extended explicitness that requires and permits extended and explicit comment. . . . I was more than half way through the book before it came to me: as a matter of literary genre (though most likely not as a matter of literary influence) the closest analogue to Economic Analysis of Law is the picaresque novel.

Think of the great ones, Tom Jones, for instance, or Huckleberry Finn, or Don Quixote. In each case the eponymous hero sets out into a world of complexity and brings to bear on successive segments of it the power of his own particular personal vision. The world presents itself as a series of problems; to each problem that vision acts as a form of solution; and the problem having been dispatched, our hero passes on to the next adventure. The particular interactions are essentially invariant because the central vision is single. No matter what comes up or comes by, Tom’s sensual vigor, Huck’s cynical innocence, or the Don’s aggressive romanticism is brought into play . . .

Richard Posner’s hero is also eponymous. He is Economic Analysis. In the book we watch him ride out into the world of law, encountering one after another almost all of the ambiguous villains of legal thought, from the fire-spewing choo-choo dragon to the multi-headed ogre who imprisons fair Efficiency in his castle keep for stupid and selfish reasons. . . .

One should not knock the genre. To hold the mind-set constant while the world is played in manageable chunks before its searching single light is a powerful analytic idea, the literary equivalent of dropping a hundred metals successively into the same acid to see what happens. The analytic move, just as a strategy, has its uses, no matter which mind-set is chosen, be it ethics or psychology or economics or even law. . . .

Leff quotes Posner:

Efficiency is a technical term: it means exploiting economic resources in such a way that human satisfaction as measured by aggregate consumer willingness to pay for goods and services is maximized. Value too is defined by willingness to pay.

And then Leff continues:

Given this initial position about the autonomy of people’s aims and their qualified rationality in reaching them, one is struck by the picture of American society presented by Posner. For it seems to be one which regulates its affairs in rather a bizarre fashion: it has created one grand system—the market, and those market-supportive aspects of law (notably “common,” judge-made law)—which is almost flawless in achieving human happiness; it has created another—the political process, and the rest of “the law” (roughly legislation and administration)—which is apparently almost wholly pernicious of those aims.

An anthropologist coming upon such a society would be fascinated. It would seem to him like one of those cultures which, existing in a country of plenty, having developed mechanisms to adjust all intracultural disputes in peace and harmony, lacking any important enemies, nevertheless comes up with some set of practices, a religion say, simultaneously so barbaric and all-pervasive as to poison almost every moment of what would otherwise appear to hold potential for the purest existential joy. If he were a bad anthropologist, he would cluck and depart. If he were a real anthropologist, I suspect he would instead stay and wonder what it was about the culture that he was missing. That is, he would ponder why they wanted that religion, what was in it for them, what it looked like and felt like to be inside the culture. And he would be especially careful not to stand outside and scoff if, like Posner, he too leaned to the proposition that “most people in most affairs of life are guided by what they conceive to be their self-interest and . . . choose means reasonably (not perfectly) designed to promote it.”

I’m reminded of our discussion from a few years ago regarding the two ways of thinking in economics.

Economists are often making one of two arguments:

1. People are rational and respond to incentives. Behavior that looks irrational is actually completely rational once you think like an economist.

2. People are irrational and they need economists, with their open minds, to show them how to be rational and efficient.

Argument 1 is associated with “why do they do that?” sorts of puzzles. Why do they charge so much for candy at the movie theater, why are airline ticket prices such a mess, why are people drug addicts, etc. The usual answer is that there’s some rational reason for what seems like silly or self-destructive behavior.

Argument 2 is associated with “we can do better” claims such as why we should fire 80% of public-schools teachers or Moneyball-style stories about how some clever entrepreneur has made a zillion dollars by exploiting some inefficiency in the market.

Despite going in the opposite directions, Arguments 1 and 2 have a similarity, in that both used to make the point that economists stand from a superior position by which they can evaluate the actions of others.

But Leff said it all back in 1974.

P.S. We seem to have mislaid our original topic, which is the study of the change in homicide rates in Chicago. I read the article by Casell and Fowles and their analysis of Chicago seemed reasonable to me. At the same time, I agree with Masinter that the comparisons to other cities didn’t seem so compelling.

## Limitations of “Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection”

“If you will believe in your heart and confess with your lips, surely you will be saved one day”The Mountain Goats paraphrasing Romans 10:9

One of the weird things about working with people a lot is that it doesn’t always translate into multiple opportunities to see them talk.  I’m pretty sure the only time I’ve seen Andrew talk was at a fancy lecture he gave in Columbia. He talked about many things that day, but the one that stuck with me (because I’d not heard it phrased that well before, but as a side-note this is a memory of the gist of what he was saying. Do not hold him to this opinion!) was that the problem with p-values and null-hypothesis wasn’t so much that the procedure was bad. The problem is that people are taught to believe that there exists a procedure that can, given any set of data, produce a “yes/no” answer to a fairly difficult question. So the problem isn’t the specific decision rule that NHST produces, so much as the idea that a universally applicable decision rule exists at all. (And yes, I know the maths. But the problem with p-values was never the maths.)

This popped into my head again this week as Aki, Andrew, Yuling, and I were working on a discussion to Gronau and Wagenmakers’ (GW) paper “Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection”.

Our discussion is titled “Limitations of ‘Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection'” and it extends various points that Aki and I have made at various points on this blog.

To summarize our key points:

1. It is a bad thing for GW to introduce LOO model selection in a way that doesn’t account for its randomness. In their very specialized examples this turns out not to matter because they choose such odd data that the LOO estimates have zero variance. But it is nevertheless bad practice.
2. Stacking is a way to get model weights that is more in line with the LOO-predictive concept than GW’s ad hoc pseudo-BMA weights. Although stacking is also not consistent for nested models, in the cases considered in GW’s paper it consistently picks the correct model. In fact, the model weight for the true model in each of their cases is $w_0=1$ independent of the number of data points.
3. By not recognizing this, GW missed an opportunity to discuss the limitations of the assumptions underlying LOO (namely that the observed data is representative of the future data, and each individual data point is conditionally exchangeable).  We spent some time laying these out and proposed some modifications to their experiments that would make these limitations clearer.
4. Because LOO is formulated under much weaker assumptions than is used in this paper, namely LOO does not assume that the data is generated by one of the models under consideration (the so-called “M-Closed assumption”), it is a little odd that GW only assess its performance under this assumption. This assumption almost never holds. If you’ve ever used the famous George Box quote, you’ve explicitly stated that the M-Closed assumption does not hold!
5. GW’s assertion that when two models can support identical models (such as in the case of nested models), the simplest model should be preferred is not a universal truth, but rather a specific choice that is being made. This can be enforced for LOO methods, but like all choices in statistical modelling, it shouldn’t be made automatically or by authority, but should instead be critically assessed in the context of the task being performed.

All of this has made me think about the idea of doing model selection. Or, more specifically, it’s made me question whether or not we should try to find universal tools for solving this problem. Is model selection even possible? (Danielle Navarro from UNSW has a particularly excellent blog post outlining her experiences with various existing model selection methods that you all should read.)

So I guess my very nebulous view is that we can’t do model selection, but we can’t not do model selection, but we also can’t not not do model selection.

In the end we need to work out how to do model selection for specific circumstances and to think critically about our assumptions. LOO helps us do some of that work.

To close off, I’m going to reproduce the final section of our paper because what’s the point of having a blog post (or writing a discussion) if you can’t have a bit of fun.

Can you do open science with M-Closed tools?

One of the great joys of writing a discussion is that we can pose a very difficult question that we have no real intention of answering. The question that is well worth pondering is the extent to which our chosen statistical tools influence how scientific decisions are made. And it’s relevant in this context because of a key difference between model selection tools based on LOO and tools based on marginal likelihoods is what happens when none of the models could reasonably generate the data.

In this context, marginal likelihood-based model selection tools will, as the amount of data increases, choose the model that best represents the data, even if it doesn’t represent the data particularly well. LOO-based methods, on the other hand, are quite comfortable expressing that they can not determine a single model that should be selected. To put it more bluntly, marginal likelihood will always confidently select the wrong model, while LOO is able to express that no one model is correct.

We leave it for each individual statistician to work out how the shortcomings of marginal likelihood-based model selection balance with the shortcomings of cross-validation methods. There is no simple answer.

## Stan on the web! (thanks to RStudio)

This is big news. Thanks to RStudio, you can now run Stan effortlessly on the web.

So you can get started on Stan without any investment in set-up time, no need to install C++ on your computer, etc.

As Ben Goodrich writes, “RStudio Cloud is particularly useful for Stan tutorials where a lot of time can otherwise be wasted getting C++ toolchains installed and configured on everyone’s laptops.”

We’ve pre-loaded this particular RStudio session with a regression model and an R script to simulate fake data and run the model. In your online RStudio Cloud session (which will appear within your browser when you click the above link), just go the lower-right window with Files, and click on simple_regression.stan and simple_regression.R. This will open up those files. Run simple_regression.R and it will simulate the data, run the Stan program, and produce a graph.

Now you can play around.

Create your own Stan program: just work in the upper-left window, click on File, New File, Stan File, then click on File, Save As, and give it a .stan extension. The RStudio editor already has highlighting and autocomplete for Stan files.

Or edit the existing Stan program (simple_regression.stan, sitting there in the lower right of your RStudio Cloud window), change it how you’d like, then edit the R script or create a new one. You can upload data to your session too.

When you run a new or newly edited Stan program, it will take some time to compile. But then next time you run it, R will access the compiled version and it will run faster.

You can also save your session and get back to it later.

Some jargon for ya—but I mean every word of it!

This is a real game-changer as it significantly lowers the barriers to entry for people to start using Stan.

Limitations

Ultimately I recommend you set up Stan on your own computer so you have full control over your modeling, but RStudio’s Cloud is a wonderful way to get started.

Here’s what Rstudio says:

Each project is allocated 1GB of RAM. Each account is allocated one private space, with up to 3 members and 5 projects. You can submit a request to the RStudio Cloud team for more capacity if you hit one of these space limits, and we will do our best accomodate you. If you are using a Professional shinyapps.io 2 account, you will not encounter these space limits.

In addition to the private space (where you can collaborate with a selected group of other users), every user also gets a personal workspace (titled “Your Workspace”), where there is virtually no limit to the number of projects you can create. Only you can work on projects in your personal workspace, but you can grant view & copy access to them to any other RStudio Cloud user.

This is just amazing. I’m not the most computer-savvy user, but I was able to get this working right away.

It also comes with
* The other 500-ish examples in the examples/ subdirectory
* Most of the R packages that use Stan, including McElreath’s rethinking package from GitHub and all the ones under stan-dev, namely
– rstanarm (comes with compiled Stan programs for regression)
– brms (generates Stan programs for regression)
– projpred (for selecting a submodel of a GLM)
– bayesplot and shinystan (for visualizing posterior output)
– loo (for model comparison using expected log predictive density)
– rstantools (for creating your own R packages like rstanarm)
* Saving new / modified compiled Stan programs to the disk to use across sessions first requires the user to do rstan_options(auto_write = TRUE)

I’m so excited. You can now play with Stan yourself with no start-up effort. Or, if you’re already a Stan user, you can demonstrate it to your friends. Also, you can put your own favorite models in an RStudio Cloud environment (as Ben did for my simple regression model) and then share the link with other people, who can use your model on your data, upload their own data, alter your model, etc.

P.S. It seems that for now this is really only for playing around with very simple models, to give you a taste of Stan, and then you’ll have to install it on your computer to do more. See this post from Aki. That’s fine. as this demo is intended to be a show horse, not a work horse. I think there is also a way to pay RStudio for cycles on the cloud and then you can run bigger Stan models through the web interface. So that could be an option too, for example if you want to use Stan as a back-end for some computing that you’d like others to access remotely.

## Why are functional programming languages so popular in the programming languages community?

Matthijs Vákár writes:

Re the popularity of functional programming and Church-style languages in the programming languages community: there is a strong sentiment in that community that functional programming provides important high-level primitives that make it easy to write correct programs. This is because functional code tends to be very short and easy to reason about because of referential transparency, making it quick to review and maintain. This becomes even more important in domains where correctness is non-trivial, like in the presence of probabilistic choice and conditioning (or concurrency).

The sense – I think – is that for the vast majority of your code, correctness and programmer-time are more important considerations than run-time efficiency. A typical workflow should consist of quickly writing an obviously correct but probably inefficient program, profiling it to locate the bottlenecks and finally – but only then – thinking hard about how to use computational effects like mutable state and concurrency to optimise those bottlenecks without sacrificing correctness.

There is also a sense that compilers will get smarter with time, which should result in pretty fast functional code, allowing the programmer to think more about what she wants to do rather than about the how. I think you could also see the lack of concern for performance of inference algorithms in this light. This community is primarily concerned with correctness over efficiency. I’m sure they will get to more efficient inference algorithms eventually. (Don’t forget that they are not statisticians by training so it may take some time for knowledge about inference algorithms to percolate into the PL community.)

Justified or not, there is a real conviction in the programming languages community that functional ideas will become more and more important in mainstream programming. People will point to the increase of functional features in Scala, Java, C# and even C++. Also, people note that OCaml and Haskell these days are getting to the point where they are really quite fast. Jane Street would not be using OCaml if it weren’t competitive. In fact, one of the reasons they use it – as I understand it – is that their code involves a lot of maths which functional languages lend themselves well to writing. Presumably, this is also part of the motivation of using functional programming for prob prog? In a way, functional programming feels closer to maths.

Another reason I think people care about language designs in the style of Church is that they make it easy to write certain programs that are hard in Stan. For instance, perhaps you care about some likelihood-free model, like a model where some probabilistic choices are made, then a complicated deterministic program is run and only then the results are observed (conditioned on). An example that we have here in Oxford is a simulation of the standard model of particle physics. This effectively involves pushing forward your priors through some complicated simulator and then observing. It would be difficult to write down a likelihood for this model. Relatedly, when I was talking to Erik Meijer, I got the sense that Facebook wants to integrate prob prog into existing large software systems. The resulting models would not necessarily have a smooth likelihood wrt the Lebesgue measure. I am told that in some cases inference in these models is quite possible with rejection sampling or importance sampling or some methods that you might consider savage. The models might not necessarily be very hard statistically, but they are hard computationally in the sense that they do involve running a non-trivial program. (Note though that this has less to do with functional programming than it does with combining probabilistic constructs with more general programming constructs!)

A different example that people often bring up is given by models where observations might be of variable length, like in language models. Of course, you could do it in Stan with padding, but some people don’t seem to like this.

Of course, there is the question whether all of this effort is wasted if eventually inference is completely intractable in basically all but the simplest models. What I would hope to eventually see – and many people in the PL community with me – is a high-level language in which you can do serious programming (like in OCaml) and have access to serious inference capabilities (like in Stan). Ideally, the compiler should decide/give hints as to which inference algorithms to try — use NUTS when you can, but otherwise back-off and try something else. And there should be plenty of diagnostics to figure out when to distrust your results.

Finally, though, something I have to note is that programming language people are the folks writing compilers. And compilers are basically the one thing that functional languages do best because of their good support for user-defined data structures like trees and recursing over such data structures using pattern matching. Obviously, therefore, programming language folks are going to favour functional languages. Similarly, because of their rich type systems and high-level abstractions like higher-order functions, polymorphism and abstract data types, functional languages serve as great hosts for DSLs like PPLs. They make it super easy for the implementer to write a PPL even if they are a single academic and do not have the team required to write a C++ project.

The question now is whether they also genuinely make things easier for the end user. I believe they ultimately have the potential to do so, provided that you have a good optimising compiler, especially if you are trying to write a complicated mathematical program.

I replied:

On one hand, people are setting up models in Church etc. that they may never be able to fit—it’s not just that Church etc are too slow to fit these models, it’s that they’re unidentified or just don’t make sense or have never really been thought out. (My problem here is not with the programming language called Church; I’m just here concerned about the difficulty with fitting any model correctly the first time, hence the need for flexible tools for model fitting and exploration.)

But, on the other hand, people are doing inference and making decisions all the time, using crude regressions or t-tests or whatever.

To me, it’s close to meaningless for someone to say they can write “an obviously correct but probably inefficient program” if the program is fitting a model that you can’t really fit!

Matthijs responded:

When I say “write an obviously correct but probably inefficient program”, I am talking about usual functional programming in a deterministic setting. I think many programming language (PL) folks do not fully appreciate what correctness for a probabilistic program means. I think they are happy when they can prove that their program asymptotically does the right thing. I do not think they quite realise yet that that gives no guarantees about what to think about your results when you run your program for a finite amount of time. It sounds silly, but I think at least part of the community is still to realise that. Personally, I expect that people will wake up eventually and will realise that correctness in a probabilistic setting is a much more hairy beast. In particular, I expect that the PL community will realise that run-time diagnostics and model checking are the things that they should have been doing all along. I am hopeful though that at some point enough knowledge of statistics will trickle through to them such that genuinely useful collaboration is possible. I strongly feel that there is mostly just a lot of confusion rather than wilful disregard.

## The Golden Rule of Nudge

Nudge unto others as you would have them nudge unto you.

Do not recommend to apply incentives to others that you would not want for yourself.

Background

The background, roughly, is this: Various English politicians promised that the net flow of immigrants to the U.K. would decrease. But the British government had little control over the flow of immigrants to the U.K., because most of them were coming from other E.U. countries. So they’d pretty much made a promise they couldn’t keep. The only way the government could even try to reach their targets was to convince people currently living in the U.K. to leave. And one way to do this was to make their lives more difficult. Apparently the government focused this effort on people living in Britain who’d originally come from former British colonies in the Caribbean, throwing paperwork at them and threatening their immigration status.

Davies explains:

The Windrush generation’s immigration status should never have been in question, and the cause of their predicament is recent: the 2014 Immigration Act, which contained the flagship policies of the then home secretary, Theresa May. Foremost among them was the plan to create a ‘hostile environment’, with the aim of making it harder for illegal immigrants to work and live in the UK . . .

It’s almost as if, on discovering that law alone was too blunt an instrument for deterring and excluding immigrants, May decided to weaponise paperwork instead. The ‘hostile environment’ strategy was never presented just as an effective way of identifying and deporting illegal immigrants: more important, it was intended as a way of destroying their ability to build normal lives. The hope, it seemed, was that people might decide that living in Britain wasn’t worth the hassle. . . .

The thread linking benefit sanctions and the ‘hostile environment’ is that both are policies designed to alter behaviour dressed up as audits. Neither is really concerned with accumulating information per se: the idea is to use a process of constant auditing to punish and deter . . .

And then he continues:

The coalition government was fond of the idea of ‘nudges’, interventions that seek to change behaviour by subtle manipulation of the way things look and feel, rather than through regulation. Nudgers celebrate the sunnier success stories, such as getting more people to recycle or to quit smoking, but it’s easy to see how the same mentality might be applied in a more menacing way.

Nudge for thee but not for me?

This all reminds me of a general phenomenon, that “incentives matter” always seems like a good motto for other people, but we rarely seem to want it for ourselves. For example, there’s lots of talk about how being worried about losing your job is a good motivator to work hard (or, conversely, that a secure job is a recipe for laziness), but we don’t want job insecurity for ourselves. (Sure, lots of people who aren’t tenured professors or secure government employees envy those of us with secure jobs, and maybe they think we don’t deserve them, but I don’t think these people are generally asking for less security in their own jobs.)

More generally, it’s my impression that people often think that “nudges” are a good way to get other people to behave in desirable ways, without these people wanting to be “nudged” themselves. For example, did Jeremy Bentham put himself in a Panopticon to ensure his own good behavior?

There are exceptions, though. I like to stick other people in my office so it’s harder for me to drift off and be less productive. And lots of smokers are supportive of polities that make it less convenient to smoke, as this can make it easier for them to quit and harder for them to relapse.

Summary

With all that in mind, I’d like to propose a Golden Rule of Nudge: Nudge unto others as you would have them nudge unto you.

Do not recommend to apply incentives to others that you would not want for yourself.

## Perhaps you could try a big scatterplot with one dot per dataset?

We are studying variation in both means and variances in metabolic conditions. We have access to nearly 200 datasets that involve a range of metabolic traits and vary in sample size, mean effects, and variance. Some traits differ in mean but not variance, others in variance but not mean, still others in both, and of course some in neither. These studies are based on animal models where genetics and environmental conditions are well-controlled and where pretty rigorous study designs are possible. We plan to report the full range of results, but would like to rank results according to confidence (power?) for each dataset. Some are obviously more robust than others. Confidence limits. which will be reported, don’t seem to quite right for ranking. We feel an obligation to share some sense of confidence, which should be based on sample size, variance in the contrast groups – between genotypes, between diets, between treatments, …… . We are of course aware of the trickiness in studies like these. Our question is based on getting the rest right, and wanting to share with readers and reviewers a sense of the ‘strengths, limitations’ across the datasets. Suggestions?

My reply: Perhaps you could try a big scatterplot with one dot per dataset? Also I would doubt that there are really traits that do not differ in mean or variance. Maybe it would help to look at the big picture rather than to be categorizing individual cases, which can be noisy. Similarly, rather than ranking the results, I think it would be better to just consider ways of displaying all of them.

## Podcast interview on polling (mostly), also some Bayesian stuff

Hugo Bowne-Anderson interviewed me for a DataCamp podcast. Transcript is here.

## Rising test scores . . . reported as stagnant test scores

Joseph Delaney points to a post by Kevin Drum pointing to a post by Bob Somerby pointing to a magazine article by Natalie Wexler that reported on the latest NAEP (National Assessment of Educational Progress) test results.

In an article entitled, “Why American Students Haven’t Gotten Better at Reading in 20 Years,” Wexler asks, “what’s the reason for the utter lack of progress in reading scores?”

The odd thing is, though, is that reading scores have clearly gone up in the past twenty years, as Somerby points out in text and Drum shows in this graph:

Drum summarizes:

Asian: +15 points
White: +5 points
Hispanic: +10 points
Black: +5 points

. . . Using the usual rule of thumb that 10 points equals one grade level, black and white kids have improved half a grade level; Hispanic kids have improved a full grade level; and Asian kids have improved 1½ grade levels.

Why does this sort of thing get consistently misreported? Delaney writes:

My [Delaney’s] opinion: because there is a lot of money in education and it won’t be possible to “disrupt” education and redirect this money if the current system is doing well. . . .

It also moves the goalposts. If everything is falling apart then it isn’t such a crisis if the disrupted industry has teething issues once they strip cash out of it to pay for the heroes who are reinventing the system.

But if current educational systems are doing well, and slowly improving through incremental change, then it is a lot harder to argue that there is a crisis in education, isn’t it?

Could be. The other thing is that it can be hard to get unstuck from a conventional story. We discussed a similar example a few years ago: that time it was math test scores, which economist Roland Fryer stated had been “largely constant over the past thirty years,” even while he’d included a graph showing solid improvements.

## Bayesian inference and religious belief

We’re speaking here not of Bayesianism as a religion but of the use of Bayesian inference to assess or validate the evidence regarding religious belief, in short, the probability that God !=0 or the probability that the Pope is Catholic or, as Tyler Cowen put it, the probability that Lutheranism is true.

As a statistician and social scientist I have a problem with these formulations in large part because the states in question do not seem at all clearly defined—for instance, Lutheranism is a set of practices as much as it is a set of doctrines, and what does it mean for a practice to be “true”; also the doctrines themselves are vague enough that it seems pretty much meaningless to declare them true or false.

Nonetheless, people have wrestled with these issues, and it may be that something can be learned from such explorations.

So the following email from political philosopher Kevin Vallier might be of interest, following up on my above-linked reply to Cowen. Here’s Vallier:

Philosophers of religion have been using Bayesian reasoning to determine the rationality of theistic belief in particular for many years. Richard Swinburne introduced Bayesian analysis in his many books defending the rationality of theistic and Christian belief. But many others, theist and atheist, use it as well. So there actually are folks out there who think about their religious commitments in Bayesian terms, just not very many of us. But I bet there are at least 1000 philosophers, theologians, and lay apologists who find thinking in Bayesian terms useful on religious matters and debates.

A nice, accessible use of Bayesian reasoning with regard to religious belief is Richard Swinburne’s The Existence of God, 2nd ed.

Wow—I had no idea there were 1000 philosophers in the world, period. Learn something new every day.

In all seriousness . . . As indicated in my post replying to Cowen, I’m interested in the topic not so much out of an interest in religion but rather because of the analogy to political and social affiliation. It seems to me that many affiliations are nominally about adherence to doctrine but actually are more about belonging. This idea is commonplace (for example, when people speak of political “tribes”), but complications start to arise when the doctrines have real-world implications, as in our current environment of political polarization.

## Present each others’ posters

It seems that I’ll be judging a poster session next week. So this seems like a good time to repost this from 2009:

I was at a conference that had an excellent poster session. I realized the session would have been even better if the students with posters had been randomly assigned to stand next to and explain other students’ posters. Some of the benefits:

1. The process of reading a poster and learning about its material would be more fun if it was a collaborative effort with the presenter.

2. If you know that someone else will be presenting your poster, you’ll be motivated to make the poster more clear.

3. When presenting somebody else’s poster, you’ll learn the material. As the saying goes, the best way to learn a subject is to teach it.

4. The random assignment will lead to more inderdisciplinary understanding and, ultimately, collaboration.

I think just about all poster sessions should be done this way.

My post elicited some comments to which I replied:

– David wrote that my idea “misses the potential benefit to the owner of the poster of geting critical responses to their work.” The solution: instead of complete randoimization, randoimize the poster presenteres into pairs, then put pairs next to each other. Student A can explain poster B, student B can explain poster A, and spectators can give their suggestions to the poster preparers.

– Mike wrote that “one strong motivation for presenters is the opportunity to stand in front of you (and other members of the evaluation committee) and explain *their* work to you. Personally.” Sure, but I don’t think it’s bad if instead they’re explaining somebody else’s work. If I were a student, I think I’d enjoy explaining my tellow-students’ work to an outsider. The ensuing conversation might even result in some useful new ideas.

– Lawrence suggested that “the logic of your post apply to conference papers, too.” Maybe so.

## “Fudged statistics on the Iraq War death toll are still circulating today”

Mike Spagat shares this story entitled, “Fudged statistics on the Iraq War death toll are still circulating today,” which discusses problems with a paper published in a scientific journal in 2006, and errors that a reporter inadvertently included in a recent news article. Spagat writes:

The Lancet could argue that if [Washington Post reporter Philip] Bump had only read the follow-up letters it published, he never would have reprinted the discredited graph. But this argument is akin to saying that there is no need for warning labels on cigarettes because people can just read the scientific literature on smoking and consider themselves warned. But in practice, many people will just assume the graph is kosher because it sits on the Lancet website with no warning attached. . . .

That said, this particular chapter at least has a happy ending. I wrote to Bump and the Washington Post and they fixed the story, in the process demonstrating an admirable respect for evidence and a commitment to the truth. The Lancet would do well to follow their example.

The Lancet declined to comment on this piece.

And here’s Spagat’s summary of the whole process:

1. The Lancet publishes a false graph.
2. The problems of the graph are exposed, several even in letters to the Lancet.
3. The Lancet just leaves the graph up.
4. A Washington Post reporter stumbles onto the false graph, thinks it’s cool and reprints it.
5. I tell the reporter that he just published a false graph.
6. The reporter does a mea culpa and pulls the graph down.
7. I write up this sequence of events for The Conversation [see above link].
8. The Conversation sends it to the Lancet.
9. The Lancet declines to comment and leaves the false graph up.
10. The Conversation publishes the piece.
11. Someone else sees and believes in the graph?

Over the years I’ve had bad experiences trying to get both academic journals and newspapers/magazines to make corrections but the journals have been the more reluctant of the two. A low water mark was when an editor at Public Opinion Quarterly told me, after a long exchange, that POQ policy is that they don’t correct errors.

The Lancet, of course, publishes lots of good stuff too (including, for example, this article). So it’s too bad to see them duck this one. Or maybe there’s more to the story. Anyone can feel free to add information in the comments.

## “Ivy League Football Saw Large Reduction in Concussions After New Kickoff Rules”

I noticed this article in the newspaper today:

A simple rule change in Ivy League football games has led to a significant drop in concussions, a study released this week found.

After the Ivy League changed its kickoff rules in 2016, adjusting the kickoff and touchback lines by just five yards, the rate of concussions per 1,000 kickoff plays fell to two from 11, according to the study, which was published Monday in the Journal of the American Medical Association. . . .

Under the new system, teams kicked off from the 40-yard line, instead of the 35, and touchbacks started from the 20-yard line, rather than the 25.

The result? A spike in the number of touchbacks — and “a dramatic reduction in the rate of concussions” . . .

The study looked at the rate of concussions over three seasons before the rule change (2013 to 2015) and two seasons after it (2016 to 2017). Researchers saw a larger reduction in concussions during kickoffs after the rule change than they did with other types of plays, like scrimmages and punts, which saw only a slight decline. . . .

I was curious so I followed the link to the research article, “Association Between the Experimental Kickoff Rule and Concussion Rates in Ivy League Football,” by Douglas Wiebe, Bernadette D’Alonzo, Robin Harris, et al.

From the Results section:

Kickoffs resulting in touchbacks increased from a mean of 17.9% annually before the rule change to 48.0% after. The mean annual concussion rate per 1000 plays during kickoff plays was 10.93 before the rule change and 2.04 after (difference, −8.88; 95% CI, −13.68 to −4.09). For other play types, the concussion rate was 2.56 before the rule change and 1.18 after (difference, −1.38; 95% CI, −3.68 to 0.92). The difference-in-differences analysis showed that 7.51 (95% CI, −12.88 to −2.14) fewer concussions occurred for every 1000 kickoff plays after vs before the rule change.

I took a look at the table and noticed some things.

First, the number of concussions was pretty high and the drop was obviously not statistical noise: 126 (that is, 42 per year) for the first three years, down to 33 (15.5 per year) for the last two years. With the exception of punts and FG/PATs, the number of cases was large enough that the drop was clear.

Second, I took a look at the confidence intervals. The confidence interval for “other play types combined” includes zero: see the bottom line of the table. Whassup with that?

I shared this example with my classes this morning and we also had some questions about the data:

– How is “concussion” defined? Could the classification be changing over time? I’d expect that, what with increased concussion awareness, that concussion would be diagnosed more than before, which would make the declining trend in the data even more impressive. But I don’t really know.

– Why data only since 2013? Maybe that’s only how long they’ve been recording concussions.

– We’d like to see the data for each year. To calibrate the effect of a change over time, you want to see year-to-year variation, in this case a time series of the 5 years of data. Obviously the years of the concussions are available, and they might have even been used in the analysis. In the published article, it says, “Annual concussion counts were modeled by year and play type, with play counts as exposures, using maximum likelihood Poisson regression . . .” I’m not clear on what exactly was done here.

– We’d also like to see similar data from other conferences, not just the Ivy League, to see changes in games that did not follow these rules.

– Even simpler than all that, we’d like to see the raw data on which the above analysis was based. Releasing the raw data—that would be trivial. Indeed, the dataset may already be accessible—I just don’t know where to look for it. Ideally we’d move to a norm in which it was just expected that every publication came with its data and code attached (except when not possible for reasons of privacy, trade secrets, etc.). It just wouldn’t be a question.

The above requests are not meant to represent devastating criticisms of the research under discussion. It’s just hard to make informed decisions without the data.

Checking the calculations

Anyway, I was concerned about the last row of the above table so I thought I’d do my best to replicate the analysis in R.

First, I put the data from the table into a file, football_concussions.txt:

```y1 n1 y2 n2
kickoff 26 2379 3 1467
scrimmage 92 34521 28 22467
punt 6 2496 2 1791
field_goal_pat 2 2090 0 1268
```

Then I read the data into R, added a new row for “Other plays” summing all the non-kickoff data, and computed the classical summaries. For each row, I computed the raw proportions and standard errors, the difference between the proportion, and the standard errors of that difference. I also computed the difference in differences, comparing the change in the concussion rate for kickoffs to the change in concussion rate for non-kickoff plays, as this comparison was mentioned in the article’s results section. I multiplied all the estimated differences and standard errors by 1000 to get the data in rates per thousand.

Here’s the (ugly) R code:

```concussions <- read.table("football_concussions.txt", header=TRUE)
concussions <- rbind(concussions, apply(concussions[2:4,], 2, sum))
rownames(concussions)[5] <- "other_plays"
compare <- function(data) {
p1 <- data\$y1/data\$n1
p2 <- data\$y2/data\$n2
diff <- p2 - p1
se_diff <- sqrt(p1*(1-p1)/data\$n1 + p2*(1-p2)/data\$n2)
diff_in_diff <- diff[1] - diff[5]
se_diff_in_diff <- sqrt(se_diff[1]^2 + se_diff[5]^2)
return(list(diffs=data.frame(data, diff=diff*1000,
diff_ci_lower=(diff - 2*se_diff)*1000, diff_ci_upper=(diff + 2*se_diff)*1000),
diff_in_diff=diff_in_diff*1000, se_diff_in_diff=se_diff_in_diff*1000))
}
print(lapply(compare(concussions), round, 2))
```

And here's the result:

```\$diffs
y1    n1 y2    n2  diff diff_ci_lower diff_ci_upper
kickoff         26  2379  3  1467 -8.88        -13.76         -4.01
scrimmage       92 34521 28 22467 -1.42         -2.15         -0.69
punt             6  2496  2  1791 -1.29         -3.80          1.23
field_goal_pat   2  2090  0  1268 -0.96         -2.31          0.40
other_plays    100 39107 30 25526 -1.38         -2.05         -0.71

\$diff_in_diff
[1] -7.5

\$se_diff_in_diff
[1] 2.46
```

The differences and 95% intervals computed this way are similar, although not identical to, the results in the published table--but there are some differences that baffle me. Let's go through the estimates, line by line:

- Kickoffs. Estimated difference is identical 95% interval is correct to two significant digits. This unimportant discrepancy could be coming because I used a binomial model and the published analysis used a Poisson regression.

- Plays from scrimmage. Estimated difference is the same for both analyses; lower confidence bound is almost the same. But the upper bounds are different: I have -0.69; the published analysis is -0.07. I'm guessing they got -0.70 and they just made a typo when entering the result into the paper. Not as bad as the famous Excel error, but these slip-ups indicate a problem with workflow. A problem I often have in my workflow too, as in the short term it is often easier to copy numbers from one place to another than to write a full script.

- Punts. Estimated difference and confidence interval work out to within 2 significant digits.

- Field goals and extra points. Same point estimate, but confidence bounds are much different, with the published intervals much wider than mine. This makes sense: There's a zero count in these data, and I was using the simple sqrt(p_hat*(1-p_hat)) standard deviation formula which gives too low a value when p_hat=0. Their Poisson regression would be better.

- All non-kickoffs. Same point estimate but much different confidence intervals. I think they made a mistake here: for one thing, their confidence interval doesn't exclude zero, even though the raw numbers show very strong evidence of a difference (30 concussions after the rule change and 100 before; even if you proportionally scale down the 100 to 60 or so, that's still sooo much more than 30; it could not be the result of noise). I have no idea what happened here; perhaps this was some artifact of their regression model?

This last error is somewhat consequential, in that it could lead readers to believe that there's no strong evidence that the underlying rate of concussions was declining for non-kickoff plays.

- Finally, the diff-in-diff is three standard errors away from zero, implying that the relatively larger decline in concussion rate in kickoffs, compared to non-kickoffs, could not be simply explained by chance variation.

Difference in difference or ratio of ratios?

But is the simple difference the right thing to look at?

Consider what was going on. Before the rule change, concussion rates were much higher for kickoffs than for other plays. After the rule change, concussion rates declined for all plays.

As a student in my class pointed out, it really makes more sense to compare the rates of decline than the absolute declines.

Put it this way. If the probabilities of concussion are:
- p_K1: Probability of concussion for a kickoff play in 2013-2015
- p_K2: Probability of concussion for a kickoff play in 2016-2017
- p_N1: Probability of concussion for a non-kickoff play in 2013-2015
- p_N2: Probability of concussion for a non-kickoff play in 2016-2017.

Following the published paper, we estimated the difference in differences, (p_K2 - p_K1) - (p_N2 - p_N1).

But I think it makes more sense to think multiplicatively, to work with the ratio of ratios, (p_K2/p_K1) / (p_N2/p_N1).

Or, on the log scale, (log p_K2 - log p_K1) - (log p_N2 - log p_N1).

What's the estimate and standard error of this comparison?

The key step is that we can use relative errors. From the Poisson distribution, the relative sd of an estimated rate is 1/sqrt(y), so this is the approx sd on the log scale. So, the estimated difference in difference of log probabilities is (log(3/1467) - log(26/2379)) - (log(30/25526) - log(100/39107)) = -0.90. That's a big number: exp(0.90) = 2.46, which means that the concussion rate fell over twice as fast in kickoff than in non-kickoff plays.

But the standard error of this difference in difference of logs is sqrt(1/3 + 1/26 + 1/30 + 1/100) = 0.64. The observed d-in-d-of-logs is -0.90, which is less than two standard errors from zero, thus not conventionally statistically significant.

So, from these data alone, we cannot confidently conclude that the relative decline in concussion rates is more for kickoffs than for other plays. That estimate of 3/1467 is just too noisy.

We could also see this using a Poisson regression with four data points. We create the data frame using this (ugly) R code:

```data <- data.frame(y = c(concussions\$y1[c(1,5)], concussions\$y2[c(1,5)]),
n = c(concussions\$n1[c(1,5)], concussions\$n2[c(1,5)]),
time = c(0, 0, 1, 1),
kickoff = c(1, 0, 1, 0))
```

Which produces this:

```    y     n time kickoff
1  26  2379    0       1
2 100 39107    0       0
3   3  1467    1       1
4  30 25526    1       0
```

Then we load Rstanarm:

```library("rstanarm")
options(mc.cores = parallel::detectCores())
```

And run the regression:

```fit <- stan_glm(y ~ time*kickoff, offset=log(n), family=poisson, data=data)
print(fit)
```

Which yields:

```             Median MAD_SD
(Intercept)  -6.0    0.1
time         -0.8    0.2
kickoff       1.4    0.2
time:kickoff -0.9    0.6
```

The coefficient for time is negative: concussion rates have gone down a lot. The coefficient for kickoff is positive: kickoffs have higher concussion rates. The coefficient for the interaction of time and kickoff is negative: concussion rates have been going down faster for kickoffs. But that last coefficient is less than two standard errors from zero. This is just confirming with our regression analysis what we already saw with our simple standard error calculations.

OK, but don't I think that the concussion rate really was going down faster, even proportionally, in kickoffs than in other plays? Yes, I do, for three reasons. In no particular order:
- The data do show a faster relative decline, albeit with uncertainty;
- The story makes sense given that the Ivy League directly changed the rules to make kickoffs safer;
- Also, there were many more touchbacks, and this directly reduces the risks.

That's fine.

Summary

- There seem to be two mistakes in the published paper: (1) some error in the analysis of the plays from scrimmage which led to too wide a standard error, and (2) a comparison of absolute rather than relative probabilities.

- It would be good to see the raw data for each year.

- The paper's main finding---a sharp drop in concussions, especially in kickoff plays---is supported both by the data and our substantive understanding.

- Concussion rates fell faster for kickoff plays, but they also dropped a lot in non-kickoff plays too. I think the published article should've emphasized this finding more than they did, but they perhaps here were hindered by the error they made in their analysis leading to an inappropriately wide confidence interval for non-kickoff plays. The decline in concussions for non-kickoff plays is consistent with a general rise in concussion awareness.

P.S. I corrected a mistake in the above Poisson regression code, pointed out by commenter TJ (see here).

## Strategic choice of where to vote in November

Darcy Kelley sends along a link to this site, Make My Vote Matter, in which you can enter two different addresses where you might vote, and it will tell you in which (if any) of these addresses has elections that are predicted to be close. The site is aimed at students; according to the site, they can register to vote at either their home or school address.

## “Six Signs of Scientism”: where I disagree with Haack

I came across this article, “Six Signs of Scientism,” by philosopher Susan Haack from 2009. I think I’m in general agreement with Haack’s views—science has made amazing progress over the centuries but “like all human enterprises, science is ineradicably is fallible and imperfect. At best its progress is ragged, uneven, and unpredictable; moreover, much scientific work is unimaginative or banal, some is weak or careless, and some is outright corrupt . . .”—and I’ll go with her definition of “scientism” as “a kind of over-enthusiastic and uncritically deferential attitude towards science, an inability to see or an unwillingness to acknowledge its fallibility, its limitations, and its potential dangers.”

But I felt something was wrong with Haack’s list of the six signs of scientism, which she summarizes as:

1. Using the words “science,” “scientific,” “scientifically,” “scientist,” etc., honorifically, as generic terms of epistemic praise.

2. Adopting the manners, the trappings, the technical terminology, etc., of the sciences, irrespective of their real usefulness.

3. A preoccupation with demarcation, i.e., with drawing a sharp line between genuine science, the real thing, and “pseudo-scientific” imposters.

4. A corresponding preoccupation with identifying the “scientific method,” presumed to explain how the sciences have been so successful.

5. Looking to the sciences for answers to questions beyond their scope.

6. Denying or denigrating the legitimacy or the worth of other kinds of inquiry besides the scientific, or the value of human activities other than inquiry, such as poetry or art.

Yes, these six signs can be a problem. But, to me, these six signs are associated with a particular sort of scientism, which one might call active scientism, a Richard Dawkins-like all-out offensive against views that are considered anti-scientific.

I spend a lot of time thinking about something different: a passive scientism which is not concerned about turf (thus, not showing signs 1 and 3 above); not concerned about the scientific method, indeed often displaying little interest in the foundations of scientific logic and reasoning (thus, not showing sign 4 above); and not showing any imperialistic inclinations to bring the humanities into the scientific orbit (thus, not showing signs 5 or 6 above). Passive scientism does involve adopting the trappings and terminology of science in a thoughtless way, so there is a bit of sign 2 above, but that’s it.

A familiar examples of passive scientism is “pizzagate”: the work, publication, and promotion, of the studies conducted by the Food and Brand Lab at Cornell University. Other examples include papers published in the Proceedings of the National Academy of Sciences on himmicanes, air rage, and ages ending in 9.

In all these cases, topics were being studied that clearly can be studied by science. So the “scientism” here is not coming into the decision to pursue this research. Rather, the scientism arises from blind trust in various processes associated with science, such as randomized treatment assignment, statistical significance, and peer review.

In this particular manifestation of scientism—claims that bounce around between scientific journals, textbooks, and general media outlets such as NPR and Ted talks—there is no preoccupation with identifying the scientific method or preoccupation with demarcation, but rather the near-opposite, an all-too-calm acceptance of wacky claims that happen to be in the proximity to various tokens of science.

So I think that when talking about scientism, we need to consider passive as well as active scientism. For every Dawkins, there is a Gladwell.


text:
# The meaning of the word “superposition”

This is from the Wikipedia article on Hilbert's 13th Problem as it was on 31 March 2012:

[Hilbert’s 13th Problem suggests this] question: can every continuous function of three variables be expressed as a composition  of finitely many continuous functions of two variables? The affirmative answer to this general question was given in 1957 by Vladimir Arnold, then only nineteen years old and a student of Andrey Kolmogorov. Kolmogorov had shown in the previous year that any function of several variables can be constructed with a finite number of three-variable functions. Arnold then expanded on this work to show that only two-variable functions were in fact required, thus answering Hilbert's question.

In their paper A relation between multidimensional data compression and Hilbert’s 13th  problem,  Masahiro Yamada and Shigeo Akashi describe an example of Arnold's theorem this way:

Let $f ( \cdot , \cdot, \cdot )$ be the function of three variable defined as $f(x, y, z)=xy+yz+zx$, $x ,y , z\in \mathbb{C}$ . Then, we can easily prove that there do not exist functions of two variables $g(\cdot , \cdot )$ , $u(\cdot, \cdot)$ and $v(\cdot , \cdot )$ satisfying the following equality: $f(x, y, z)=g(u(x, y),v(x, z)) , x , y , z\in \mathbb{C}$ . This result shows us that $f$ cannot be represented any 1-time nested superposition constructed from three complex-valued functions of two variables. But it is clear that the following equality holds: $f(x, y, z)=x(y+z)+(yz)$ , $x,y,z\in \mathbb{C}$ . This result shows us that $f$ can be represented as a 2-time nested superposition.

The strategy used in the Superposition Theorem is to eliminate all but one source of power within a network at a time, using series/parallel analysis to determine voltage drops (and/or currents) within the modified network for each power source separately. Then, once voltage drops and/or currents have been determined for each power source working separately, the values are all “superimposed” on top of each other (added algebraically) to find the actual voltage drops/currents with all sources active.

Superposition Theorem in Wikipedia:

The superposition theorem for electrical circuits states that for a linear system the response (Voltage or Current) in any branch of a bilateral linear circuit having more than one independent source equals the algebraic sum of the responses caused by each independent source acting alone, while all other independent sources are replaced by their internal impedances.

Quantum superposition is a fundamental principle of quantum mechanics. It holds that a physical system — such as an electron — exists partly in all its particular, theoretically possible states (or, configuration of its properties) simultaneously; but, when measured, it gives a result corresponding to only one of the possible configurations (as described in interpretation of quantum mechanics).

Mathematically, it refers to a property of solutions to the Schrödinger equation; since theSchrödinger equation is linear, any linear combination of solutions to a particular equation will also be a solution of it. Such solutions are often made to be orthogonal (i.e. the vectors are at right-angles to each other), such as the energy levels of an electron. By doing so the overlap energy of the states is nullified, and the expectation value of an operator (any superposition state) is the expectation value of the operator in the individual states, multiplied by the fraction of the superposition state that is "in" that state

The CIO midmarket site says much the same thing as the first paragraph of the Wikipedia Quantum Superposition entry but does not mention the stuff in the second paragraph.

In particular, the  Yamada & Akashi article describes the way the functions of two variables are put together as "superposition", whereas the Wikipedia article on Hilbert's 13th calls it composition.  Of course, superposition in the sense of the Superposition Principle is a composition of multivalued functions with the top function being addition.  Both of Yamada & Akashi's examples have addition at the top.  But the Arnold theorem allows any continuous function at the top (and anywhere else in the composite).

So one question is: is the word "superposition" ever used for general composition of multivariable functions? This requires the kind of research I proposed in the introduction of The Handbook of Mathematical Discourse, which I am not about to do myself.

The first Wikipedia article above uses "composition" where I would use "composite".  This is part of a general phenomenon of using the operation name for the result of the operation; for examples, students, even college students, sometimes refer to the "plus of 2 and 3" instead of the "sum of 2 and 3". (See "name and value" in abstractmath.org.)  Using "composite" for "composition" is analogous to this, although the analogy is not perfect.  This may be a change in progress in the language which simplifies things without doing much harm.  Even so, I am irritated when "composition" is used for "composite".

Quantum superposition seems to be a separate idea.  The second paragraph of the Wikipedia article on quantum superposition probably explains the use of the word in quantum mechanics.

Send to Kindle

# Composites of functions

In my post on automatic spelling reform, I mentioned the various attempts at spelling reform that have resulted in both the old and new systems being used, which only makes things worse.  This happens in Christian denominations, too.  Someone (Martin Luther, John Wesley) tries to reform things; result: two denominations.   But a lot of the time the reform effort simply disappears.  The Chicago Tribune tried for years to get us to write “thru” and “tho” —  and failed.  Nynorsk (really a language reform rather than a spelling reform) is down to 18% of the population and the result of allowing Nynorsk forms to be used in the standard language have mostly been nil.  (See Note 1.)

In my early years as a mathematician I wrote a bunch of papers writing functions on the right (including the one mentioned in the last post).  I was inspired by some algebraists and particularly by Beck’s Thesis (available online via TAC), which I thought was exceptionally well-written.  This makes function composition read left to right and makes the pronunciation of commutative diagrams get along with notation, so when you see the diagram below you naturally write h = fg instead of h = gf.

Sadly, I gave all that up before 1980 (I just looked at some of my old papers to check).  People kept complaining.  I even completely rewrote one long paper (Reference [3]) changing from right hand to left hand (just like Samoa).  I did this in Zürich when I had the gout, and I was happy to do it because it was very complicated and I had a chance to check for errors.

Well, I adapted.  I have learned to read the arrows backward (g then f in the diagram above).  Some French category theorists write the diagram backward, thus:

But I was co-authoring books on category theory in those days and didn’t think people would accept it. Not to mention Mike Barr (not that he is not a people, oh, never mind).

Nevertheless, we should have gone the other way.  We should have adopted the Dvorak keyboard and Betamax, too.

Notes

[1] A lifelong Norwegian friend of ours said that when her children say “boka” instead of “boken” it sound like hillbilly talk does to Americans.  I kind of regretted this, since I grew up in north Georgia and have been a kind of hillbilly-wannabe (mostly because of the music); I don’t share that negative reaction to hillbillies.  On the other hand, you can fageddabout “ho” for “hun”.

References

[1] Charles Wells, Automorphisms of group extensions, Trans. Amer. Math. Soc, 155 (1970), 189-194.

[2] John Martino and Stewart Priddy, Group extensions and automorphism group rings. Homology, Homotopy and Applications 5 (2003), 53-70.

[3] Charles Wells, Wreath product decomposition of categories 1, Acta Sci. Math. Szeged 52 (1988), 307 – 319.

Send to Kindle

format: plain


text:
Homework Help: What is sign

1. Sep 17, 2011

pyroknife

I've seen some equations written with a sign for example the e field due to a uniform disk of charge is sign(z)*2*pi*etc... what does the sign(z) do in that equation? When I see the equation actually used, I just see them ignore the sign.

2. Sep 17, 2011

ohms law

wouldn't the answer here depend on the frame of reference that you decided to use? I mean, if you decide that coordinate 0,0(,0) is outside of the possible area that is of interest to the problem, then signs wouldn't really matter much, right? On the other hand, if the thing being modeled crosses from a positive coordinate to a negative one (or vice versa)...


text:
# Properties

 Label 189618.bp Number of curves $1$ Conductor $189618$ CM no Rank $0$

# Related objects

Show commands: SageMath
sage: E = EllipticCurve("bp1")

sage: E.isogeny_class()

## Elliptic curves in class 189618.bp

sage: E.isogeny_class().curves

LMFDB label Cremona label Weierstrass coefficients j-invariant Discriminant Torsion structure Modular degree Faltings height Optimality
189618.bp1 189618h1 $$[1, 0, 0, -9396, -351378]$$ $$-41756642210557/4634982$$ $$-10183055454$$ $$[]$$ $$228096$$ $$0.94915$$ $$\Gamma_0(N)$$-optimal

## Rank

sage: E.rank()

The elliptic curve 189618.bp1 has rank $$0$$.

## Complex multiplication

The elliptic curves in class 189618.bp do not have complex multiplication.

## Modular form 189618.2.a.bp

sage: E.q_eigenform(10)

$$q + q^{2} + q^{3} + q^{4} - q^{5} + q^{6} + q^{7} + q^{8} + q^{9} - q^{10} + q^{11} + q^{12} + q^{14} - q^{15} + q^{16} + q^{17} + q^{18} + 6q^{19} + O(q^{20})$$


text:
Home > Olympiad > IMC 2011 Day 2 Problem 2

## IMC 2011 Day 2 Problem 2

An alien race has three genders: male, female and emale. A married triple consists of three persons, one from each gender who all like each other. Any person is allowed to belong to at most one married triple. The feelings are always mutual ( if $x$ likes $y$ then $y$ likes $x$).

The race wants to colonize a planet and sends $n$ males, $n$ females and $n$ emales. Every expedition member likes at least $k$ persons of each of the two other genders. The problem is to create as many married triples so that the colony could grow.

a) Prove that if $n$ is even and $k\geq n/2$ then there might be no married triple.

b) Prove that if $k \geq 3n/4$ then there can be formed $n$ married triple ( i.e. everybody is in a triple).

1. August 4, 2011 at 6:20 pm

I think this problem is not the case in question a), may be k> = n / 2 rather than k> = 1 / 2. Be sure to check back and see more here http://www.mathdoi.com/2011/07/de-thi-olympic-toan-sinh-vien-quoc-te-nam-2011-imc-2011-problems-of-2st-day/

• August 4, 2011 at 10:16 pm

Yes, you are right. There was $k/2$ instead of $1/2$. Thank you for your correction.


text:
# Mework i Help Save & Exit Sube Check my wo Required information Problem 15-1A Production costs...

###### Question:

mework i Help Save & Exit Sube Check my wo Required information Problem 15-1A Production costs computed and recorded; reports prepared LO P1, P2, P3, P4 [The following information applies to the questions displayed below.) Marcelino Co.'s March 31 inventory of raw materials is $87.000. Raw materials purchases in April are$590.000, and factory payroll cost in April is $386.000. Overhead costs incurred in April are: indirect materials,$58,000; indirect labor, $29.000; factory rent,$40,000; factory utilities, $22,000; and factory equipment depreciation,$55,000. The predetermined overhead rate is 50% of direct labor cost. Job 306 is sold for $655.000 cash in April. Costs of the three jobs worked on in April follow. Job 306 Job 307 Job 308$31.000 24,000 12,000 $38,000 15,000 7,500 Balances on March 31 Direct materials Direct labor Applied overhead Costs during April Direct materials Direct labor Applied overhead Status on April 30 132,000 102,000 215,000 154,000$105,000 101.000 In Finished (sold) Finished (unsold) process < Prev 67 of 7 Next >
##### Q1 Need this solved asap. Please help me with solution. 1. What is the wavelength of...
Q1 Need this solved asap. Please help me with solution. 1. What is the wavelength of a neutron (mo = 1.67 x 10-27 kg) traveling at a speed of 4.20 x 10 m/s?...
##### The coefficient of determination of a set of data points is 0.559 and the slope of...
The coefficient of determination of a set of data points is 0.559 and the slope of the regression line is 4.45. Determine the linear correlation coefficient of the data. What is the linear correlation coefficient? requals=nothing (Round to three decimal places as needed.)...
##### How do you use the laws of exponents to simplify the expression b^8(2b)^4?
How do you use the laws of exponents to simplify the expression b^8(2b)^4?...
##### Two people carry a heavy electric motor by placing it on a light board 2.80 m...
Two people carry a heavy electric motor by placing it on a light board 2.80 m long. One person lifts at one end with a force of 450 N. and the other lifts the opposite end with a force of 670 N. What is the weight of the motor? Part B Where along the board is its center of gravity located? PartC Sup...
##### Determine the equivalent capacitance between A and B for the group of capacitors in the drawing....
Determine the equivalent capacitance between A and B for the group of capacitors in the drawing. Let C1-9 μF and C2 = 4.0 5.0 gF 24 jaF 4,0 μF Ca...
##### Question 4 The SEC has the authority to set accounting standards for A) All companies listed...
Question 4 The SEC has the authority to set accounting standards for A) All companies listed on the securities exchange as 1929 B) All companies as of the passing of the Sarbanes-Oxley Act C) US Companies, however the SEC delegates this authority to FASB D) Only those companies that have both US and...
##### Explain when will two events be independent and when will two events be mutually exclusive. Can...
Explain when will two events be independent and when will two events be mutually exclusive. Can two mutually exclusive events be independent also? Can two independent events be mutually exclusive? Suppose the experiment is roll two dice. Consider events E= both numbers are even. F = both numbers are...
##### C++. Test if a Tree is Balanced Use the given Tree class and implement a function...
C++. Test if a Tree is Balanced Use the given Tree class and implement a function to test if the binary tree is balanced or not. An empty tree is height-balanced. A non-empty binary tree T is balanced if: 1) Left subtree of T is balanced 2) Right subtree of T is balanced 3) The difference between he...
##### Wexpro, Inc., produces several products from processing 1 ton of clypton, a rare mineral. Material and...
Wexpro, Inc., produces several products from processing 1 ton of clypton, a rare mineral. Material and processing costs total \$63,000 per ton, one-fourth of which is allocated to product X15. Eight thousand two hundred units of product X15 are produced from each ton of clypton. The units can either ...
##### Temperature98.4,Blood pressure 200/100, Pulse 100 beats/min, Respiratory rate 32, She states that she is 5 ft,...
Temperature98.4,Blood pressure 200/100, Pulse 100 beats/min, Respiratory rate 32, She states that she is 5 ft, 6 inches tall,body weight 210 lbs. Patient was sent Chest X-rays and the report shows Right middle Lobs atelectasis with elevated right hemi diaphragm The Nurse place patient on Electrocard...


text:
# DM-Sets,Relations and Functions-Q12

Let $R$ denote a relation on the set of all ordered pairs of positive integers by $(x,y)R(u,v)$ if and only if $xv=yu$. Which of the following is true?

(A). $R$ is a partial order relation

(B). $R$ is an equivalence relation

(C). $R$ is neither partial order relation nor equivalence relation

(D). $R$ is an antisymmetric relation

reshown Jul 6

So correct Answer is B

answered Jul 8 by (33,170 points)
selected Jul 9
B is correct one.
answered Jul 6 by (31,600 points)
can anyone provide an expalnation, i couldnt visualise because of the ordered pair.
Can you please explain why is it symmetric? for example (1,2) and (2,1)
here,ordered pair present in relation does not contain single elements,instead of that it contains ordered pairs inside of that ordered pair,                                                 for eg.,if relation is {((a,b),(c,d)),((c,d),(a,b))},it is symmetric                                                           since in first pair ,ad=bc                                                                                ..in second pair also,bc=ad..
such that both are present in relation and hence it is symmetric.


text:
Lays Chips Complaint, Asus Zenfone Max Pro M5, Wisp Internet Speeds, Lead Verb Forms, Bh Cosmetics Sand Foundation, Coming In Hot Saying, Model United Nations Conferences 2020, Creamy Chicken Soup Recipe, Shaadi Mein Zaroor Aana Full Movie Movierulz, Black Is The Color Folk Song, Jihad Dib Inaugural Speech, Peppermint Chocolate Bar, Vidal Sassoon Hair Dryer, Best Public Relations Undergraduate Programs, Society Must Be Defended Wiki, Healthy Coffee Creamer Alternatives, Watch Science Of Babies, Demons Souls Intelligence, Sinugbang Baboy Bisaya Recipe, Cost Of Living For A Family Of 4, How To Get To Vashj'ir, Abdominal Pain Metallic Taste In Mouth Diarrhea, " />

## salt water boiling point

When did organ music become associated with baseball? Was Greta Van Susteren a defense attorney in the OJ Simpson case? This point is by definition the origin of the freezing (solid-liquid) line, which intersects the 1-atm line at a reduced freezing point ΔTf, indicated by . Pressure must be within the ranges 1-220 bara, 14.7-3200 psia, 760-165 000 mm Hg or 30-6500 in Hg. You would have to add 58 grams of salt just to raise the boiling point of a liter of water by one half of a degree Celsius. A pot of water with 20 percent salt boils faster because the salt in the water does not take much of the heat being directed into the solution. Unless otherwise noted, LibreTexts content is licensed by CC BY-NC-SA 3.0. The usual boiling point of water is 100 °C or 212 °F at 1 atmosphere of pressure (at sea level). Firstly, we conducted an experiment to find out the boiling point of water. Fact Check: What Power Does the President Really Have Over State Governors? When you dissolve salt in water, it breaks into sodium and chloride ions. This has the interesting effect of making the water that remains more saline, and hence more dense, causing it to sink to the bottom part of the ocean where it gets taken up by the south-flowing deep current. We don't save this data. Can you take flexeril and diclofenac together? Advertisement. If you add salt to water, be sure to add it before boiling the water. Does Adding Salt Lower the Boiling Point of Water? 1 2 3 0 If we let x = [H+] = [NO2–], then by stoichiometry, [HNO2] = 0.100 - x and .106 - x = 2x and x = .0355. The calculator below can be used to calculate the water boiling point at given absolute pressures. Dilution of the liquid (the solvent) by the nonvolatile solute reduces the vapor pressure of the solvent according to Raoult’s law, thus reducing the temperature at which the vapor pressures of the liquid and frozen forms of the solution will be equal. The fraction of HNO2 that is dissociated is .0355 ÷ 0.100 = .355, corresponding to 35.5% dissociation of the acid. You may already be familiar with the phase map for water below. These applications will - due to browser restrictions - send data between your browser and our server. The rule is that the phase with the most negative free energy rules. ThoughtCo uses cookies to provide you with a great user experience. Adding a small amount of salt to a pot of boiling water does not make it boil significantly faster. Make sure you thoroughly understand the following essential ideas: The colligative properties really depend on the escaping tendency of solvent molecules from the liquid phase. However, there is no danger of boiling the NaCl: The boiling point of sodium chloride is 2575 F or 1413 C. Salt, like other ionic solids, has an extremely high boiling point. The nominal molality of the solution is (.001 mol) ÷ (1.00 kg) = 0.001 mol kg–1. Salt waters boiling point is 200 degrees C. It varies depending upon the salt concentration. Addition of a non-volatile solute reduces the vapor pressures to the values given by the blue line. Legal. If you want to promote your products or services in the Engineering ToolBox - please use Google Adwords. Suppose that a liquid solvent and its solid (water and ice, for example) are in equilibrium ( below), and we add a non-volatile solute (such as salt, sugar, or automotive antifreeze liquid) to the water. The aims of our experiment were: To learn how to heat a liquid in a beaker. Note that this plot is not to scale. What instrument plays the main melody fom Nickelback? Engineering ToolBox - Resources, Tools and Basic Information for Engineering and Design of Technical Applications! An aqueous solution of nitrous acid (HNO2, MW = 47) freezes at –0.198 .C. Boiling Point of Water On the 5th of February, we learnt about the boiling point of water. Explaining freezing point depression is admittedly a bit more difficult, but you should nevertheless be able to explain how the application of salt on an ice-covered road can cause the ice to melt. Note! How did Eli Whitney change things socially? As with boiling point elevation, in dilute solutions there is a simple linear relation between the freezing point depression and the molality of the solute: $\Delta T_f = K_f \dfrac{\text{moles of solute}}{\text{kg of solvent}}$. The boiling point is raised by 0.5 degrees Celsius for water with 29.2 grams of salt dissolved in each kg of water.

#### Aurora, North Aurora, Boulder Hill, Montgomery, Oswego, Sugar Grove and portions of Yorkville and Batavia

© 2020 Fox Metro Water Reclamation District - Contact Us - Public Notices - Careers - FOIA - Accessibility


text:
# Build embedded XML based on string containment

I have a list of strings that represent a path. These are unique, unsorted values that look like this:

li = ['1/', '1/2/4/', '1/23/4/', '1/2/', '1/1/3/', '1/2/3/4/', '1/1/', '1/23/', '1/2/3/']

It can also be assumed that every path has a parent, e.g. 1/2/ has parent 1/.

The goal is to organise these items in a structured XML format, where the parent-relation is clear, i.e. where deeper elements are part of their parents, mimicking a directory tree. Desired output for the list above would be:

<root>
<include path="1/">
<include path="1/1/">
<include path="1/1/3/"/>
</include>
<include path="1/2/">
<include path="1/2/3/">
<include path="1/2/3/4/"/>
</include>
<include path="1/2/4/"/>
</include>
<include path="1/23/">
<include path="1/23/4/"/>
</include>
</include>
</root>

The following works (run it here), but I am not sure about efficiency. The actual list can be thousands of values long. I am also curious about memory management, because I am writing all XML nodes in-memory. The eventual goal is to print the final XML to a file.

from xml.etree import ElementTree as ET
import re

li = ['1/', '1/2/4/', '1/23/4/', '1/2/', '1/1/3/', '1/2/3/4/', '1/1/', '1/23/', '1/2/3/']
li = sorted(li)
# In Python >= 3.6 dicts maintain order
# Sets don't? So use best of both worlds, dict: speed and order
di = {l: False for l in li}

root = ET.Element('root')

# First build the root nodes, remove them from dict
for p in list(di.keys()):
if len(list(filter(None, p.split('/')))) == 1:
del di[p]
root.append(ET.Element('include', {'path': p}))

# Because dict (and so keys()) are ordered, we can assume that the nodes higher
# in the include tree are created before the more in-depth ones
for p in list(di.keys()):
parent_path = re.sub('\d+\/$', '', p) try: parent = root.find(f'.//include[@path="{parent_path}"]') parent.append(ET.Element('include', {'path': p})) except Exception: print(f"parent not found for {p}") print(ET.tostring(root))  This indeed returns the expected output (not pretty-printed). I am wondering if there is a better option and whether there are flaws in my approach. I'll only be using Python >= 3.6, which is quite important for the order of the dictionary. Update I noticed a bug with the sorting of the list/dict because actually the sorting was done on the strings which isn't what we want. (e.g. 1001/ would be before 134/) I rewrote part of the code, and use sets assuming these are faster. from xml.etree import ElementTree as ET import re def get_xml(paths): root = ET.Element('paths') root_paths = set() for p in paths: # If only one item, use as first-level node - e.g. 1/ if len(list(filter(None, p.split('/')))) == 1: root_paths.add(p) root.append(ET.Element('include', {'path': p})) # Remove root_paths from set paths = paths.difference(root_paths) # We can't use regular sort because that'll sort by string # Instead, sort by array of ints representation. E.g. 12/1001/14/ -> [12, 1001, 14] sorted_paths = sorted(paths, key=lambda item: [int(n) for n in list(filter(None, item.split('/')))]) for p in sorted_paths: # Find parent_path by removing last part of string parent_path = re.sub('\d+\/$', '', p)
try:
parent = root.find(f'.//include[@path="{parent_path}"]')
parent.append(ET.Element('include', {'path': p}))
except AttributeError:

return root

s = {'1/', '1/2/4/', '1/23/4/', '1/2/', '1/1/3/', '1/2/3/4/', '1/1/', '1/23/', '1/2/3/'}

xml = get_xml(s)

print(ET.tostring(xml))

Because you have the overall root node, (a bit confusing that it changes name from root to paths between the two versions) there shouldn't actually be any inherent difference between nodes of the form "1/" and nodes of the form "1/2/" or longer. It should be possible to completely remove the first loop, and just have a special case definition of parent_path to point it to the actual root.
Memory consumption is something that I can't helpfully speculate about. It will depend on the particulars of your xml library and data. It's a problem if you're running out of memory, in which case profile it to check the usage is where you expect. If you're allowed to mutate the input array, and are worried about memory, you'd get some small savings and possibly a small efficiency boost for doing the sort in place rather than using sorted. However, you'd need a streaming XML writer (and some moderately knotty logic about when things can be committed) to make a significant difference. As far as I can see from the docs, xml.etree.ElementTree does not have inbuilt support for such streaming.


text:
A golden ring of radius 0.5cm and cross section area $2mm^2$ is shifted to a wooden of disk of radius 8cm.If young modulus is $3\times 10^{10}N/m^2$,what is force along its length?

$\begin{array}{1 1}(A)\;24KN\\(B)\;36KN\\(C)\;18KN\\(D)\;20KN\end{array}$

$\large\frac{\Delta l}{l}=\frac{2\pi r_2-2\pi r_1}{2\pi r_1}$
$\Rightarrow \large\frac{.8-.5}{.5}$
$\Rightarrow \large\frac{3}{5}$
As stress=Y$\times$ strain
$F=3\times 10^{10}\times \large\frac{3}{5}$$\times 2\times 10^{-6} \;\;\;=\large\frac{18}{5}$$\times 10^4$
$\;\;\;=36KN$
Hence (B) is the correct answer.


text:
# The rationals form a field

The set $$\mathbb{Q}$$ of rational numbers is a field.

# Proof

$$\mathbb{Q}$$ is a (commutative) ring with additive identity $$\frac{0}{1}$$ (which we will write as $$0$$ for short) and multiplicative identity $$\frac{1}{1}$$ (which we will write as $$1$$ for short): we check the axioms individually.

• $$+$$ is commutative: $$\frac{a}{b} + \frac{c}{d} = \frac{ad+bc}{bd}$$, which by commutativity of addition and multiplication in $$\mathbb{Z}$$ is $$\frac{cb+da}{db} = \frac{c}{d} + \frac{a}{b}$$

• $$0$$ is an identity for $$+$$: have $$\frac{a}{b}+0 = \frac{a}{b} + \frac{0}{1} = \frac{a \times 1 + 0 \times b}{b \times 1}$$, which is $$\frac{a}{b}$$ because $$1$$ is a multiplicative identity in $$\mathbb{Z}$$ and $$0 \times n = 0$$ for every integer $$n$$.

• Every rational has an additive inverse: $$\frac{a}{b}$$ has additive inverse $$\frac{-a}{b}$$.

• $$+$$ is associative:

$$\left(\frac{a_1}{b_1}+\frac{a_2}{b_2}\right)+\frac{a_3}{b_3} = \frac{a_1 b_2 + b_1 a_2}{b_1 b_2} + \frac{a_3}{b_3} = \frac{a_1 b_2 b_3 + b_1 a_2 b_3 + a_3 b_1 b_2}{b_1 b_2 b_3}$$
which we can easily check is equal to $$\frac{a_1}{b_1}+\left(\frac{a_2}{b_2}+\frac{a_3}{b_3}\right)$$. actually do this

• $$\times$$ is associative, trivially:

$$\left(\frac{a_1}{b_1} \frac{a_2}{b_2}\right) \frac{a_3}{b_3} = \frac{a_1 a_2}{b_1 b_2} \frac{a_3}{b_3} = \frac{a_1 a_2 a_3}{b_1 b_2 b_3} = \frac{a_1}{b_1} \left(\frac{a_2 a_3}{b_2 b_3}\right) = \frac{a_1}{b_1} \left(\frac{a_2}{b_2} \frac{a_3}{b_3}\right)$$

• $$\times$$ is commutative, again trivially:

$$\frac{a}{b} \frac{c}{d} = \frac{ac}{bd} = \frac{ca}{db} = \frac{c}{d} \frac{a}{b}$$

• $$1$$ is an identity for $$\times$$:

$$\frac{a}{b} \times 1 = \frac{a}{b} \times \frac{1}{1} = \frac{a \times 1}{b \times 1} = \frac{a}{b}$$
by the fact that $$1$$ is an identity for $$\times$$ in $$\mathbb{Z}$$.

• $$+$$ distributes over $$\times$$:

$$\frac{a}{b} \left(\frac{x_1}{y_1}+\frac{x_2}{y_2}\right) = \frac{a}{b} \frac{x_1 y_2 + x_2 y_1}{y_1 y_2} = \frac{a \left(x_1 y_2 + x_2 y_1\right)}{b y_1 y_2}$$
while
$$\frac{a}{b} \frac{x_1}{y_1} + \frac{a}{b} \frac{x_2}{y_2} = \frac{a x_1}{b y_1} + \frac{a x_2}{b y_2} = \frac{a x_1 b y_2 + b y_1 a x_2}{b^2 y_1 y_2} = \frac{a x_1 y_2 + a y_1 x_2}{b y_1 y_2}$$
so we are done by distributivity of $$+$$ over $$\times$$ in $$\mathbb{Z}$$.

So far we have shown that $$\mathbb{Q}$$ is a ring; to show that it is a field, we need all nonzero fractions to have inverses under multiplication. But if $$\frac{a}{b}$$ is not $$0$$ (equivalently, $$a \not = 0$$), then $$\frac{a}{b}$$ has inverse $$\frac{b}{a}$$, which does indeed exist since $$a \not = 0$$.

This completes the proof.

Parents: