text:
# Lale: Library for Semi-Automated Data Science¶

Martin Hirzel, Kiran Kate, Avi Shinnar, Guillaume Baudart, and Pari Ram

5 November 2019

Examples, documentation, code: https://github.com/ibm/lale

This notebook is the basis for a talk about Lale. Lale is an open-source Python library for semi-automated data science. Lale is compatible with scikit-learn, adding a simple interface to existing machine-learning automation tools. Lale lets you search over possible pipelines in just a few lines of code while remaining in control of your work.

## Value Proposition¶

When writing machine-learning pipelines, you have a lot of decisions to make, such as picking transformers, estimators, and hyperparameters. Since some of these decisions are tricky, you will likely find yourself searching over many possible pipelines. Machine-learning automation tools help with this search. Unfortunately, each of these tools has its own API, and the search spaces are not necessarily consistent nor even correct. We have discovered that types (such as enum, float, or dictionary) can both check the correctness of, and help automatically search over, hyperparameters and pipeline configurations.

To address this issue, we have open-sourced Lale, an open-source Python library for semi-automated data science. Lale is compatible with scikit-learn, adding a simple interface to existing machine-learning automation tools. Lale is designed to augment, but not replace, the data scientist.

The target user of Lale is the working data scientist. The scope of Lale includes machine learning (both deep learning and non-DL) and data preparation. The value of Lale encompasses:

## Categorical + Continuous Dataset¶

To demonstrate automated machine learning (AutoML), we first need a dataset. We will use a tabular dataset that has two kinds of features: categorical features (columns that can contain one of a small set of strings) and continuous features (numerical columns). In particular, we use the credit-g dataset from OpenML. After fetching the data, we display a few rows to get a better understanding of its labels and features.

In [1]:
!pip install 'liac-arff>=2.4.0'
Requirement already satisfied: liac-arff>=2.4.0 in /usr/local/lib/python3.7/site-packages (2.4.0)
WARNING: You are using pip version 19.3.1; however, version 21.0.1 is available.
In [2]:
import lale.datasets.openml
import pandas as pd
(train_X, train_y), (test_X, test_y) = lale.datasets.openml.fetch(
'credit-g', 'classification', preprocess=False)
# print last five rows of labels in train_y and features in train_X
pd.options.display.max_columns = None
pd.concat([train_y.tail(5), train_X.tail(5)], axis=1)
Out[2]:
class checking_status duration credit_history purpose credit_amount savings_status employment installment_commitment personal_status other_parties residence_since property_magnitude age other_payment_plans housing existing_credits job num_dependents own_telephone foreign_worker
835 bad <0 12.0 no credits/all paid new car 1082.0 <100 1<=X<4 4.0 male single none 4.0 car 48.0 bank own 2.0 skilled 1.0 none yes
192 bad 0<=X<200 27.0 existing paid business 3915.0 <100 1<=X<4 4.0 male single none 2.0 car 36.0 none own 1.0 skilled 2.0 yes yes
629 good no checking 9.0 existing paid education 3832.0 no known savings >=7 1.0 male single none 4.0 real estate 64.0 none own 1.0 unskilled resident 1.0 none yes
559 bad 0<=X<200 18.0 critical/other existing credit furniture/equipment 1928.0 <100 <1 2.0 male single none 2.0 real estate 31.0 none own 2.0 unskilled resident 1.0 none yes
684 good 0<=X<200 36.0 delayed previously business 9857.0 100<=X<500 4<=X<7 1.0 male single none 3.0 life insurance 31.0 none own 2.0 unskilled resident 2.0 yes yes

The labels y are either 0 or 1, which means that this is a binary classification task. The remaining 20 columns are features. Some of the features are categorical, such as checking_status or credit_history. Other features are continuous, such as duration or credit_amount.

## Manual Pipeline¶

Lale is designed to support both manual and automated data science, with a consistent API that works for both of these cases as well as for the spectrum of semi-automated cases in between. A machine-learning pipeline is a computational graph, where each node is an operator (which transforms data or makes predictions) and each edge is an intermediary dataset (outputs from previous operators are piped as inputs to the next operators). We first import some operators from scikit-learn and ask Lale to wrap them for us.

In [3]:
from sklearn.preprocessing import Normalizer as Norm
from sklearn.preprocessing import OneHotEncoder as OneHot
from sklearn.linear_model import LogisticRegression as LR
import lale
lale.wrap_imported_operators()

The next cell imports a couple of utility operators from Lale's standard library.

In [4]:
from lale.lib.lale import Project
from lale.lib.lale import ConcatFeatures as Concat
/usr/local/lib/python3.7/site-packages/lightgbm/__init__.py:46: UserWarning: Starting from version 2.2.1, the library file in distribution wheels for macOS is built by the Apple Clang (Xcode_8.3.3) compiler.
This means that in case of installing LightGBM from PyPI via the pip install lightgbm command, you don't need to install the gcc compiler anymore.
Instead of that, you need to install the OpenMP library, which is required for running LightGBM on the system with the Apple Clang compiler.
You can install the OpenMP library by the following command: brew install libomp.
"You can install the OpenMP library by the following command: brew install libomp.", UserWarning)
/usr/local/lib/python3.7/site-packages/pyparsing.py:3190: FutureWarning: Possible set intersection at position 3
self.re = re.compile(self.reString)

Now that we have all the operators we need, we arrange them into a pipeline. The Project operator works like the corresponding relational algebra primitive, picking a subset of the columns of the dataset. An sub-pipeline of the form op1 >> op2 pipes the output from op1 into op2. An sub-pipeline of the form op1 & op2 causes both op1 and op2 to execute on the same data. The overall pipeline preprocesses numbers with a normalizer and strings with a one-hot encoder, then concatenates the corresponding columns and pipes the result into an LR classifier.

In [5]:
manual_trainable = (
(  Project(columns={'type': 'number'}) >> Norm()
& Project(columns={'type': 'string'}) >> OneHot())
>> Concat
>> LR(LR.enum.penalty.l2, C=0.001))
manual_trainable.visualize()

In this example, we have manually chosen all operators and configured their hyperparameters. For the LR operator, we configured the penalty hyperparameter with l2 and the regularization constant C with 0.001. Depending on what tool you use to you view this notebook, you can explore these hyperparameters by hovering over the visualization above and observing the tooltips that pop up. Furthermore, each node in the visualization is a hyperlink that takes you to the documentation of the corresponding operator. Calling fit on the trainable pipeline returns a trained pipeline, and calling predict on the trained pipeline returns predictions. We can use off-the-shelf scikit-learn metrics to evaluate the result. In this case, the accuracy is poor.

In [6]:
import sklearn.metrics
manual_trained = manual_trainable.fit(train_X, train_y)
manual_y = manual_trained.predict(test_X)
print(f'accuracy {sklearn.metrics.accuracy_score(test_y, manual_y):.1%}')
accuracy 70.9%

## Pipeline Combinators¶

As the previous example demonstrates, Lale provides combinators >> and & for arranging operators into a pipeline. These combinators are actually syntactic sugar for functions make_pipeline and make_union, which were inspired by the corresponding functions in scikit-learn. To support AutoML, Lale supports a third combinator | that specifies an algorithmic choice. Scikit-learn does not have a corresponding function. Various AutoML tools support algorithmic choice in some form or other, using their own tool-specific syntax. Lale's | combinator is syntactic sugar for Lale's make_choice function.

Lale feature Name Description Scikit-learn feature
>> or make_pipeline pipe feed to next make_pipeline
& or make_union and run both make_union or ColumnTransformer
| or make_choice or choose one N/A (specific to given AutoML tool)

## Automated Pipeline¶

Next, we will use automation to search for a better pipeline for the credit-g dataset. Specifically, we will perform combined hyperparameter optimization and algorithm selection (CASH). To do this, we first import a couple more operators to serve as choices in the algorithm selection.

In [7]:
from xgboost import XGBClassifier as XGBoost
from sklearn.svm import LinearSVC
lale.wrap_imported_operators()

Next, we arrange the pipeline that specifies the search space for CASH. As promised, the look-and-feel for the automated case resembles that for the manual case from earlier. The main difference is the use of the | combinator to provide algorithmic choice (LR | XGBoost | LinearSVC). However, there is another difference: several of the operators are not configured with hyperparameters. For instance, this pipeline writes LR instead of LR(LR.penalty.l1, C=0.001). Omitting the arguments means that as the data scientist, we do not bind the hyperparameters by hand in this pipeline. Instead, we leave these bindings free for CASH to search over. Similarly, the code contains Norm instead of Norm() and OneHot instead of OneHot(). This illustrates that automated hyperparameter tuning can be used not only on the final classifier but also inside of nested preprocessing sub-pipelines.

In [8]:
auto_planned = (
(  Project(columns={'type': 'number'}) >> Norm
& Project(columns={'type': 'string'}) >> OneHot)
>> Concat
>> (LR | XGBoost | LinearSVC))
auto_planned.visualize()

The colors in the visualization indicate the particular mix of manual vs. automated bindings and will be explained in more detail below. For now, we look at how to actually invoke an AutoML tool from Lale. Lale provides bindings for multiple such tools; here we use the popular hyperopt open-source library. Specifically, Hyperopt takes the auto_planned pipeline from above as an argument, along with optional specifications for the number of cross-validation folds and trials to run. Calling fit yields a trained pipeline. The code that uses that trained pipeline for prediction and evaluation is the same as in the manual use case we saw before.

In [9]:
from lale.lib.lale.hyperopt import Hyperopt
auto_optimizer = Hyperopt(estimator=auto_planned, cv=3, max_evals=10)
auto_trained = auto_optimizer.fit(train_X, train_y)
auto_y = auto_trained.predict(test_X)
print(f'accuracy {sklearn.metrics.accuracy_score(test_y, auto_y):.1%}')
[15:01:49] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:01:51] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:01:52] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:06] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:08] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:11] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:12] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:14] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:16] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
100%|██████████| 10/10 [01:01<00:00,  6.12s/trial, best loss: -0.7537102284860132]
accuracy 71.2%

The predictive performance is better, and in fact, it is in the same ball-park as the state-of-the-art performance reported for this dataset in OpenML. Since we left various choices to automation, at this point, we probably want to inspect what the automation chose for us. One way to do that is by visualizing the pipeline as before. This reveals that hyperopt chose XGBoost instead of the scikit-learn classifiers. Also, by hovering over the nodes in the visualization, we can explore how hyperopt configured the hyperparameters. Note that all nodes are visualized in white to indicate they are fully trained.

In [10]:
auto_trained.get_pipeline().visualize()

Another way to inspect the results of automation is to pretty-print the trained pipeline back as Python source code. This lets us look at the output of automation in the same syntax we used to specify the input to automation in the first place.

In [11]:
auto_trained.get_pipeline().pretty_print(ipython_display=True, show_imports=False)
project_0 = Project(columns={"type": "number"})
norm = Norm(norm="max")
project_1 = Project(columns={"type": "string"})
lr = LR(
fit_intercept=False,
intercept_scaling=0.38106476479749274,
max_iter=422,
multi_class="multinomial",
solver="saga",
tol=0.004851580781685862,
)
pipeline = ((project_0 >> norm) & (project_1 >> OneHot())) >> Concat() >> lr

## Bindings as Lifecycle¶

The previous example already alluded to the notion of bindings and that those bindings are reflected in node visualization colors. Bindings here relate to the mathematical notion of free or bound variables in formulas. Bindings as lifecycle is one of the fundamental novel concepts around which we designed Lale. Lale distinguishes two kinds of operators: individual operators are data-science primitives such as data preprocessors or predictive models, whereas pipelines are compositions of operators. Each operator, whether individual or composite, can be in one of four lifecycle states: meta-model, planned, trainable, and trained. The lifecycle state of an operator is defined by which bindings it has. Specifically:

• Meta-model: Individual operators at the meta-model state have schemas and priors for their hyperparameters. Pipelines at the meta-model state have steps (which are other operators) and a grammar.

• Planned: To get a planned pipeline, we need to arrange it by binding a specific graph topology. This topology is consistent with but more concrete than the steps and grammar from the meta-model state.

• Trainable: To get a trainable individual operator, we need to initialize the concrete bindings for its hyperparameters. Scikit-learn does this using __init__; Lale emulates that syntax using __call__. To get a trainable pipeline, we need to bind concrete operator choices given by the | combinator.

• Trained: To get a trained individual operator, we need to fit it to the data, thus binding its learnable coefficients. A trained pipeline is like a trainable pipeline where all steps are trained. More generally, the lifecycle state of a pipeline is limited by the least upper bound of the states of its steps.

The interested reader can explore Lale's concept of bindings as lifecycle in more detail in our paper.

The above diagram is actually a Venn diagram. Each lifecycle state is a subset of its predecessor. For instance, a trained operator has all the bindings of a planned operator and in addition also binds learned coefficients. Since a trained operator has all bindings required for being a trainable operator, it can be used where a trainable operator is expected. Thus, the set of trained operators is a subset of the set of trainable operators. This relationship between the lifecycle states is essential for offering a flexible semi-automated data science experience.

## Semi-Automated Data Science¶

Why would data scientists prefer semi-automation over full automation? The main reason is to excert control over what kind of pipelines they find. Below are several scenarios motivating semi-automated data science.

Manual control over automation Examples
Restrict available operator choices Interpretable, or based on licenses, or based on GPU requirements, ...
Tweak graph topology Custom preprocessing, or multi-modal data, or fairness mitigation, ...
Tweak hyperparametrer schemas Adjust range for continuous, or restrict choices for categorical, ...
Expand available operator choices Wrap existing library, or write your own operators, ...

In fact, we envision that this will often happen via trial-and-error, where you as the data scientist specify a first planned pipeline, let AutoML do its search, then inspect the results, edit your code, and try again. You already saw some of the Lale features for inspecting the results of automation above. Lale further supports this workflow by letting you specify partial bindings of some hyperparameters while allowing other to remain free, and by letting you freeze an operator at the trainable or trained state.

## 20 Newsgroups Dataset¶

For the next examples in this notebook, we will need a different dataset. The following code fetches the 20 newsgroups data, using a function that comes with scikit-learn. Then, it prints a representative sample of the labels and features.

In [12]:
import sklearn.datasets
news = sklearn.datasets.fetch_20newsgroups()
news_X, news_y = news.data, news.target
pd.DataFrame({'y': [news.target_names[i] for i in news_y], 'X': news_X}).head()
Out[12]:
y X
0 rec.autos From: [email protected] (where's my thing)\nS...
1 comp.sys.mac.hardware From: [email protected] (Guy Kuo)...
2 comp.sys.mac.hardware From: [email protected] (Thomas E Will...
3 comp.graphics From: [email protected] (Joe Green)\nSubject: Re: W...
4 sci.space From: [email protected] (Jonathan McDow...

The labels y are strings taken from a set of 20 different newsgroup names. That means that this is a 20-way classification dataset. The features X consist of a single string with a message posted to one of the newsgroups. The task is to use the message to predict which group it was posted on.

## Constraints in Scikit-learn¶

As implied by the title of this notebook and the corresponding talk, Lale is built around the concept of types. In programming languages, a type specifies a set of valid values for a variable (e.g., for a hyperparameter). While types are in the foreground of some programming languages, Python keeps types more in the background. Python is dynamically typed, and libraries such as scikit-learn rely more on exceptions than on types for their error checking. To demonstrate this, we first import some scikit-learn modules.

In [13]:
import sklearn.feature_extraction.text
import sklearn.pipeline
import sklearn.linear_model

Next, we create a scikit-learn pipeline, using fully qualified names to ensure and clarify that we are not using any Lale facilities. The pipeline consists of just two operators, a TFIDF transformer for extracting features from the message text followed by a logistic regression for classifying the message by newsgroup.

In [14]:
sklearn_misconfigured = sklearn.pipeline.make_pipeline(
sklearn.feature_extraction.text.TfidfVectorizer(),
sklearn.linear_model.LogisticRegression(solver='sag', penalty='l1'))
print('no error detected yet')
no error detected yet

The above code actually contains a mistake in the hyperparameters for LogisticRegression. Unfortunately, scikit-learn does not detect this mistake when the hyperparameters are configured in the code. Instead, it delays its error checking until we attempt to fit the pipeline to data.

In [15]:
%%time
import sys
try:
sklearn_misconfigured.fit(news_X, news_y)
except ValueError as e:
print(e, file=sys.stderr)
CPU times: user 3.45 s, sys: 92.9 ms, total: 3.54 s
Wall time: 3.59 s
Solver sag supports only 'l2' or 'none' penalties, got l1 penalty.

The above output demonstrates that scikit-learn takes a few seconds to report the error. This is because it first trains the TFIDF transformer on the data, and only when that is done, it attempts to train the LogisticRegression, triggering the exception. While a few seconds are no big deal, the 20 newsgroup dataset is small, and for larger datasets, the delay is larger. Furthermore, while in this case, the erroneous code is not far away from the code that triggers the error report, for larger code bases, that distance is also further. The error message is nice and clear about the erroneous hyperpareters, but does not mention which operator was misconfigured (here, LogisticRegression).

## Constraints in AutoML¶

The above example illustrates an exception caused by manually misconfigured hyperparameters. Tools for automated machine learning usually run many trials with different hyperparameter configurations. Just as in the manual case, in general, some of these automated trials may raise exceptions.

Solution 1: Unconstrained search space

• {solver: [linear, sag, lbfgs], penalty: [l1, l2]}
• catch exception (after some time)

This first solution has the benefit of being simple, but the drawback that it may waste computational resources on training upstream operators before detecting erroneous downstream operators. It also leads to a larger-than-necessary search space. While we can use a high loss to steer the AutoML tool away from poorly performing points in the search space, in our experiments, we have encountered cases where this adversely affects convergence.

Solution 2: Constrained search space

• {solver: [linear, sag, lbfgs], penalty: [l1, l2]} and (if solver: [sag, lbfgs] then penalty: [l2])}
• no exceptions (no time wasted)

The second solution is the one we advocate in Lale. The Lale library contains hyperparameter schemas for a large number of operators. These schemas are type specifications that encode not just the valid values for each hyperparameter in isolation, but also constraints cutting across multiple hyperparameters such as solver and penalty in the example.

## Constraints in Lale¶

To demonstrate how Lale handles constraints, we first import Lale's wrapper for TFIDF. We need not import LogisticRegression here since we already imported it earlier in the notebook.

In [16]:
from lale.lib.sklearn import TfidfVectorizer as Tfidf

Lale uses JSON Schema to express types. JSON Schema is a widely supported and adopted standard proposal for specifying the valid structure of JSON documents. By using JSON Schema for hyperparameters, we avoided the need to implement any custom schema validation code. Instead, we simply put the concrete hyperparameters into a JSON document and then use an off-the-shelf validator to check that against the schema.

In [17]:
%%time
import jsonschema
try:
lale_misconfigured = Tfidf >> LR(LR.enum.solver.sag, LR.enum.penalty.l1)
except jsonschema.ValidationError as e:
print(e.message, file=sys.stderr)
CPU times: user 23 ms, sys: 2.6 ms, total: 25.6 ms
Wall time: 25.5 ms
Invalid configuration for LR(solver='sag', penalty='l1') due to constraint the newton-cg, sag, and lbfgs solvers support only l2 or no penalties.
Schema of constraint 1: {
"description": "The newton-cg, sag, and lbfgs solvers support only l2 or no penalties.",
"anyOf": [
{
"type": "object",
"properties": {
"solver": {"not": {"enum": ["newton-cg", "sag", "lbfgs"]}}
},
},
{
"type": "object",
"properties": {"penalty": {"enum": ["l2", "none"]}},
},
],
}
Value: {'solver': 'sag', 'penalty': 'l1', 'dual': False, 'C': 1.0, 'tol': 0.0001, 'fit_intercept': True, 'intercept_scaling': 1.0, 'class_weight': None, 'random_state': None, 'max_iter': 100, 'multi_class': 'auto', 'verbose': 0, 'warm_start': False, 'n_jobs': None, 'l1_ratio': None}

The output demonstrates that the error check happened more than 100 times faster than in the previous example. This is because the code did not need to train TFIDF. The error check also happened closer to the root cause of the error, so the code dinstance is smaller. Furthermore, the error message indicates not just the wrong hyperparameters but also which operator was misconfigured (here, LR).

## Schemas as Documentation¶

The above code example demonstrates that Lale uses JSON Schema for error checking. But we can do better than just using schemas for one purpose. Since all Lale operators carry hyperparameter schemas, we can also use those same schemas for interactive documentation. The following code illustrates that by inspecting the schema of a continuous hyperparameter.

In [18]:
XGBoost.hyperparam_schema('n_estimators')
Out[18]:
{'description': 'Number of trees to fit.',
'type': 'integer',
'default': 100,
'minimumForOptimizer': 50,
'maximumForOptimizer': 1000}

Of course, the same interactive documentation approach also works for categorical hyperparameters.

In [19]:
XGBoost.hyperparam_schema('booster')
Out[19]:
{'description': 'Specify which booster to use.',
'enum': ['gbtree', 'gblinear', 'dart', None],
'default': None}

Using the same schemas for two purposes (validation and documentation) makes them stronger. As a user, you can be confident that the documentation you read is in sync with the error checks in the code. Furthermore, while schemas check hyperparameters, the reverse is also true: when tests reveal mistakes in the schemas themselves, they can be corrected in a single location, thus improving both validation and documentation. However, we can do even better than using the same schemas for two purposes: we can also use them for a third purpose (automated search).

## Types as Search Spaces¶

The following picture illustrates how Lale uses types for AutoML. The data scientist is shown on the left. They interact with a pipeline at either the planned or trainable lifecycle state, or at some mixed state as discussed above under semi-automated AutoML. Every individual operator in Lale carries types in the form of a schema. Users rarely have to write these schemas by hand, since the Lale library already includes schemas for, at last count, 133 individual operators. Given a planned pipeline, when the data scientist kicks off a search, Lale automatically generates a search space appropriate for the given combined algorithm selection and hyperparameter optimization tool, e.g., hyperopt. For each trial, the CASH tool then acquires a point in the search space, e.g., using an acquisition function in Bayesian optimization. Then, Lale automatically decodes that search point into a trainable Lale pipeline. By construction, this trainable pipeline validates against the schemas. We can train and score the pipeline, usually with cross-validation, to return a loss value to the CASH tool. If this is the best pipeline found so far, the CASH tool will remember it as the incumbent. When the search terminates, the CASH tool returns the final incumbent, minimizing the loss. Types as search spaces is another fundamental novel concept around which we designed Lale, described in more detail in our paper.

## Customizing Schemas¶

While you can use Lale operators with their schemas as-is, you can also customize the schemas to excert more control over the automation. For instance, you might want to reduce the number of trees in an XGBoost forest to reduce memory consumption or to improve explainability. Or you might want to hand-pick one of the boosters to reduce the search space and thus hopefully speed up the search.

In [20]:
import lale.schemas as schemas
Grove = XGBoost.customize_schema(
n_estimators=schemas.Int(minimum=2, maximum=6),
booster=schemas.Enum(['gbtree'], default='gbtree'))

As this example demonstrates, Lale provides a simple Python API for writing schemas, which it then converts to JSON Schema internally. The result of customization is a new copy of the operator that can be used in the same way as any other operator in Lale. In particular, it can be part of a pipeline as before.

In [21]:
grove_planned = ( Project(columns={'type': 'number'}) >> Norm
& Project(columns={'type': 'string'}) >> OneHot
) >> Concat >> Grove

Given this new planned pipeline, we use hyperopt as before to search for a good trained pipeline.

In [22]:
grove_optimizer = Hyperopt(estimator=grove_planned, cv=3, max_evals=10, verbose=True)
grove_trained = grove_optimizer.fit(train_X, train_y)
grove_y = grove_trained.predict(test_X)
print(f'accuracy {sklearn.metrics.accuracy_score(test_y, grove_y):.1%}')
[15:02:56] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:57] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:58] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:59] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:02:59] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:00] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:01] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:02] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:03] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:03] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:04] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:05] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:06] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:07] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:07] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:08] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:09] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:10] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:11] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:12] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:12] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:13] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:14] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:15] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:16] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:16] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:17] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:18] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:19] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
[15:03:20] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
100%|██████████| 10/10 [00:24<00:00,  2.49s/trial, best loss: -0.7313420884048686]
[15:03:20] WARNING: /Users/travis/build/dmlc/xgboost/src/learner.cc:1061: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.
accuracy 73.0%

The result shows that the predictive performance is not quite as good as before. As a data scientist, you can weigh that against other needs, and possibly experiment more. Of course, you can also display the result of automation, e.g., by pretty-printing it back as code.

In [23]:
grove_trained.get_pipeline().pretty_print(ipython_display=True, show_imports=False)
project_0 = Project(columns={"type": "number"})
norm = Norm(norm="l1")
project_1 = Project(columns={"type": "string"})
grove = Grove(
gamma=0.8726303533099419,
learning_rate=0.7606845221791637,
max_depth=2,
min_child_weight=11,
n_estimators=5,
reg_alpha=0.5980470775121279,
reg_lambda=0.2546844052569046,
subsample=0.8142720284737895,
)
pipeline = (
((project_0 >> norm) & (project_1 >> OneHot())) >> Concat() >> grove
)

## Scikit-learn Compatible Interoperability¶

Scikit-learn has a big following, which is well-earned: it is one of the most complete and most usable machine-learning libraries available. In developing Lale, we spent a lot of effort on maintaining scikit-learn compatibility. Earlier on, you already saw that Lale pipelines work with off-the-shelf scikit-learn functions such as metrics, and with other libraries such as XGBoost. To demonstrate interoperability with additional libraries, we ran experiments with pipelines from various different data modalities. For the movies review text dataset, the best pipeline used a PyTorch implementation of the BERT embedding. For the car tabular dataset, the best pipeline used a Java implementation of the J48 decision tree. For the CIFAR-10 images dataset, the best pipeline used a PyTorch implementation of a ResNet50 neural network. And finally, for the epilepsy time-series dataset, the best pipeline used a window transformer and voting operator pair that was written for the task and then made available in the Lale library. That demonstrates that Lale can use operators from other libraries beyond scikit-learn and even from other languages than Python. It also demonstrates that for some tasks, this interoperability is necessary for better predictive performance.

## Summary¶

Please check out the Lale github repository for examples, documentation, and code: https://github.com/ibm/lale

This notebook started by listing three values that Lale provides: automation, interoperability, and usability. Then, this notebook introduced three fundamental concepts around which Lale is designed: bindings as lifecycle, types as search spaces, and scikit-learn compatible interoperability. As the following picture illustrates, the three concepts sit at the intersection of the three values.


text:
# Resolve Ax=b using preconditionner for the conjugate gradient

I would like to resolve the equation $$\left[ \begin{array}{ c c } A & B\\ B' & 0 \end{array} \right] \left[ \begin{array}{ c} x \\ y \end{array} \right] = \left[ \begin{array}{ c} b \\ 0 \end{array} \right]$$ Where $A$ is Symmetric Positive Definite.

I tried with a conjugate gradient method and it works for a very small number of data. It diverges very easily if I try to increase the size of my matrix. So I was thinking about using a preconditionner, for example the simplest Jacobi preconditionner $C(A) = diag(A)$. But $C$ contains $0$ on the diagonal therefore it is singular. Is anyone has an idea how to resolve this equation ?

-
Conjugate gradient (preconditioned or not) is not likely to converge at the coefficient matrix is indefinite, even if $A$ is spd. Try MINRES if the coefficient matrix is not too badly conditioned. –  Daryl Aug 7 '12 at 8:01
Why don't you split equations for $x$ and for $y$? –  Konrad Sakowski Aug 7 '12 at 8:06
@KonradSakowski you mean that I should resolve first $B'x=0$ and after $Ax+By=b$ ? –  Seltymar Aug 7 '12 at 8:15
@Seltymar Yes. But I realized that probably the reason is your B is not square. –  Konrad Sakowski Aug 7 '12 at 8:26
@KonradSakowski Yes, B is not square so I will have too many unknown for the number of equations. –  Seltymar Aug 7 '12 at 8:38

format: plain


text:
# For every partition $\pi$ of a fixed integer $n$, $\sum{F(\pi)}=\sum{G(\pi)}$

I need to prove the following question.

For every partition $\pi$ of a fixed integer $n$, define $F(\pi)$=number of occurrences of 1 as a summand, and $G(\pi)$=no. of distinct summands in the partition. Prove that $\sum{F(\pi)}=\sum{G(\pi)}$.

I was trying to solve this taking an example of 5 with Ferrers graph but not getting how to prove it.

• Hint: dual partition. (Draw Young diagrams to see the bijection.) – Sammy Black Apr 8 '13 at 19:34
• Can you please provide some more hint. – user71613 Apr 9 '13 at 5:21
• @SammyBlack: Could you elaborate on the hint? It is certainly not true that $F(\pi)$ equals $G(\pi^*)$. – Shahab Apr 9 '13 at 16:26
• @Shahab, you're certainly correct. I misread one of the functions. I'm not so sure how to proceed now. Perhaps look at the generating function in two different ways and equate coefficients? – Sammy Black Apr 9 '13 at 18:52
• It can be shown that $\sum F(\pi)=p(0)+\cdots p(n-1)$. I am having trouble proving that the RHS equals $\sum G(\pi)$. – Shahab Apr 10 '13 at 5:54

The following proof uses ordinary generating functions (bivariate). The partition generating function where ones are marked is $$Q_1(z, u) = \frac{1}{1-uz} \prod_{k\ge 2} \frac{1}{1-z^k}.$$ If we want to sum the occurrences of one as a summand for all partitions of $n$ we need to compute $$\left.\frac{d}{du} Q_1(z, u) \right|_{u=1} = \left. -1\times \frac{1}{(1-uz)^2} \times -z \times \prod_{k\ge 2} \frac{1}{1-z^k} \right|_{u=1} = \frac{z}{1-z} \prod_{k\ge 1} \frac{1}{1-z^k}.$$ On the other hand, marking distinct summands looks like this: $$Q_2(z,u) = \prod_{k\ge 1}\left(1 + u z^k + u z^{2k} + u z^{3k} + \cdots\right).$$ Once more differentiating and setting $u=1$ gives this time $$\left.\frac{d}{du} Q_2(z, u) \right|_{u=1} = \left.\prod_{k\ge 1}\left(1 + u z^k + u z^{2k} + u z^{3k} + \cdots\right) \times \sum_{k\ge 1} \frac{z^k + z^{2k} + z^{3k} + \cdots} {1 + u z^k + u z^{2k} + u z^{3k} + \cdots}\right|_{u=1} \\= \prod_{k\ge 1}\frac{1}{1-z^k} \sum_{k\ge 1} \frac{z^k/(1-z^k)}{1/(1-z^k)} = \prod_{k\ge 1}\frac{1}{1-z^k} \sum_{k\ge 1} z^k = \frac{z}{1-z} \prod_{k\ge 1}\frac{1}{1-z^k}.$$ The two generating functions are exactly the same, QED.

There are similar computations at this MSE link I and this MSE link II.

A counting argument attributed to Richard Stanley appears here.

Briefly, it goes like this:

$$\displaystyle\sum_{\pi\,\vdash n} {F(\pi)}=\displaystyle\sum_{i=1}^n p(n-i)\textrm,$$

because $p(n-i)$ is the number of partitions of $n$ with at least $i$ ones. The sum counts the number of partitions with $k$ ones $k$ times: once in $p(n-1)$, once in $p(n-2)$, and so on up to one final time in $p(n-k)$.

Then $\displaystyle\sum{G(\pi)}$ is shown to be the same thing by a clever observation.

Let $True(P)$ have value $1$ if $P$ is true and $0$ if $P$ is false, and let $H(i)$ be the number of partitions of $n$ that contain (at least one) $i$. Then

$$\sum_{\pi\,\vdash n}{G(\pi)}=\sum_{\pi\,\vdash n}\sum_{i=1}^nTrue\left(\pi \textrm{ contains an }i \right) = \sum_{i=1}^n\sum_{\pi\,\vdash n}True\left(\pi \textrm{ contains an }i \right) = \sum_{i=1}^n H(i)\textrm.$$

The observation that $H(i)=p(n-i)$ completes the proof.


text:
## University Calculus: Early Transcendentals (3rd Edition)

Published by Pearson

# Chapter 1 - Section 1.1 - Functions and Their Graphs - Exercises - Page 12: 12

#### Answer

$P(m)=(\frac{1}{m^2},\frac{1}{m})$

#### Work Step by Step

$\mathrm{See\: the\:figure\:below}$. Let's say $\ P(a,b)\$ is any point on the graph of $\ f(x)=\sqrt{x}\$. If we input $\ x\$ into the function $\ f(x)=\sqrt{x}\$, we get the corresponding value of $\ y\$ to be $\ \sqrt{x}\$. And, since $\ P\$ is on the graph of the function $\ \sqrt{x}\$, we have a relation for its coordinates. $P=(x,\sqrt{x})$ The slope of the line joining the point $\ P\$ and the origin is $\ m=\frac{\sqrt{x}}{x}.$ After rearranging the equation, we have: $\ m=\frac{\sqrt{x}}{x}$ $\Rightarrow\ m=\frac{1}{\sqrt{x}}$ $\Rightarrow\ \sqrt{x}=\frac{1}{m}$ $\Rightarrow\ x=\frac{1}{m^2}$ So, the coordinates of $\ P\$ as a function of the slope $\ m\$ can be written as: $P(m)=(\frac{1}{m^2},\frac{1}{m})$ where $\ m\$ represents the slope of the line.

After you claim an answer you’ll have 24 hours to send in a draft. An editor will review the submission and either publish your submission or provide feedback.


text:
# Identify the structure of compound with C₉H₁₀ formula using IR and NMR data [closed]

Use IR and NMR data to define the structure of the molecule, the formula of the given structure $$\ce{C9H10}.$$

On what basis do we conclude that its structure is that (the answer) only? On what points can we identify the structure of this molecule?

I have found out that since its frequency is just less than $$\pu{3000 cm-1}$$ it is an alkane. Also, the approximate value of the chemical shift is near $$7.2,$$ hence it is aromatic.

But what I did not get was the meaning of two peaks at $$7.2$$ and the shifts at $$2.9$$ and $$2.1.$$

• There are peaks beyond $\pu{3000 cm-1}$ in FT-IR. What's that means? – Mathew Mahindaratne Jun 13 '20 at 16:09
• Good. Actually they are $\mathrm{sp^2}$-$\ce{C-H}$ stretching. You have an aromatic ring so that's it for IR. Now count how many unsaturation in your compound. You can count out 4 for aromatic ring. – Mathew Mahindaratne Jun 13 '20 at 16:14
• @Andrews: I have given the explanation for peaks in NMR for you to learn. – Mathew Mahindaratne Jun 13 '20 at 17:09

The molecular formula of the compound is $$\ce{C9H10}$$. The number of unsaturations in the compound of $$\ce{C_nH_m}$$ is $$\frac{2n+1-n}{2}$$. Thus, the number of unsaturations in $$\ce{C9H10}$$ is $$\frac{2\times 9+1-10}{2}=5$$. You already found out that compound is aromatic by FT-IR, and hence $$\ce{C6}$$ of the formula counted out with 4 unsaturaations (3 double bond and a ring for aromatic nucleus).The remaining number of unsaturarion is $$5-4=1$$, and remaining number of $$\ce{C}$$ in the formula is 3. The remaining unsaturation could be a double bond or a ring.
When look at $$\ce{^1H}$$-$$\mathrm{NMR}$$ spectrum, chemical shifts of all but aromatic peaks are less than $$\pu{3.00 ppm}$$. Therefore, it is safe to conclude that the compound is a aromatic bicyclic rather than aromatic alkene. Therefore, the compound is indane:
The protons on 1-$$\ce{C}$$ and 3-$$\ce{C}$$ are chemically and magnetically equivalent, and benzylic, hence the their relevant signal would be down-field to that of protons on 2-$$\ce{C}$$, which is usual acyclic aliphatic carbon. The benzylic protons have two vicinal protons, thus the corresponding peak is a triplet. Meanwhile, protons on 2-$$\ce{C}$$ have four vicinal protons (benzylic protons), thus the corresponding peak is a quintet (5 peaks).


text:
# 7.2: Series Solutions Near an Ordinary Point I

Many physical applications give rise to second order homogeneous linear differential equations of the form

$\label{eq:7.2.1} P_0(x)y''+P_1(x)y'+P_2(x)y=0,$

where $$P_0$$, $$P_1$$, and $$P_2$$ are polynomials. Usually the solutions of these equations can’t be expressed in terms of familiar elementary functions. Therefore we’ll consider the problem of representing solutions of Equation \ref{eq:7.2.1} with series.

We assume throughout that $$P_0$$, $$P_1$$ and $$P_2$$ have no common factors. Then we say that $$x_0$$ is an ordinary point of Equation \ref{eq:7.2.1} if $$P_0(x_0)\ne0$$, or a singular point if $$P_0(x_0)=0$$. For Legendre’s equation,

$\label{eq:7.2.2} (1-x^2)y''-2xy'+\alpha(\alpha+1)y=0,$

$$x_0=1$$ and $$x_0=-1$$ are singular points and all other points are ordinary points. For Bessel’s equation,

$x^2y''+xy'+(x^2-\nu^2)y=0, \nonumber$

$$x_0=0$$ is a singular point and all other points are ordinary points. If $$P_0$$ is a nonzero constant as in Airy’s equation,

$\label{eq:7.2.3} y''-xy=0,$

then every point is an ordinary point.

Since polynomials are continuous everywhere, $$P_1/P_0$$ and $$P_2/P_0$$ are continuous at any point $$x_0$$ that isn’t a zero of $$P_0$$. Therefore, if $$x_0$$ is an ordinary point of Equation \ref{eq:7.2.1} and $$a_0$$ and $$a_1$$ are arbitrary real numbers, then the initial value problem

$\label{eq:7.2.4} P_0(x)y''+P_1(x)y'+P_2(x)y=0, \quad y(x_0)=a_0,\quad y'(x_0)=a_1$

has a unique solution on the largest open interval that contains $$x_0$$ and does not contain any zeros of $$P_0$$. To see this, we rewrite the differential equation in Equation \ref{eq:7.2.4} as

$y''+{P_1(x)\over P_0(x)}y'+{P_2(x)\over P_0(x)}y=0 \nonumber$

and apply Theorem 5.1.1 with $$p=P_1/P_0$$ and $$q=P_2/P_0$$. In this section and the next we consider the problem of representing solutions of Equation \ref{eq:7.2.1} by power series that converge for values of $$x$$ near an ordinary point $$x_0$$.

We state the next theorem without proof.

##### Theorem 7.3.1

Suppose $$P_0$$, $$P_1$$, and $$P_2$$ are polynomials with no common factor and $$P_0$$ isn’t identically zero$$.$$ Let $$x_0$$ be a point such that $$P_0(x_0)\ne0,$$ and let $$\rho$$ be the distance from $$x_0$$ to the nearest zero of $$P_0$$ in the complex plane. $$($$If $$P_0$$ is constant, then $$\rho=\infty$$.$$)$$ Then every solution of

$\label{eq:7.2.5} P_0(x)y''+P_1(x)y'+P_2(x)y=0$

can be represented by a power series

$\label{eq:7.2.6} y=\sum_{n=0}^\infty a_n(x-x_0)^n$

that converges at least on the open interval $$(x_0-\rho,x_0+\rho)$$. $$($$ If $$P_0$$ is nonconstant$$,$$ so that $$\rho$$ is necessarily finite$$,$$ then the open interval of convergence of $$\eqref{eq:7.2.6}$$ may be larger than $$(x_0-\rho,x_0+\rho).$$ If $$P_0$$ is constant then $$\rho=\infty$$ and $$(x_0-\rho,x_0+\rho)=(-\infty,\infty)$$.

We call Equation \ref{eq:7.2.6} a power series solution in $$x-x_0$$ of Equation \ref{eq:7.2.5}. We’ll now develop a method for finding power series solutions of Equation \ref{eq:7.2.5}. For this purpose we write Equation \ref{eq:7.2.5} as $$Ly=0$$, where

$\label{eq:7.2.7} Ly=P_0y''+P_1y'+P_2y.$

Theorem 7.3.1 implies that every solution of $$Ly=0$$ on $$(x_0-\rho,x_0+\rho)$$ can be written as

$y=\sum_{n=0}^\infty a_n(x-x_0)^n. \nonumber$

Setting $$x=x_0$$ in this series and in the series

$y'=\sum_{n=1}^\infty na_n(x-x_0)^{n-1} \nonumber$

shows that $$y(x_0)=a_0$$ and $$y'(x_0)=a_1$$. Since every initial value problem Equation \ref{eq:7.2.4} has a unique solution, this means that $$a_0$$ and $$a_1$$ can be chosen arbitrarily, and $$a_2$$, $$a_3$$, …are uniquely determined by them.

To find $$a_2$$, $$a_3$$, …, we write $$P_0$$, $$P_1$$, and $$P_2$$ in powers of $$x-x_0$$, substitute

\begin{align*} y &=\sum^\infty_{n=0}a_n(x-x_0)^n, \\[4pt] y' &=\sum^\infty_{n=1}na_n(x-x_0)^{n-1}, \\[4pt] y''&=\sum^\infty_{n=2}n(n-1)a_n(x-x_0)^{n-2} \end{align*}

into Equation \ref{eq:7.2.7}, and collect the coefficients of like powers of $$x-x_0$$. This yields

$\label{eq:7.2.8} Ly=\sum^\infty_{n=0}b_n(x-x_0)^n,$

where $$\{b_0, b_1, \dots, b_n, \dots\}$$ are expressed in terms of $$\{a_0, a_1, \dots,a_n, \dots\}$$ and the coefficients of $$P_0$$, $$P_1$$, and $$P_2$$, written in powers of $$x-x_0$$. Since Equation \ref{eq:7.2.8} and (a) of Theorem 7.1.6 imply that $$Ly=0$$ if and only if $$b_n=0$$ for $$n\ge0$$, all power series solutions in $$x-x_0$$ of $$Ly=0$$ can be obtained by choosing $$a_0$$ and $$a_1$$ arbitrarily and computing $$a_2$$, $$a_3$$, …, successively so that $$b_n=0$$ for $$n\ge0$$. For simplicity, we call the power series obtained this way the power series in $$x-x_0$$ for the general solution of $$Ly=0$$, without explicitly identifying the open interval of convergence of the series.

##### Example 7.3.1

Let $$x_0$$ be an arbitrary real number. Find the power series in $$x-x_0$$ for the general solution of

$\label{eq:7.2.9} y''+ y=0.$

Solution

Here

$Ly=y''+y. \nonumber$

If

$y=\sum_{n=0}^\infty a_n(x-x_0)^n, \nonumber$

then

$y''=\sum_{n=2}^\infty n(n-1)a_n(x-x_0)^{n-2}, \nonumber$

so

$Ly=\sum_{n=2}^\infty n(n-1)a_n(x-x_0)^{n-2}+\sum_{n=0}^\infty a_n(x-x_0)^n. \nonumber$

To collect coefficients of like powers of $$x-x_0$$, we shift the summation index in the first sum. This yields

$Ly=\sum^\infty_{n=0}(n+2)(n+1)a_{n+2}(x-x_0)^n + \sum^\infty_{n=0}a_n(x-x_0)^n =\sum^\infty_{n=0}b_n(x-x_0)^n, \nonumber$

with

$b_n=(n+2)(n+1)a_{n+2}+a_n. \nonumber$

Therefore $$Ly=0$$ if and only if

$\label{eq:7.2.10} a_{n+2}={-a_n\over(n+2)(n+1)},\quad n\ge0,$

where $$a_0$$ and $$a_1$$ are arbitrary. Since the indices on the left and right sides of Equation \ref{eq:7.2.10} differ by two, we write Equation \ref{eq:7.2.10} separately for $$n$$ even $$(n=2m)$$ and $$n$$ odd $$(n=2m+1)$$. This yields

$a_{2m+2} = \dfrac{-a_{2m}}{(2m+2)(2m+1)}, \quad m \ge 0, \label{eq:7.2.11}$

and

$a_{2m+3} = {-a_{2m+1}\over(2m+3)(2m+2)},\quad m\ge0. \label{eq:7.2.12}$

Computing the coefficients of the even powers of $$x-x_0$$ from Equation \ref{eq:7.2.11} yields

\begin{aligned} a_2 &= -{a_0\over2\cdot1}\\[4pt] a_4 &= -{a_2\over4\cdot3}=-{1\over4\cdot3} \left(-{a_0\over2\cdot1}\right)= {a_0\over4\cdot3\cdot2\cdot1}, \\ a_6 &= -{a_4\over6\cdot5}=-{1\over6\cdot5} \left({a_0\over4\cdot3\cdot2\cdot1}\right) =-{a_0\over6\cdot5\cdot4\cdot3\cdot 2\cdot1},\end{aligned}\nonumber

and, in general,

$\label{eq:7.2.13} a_{2m}=(-1)^m {a_0\over(2m)!}\;,\quad m\ge0.$

Computing the coefficients of the odd powers of $$x-x_0$$ from Equation \ref{eq:7.2.12} yields

\begin{aligned} a_3 &= -{a_1\over3\cdot2}\\[4pt] a_5 &= -{a_3\over5\cdot4}=-{1\over5\cdot4} \left(-{a_1\over3\cdot2}\right)= {a_1\over5\cdot4\cdot3\cdot2}, \\ a_7 &= -{a_5\over7\cdot6}=-{1\over7\cdot6} \left({a_1\over5\cdot4\cdot3\cdot2}\right) =-{a_1\over7\cdot6\cdot5\cdot4\cdot 3\cdot2},\end{aligned}\nonumber

and, in general,

$\label{eq:7.2.14} a_{2m+1}={(-1)^ma_1\over(2m+1)!}\quad m\ge0.$

Thus, the general solution of Equation \ref{eq:7.2.9} can be written as

$y=\sum_{m=0}^\infty a_{2m}(x-x_0)^{2m}+\sum_{m=0}^\infty a_{2m+1}(x-x_0)^{2m+1}, \nonumber$

or, from Equation \ref{eq:7.2.13} and Equation \ref{eq:7.2.14}, as

$\label{eq:7.2.15} y=a_0\sum_{m=0}^\infty(-1)^m{(x-x_0)^{2m}\over(2m)!} +a_1\sum_{m=0}^\infty(-1)^m{(x-x_0)^{2m+1}\over(2m+1)!}.$

If we recall from calculus that

$\sum_{m=0}^\infty(-1)^m{(x-x_0)^{2m}\over(2m)!}=\cos(x-x_0) \quad \text{and} \quad \sum_{m=0}^\infty(-1)^m{(x-x_0)^{2m+1}\over(2m+1)!}=\sin(x-x_0), \nonumber$

then Equation \ref{eq:7.2.15} becomes

$y=a_0\cos(x-x_0)+a_1\sin(x-x_0), \nonumber$

which should look familiar.

Equations like Equations \ref{eq:7.2.10}, \ref{eq:7.2.11}, and \ref{eq:7.2.12}, which define a given coefficient in the sequence $$\{a_n\}$$ in terms of one or more coefficients with lesser indices are called recurrence relations. When we use a recurrence relation to compute terms of a sequence we are computing recursively.

In the remainder of this section we consider the problem of finding power series solutions in $$x-x_0$$ for equations of the form

$\label{eq:7.2.16} \left(1+\alpha(x-x_0)^2\right)y''+\beta(x-x_0) y'+\gamma y=0.$

Many important equations that arise in applications are of this form with $$x_0=0$$, including Legendre’s equation Equation \ref{eq:7.2.2}, Airy’s equation Equation \ref{eq:7.2.3}, Chebyshev’s equation,

$(1-x^2)y''-xy'+\alpha^2 y=0, \nonumber$

and Hermite’s equation,

$y''-2xy'+2\alpha y=0. \nonumber$

Since

$P_0(x)=1+\alpha(x-x_0)^2 \nonumber$

in Equation \ref{eq:7.2.16}, the point $$x_0$$ is an ordinary point of Equation \ref{eq:7.2.16}, and Theorem 7.3.1 implies that the solutions of Equation \ref{eq:7.2.16} can be written as power series in $$x-x_0$$ that converge on the interval $$(x_0-1/\sqrt|\alpha|,x_0+1/\sqrt|\alpha|)$$ if $$\alpha\ne0$$, or on $$(-\infty,\infty)$$ if $$\alpha=0$$. We’ll see that the coefficients in these power series can be obtained by methods similar to the one used in Example 7.3.1 .

To simplify finding the coefficients, we introduce some notation for products:

$\prod^s_{j=r}b_j=b_rb_{r+1}\cdots b_s\quad \mbox{if} \quad s\ge r. \nonumber$

Thus,

$\prod^7_{j=2}b_j=b_2b_3b_4b_5b_6b_7, \nonumber$

$\prod^4_{j=0}(2j+1)=(1)(3)(5)(7)(9)=945, \nonumber$

and

$\prod^2_{j=2}j^2=2^2=4. \nonumber$

We define

$\prod^s_{j=r}b_j=1\quad \mbox{if}\quad s < r, \nonumber$

no matter what the form of $$b_j$$.
##### Example 7.3.2

Find the power series in $$x$$ for the general solution of

$\label{eq:7.2.17} (1+2x^2)y''+6xy'+2y=0.$

Solution

Here

$Ly=(1+2x^2)y''+6xy'+2y. \nonumber$

If

$y=\sum_{n=0}^\infty a_nx^n \nonumber$

then

$y'=\sum_{n=1}^\infty na_nx^{n-1}\quad\mbox{ and }\quad y''=\sum_{n=2}^\infty n(n-1)a_nx^{n-2},\nonumber$

so

\begin{aligned} Ly &= (1+2x^2) \sum^\infty_{n=2}n(n-1)a_nx^{n-2}+ 6x \sum^\infty_{n=1}na_nx^{n-1} +2 \sum^\infty_{n=0}a_nx^n\\[4pt] &= \sum_{n=2}^\infty n(n-1)a_nx^{n-2}+\sum_{n=0}^\infty \left[2n(n-1)+6n+2\right]a_nx^n\\[4pt] &= \sum_{n=2}^\infty n(n-1)a_nx^{n-2}+2\sum_{n=0}^\infty(n+1)^2a_nx^n.\end{aligned}\nonumber

To collect coefficients of $$x^n$$, we shift the summation index in the first sum. This yields

$Ly=\sum_{n=0}^\infty(n+2)(n+1)a_{n+2}x^n+2\sum_{n=0}^\infty(n+1)^2a_nx^n =\sum_{n=0}^\infty b_nx^n, \nonumber$

with

$b_n=(n+2)(n+1)a_{n+2}+2(n+1)^2a_n,\quad n\ge0. \nonumber$

To obtain solutions of Equation \ref{eq:7.2.17}, we set $$b_n=0$$ for $$n\ge0$$. This is equivalent to the recurrence relation

$\label{eq:7.2.18} a_{n+2}=-2{n+1\over n+2}a_n,\quad n\ge0.$

Since the indices on the left and right differ by two, we write Equation \ref{eq:7.2.18} separately for $$n=2m$$ and $$n=2m+1$$, as in Example 7.3.1 . This yields

\begin{align} a_{2m+2} &= -2 {2m+1\over2m+2}a_{2m} =-{2m+1\over m+1}a_{2m},\quad m \ge0,\label{eq:7.2.19} \end{align}

and

\begin{align} a_{2m+3} &= -2{2m+2\over2m+3}a_{2m+1} =-4{m+1\over2m+3}a_{2m+1},\quad m\ge0. \label{eq:7.2.20}\end{align}

Computing the coefficients of even powers of $$x$$ from Equation \ref{eq:7.2.19} yields

\begin{aligned} a_2 &= -{1\over1}a_0,\\[4pt] a_4 &= -{3\over2}a_2=\left(-{3\over2}\right)\left(-{1\over1}\right)a_0 ={1\cdot3\over1\cdot2}a_0, \\[4pt] a_6 &= -{5\over3}a_4= -{5\over3}\left(1\cdot3\over1\cdot2\right)a_0 =-{1\cdot3\cdot5\over1\cdot2\cdot3}a_0, \\[4pt] a_8 &= -{7\over4}a_6=-{7\over4} \left(-{1\cdot3\cdot5\over1\cdot2\cdot3}\right)a_0= {1\cdot3\cdot5\cdot7\over1\cdot2\cdot3\cdot4}a_0.\\\end{aligned}\nonumber In general,

$\label{eq:7.2.21} a_{2m}=(-1)^m{\prod_{j=1}^m(2j-1)\over m!}a_0,\quad m\ge0.$

(Note that Equation \ref{eq:7.2.21} is correct for $$m=0$$ because we defined $$\prod_{j=1}^0b_j=1$$ for any $$b_j$$.)

Computing the coefficients of odd powers of $$x$$ from Equation \ref{eq:7.2.20} yields

\begin{aligned} a_3 &= -4\,{1\over3}a_1, \\[4pt] a_5 &= -4\,{2\over5}a_3=-4\,{2\over5}\left(-4{1\over3}\right)a_1 =4^2{1\cdot2\over3\cdot5}a_1, \\[4pt] a_7 &= -4\,{3\over7}a_5=-4\,{3\over7}\left( 4^2{1\cdot2\over3\cdot5}\right)a_1= -4^3{1\cdot2\cdot3\over3\cdot5\cdot7}a_1,\\[4pt] a_9 &= -4\, {4\over9}a_7=-4\, {4\over9}\left( 4^3{1\cdot2\cdot3\over3\cdot5\cdot7}\right)a_1= 4^4{1\cdot2\cdot3\cdot4\over3\cdot5\cdot7\cdot9}a_1.\end{aligned}\nonumber

In general,

$\label{eq:7.2.22} a_{2m+1}={(-1)^m4^m m!\over\prod_{j=1}^m(2j+1)}a_1,\quad m\ge0.$

From Equation \ref{eq:7.2.21} and Equation \ref{eq:7.2.22},

$y=a_0 \sum^\infty_{m=0}(-1)^m {\prod_{j=1}^m(2j-1)\over m!}x^{2m} +a_1 \sum^\infty_{m=0}(-1)^m {4^mm!\over\prod_{j=1}^m(2j+1)} x^{2m+1}. \nonumber$

is the power series in $$x$$ for the general solution of Equation \ref{eq:7.2.17}. Since $$P_0(x)=1+2x^2$$ has no real zeros, Theorem 5.1.1 implies that every solution of Equation \ref{eq:7.2.17} is defined on $$(-\infty,\infty)$$. However, since $$P_0(\pm i/\sqrt2)=0$$, Theorem 7.3.1 implies only that the power series converges in $$(-1/\sqrt2,1/\sqrt2)$$ for any choice of $$a_0$$ and $$a_1$$.

The results in Examples 7.3.1 and 7.3.2 are consequences of the following general theorem.

##### Theorem 7.3.2

The coefficients $$\{a_n\}$$ in any solution $$y=\sum_{n=0}^\infty a_n(x-x_0)^n$$ of

$\label{eq:7.2.23} \left(1+\alpha(x-x_0)^2\right)y''+\beta(x-x_0) y'+\gamma y=0$

satisfy the recurrence relation

$\label{eq:7.2.24} a_{n+2}=-{p(n) \over(n+2)(n+1)}a_n,\quad n\ge0,$

where

$\label{eq:7.2.25} p(n)=\alpha n(n-1) +\beta n+\gamma.$

Moreover$$,$$ the coefficients of the even and odd powers of $$x-x_0$$ can be computed separately as

$a_{2m+2} = -{p(2m)\over(2m+2)(2m+1)}a_{2m},\quad m\ge0\label{eq:7.2.26}$

$a_{2m+3} = -{p(2m+1)\over(2m+3)(2m+2)}a_{2m+1},\quad m\ge0, \label{eq:7.2.27}$

where $$a_0$$ and $$a_1$$ are arbitrary.

Proof

Here

$Ly=\left(1+\alpha(x-x_0\right)^2)y''+\beta(x-x_0) y'+\gamma y. \nonumber$

If

$y=\sum_{n=0}^\infty a_n(x-x_0)^n,\nonumber$

then

$y'=\sum_{n=1}^\infty na_n(x-x_0)^{n-1} \quad\mbox{ and }\quad y''=\sum_{n=2}^\infty n(n-1)a_n(x-x_0)^{n-2}.\nonumber$

Hence,

$\begin{array}{ccl} Ly &= {\sum_{n=2}^\infty n(n-1)a_n(x-x_0)^{n-2}+ \sum_{n=0}^\infty \left[\alpha n(n-1) +\beta n+\gamma\right]a_n(x-x_0)^n}\\[4pt] &= {\sum_{n=2}^\infty n(n-1)a_n(x-x_0)^{n-2}+\sum_{n=0}^\infty p(n)a_n(x-x_0)^n}, \end{array}\nonumber$

from Equation \ref{eq:7.2.25}. To collect coefficients of powers of $$x-x_0$$, we shift the summation index in the first sum. This yields

$Ly=\sum_{n=0}^\infty \left[(n+2)(n+1)a_{n+2}+p(n)a_n\right](x-x_0)^n.\nonumber$

Thus, $$Ly=0$$ if and only if

$(n+2)(n+1)a_{n+2}+p(n)a_n=0,\quad n\ge0,\nonumber$

which is equivalent to Equation \ref{eq:7.2.24}. Writing Equation \ref{eq:7.2.24} separately for the cases where $$n=2m$$ and $$n=2m+1$$ yields Equation \ref{eq:7.2.26} and Equation \ref{eq:7.2.27}.

##### Example 7.3.3

Find the power series in $$x-1$$ for the general solution of

$\label{eq:7.2.28} (2+4x-2x^2)y''-12(x-1)y'-12y=0.$

Solution

We must first write the coefficient $$P_0(x)=2+4x-x^2$$ in powers of $$x-1$$. To do this, we write $$x=(x-1)+1$$ in $$P_0(x)$$ and then expand the terms, collecting powers of $$x-1$$; thus,

\begin{aligned} 2+4x-2x^2 &= 2+4[(x-1)+1]-2[(x-1)+1]^2\\ &= 4-2(x-1)^2.\end{aligned}\nonumber

Therefore we can rewrite Equation \ref{eq:7.2.28} as

$\left(4-2(x-1)^2\right)y''-12(x-1)y'-12y=0, \nonumber$

or, equivalently,

$\left(1-{1\over2}(x-1)^2\right)y''-3(x-1)y'-3y=0. \nonumber$

This is of the form Equation \ref{eq:7.2.23} with $$\alpha=-1/2$$, $$\beta=-3$$, and $$\gamma=-3$$. Therefore, from Equation \ref{eq:7.2.25}

$p(n)=-{n(n-1)\over2}-3n-3=-{(n+2)(n+3)\over2}.\nonumber$

Hence, Theorem7.3.2 implies that

\begin{aligned} a_{2m+2} &= -{p(2m)\over(2m+2)(2m+1)}a_{2m}\\ &= {(2m+2)(2m+3)\over2(2m+2)(2m+1)} a_{2m}={2m+3\over2(2m+1)}a_{2m},\quad m\ge0 \end{aligned} \nonumber

and

\begin{aligned} a_{2m+3} &= -{p(2m+1)\over(2m+3)(2m+2)}a_{2m+1}\\ &= {(2m+3)(2m+4)\over2 (2m+3)(2m+2)}a_{2m+1}={m+2\over2(m+1)}a_{2m+1},\quad m\ge0.\end{aligned}\nonumber

We leave it to you to show that

$a_{2m}={2m+1\over2^m}a_0\quad\mbox{ and }\quad a_{2m+1}={m+1\over2^m}a_1,\quad m\ge0,\nonumber$

which implies that the power series in $$x-1$$ for the general solution of Equation \ref{eq:7.2.28} is

$y=a_0\sum_{m=0}^\infty{2m+1\over2^m}(x-1)^{2m}+a_1\sum_{m=0}^\infty {m+1\over2^m}(x-1)^{2m+1}.\nonumber$

In the examples considered so far we were able to obtain closed formulas for coefficients in the power series solutions. In some cases this is impossible, and we must settle for computing a finite number of terms in the series. The next example illustrates this with an initial value problem.

##### Example 7.3.4

Compute $$a_0$$, $$a_1$$, …, $$a_7$$ in the series solution $$y=\sum_{n=0}^\infty a_nx^n$$ of the initial value problem

$\label{eq:7.2.29} (1+2x^2)y''+10xy'+8y=0,\quad y(0)=2,\quad y'(0)=-3.$

Solution

Since $$\alpha=2$$, $$\beta=10$$, and $$\gamma=8$$ in Equation \ref{eq:7.2.29},

$p(n)=2n(n-1)+10n+8=2(n+2)^2.\nonumber$

Therefore

$a_{n+2}=-2{(n+2)^2\over(n+2)(n+1)}a_n=-2{n+2\over n+1}a_n,\quad n\ge0.\nonumber$

Writing this equation separately for $$n=2m$$ and $$n=2m+1$$ yields

\begin{align} a_{2m+2} = -2{(2m+2)\over2m+1}a_{2m}=-4{m+1\over2m+1}a_{2m},\quad m\ge 0\label{eq:7.2.30}\end{align}

\begin{align} a_{2m+3} = -2{2m+3\over2m+2}a_{2m+1}=-{2m+3\over m+1}a_{2m+1},\quad m\ge0. \label{eq:7.2.31}\end{align}

Starting with $$a_0=y(0)=2$$, we compute $$a_2, a_4$$, and $$a_6$$ from Equation \ref{eq:7.2.30} :

\begin{aligned} a_2 &= -4\,{1\over1}2=-8,\\[4pt] a_4 &= -4\,{2\over3}(-8)={64\over3},\\[4pt] a_6 &= -4\,{3\over5}\left(64\over3\right)=-{256\over5}.\end{aligned}\nonumber

Starting with $$a_1=y'(0)=-3$$, we compute $$a_3,a_5$$ and $$a_7$$ from Equation \ref{eq:7.2.31} :

\begin{aligned} a_3 &= -{3\over1}(-3)=9,\\[4pt] a_5 &= -{5\over2}9=-{45\over2},\\[4pt] a_7 &= -{7\over3}\left(-{45\over2}\right)={105\over2}.\end{aligned}\nonumber

Therefore the solution of Equation \ref{eq:7.2.29} is

$y=2-3x-8x^2+9x^3+{64\over3}x^4-{45\over2}x^5-{256\over5} x^6+{105\over2}x^7+\cdots\;.\nonumber$

## Using Technology

Computing coefficients recursively as in Example 7.3.4 is tedious. We recommend that you do this kind of computation by writing a short program to implement the appropriate recurrence relation on a calculator or computer. You may wish to do this in verifying examples and doing exercises (identified by the symbol $$C$$) in this chapter that call for numerical computation of the coefficients in series solutions. We obtained the answers to these exercises by using software that can produce answers in the form of rational numbers. However, it is perfectly acceptable - and more practical - to get your answers in decimal form. You can always check them by converting our fractions to decimals.

If you’re interested in actually using series to compute numerical approximations to solutions of a differential equation, then whether or not there’s a simple closed form for the coefficients is essentially irrelevant. For computational purposes it is usually more efficient to start with the given coefficients $$a_0=y(x_0)$$ and $$a_1=y'(x_0)$$, compute $$a_2$$, …, $$a_N$$ recursively, and then compute approximate values of the solution from the Taylor polynomial

$T_N(x)=\sum_{n=0}^Na_n(x-x_0)^n.\nonumber$

The trick is to decide how to choose $$N$$ so the approximation $$y(x)\approx T_N(x)$$ is sufficiently accurate on the subinterval of the interval of convergence that you’re interested in. In the computational exercises in this and the next two sections, you will often be asked to obtain the solution of a given problem by numerical integration with software of your choice (see Section 10.1 for a brief discussion of one such method), and to compare the solution obtained in this way with the approximations obtained with $$T_N$$ for various values of $$N$$. This is a typical textbook kind of exercise, designed to give you insight into how the accuracy of the approximation $$y(x)\approx T_N(x)$$ behaves as a function of $$N$$ and the interval that you’re working on. In real life, you would choose one or the other of the two methods (numerical integration or series solution). If you choose the method of series solution, then a practical procedure for determining a suitable value of $$N$$ is to continue increasing $$N$$ until the maximum of $$|T_N-T_{N-1}|$$ on the interval of interest is within the margin of error that you’re willing to accept.

In doing computational problems that call for numerical solution of differential equations you should choose the most accurate numerical integration procedure your software supports, and experiment with the step size until you’re confident that the numerical results are sufficiently accurate for the problem at hand.

7.2: Series Solutions Near an Ordinary Point I is shared under a CC BY-NC-SA 3.0 license and was authored, remixed, and/or curated by William F. Trench.


text:
# For every non-square odd integer $n>1$ , does there exist prime $p$ with $p+n$ a square?

Let $n>1$ be a non-square odd integer. Then does there necessarily exist a prime $p$ such that $p+n$ is a perfect square ?

I know that the existence of infinitely many such primes is open, but can we ensure at least one prime ?

• You could start by considering just the primes between two consecutive squares e.g. between $2^2$ and $3^2$ there are two primes, $5$ and $7$, with $n=(2,4)$ then between $3^2$ and $4^2$ with $n=(3,5)$. You could use a computer to test this stronger and simpler hypothesis first up to some large number to see if there are any gaps in $n$. – James Arathoon Dec 15 '17 at 12:06
• See also Legendre's conjecture. – lhf Dec 15 '17 at 12:32

The question is whether for given $n\ge 3$ as above the set $\{m^2-n\mid m\in \mathbb{N}\}$ contains a prime. Prime numbers of this form are called Near-Square Primes. They have been studied, and the question is again related to the Bunyakovsky conjecture, see here, which predicts infinitely many such primes for given $n$ not a perfect square. It seems to me, that even finding always one prime is open, but I need to search the literature more carefully for a reference.