# PySRRegressor Reference¶

High-performance symbolic regression algorithm.

This is the scikit-learn interface for SymbolicRegression.jl. This model will automatically search for equations which fit a given dataset subject to a particular loss and set of constraints.

Most default parameters have been tuned over several example equations, but you should adjust niterations, binary_operators, unary_operators to your requirements. You can view more detailed explanations of the options on the options page of the documentation.

Parameters:

Name Type Description Default
model_selection str

Model selection criterion when selecting a final expression from the list of best expression at each complexity. Can be 'accuracy', 'best', or 'score'. Default is 'best'. 'accuracy' selects the candidate model with the lowest loss (highest accuracy). 'score' selects the candidate model with the highest score. Score is defined as the negated derivative of the log-loss with respect to complexity - if an expression has a much better loss at a slightly higher complexity, it is preferred. 'best' selects the candidate model with the highest score among expressions with a loss better than at least 1.5x the most accurate model.

'best'
binary_operators list[str]

List of strings for binary operators used in the search. See the operators page for more details. Default is ["+", "-", "*", "/"].

None
unary_operators list[str]

Operators which only take a single scalar as input. For example, "cos" or "exp". Default is None.

None
niterations int

Number of iterations of the algorithm to run. The best equations are printed and migrate between populations at the end of each iteration. Default is 40.

40
populations int

Number of populations running. Default is 15.

15
population_size int

Number of individuals in each population. Default is 33.

33
max_evals int

Limits the total number of evaluations of expressions to this number. Default is None.

None
maxsize int

Max complexity of an equation. Default is 20.

20
maxdepth int

Max depth of an equation. You can use both maxsize and maxdepth. maxdepth is by default not used. Default is None.

None
warmup_maxsize_by float

Whether to slowly increase max size from a small number up to the maxsize (if greater than 0). If greater than 0, says the fraction of training time at which the current maxsize will reach the user-passed maxsize. Default is 0.0.

0.0
timeout_in_seconds float

Make the search return early once this many seconds have passed. Default is None.

None
constraints dict[str, int | tuple[int, int]]

Dictionary of int (unary) or 2-tuples (binary), this enforces maxsize constraints on the individual arguments of operators. E.g., 'pow': (-1, 1) says that power laws can have any complexity left argument, but only 1 complexity in the right argument. Use this to force more interpretable solutions. Default is None.

None
nested_constraints dict[str, dict]

Specifies how many times a combination of operators can be nested. For example, {"sin": {"cos": 0}}, "cos": {"cos": 2}} specifies that cos may never appear within a sin, but sin can be nested with itself an unlimited number of times. The second term specifies that cos can be nested up to 2 times within a cos, so that cos(cos(cos(x))) is allowed (as well as any combination of + or - within it), but cos(cos(cos(cos(x)))) is not allowed. When an operator is not specified, it is assumed that it can be nested an unlimited number of times. This requires that there is no operator which is used both in the unary operators and the binary operators (e.g., - could be both subtract, and negation). For binary operators, you only need to provide a single number: both arguments are treated the same way, and the max of each argument is constrained. Default is None.

None
loss str

String of Julia code specifying the loss function. Can either be a loss from LossFunctions.jl, or your own loss written as a function. Examples of custom written losses include: myloss(x, y) = abs(x-y) for non-weighted, or myloss(x, y, w) = w*abs(x-y) for weighted. The included losses include: Regression: LPDistLoss{P}(), L1DistLoss(), L2DistLoss() (mean square), LogitDistLoss(), HuberLoss(d), L1EpsilonInsLoss(ϵ), L2EpsilonInsLoss(ϵ), PeriodicLoss(c), QuantileLoss(τ). Classification: ZeroOneLoss(), PerceptronLoss(), L1HingeLoss(), SmoothedL1HingeLoss(γ), ModifiedHuberLoss(), L2MarginLoss(), ExpLoss(), SigmoidLoss(), DWDMarginLoss(q). Default is "L2DistLoss()".

'L2DistLoss()'
complexity_of_operators dict[str, float]

If you would like to use a complexity other than 1 for an operator, specify the complexity here. For example, {"sin": 2, "+": 1} would give a complexity of 2 for each use of the sin operator, and a complexity of 1 for each use of the + operator (which is the default). You may specify real numbers for a complexity, and the total complexity of a tree will be rounded to the nearest integer after computing. Default is None.

None
complexity_of_constants float

Complexity of constants. Default is 1.

1
complexity_of_variables float

Complexity of variables. Default is 1.

1
parsimony float

Multiplicative factor for how much to punish complexity. Default is 0.0032.

0.0032
use_frequency bool

Whether to measure the frequency of complexities, and use that instead of parsimony to explore equation space. Will naturally find equations of all complexities. Default is True.

True
use_frequency_in_tournament bool

Whether to use the frequency mentioned above in the tournament, rather than just the simulated annealing. Default is True.

True
alpha float

Initial temperature for simulated annealing (requires annealing to be True). Default is 0.1.

0.1
annealing bool

Whether to use annealing. Default is False.

False
early_stop_condition float | str

Stop the search early if this loss is reached. You may also pass a string containing a Julia function which takes a loss and complexity as input, for example: "f(loss, complexity) = (loss < 0.1) && (complexity < 10)". Default is None.

None
ncyclesperiteration int

Number of total mutations to run, per 10 samples of the population, per iteration. Default is 550.

550
fraction_replaced float

How much of population to replace with migrating equations from other populations. Default is 0.000364.

0.000364
fraction_replaced_hof float

How much of population to replace with migrating equations from hall of fame. Default is 0.035.

0.035
weight_add_node float

Relative likelihood for mutation to add a node. Default is 0.79.

0.79
weight_insert_node float

Relative likelihood for mutation to insert a node. Default is 5.1.

5.1
weight_delete_node float

Relative likelihood for mutation to delete a node. Default is 1.7.

1.7
weight_do_nothing float

Relative likelihood for mutation to leave the individual. Default is 0.21.

0.21
weight_mutate_constant float

Relative likelihood for mutation to change the constant slightly in a random direction. Default is 0.048.

0.048
weight_mutate_operator float

Relative likelihood for mutation to swap an operator. Default is 0.47.

0.47
weight_randomize float

Relative likelihood for mutation to completely delete and then randomly generate the equation Default is 0.00023.

0.00023
weight_simplify float

Relative likelihood for mutation to simplify constant parts by evaluation Default is 0.0020.

0.002
crossover_probability float

Absolute probability of crossover-type genetic operation, instead of a mutation. Default is 0.066.

0.066
skip_mutation_failures bool

Whether to skip mutation and crossover failures, rather than simply re-sampling the current member. Default is True.

True
migration bool

Whether to migrate. Default is True.

True
hof_migration bool

Whether to have the hall of fame migrate. Default is True.

True
topn int

How many top individuals migrate from each population. Default is 12.

12
should_optimize_constants bool

Whether to numerically optimize constants (Nelder-Mead/Newton) at the end of each iteration. Default is True.

True
optimizer_algorithm str

Optimization scheme to use for optimizing constants. Can currently be NelderMead or BFGS. Default is "BFGS".

'BFGS'
optimizer_nrestarts int

Number of time to restart the constants optimization process with different initial conditions. Default is 2.

2
optimize_probability float

Probability of optimizing the constants during a single iteration of the evolutionary algorithm. Default is 0.14.

0.14
optimizer_iterations int

Number of iterations that the constants optimizer can take. Default is 8.

8
perturbation_factor float

Constants are perturbed by a max factor of (perturbation_factor*T + 1). Either multiplied by this or divided by this. Default is 0.076.

0.076
tournament_selection_n int

Number of expressions to consider in each tournament. Default is 10.

10
tournament_selection_p float

Probability of selecting the best expression in each tournament. The probability will decay as p*(1-p)^n for other expressions, sorted by loss. Default is 0.86.

0.86
procs int

Number of processes (=number of populations running). Default is cpu_count().

cpu_count()
multithreading bool

Use multithreading instead of distributed backend. Using procs=0 will turn off both. Default is True.

None
cluster_manager str

For distributed computing, this sets the job queue system. Set to one of "slurm", "pbs", "lsf", "sge", "qrsh", "scyld", or "htc". If set to one of these, PySR will run in distributed mode, and use procs to figure out how many processes to launch. Default is None.

None
batching bool

Whether to compare population members on small batches during evolution. Still uses full dataset for comparing against hall of fame. Default is False.

False
batch_size int

The amount of data to use if doing batching. Default is 50.

50
fast_cycle bool

Batch over population subsamples. This is a slightly different algorithm than regularized evolution, but does cycles 15% faster. May be algorithmically less efficient. Default is False.

False
precision int

What precision to use for the data. By default this is 32 (float32), but you can select 64 or 16 as well, giving you 64 or 16 bits of floating point precision, respectively. Default is 32.

32
random_state int, Numpy RandomState instance or None

Pass an int for reproducible results across multiple function calls. See :term:Glossary <random_state>. Default is None.

None
deterministic bool

Make a PySR search give the same result every run. To use this, you must turn off parallelism (with procs=0, multithreading=False), and set random_state to a fixed seed. Default is False.

False
warm_start bool

Tells fit to continue from where the last call to fit finished. If false, each call to fit will be fresh, overwriting previous results. Default is False.

False
verbosity int

What verbosity level to use. 0 means minimal print statements. Default is 1e9.

1000000000.0
update_verbosity int

What verbosity level to use for package updates. Will take value of verbosity if not given. Default is None.

None
progress bool

Whether to use a progress bar instead of printing to stdout. Default is True.

True
equation_file str

Where to save the files (.csv extension). Default is None.

None
temp_equation_file bool

Whether to put the hall of fame file in the temp directory. Deletion is then controlled with the delete_tempfiles parameter. Default is False.

False
tempdir str

directory for the temporary files. Default is None.

None
delete_tempfiles bool

Whether to delete the temporary files after finishing. Default is True.

True
julia_project str

A Julia environment location containing a Project.toml (and potentially the source code for SymbolicRegression.jl). Default gives the Python package directory, where a Project.toml file should be present from the install.

None
update

Whether to automatically update Julia packages. Default is True.

True
output_jax_format bool

Whether to create a 'jax_format' column in the output, containing jax-callable functions and the default parameters in a jax array. Default is False.

False
output_torch_format bool

Whether to create a 'torch_format' column in the output, containing a torch module with trainable parameters. Default is False.

False
extra_sympy_mappings dict[str, Callable]

Provides mappings between custom binary_operators or unary_operators defined in julia strings, to those same operators defined in sympy. E.G if unary_operators=["inv(x)=1/x"], then for the fitted model to be export to sympy, extra_sympy_mappings would be {"inv": lambda x: 1/x}. Default is None.

None
extra_jax_mappings dict[Callable, str]

Similar to extra_sympy_mappings but for model export to jax. The dictionary maps sympy functions to jax functions. For example: extra_jax_mappings={sympy.sin: "jnp.sin"} maps the sympy.sin function to the equivalent jax expression jnp.sin. Default is None.

None
extra_torch_mappings dict[Callable, Callable]

The same as extra_jax_mappings but for model export to pytorch. Note that the dictionary keys should be callable pytorch expressions. For example: extra_torch_mappings={sympy.sin: torch.sin}. Default is None.

None
denoise bool

Whether to use a Gaussian Process to denoise the data before inputting to PySR. Can help PySR fit noisy data. Default is False.

False
select_k_features int

whether to run feature selection in Python using random forests, before passing to the symbolic regression code. None means no feature selection; an int means select that many features. Default is None.

None
**kwargs dict

Supports deprecated keyword arguments. Other arguments will result in an error.

{}
Attributes required
equations_ pandas.DataFrame | list[pandas.DataFrame]

Processed DataFrame containing the results of model fitting.

required
n_features_in_ int

Number of features seen during :term:fit.

required
feature_names_in_ ndarray of shape (

Names of features seen during :term:fit. Defined only when X has feature names that are all strings.

required
nout_ int

Number of output dimensions.

required
selection_mask_ list[int] of length

List of indices for input features that are selected when select_k_features is set.

required
tempdir_ Path

Path to the temporary equations directory.

required
equation_file_ str

Output equation file name produced by the julia backend.

required
raw_julia_state_ tuple[list[PyCall.jlwrap], PyCall.jlwrap]

The state for the julia SymbolicRegression.jl backend post fitting.

required
equation_file_contents_ list[pandas.DataFrame]

Contents of the equation file output by the Julia backend.

required
show_pickle_warnings_ bool

Whether to show warnings about what attributes can be pickled.

required

Examples:

>>> import numpy as np
>>> from pysr import PySRRegressor
>>> randstate = np.random.RandomState(0)
>>> X = 2 * randstate.randn(100, 5)
>>> # y = 2.5382 * cos(x_3) + x_0 - 0.5
>>> y = 2.5382 * np.cos(X[:, 3]) + X[:, 0] ** 2 - 0.5
>>> model = PySRRegressor(
...     niterations=40,
...     binary_operators=["+", "*"],
...     unary_operators=[
...         "cos",
...         "exp",
...         "sin",
...         "inv(x) = 1/x",  # Custom operator (julia syntax)
...     ],
...     model_selection="best",
...     loss="loss(x, y) = (x - y)^2",  # Custom loss function (julia syntax)
... )
>>> model.fit(X, y)
>>> model
PySRRegressor.equations_ = [
0         0.000000                                          3.8552167  3.360272e+01           1
1         1.189847                                          (x0 * x0)  3.110905e+00           3
2         0.010626                          ((x0 * x0) + -0.25573406)  3.045491e+00           5
3         0.896632                              (cos(x3) + (x0 * x0))  1.242382e+00           6
4         0.811362                ((x0 * x0) + (cos(x3) * 2.4384754))  2.451971e-01           8
5  >>>>  13.733371          (((cos(x3) * 2.5382) + (x0 * x0)) + -0.5)  2.889755e-13          10
6         0.194695  ((x0 * x0) + (((cos(x3) + -0.063180044) * 2.53...  1.957723e-13          12
7         0.006988  ((x0 * x0) + (((cos(x3) + -0.32505524) * 1.538...  1.944089e-13          13
8         0.000955  (((((x0 * x0) + cos(x3)) + -0.8251649) + (cos(...  1.940381e-13          15
]
>>> model.score(X, y)
1.0
>>> model.predict(np.array([1,2,3,4,5]))
array([-1.15907818, -1.15907818, -1.15907818, -1.15907818, -1.15907818])

Source code in pysr/sr.py
 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 def __init__( self, model_selection="best", *, binary_operators=None, unary_operators=None, niterations=40, populations=15, population_size=33, max_evals=None, maxsize=20, maxdepth=None, warmup_maxsize_by=0.0, timeout_in_seconds=None, constraints=None, nested_constraints=None, loss="L2DistLoss()", complexity_of_operators=None, complexity_of_constants=1, complexity_of_variables=1, parsimony=0.0032, use_frequency=True, use_frequency_in_tournament=True, alpha=0.1, annealing=False, early_stop_condition=None, ncyclesperiteration=550, fraction_replaced=0.000364, fraction_replaced_hof=0.035, weight_add_node=0.79, weight_insert_node=5.1, weight_delete_node=1.7, weight_do_nothing=0.21, weight_mutate_constant=0.048, weight_mutate_operator=0.47, weight_randomize=0.00023, weight_simplify=0.0020, crossover_probability=0.066, skip_mutation_failures=True, migration=True, hof_migration=True, topn=12, should_optimize_constants=True, optimizer_algorithm="BFGS", optimizer_nrestarts=2, optimize_probability=0.14, optimizer_iterations=8, perturbation_factor=0.076, tournament_selection_n=10, tournament_selection_p=0.86, procs=cpu_count(), multithreading=None, cluster_manager=None, batching=False, batch_size=50, fast_cycle=False, precision=32, random_state=None, deterministic=False, warm_start=False, verbosity=1e9, update_verbosity=None, progress=True, equation_file=None, temp_equation_file=False, tempdir=None, delete_tempfiles=True, julia_project=None, update=True, output_jax_format=False, output_torch_format=False, extra_sympy_mappings=None, extra_torch_mappings=None, extra_jax_mappings=None, denoise=False, select_k_features=None, **kwargs, ): # Hyperparameters # - Model search parameters self.model_selection = model_selection self.binary_operators = binary_operators self.unary_operators = unary_operators self.niterations = niterations self.populations = populations self.population_size = population_size self.ncyclesperiteration = ncyclesperiteration # - Equation Constraints self.maxsize = maxsize self.maxdepth = maxdepth self.constraints = constraints self.nested_constraints = nested_constraints self.warmup_maxsize_by = warmup_maxsize_by # - Early exit conditions: self.max_evals = max_evals self.timeout_in_seconds = timeout_in_seconds self.early_stop_condition = early_stop_condition # - Loss parameters self.loss = loss self.complexity_of_operators = complexity_of_operators self.complexity_of_constants = complexity_of_constants self.complexity_of_variables = complexity_of_variables self.parsimony = parsimony self.use_frequency = use_frequency self.use_frequency_in_tournament = use_frequency_in_tournament self.alpha = alpha self.annealing = annealing # - Evolutionary search parameters # -- Mutation parameters self.weight_add_node = weight_add_node self.weight_insert_node = weight_insert_node self.weight_delete_node = weight_delete_node self.weight_do_nothing = weight_do_nothing self.weight_mutate_constant = weight_mutate_constant self.weight_mutate_operator = weight_mutate_operator self.weight_randomize = weight_randomize self.weight_simplify = weight_simplify self.crossover_probability = crossover_probability self.skip_mutation_failures = skip_mutation_failures # -- Migration parameters self.migration = migration self.hof_migration = hof_migration self.fraction_replaced = fraction_replaced self.fraction_replaced_hof = fraction_replaced_hof self.topn = topn # -- Constants parameters self.should_optimize_constants = should_optimize_constants self.optimizer_algorithm = optimizer_algorithm self.optimizer_nrestarts = optimizer_nrestarts self.optimize_probability = optimize_probability self.optimizer_iterations = optimizer_iterations self.perturbation_factor = perturbation_factor # -- Selection parameters self.tournament_selection_n = tournament_selection_n self.tournament_selection_p = tournament_selection_p # Solver parameters self.procs = procs self.multithreading = multithreading self.cluster_manager = cluster_manager self.batching = batching self.batch_size = batch_size self.fast_cycle = fast_cycle self.precision = precision self.random_state = random_state self.deterministic = deterministic self.warm_start = warm_start # Additional runtime parameters # - Runtime user interface self.verbosity = verbosity self.update_verbosity = update_verbosity self.progress = progress # - Project management self.equation_file = equation_file self.temp_equation_file = temp_equation_file self.tempdir = tempdir self.delete_tempfiles = delete_tempfiles self.julia_project = julia_project self.update = update self.output_jax_format = output_jax_format self.output_torch_format = output_torch_format self.extra_sympy_mappings = extra_sympy_mappings self.extra_jax_mappings = extra_jax_mappings self.extra_torch_mappings = extra_torch_mappings # Pre-modelling transformation self.denoise = denoise self.select_k_features = select_k_features # Once all valid parameters have been assigned handle the # deprecated kwargs if len(kwargs) > 0: # pragma: no cover deprecated_kwargs = make_deprecated_kwargs_for_pysr_regressor() for k, v in kwargs.items(): # Handle renamed kwargs if k in deprecated_kwargs: updated_kwarg_name = deprecated_kwargs[k] setattr(self, updated_kwarg_name, v) warnings.warn( f"{k} has been renamed to {updated_kwarg_name} in PySRRegressor. " "Please use that instead.", FutureWarning, ) # Handle kwargs that have been moved to the fit method elif k in ["weights", "variable_names", "Xresampled"]: warnings.warn( f"{k} is a data dependant parameter so should be passed when fit is called. " f"Ignoring parameter; please pass {k} during the call to fit instead.", FutureWarning, ) else: raise TypeError( f"{k} is not a valid keyword argument for PySRRegressor." ) 

## pysr.sr.PySRRegressor.fit(X, y, Xresampled=None, weights=None, variable_names=None)¶

Search for equations to fit the dataset and store them in self.equations_.

Parameters:

Name Type Description Default
X ndarray | pandas.DataFrame

Training data of shape (n_samples, n_features).

required
y ndarray | pandas.DataFrame

Target values of shape (n_samples,) or (n_samples, n_targets). Will be cast to X's dtype if necessary.

required
Xresampled ndarray | pandas.DataFrame

Resampled training data, of shape (n_resampled, n_features), to generate a denoised data on. This will be used as the training data, rather than X.

None
weights ndarray | pandas.DataFrame

Weight array of the same shape as y. Each element is how to weight the mean-square-error loss for that particular element of y. Alternatively, if a custom loss was set, it will can be used in arbitrary ways.

None
variable_names list[str]

A list of names for the variables, rather than "x0", "x1", etc. If X is a pandas dataframe, the column names will be used instead of variable_names. Cannot contain spaces or special characters. Avoid variable names which are also function names in sympy, such as "N".

None

Returns:

Name Type Description
self object

Fitted estimator.

Source code in pysr/sr.py
 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 def fit( self, X, y, Xresampled=None, weights=None, variable_names=None, ): """ Search for equations to fit the dataset and store them in self.equations_. Parameters ---------- X : ndarray | pandas.DataFrame Training data of shape (n_samples, n_features). y : ndarray | pandas.DataFrame Target values of shape (n_samples,) or (n_samples, n_targets). Will be cast to X's dtype if necessary. Xresampled : ndarray | pandas.DataFrame Resampled training data, of shape (n_resampled, n_features), to generate a denoised data on. This will be used as the training data, rather than X. weights : ndarray | pandas.DataFrame Weight array of the same shape as y. Each element is how to weight the mean-square-error loss for that particular element of y. Alternatively, if a custom loss was set, it will can be used in arbitrary ways. variable_names : list[str] A list of names for the variables, rather than "x0", "x1", etc. If X is a pandas dataframe, the column names will be used instead of variable_names. Cannot contain spaces or special characters. Avoid variable names which are also function names in sympy, such as "N". Returns ------- self : object Fitted estimator. """ # Init attributes that are not specified in BaseEstimator if self.warm_start and hasattr(self, "raw_julia_state_"): pass else: if hasattr(self, "raw_julia_state_"): warnings.warn( "The discovered expressions are being reset. " "Please set warm_start=True if you wish to continue " "to start a search where you left off.", ) self.equations_ = None self.nout_ = 1 self.selection_mask_ = None self.raw_julia_state_ = None random_state = check_random_state(self.random_state) # For np random seed = random_state.get_state()[1][0] # For julia random self._setup_equation_file() mutated_params = self._validate_and_set_init_params() X, y, Xresampled, weights, variable_names = self._validate_and_set_fit_params( X, y, Xresampled, weights, variable_names ) if X.shape[0] > 10000 and not self.batching: warnings.warn( "Note: you are running with more than 10,000 datapoints. " "You should consider turning on batching (https://astroautomata.com/PySR/options/#batching). " "You should also reconsider if you need that many datapoints. " "Unless you have a large amount of noise (in which case you " "should smooth your dataset first), generally < 10,000 datapoints " "is enough to find a functional form with symbolic regression. " "More datapoints will lower the search speed." ) # Pre transformations (feature selection and denoising) X, y, variable_names = self._pre_transform_training_data( X, y, Xresampled, variable_names, random_state ) # Warn about large feature counts (still warn if feature count is large # after running feature selection) if self.n_features_in_ >= 10: warnings.warn( "Note: you are running with 10 features or more. " "Genetic algorithms like used in PySR scale poorly with large numbers of features. " "Consider using feature selection techniques to select the most important features " "(you can do this automatically with the select_k_features parameter), " "or, alternatively, doing a dimensionality reduction beforehand. " "For example, X = PCA(n_components=6).fit_transform(X), " "using scikit-learn's PCA class, " "will reduce the number of features to 6 in an interpretable way, " "as each resultant feature " "will be a linear combination of the original features. " ) # Assertion checks use_custom_variable_names = variable_names is not None # TODO: this is always true. _check_assertions( X, use_custom_variable_names, variable_names, weights, y, ) # Initially, just save model parameters, so that # it can be loaded from an early exit: if not self.temp_equation_file: self._checkpoint() # Perform the search: self._run(X, y, mutated_params, weights=weights, seed=seed) # Then, after fit, we save again, so the pickle file contains # the equations: if not self.temp_equation_file: self._checkpoint() return self 

## pysr.sr.PySRRegressor.predict(X, index=None)¶

Predict y from input X using the equation chosen by model_selection.

You may see what equation is used by printing this object. X should have the same columns as the training data.

Parameters:

Name Type Description Default
X ndarray | pandas.DataFrame

Training data of shape (n_samples, n_features).

required
index int | list[int]

If you want to compute the output of an expression using a particular row of self.equations_, you may specify the index here. For multiple output equations, you must pass a list of indices in the same order.

None

Returns:

Name Type Description
y_predicted ndarray of shape (n_samples, nout_)

Values predicted by substituting X into the fitted symbolic regression model.

Raises:

Type Description
ValueError

Raises if the best_equation cannot be evaluated.

Source code in pysr/sr.py
 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 def predict(self, X, index=None): """ Predict y from input X using the equation chosen by model_selection. You may see what equation is used by printing this object. X should have the same columns as the training data. Parameters ---------- X : ndarray | pandas.DataFrame Training data of shape (n_samples, n_features). index : int | list[int] If you want to compute the output of an expression using a particular row of self.equations_, you may specify the index here. For multiple output equations, you must pass a list of indices in the same order. Returns ------- y_predicted : ndarray of shape (n_samples, nout_) Values predicted by substituting X into the fitted symbolic regression model. Raises ------ ValueError Raises if the best_equation cannot be evaluated. """ check_is_fitted( self, attributes=["selection_mask_", "feature_names_in_", "nout_"] ) best_equation = self.get_best(index=index) # When X is an numpy array or a pandas dataframe with a RangeIndex, # the self.feature_names_in_ generated during fit, for the same X, # will cause a warning to be thrown during _validate_data. # To avoid this, convert X to a dataframe, apply the selection mask, # and then set the column/feature_names of X to be equal to those # generated during fit. if not isinstance(X, pd.DataFrame): X = check_array(X) X = pd.DataFrame(X) if isinstance(X.columns, pd.RangeIndex): if self.selection_mask_ is not None: # RangeIndex enforces column order allowing columns to # be correctly filtered with self.selection_mask_ X = X.iloc[:, self.selection_mask_] X.columns = self.feature_names_in_ # Without feature information, CallableEquation/lambda_format equations # require that the column order of X matches that of the X used during # the fitting process. _validate_data removes this feature information # when it converts the dataframe to an np array. Thus, to ensure feature # order is preserved after conversion, the dataframe columns must be # reordered/reindexed to match those of the transformed (denoised and # feature selected) X in fit. X = X.reindex(columns=self.feature_names_in_) X = self._validate_data(X, reset=False) try: if self.nout_ > 1: return np.stack( [eq["lambda_format"](X) for eq in best_equation], axis=1 ) return best_equation["lambda_format"](X) except Exception as error: raise ValueError( "Failed to evaluate the expression. " "If you are using a custom operator, make sure to define it in extra_sympy_mappings, " "e.g., model.set_params(extra_sympy_mappings={'inv': lambda x: 1/x}), where " "lambda x: 1/x is a valid SymPy function defining the operator. " "You can then run model.refresh() to re-load the expressions." ) from error 

## pysr.sr.PySRRegressor.from_file(equation_file, *, binary_operators=None, unary_operators=None, n_features_in=None, feature_names_in=None, selection_mask=None, nout=1, **pysr_kwargs) classmethod ¶

Create a model from a saved model checkpoint or equation file.

Parameters:

Name Type Description Default
equation_file str

Path to a pickle file containing a saved model, or a csv file containing equations.

required
binary_operators list[str]

The same binary operators used when creating the model. Not needed if loading from a pickle file.

None
unary_operators list[str]

The same unary operators used when creating the model. Not needed if loading from a pickle file.

None
n_features_in int

Number of features passed to the model. Not needed if loading from a pickle file.

None
feature_names_in list[str]

Names of the features passed to the model. Not needed if loading from a pickle file.

None
selection_mask list[bool]

If using select_k_features, you must pass model.selection_mask_ here. Not needed if loading from a pickle file.

None
nout int

Number of outputs of the model. Not needed if loading from a pickle file. Default is 1.

1
**pysr_kwargs dict

Any other keyword arguments to initialize the PySRRegressor object. These will overwrite those stored in the pickle file. Not needed if loading from a pickle file.

{}

Returns:

Name Type Description
model PySRRegressor

The model with fitted equations.

Source code in pysr/sr.py
 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 @classmethod def from_file( cls, equation_file, *, binary_operators=None, unary_operators=None, n_features_in=None, feature_names_in=None, selection_mask=None, nout=1, **pysr_kwargs, ): """ Create a model from a saved model checkpoint or equation file. Parameters ---------- equation_file : str Path to a pickle file containing a saved model, or a csv file containing equations. binary_operators : list[str] The same binary operators used when creating the model. Not needed if loading from a pickle file. unary_operators : list[str] The same unary operators used when creating the model. Not needed if loading from a pickle file. n_features_in : int Number of features passed to the model. Not needed if loading from a pickle file. feature_names_in : list[str] Names of the features passed to the model. Not needed if loading from a pickle file. selection_mask : list[bool] If using select_k_features, you must pass model.selection_mask_ here. Not needed if loading from a pickle file. nout : int Number of outputs of the model. Not needed if loading from a pickle file. Default is 1. **pysr_kwargs : dict Any other keyword arguments to initialize the PySRRegressor object. These will overwrite those stored in the pickle file. Not needed if loading from a pickle file. Returns ------- model : PySRRegressor The model with fitted equations. """ if os.path.splitext(equation_file)[1] != ".pkl": pkl_filename = _csv_filename_to_pkl_filename(equation_file) else: pkl_filename = equation_file # Try to load model from .pkl print(f"Checking if {pkl_filename} exists...") if os.path.exists(pkl_filename): print(f"Loading model from {pkl_filename}") assert binary_operators is None assert unary_operators is None assert n_features_in is None with open(pkl_filename, "rb") as f: model = pkl.load(f) # Change equation_file_ to be in the same dir as the pickle file base_dir = os.path.dirname(pkl_filename) base_equation_file = os.path.basename(model.equation_file_) model.equation_file_ = os.path.join(base_dir, base_equation_file) # Update any parameters if necessary, such as # extra_sympy_mappings: model.set_params(**pysr_kwargs) if "equations_" not in model.__dict__ or model.equations_ is None: model.refresh() return model # Else, we re-create it. print( f"{equation_file} does not exist, " "so we must create the model from scratch." ) assert binary_operators is not None assert unary_operators is not None assert n_features_in is not None # TODO: copy .bkup file if exists. model = cls( equation_file=equation_file, binary_operators=binary_operators, unary_operators=unary_operators, **pysr_kwargs, ) model.nout_ = nout model.n_features_in_ = n_features_in if feature_names_in is None: model.feature_names_in_ = [f"x{i}" for i in range(n_features_in)] else: assert len(feature_names_in) == n_features_in model.feature_names_in_ = feature_names_in if selection_mask is None: model.selection_mask_ = np.ones(n_features_in, dtype=bool) else: model.selection_mask_ = selection_mask model.refresh(checkpoint_file=equation_file) return model 

## pysr.sr.PySRRegressor.sympy(index=None)¶

Return sympy representation of the equation(s) chosen by model_selection.

Parameters:

Name Type Description Default
index int | list[int]

If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature.

None

Returns:

Name Type Description
best_equation str, list[str] of length nout_

SymPy representation of the best equation.

Source code in pysr/sr.py
 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 def sympy(self, index=None): """ Return sympy representation of the equation(s) chosen by model_selection. Parameters ---------- index : int | list[int] If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature. Returns ------- best_equation : str, list[str] of length nout_ SymPy representation of the best equation. """ self.refresh() best_equation = self.get_best(index=index) if self.nout_ > 1: return [eq["sympy_format"] for eq in best_equation] return best_equation["sympy_format"] 

## pysr.sr.PySRRegressor.latex(index=None, precision=3)¶

Return latex representation of the equation(s) chosen by model_selection.

Parameters:

Name Type Description Default
index int | list[int]

If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature.

None
precision int

The number of significant figures shown in the LaTeX representation. Default is 3.

3

Returns:

Name Type Description
best_equation str or list[str] of length nout_

LaTeX expression of the best equation.

Source code in pysr/sr.py
 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 def latex(self, index=None, precision=3): """ Return latex representation of the equation(s) chosen by model_selection. Parameters ---------- index : int | list[int] If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature. precision : int The number of significant figures shown in the LaTeX representation. Default is 3. Returns ------- best_equation : str or list[str] of length nout_ LaTeX expression of the best equation. """ self.refresh() sympy_representation = self.sympy(index=index) if self.nout_ > 1: output = [] for s in sympy_representation: latex = to_latex(s, prec=precision) output.append(latex) return output return to_latex(sympy_representation, prec=precision) 

## pysr.sr.PySRRegressor.pytorch(index=None)¶

Return pytorch representation of the equation(s) chosen by model_selection.

Each equation (multiple given if there are multiple outputs) is a PyTorch module containing the parameters as trainable attributes. You can use the module like any other PyTorch module: module(X), where X is a tensor with the same column ordering as trained with.

Parameters:

Name Type Description Default
index int | list[int]

If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature.

None

Returns:

Name Type Description
best_equation torch.nn.Module

PyTorch module representing the expression.

Source code in pysr/sr.py
 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 def pytorch(self, index=None): """ Return pytorch representation of the equation(s) chosen by model_selection. Each equation (multiple given if there are multiple outputs) is a PyTorch module containing the parameters as trainable attributes. You can use the module like any other PyTorch module: module(X), where X is a tensor with the same column ordering as trained with. Parameters ---------- index : int | list[int] If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature. Returns ------- best_equation : torch.nn.Module PyTorch module representing the expression. """ self.set_params(output_torch_format=True) self.refresh() best_equation = self.get_best(index=index) if self.nout_ > 1: return [eq["torch_format"] for eq in best_equation] return best_equation["torch_format"] 

## pysr.sr.PySRRegressor.jax(index=None)¶

Return jax representation of the equation(s) chosen by model_selection.

Each equation (multiple given if there are multiple outputs) is a dictionary containing {"callable": func, "parameters": params}. To call func, pass func(X, params). This function is differentiable using jax.grad.

Parameters:

Name Type Description Default
index int | list[int]

If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature.

None

Returns:

Name Type Description
best_equation dict[str, Any]

Dictionary of callable jax function in "callable" key, and jax array of parameters as "parameters" key.

Source code in pysr/sr.py
 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 def jax(self, index=None): """ Return jax representation of the equation(s) chosen by model_selection. Each equation (multiple given if there are multiple outputs) is a dictionary containing {"callable": func, "parameters": params}. To call func, pass func(X, params). This function is differentiable using jax.grad. Parameters ---------- index : int | list[int] If you wish to select a particular equation from self.equations_, give the index number here. This overrides the model_selection parameter. If there are multiple output features, then pass a list of indices with the order the same as the output feature. Returns ------- best_equation : dict[str, Any] Dictionary of callable jax function in "callable" key, and jax array of parameters as "parameters" key. """ self.set_params(output_jax_format=True) self.refresh() best_equation = self.get_best(index=index) if self.nout_ > 1: return [eq["jax_format"] for eq in best_equation] return best_equation["jax_format"] 

## pysr.sr.PySRRegressor.latex_table(indices=None, precision=3, columns=['equation', 'complexity', 'loss', 'score'])¶

Create a LaTeX/booktabs table for all, or some, of the equations.

Parameters:

Name Type Description Default
indices list[int] | list[list[int]]

If you wish to select a particular subset of equations from self.equations_, give the row numbers here. By default, all equations will be used. If there are multiple output features, then pass a list of lists.

None
precision int

The number of significant figures shown in the LaTeX representations. Default is 3.

3
columns list[str]

Which columns to include in the table. Default is ["equation", "complexity", "loss", "score"].

['equation', 'complexity', 'loss', 'score']

Returns:

Name Type Description
latex_table_str str

A string that will render a table in LaTeX of the equations.

Source code in pysr/sr.py
 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 def latex_table( self, indices=None, precision=3, columns=["equation", "complexity", "loss", "score"], ): """Create a LaTeX/booktabs table for all, or some, of the equations. Parameters ---------- indices : list[int] | list[list[int]] If you wish to select a particular subset of equations from self.equations_, give the row numbers here. By default, all equations will be used. If there are multiple output features, then pass a list of lists. precision : int The number of significant figures shown in the LaTeX representations. Default is 3. columns : list[str] Which columns to include in the table. Default is ["equation", "complexity", "loss", "score"]. Returns ------- latex_table_str : str A string that will render a table in LaTeX of the equations. """ self.refresh() if self.nout_ > 1: if indices is not None: assert isinstance(indices, list) assert isinstance(indices[0], list) assert isinstance(len(indices), self.nout_) generator_fnc = generate_multiple_tables else: if indices is not None: assert isinstance(indices, list) assert isinstance(indices[0], int) generator_fnc = generate_single_table table_string = generator_fnc( self.equations_, indices=indices, precision=precision, columns=columns ) preamble_string = [ r"\usepackage{breqn}", r"\usepackage{booktabs}", "", "...", "", ] return "\n".join(preamble_string + [table_string]) 

## pysr.sr.PySRRegressor.refresh(checkpoint_file=None)¶

Update self.equations_ with any new options passed.

For example, updating extra_sympy_mappings will require a .refresh() to update the equations.

Parameters:

Name Type Description Default
checkpoint_file str

Path to checkpoint hall of fame file to be loaded. The default will use the set equation_file_.

None
Source code in pysr/sr.py
 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 def refresh(self, checkpoint_file=None): """ Update self.equations_ with any new options passed. For example, updating extra_sympy_mappings will require a .refresh() to update the equations. Parameters ---------- checkpoint_file : str Path to checkpoint hall of fame file to be loaded. The default will use the set equation_file_. """ if checkpoint_file: self.equation_file_ = checkpoint_file self.equation_file_contents_ = None check_is_fitted(self, attributes=["equation_file_"]) self.equations_ = self.get_hof()