Example 3
=========

The next example demonstrates the multi-parameter capabilities of ``cgn``. We are going to solve a nonlinear least-squares problem that depends on two parameters $x$ and $y$, where $x$ has to satisfy a nonlinear equality constraint, while $y$ is only nonnegativity constrained.

$
\begin{align}
\min_{x \in \mathbb{R}^2, y \in \mathbb{R}} \quad & ||F(x, y)||_2^2 + \beta ||R(x - m)||_2^2\\
\text{s. t.} \quad  & x_1 + x_1^3 + x_2 + x_2^2 = 0, \quad y > 0, \\
\text{where } \quad & F(x) = \left(\begin{matrix}
x_1 + e^{-x_2} + \sqrt{y} \\
x_1^2 + 2 x_2 + 1 - \sqrt{y}
\end{matrix} \right), \\
& R = \left( \begin{matrix}
1 & 2 \\
3 & 4
\end{matrix} \right), \quad m = \left(\begin{matrix}
1 \\ 1 \\
\end{matrix} \right), \quad \beta = 0.1.
\end{align}
$

Let us start by implementing the required functions and their derivatives:


```python
from math import exp, sqrt
import numpy as np
import cgn

def F(x, y):
    out = np.array([x[0] + exp(-x[1] + sqrt(y[0])),
                    x[0] ** 2 + 2 * x[1] + 1 - sqrt(y[0])])
    return out

def DF(x, y):
    jac = np.array([[1., -exp(-x[1]), 0.5 / sqrt(y[0])],
                    [2 * x[0], 2., - 0.5 / sqrt(y[0])]])
    return jac
```

Next, we set up the inequality constraint, which only depends on $x$:


```python
def g(x):
    out = x[0] + x[0] ** 3 + x[1] + x[1] ** 2
    return np.array([out])

def Dg(x):
    jac = np.array([1 + 3 * x[0] ** 2, 1. + 2 * x[1]]).reshape((1, 2))
    return jac
```

Next, we set up our ``cgn.Parameter`` objects. For the initial guess, let's just try $x = [0, 0]$ and $y = 1$:


```python
x = cgn.Parameter(name="x", start=np.zeros(2))
y = cgn.Parameter(name="y", start=np.ones(1))
```

For ``x``, we have to specify the regularization term $\beta ||R(x-m)||_2^2$:


```python
x.regop = np.array([[1., 2.], [3., 4.]])
x.mean = np.array([1., 1.])
x.beta = 0.1
```

The second parameter ``y`` is not regularized, but it has to satisfy the lower-bound constraint $y > 0$. We implement this strict inequality constraint as an equality constraint
$y \geq \epsilon$ for a small, positive $\epsilon > 0$.


```python
eps = 1e-10
y.lb = np.array([eps])
```

With this setup, we can finally create our optimization problem and solve it with ``cgn``.


```python
inequality_constraint = cgn.NonlinearConstraint(parameters=[x], fun=g, jac=Dg, ctype="ineq")

problem = cgn.Problem(parameters=[x, y], fun=F, jac=DF, constraints=[inequality_constraint])
solver = cgn.CGN()

solution = solver.solve(problem=problem)
```

Seems to work without problems. Let us look at the minimizers:


```python

x_min = solution.minimizer("x")
y_min = solution.minimizer("y")
print(f"x_min = {x_min}")
print(f"y_min = {y_min}")
```

    x_min = [-0.00522749  0.83669584]
    y_min = [1.02461374]


8d50e1f1417e18bb734ff09e610af292e603cfaf
ipynb
Jupyter Notebook
examples/example3.ipynb
FabianKP/cgn
9963e60c4a4bf4f3869e43d1dfbe11da74887ba5
MIT
2022-03-21T00:40:23.000Z
2022-03-21T00:40:23.000Z
examples/example3.ipynb
FabianKP/cgn
9963e60c4a4bf4f3869e43d1dfbe11da74887ba5
MIT
examples/example3.ipynb
FabianKP/cgn
9963e60c4a4bf4f3869e43d1dfbe11da74887ba5
MIT
Qwen/Qwen-72B
1. YES
2. YES