Given a finite set of points $(x\_1, y\_1), (x\_2, y\_2), \ldots, (x\_n, y\_n)$ in the plane, *Linear Regression* tells us how to find the straight line "$y=a+bx$" best approximating the given points, in the sense that the quantity
 $$
 E(a, b)= \sum\_{i=1}^n\big (ax\_i+b-y\_i\big )^2
 $$
 is as small as possible. However, when the given points are believed to be generated by a nonlinear phenomenon, perhaps the time series of an exponential process, one might prefer to replace "$y=a+bx$" with some other class of functions, often one that is parametrized by a small number of parameters, in which case one is often interested in finding the values of such parameters that minimize some sensible error estimate replacing our $E(a, b)$ above.


On the other hand, according to the Wikipedia, *Nonparametric Statistics* is the branch of Statistics that is not based solely on parametrized models, although the term non-parametric is not meant to imply that such models completely lack parameters but that the number and nature of the parameters are flexible and not fixed in advance.


I believe that one of the reasons for the above disclaimer is that, should one adopt a completely non-parametric approach for fitting a function to a given set of points $(x\_1, y\_1), (x\_2, y\_2), \ldots, (x\_n, y\_n)$, as above, there will be too much freedom and hence the problem will become too easy (a piecewise linear function joining the points being a trivial solution), while I think it is safe to assume that such a solution will likely shed no light on the phenomenon under study.


In order to avoid such trivialities one must therefore either choose a model (parametrized family of functions) beforehand or else impose extra conditions on the fitted function. One possible approach is to require that the fitted function should not *wiggle* too much and, since the curvature of the graph of a function is related to its second derivative, a possible measure of *wiggleness* could be taken as
 $$
 W(f) = \int\_a^bf''(x)^2\, d(x).
 $$ Note that, if $W(f)=0$, then $f$ is necessarily a straight line, which certainly does not wiggle at all.



> 
> **Question**: Given a finite set of points 
>  $(x\_1, y\_1), (x\_2, y\_2),..., (x\_n, y\_n)$ in the plane, such that the $x\_i$ all lie in the
>  interval $[a, b]$, does there exist a twice differentiable function
>  $f$ defined on $[a, b]$, such that the quantity $$L(f) = \sum\_{i=1}^n\big (f(x\_i)-y\_i\big )^2 + \int\_a^bf''(x)^2\, d(x)$$  
> 
>  is minimum among all such functions? In other words, does the
>  functional $L$ defined above attain a minimum on $C^2([a,b])$?
> 
> 
> 



Non-parametric regression and curvature
Yes, that is the cubic smoothing spline with $\lambda$ (multiplier on the integral, which controls the amount of smoothing) ) = 1. See <https://en.wikipedia.org/wiki/Smoothing_spline#Cubic_spline_definition> . 


