Estimating the second derivative of function from optimizers Consider the following optimization
$$x^*(s) = \max_{x\in X} \big(\,f(x)-sx\,\big)$$
where $f$ is assumed to be a strictly concave function and $X$ is an interval constraint, e.g $X = [0,b]$. We do not know the exact function $f$.

Assume that we can provide a parameter $s$ and obtain the corresponding (unique) optimizer $x^*(s)$.

My goal is to estimate $f''(x)$ from a collection of parameters $(s_1,s_2,\ldots)$ and associated optimizers $(x^*(s_1),x^*(s_2),\ldots)$.

How would one go about doing this? I was thinking of applying the envelope theorem or using finite differences in some way but I'm not quite sure how to proceed.
The first order condition of the maximization problem is
\begin{equation}
f'(x)-s=0\iff f'(x)=s
\end{equation}

We can then replace $x$ by $x(s)$ because this is the optimal value given $s$. Since this is true for every $s$, we can differentiate with respect to $s$ which yields 
\begin{equation}
f''(x(s))x'(s)=1
\end{equation}

Which can be rewrite as 
\begin{equation}
f''(x(s))=\frac{1}{x'(s)}
\end{equation}

Then $x'(s)$ can be estimated by finite differences which would give $f''(x)$.