Convolve a SED with a filter. Is convoluting the mathematical operation? I know that in order to get the Flux of a star (or something else) in a particular filter from its SED (luminosity per unit wavelength), I need to convolve the spectrum (SED) with the filter response. Most of the formulas I see to do this are 
$$F_{b}=\dfrac{\int f(\lambda) T_{b}(\lambda)d\lambda}{\int T_{b}(\lambda)d\lambda}\,,$$ 
where $f(\lambda)$ is the SED, and $T_b$ is the filter response in band $b$.

This formula seems very different from a mathematical convolution that I would write as 
$$f*g(\lambda)=\int_{-\infty}^{\infty} f(x) g(\lambda-x)dx\,.$$

Are these two convolutions the same thing? Or is the "astronomy convolution" a different thing (e.g. SED Fitter python package)?
To get the flux of an SED through a particular filter, you actually multiply the the SED by the filter's response. Talking about convolution in this context is a bit of a misnomer. 

Basically, for each wavelength, you look at what fraction of the light will go through the filter, and you sum up the values you get at those wavelengths. 
The division by $\int T_{b}(\lambda)d\lambda$ is just a normalization term.