# What is lost in terms of approximation, when writing a problem in terms of a Differential Algebraic Equation (DAE) system rather than an ODE system?

It may be that we have a model where the following equation holds for some phenomenon:

$$(1)\quad x + y + z = T$$

Importantly, $T$ is a constant, i.e.: $$(2) \quad \frac{\mathrm{d}T(t)}{\mathrm{d}t} = 0$$

We may be interested in the evolution of $x$, $y$ and $z$ as governed by their differential equations. If we want to make sure that we maintain condition $(2)$ (not guaranteed for any numerical integration scheme), then we may represent the problem as follows:

$$(3a)\quad \frac{\mathrm{d}x}{\mathrm{d}t} = f(x, y, z, t)$$

$$(3b)\quad \frac{\mathrm{d}y}{\mathrm{d}t} = g(x, y, z, t)$$

$$(3c)\quad \frac{\mathrm{d}z}{\mathrm{d}t} = \frac{\mathrm{d}T(t)}{\mathrm{d}t}- f(x,y,z,t) - g(x, y, z, t) = - f(x,y,z,t) - g(x, y, z, t)$$

So, some program might calculate the evolution of $x$, $y$ and $z$ by numerical integrating $x$ and $y$, while using condition $(2)$ to determine $z$ (generally, abd during the calculations for $x$ and $y$).

One takeaway I have had from a numerical analysis course I was a part of recently is that "if you're not violating some conservation rule in the phenomenon being modelled, then you have the exact solution" (and numerical solutions are not exact solutions, usually). Thus, even though I might feel nice about saving $(2)$, I know I am losing something somewhere else -- what is it that I am losing?

An idea I have have is that $(3)$ essentially allows a numerical integration scheme to assume that any difference in $T$ is solely made up by $z$, rather than by some combination of $x$, $y$ and $z$? Or, put differently, any error in the numerical integration scheme in violating $(2)$ is chalked up to $z$, so we may not be getting the actual dynamics of $z$, but rather, the dynamics of $z$ along with some error based on how $(2)$ is being violated by the numerical scheme?

How might one more precisely explain what the downside of using a DAE system is in terms of "what is lost in approximation"?

• Doesn't that imply that for a system with no conservation laws every numerical method produces the exact solution, since it would violate no conservation laws? Not every system is described fully by its conservation laws, not every system is integrable. – Kirill Dec 23 '14 at 7:07
• @Kirill I am basically just paraphrasing something as I understand it, but my understanding is limited. Also, I lack the mathematical sophistication needed to understand the link you supplied, or think of a real life phenomenon that doesn't conserve anything -- at the very least, energy is always conserved? – user89 Dec 23 '14 at 9:06
• @user89 In the shallow water equations (SWE), energy is not conserved across shocks. Indeed, energy is an entropy for the SWE. In the real physical system that SWE seek to model, kinetic and potential energy (modeled by SWE) is converted to heat (not modeled by SWE). – Jed Brown Dec 23 '14 at 22:29
• A simple example of a non-closed system with non-conserved energy is a falling particle whose height satisfies $\ddot h = -g$. – Kirill Dec 24 '14 at 7:38
• The "$T$" in your equation (3c) should not be there. – Jan Dec 24 '14 at 11:13

One takeaway I have had from a numerical analysis course I was a part of recently is that "if you're not violating some conservation rule in the phenomenon being modelled, then you have the exact solution" (and numerical solutions are not exact solutions, usually).

Kirill and JedBrown pointed out situations where conservation doesn't hold; open systems in thermodynamics are another situation. Even in those cases, you don't have the exact solution. Here's a stupidly trivial example that can't be solved exactly on a computer:

\begin{align} \dot{x} = 0, \quad x(0) = 1/3 \end{align}

One-third is not exactly representable in floating-point arithmetic because it can't be expressed as a finite decimal in base-2. So generally speaking, you should expect any numerical solution to have some amount of error in it (even if that error is small); in this case, the error is going to be in the machine representation of one-third.

An idea I have have is that (3) essentially allows a numerical integration scheme to assume that any difference in $T$ is solely made up by $z$, rather than by some combination of $x$, $y$ and $z$? Or, put differently, any error in the numerical integration scheme in violating (2) is chalked up to $z$, so we may not be getting the actual dynamics of $z$, but rather, the dynamics of $z$ along with some error based on how (2) is being violated by the numerical scheme?

I don't see why this situation would be true in general, and numerical schemes normally have tolerances so that you could control the error in the numerical solution. In many cases, you can control the error on a per-variable basis, so conceivably, you could place tight tolerances on $y$ and $z$, plus a loose tolerance on $x$, and expect that you might have more error in $x$ than in $y$ or $z$.

How might one more precisely explain what the downside of using a DAE system is in terms of "what is lost in approximation"?

As you pointed out, you can express some physical systems in multiple ways. Any ODE can be expressed as a DAE; you can also take advantage of conservation laws to rewrite your system of differential equations. In general, rewriting an ODE as a DAE (in a trivial way, re-expressing $\dot{y} = f(y)$ as $\dot{y} - f(y) = 0$) and using numerical methods is more expensive, but not always. For instance, if when using sparse direct methods to solve the linear systems for the numerical methods, the Jacobian matrix of the DAE is much more sparse than the Jacobian matrix of the right-hand-side of the ODE, then solving the DAE may be faster. For a while, this approach was advocated in combustion.

Some ODE methods don't preserve certain types of invariants (e.g., the conservation laws in your example), so the numerical solution might be mostly accurate, but fail to satisfy the invariant in a noticeable way. An example would be something like using a Runge-Kutta method for some Hamiltonian system like orbital dynamics; over short times, your numerical solution will be fine, but over long times, failure to conserve energy will start to introduce significant error in the numerical solution. This shortcoming can be corrected by using symplectic methods (e.g., Verlet), or you could solve the DAE, assuming the DAE is stable.

• "...or you could solve the DAE, assuming the DAE is stable." -- but if I do that, and now save the conservation law, where I am losing out instead? – user89 Dec 25 '14 at 0:50
• To summarize my answer, you usually give up something in execution time; solving the DAE is usually slower. You also might not be able to solve the DAE, depending on its index. – Geoff Oxberry Dec 27 '14 at 3:56