How to find the optimum finite difference method for derivatives?

Question

Related to: What are the negatives of using higher order finite diference schemes?

Problem: I have some discrete data of a trajectory $x_t$ with errors $\delta x_t$ of a physical system sampled at equal times $\Delta t$ and I want to calculate the derivative of the trajectory, i.e., the velocity $v_t$.

I know I can differentiate it numerically using a finite difference method. From the formulas, it may seem a higher order method is more accurate as its theoretical error is lower. However, I believe the derivative of a function at a given point should depend only on a local region of that point. If we take more points that are far away using a higher order finite difference scheme, I feel we would be increasing the error of the derivative.

Questions: Is this reasoning correct? If so, how to balance these two problems to find the best scheme in a particular situation?

---

Based on Superbee's answer I have thought of a way to approach the problem:

1. Compare the sampling separation $\Delta t$ with the variations on your data $\Delta x$. In order to use a higher order method you would need the sampling in time to be much less than the variations in your data. However, to compare the different variables you should remove their units. We could do it in a rough way normalizing against a characteristic length: $L$ and $T$, for $x$ and $t$ respectively. These lengths could be defined as an average of the variables. Then, to use higher order methods we would require: $$ \frac{\Delta t}{T} \ll \frac{\Delta x}{L.} $$ If we have doubts this is fulfilled, we would prefer a lower order.

2. If we do not consider Runge's phenomenon, in the case 1. is fulfilled we could use higher order methods without a problem. However, there would not be any benefit in using a higher order method if the theoretical error $\varepsilon$ of the finite difference method is lower than the error coming from your data $\delta$:

   • $\varepsilon$ depends typically on a derivative of the function (which we could estimate numerically) and a given power of $\Delta t$.

   • $\delta$ comes from the propagation of the error $\delta x_t$ to the finite difference method. We could estimate this from the form of the finite difference method using the typical formula of error propagation: $$ \delta y = \sqrt{\sum_i \left( \frac{ \partial y }{\partial x_i} \right ) ^2}$$

   From these two errors we could estimate also the real error of the derivative.

Does this approach look fine?

Answer 1

The choice of finite-difference scheme depends on several factors, such as the smoothness of your data, how uniformly-spaced the data actually is, etc. You may also want to consider just how accurate your velocity estimate actually needs to be. For example, if there are large error bars in the trajectory then it probably makes little sense to use a high-order method and then claim that the velocity is also accurate to a high-order.

Other considerations include Runge's phenomenon, the increased computational cost of higher-order methods (due to needing to sample more grid points) and the complications that arise at the boundaries when using a very wide finite-difference stencil.

In summary, I don't think there is a single optimum choice; a lot depends on your data.

Edit in response to comments:

How I would approach this problem may be to not even use finite-differences at all. In my experience finite-differences are used mainly for solving differential equations rather than computing derivatives to discrete experimental data ex post facto. If you are solving differential equations you have the advantage of being able to e.g. solve with different $h$ and explicitly check for convergence, but for your problem you don't necessarily have these advantages to verify the "reasonableness" of the finite-difference scheme so it is difficult to say whether any given choice is adequate.

It would certainly be possible to use finite-differences, and the approach in your edits may not be unreasonable, but my instinct for such a problem would be to fit a curve to the data and then just differentiate that (e.g. using polynomial interpolation and then differentiating the polynomial). This may be the simplest way to deal with a host of potential problems with your data: noise (and, therefore, lack of smoothness), steps being unevenly-spaced, how to deal with error bars, different length scales in the data, etc.