Question 1

How is the optimal step size h determined for central differences?

Accepted Answer

The total error of first-order central differences is the sum of truncation error O(h²) and round-off error O(ε/h). Minimizing gives h_opt = (3ε/|f'''|)^(1/3) ≈ ε^(1/3) ≈ 6×10⁻⁶ for float64 (ε ≈ 2.22e-16). The minimum achievable error is approximately ε^(2/3) ≈ 10⁻¹⁰–10⁻¹¹. On the log-log plot, this optimal point appears as the bottom of the V-shaped curve where truncation and round-off contributions are equal.

Question 2

When should I use the 4-point O(h⁴) central difference scheme?

Accepted Answer

The 4-point central scheme [-f(x+2h)+8f(x+h)-8f(x-h)+f(x-2h)]/(12h) with O(h⁴) truncation is best for high-accuracy sensitivity analysis, Jacobian computation, or when function evaluations are expensive. However, its larger stencil coefficient makes it more susceptible to round-off, shifting the optimal h to around 10⁻⁴. For routine CAE Jacobian computation, central differences at h = 10⁻⁵ typically achieve sufficient accuracy.

Question 3

How much more accurate is central differencing than forward differencing?

Accepted Answer

At the same step size h = 10⁻³, forward differences (O(h¹)) yield error ≈ 10⁻³, while central differences (O(h²)) yield error ≈ 10⁻⁶ — about 1,000 times more accurate. Central differences require two function evaluations vs. one for forward differences. In CFD iterative solvers where computational cost is critical, forward differences are common; for post-processing sensitivity analysis, central differences are preferred.

Question 4

Why does making h smaller eventually increase the numerical error?

Accepted Answer

When h becomes very small (e.g., h < 10⁻¹²), f(x+h) and f(x) become nearly equal in floating-point representation. Subtracting nearly equal numbers causes catastrophic cancellation, losing significant digits. The resulting round-off error grows as O(ε/h). This is visible on the log-log plot as an upward-sloping line for small h. The fixed machine epsilon ε = 2.22e-16 for IEEE 754 double precision sets this lower limit.

Truncation Error & Order of Accuracy Visualization

Function Settings

Difference Schemes

Round-off Error Constant

|Error| vs Step Size h (log-log) — Truncation vs Round-off Balance

Stencil Visualization (near f(x))

Error Decomposition: Truncation vs Round-off (Central Diff.)