Bayesian Calibration of CAE Models — Troubleshooting Guide
By Sitemap
My Bayesian Calibration simulation is giving me unexpected results — convergence issues, maybe. How do I diagnose this systematically?
Bayesian Calibration troubleshooting follows patterns once you know what to look for. Most issues fall into three buckets: convergence failures, accuracy problems, and result misinterpretation. Let me give you a systematic diagnostic framework rather than a list of random fixes.
That framing helps. Before we dive in — what's the single most common mistake engineers make with Bayesian Calibration?
Honestly, it's skipping the sanity checks. Engineers set up a Bayesian Calibration model, it converges, and they trust the result without verifying it against a hand calculation or a known benchmark. The solver gives you an answer regardless of whether your model is physically correct. Always run a simplified version first.
Bayesian calibration infers the posterior of a CAE model's unknown parameters \( \theta \) (material constants, boundary stiffness, heat transfer coefficients) from measured data \( D \).
$$ p(\theta \mid D) \propto p(D \mid \theta)\, p(\theta) $$
A CAE implementation is almost always a 4-stage pipeline: ① set the prior \( p(\theta) \), ② set the likelihood \( p(D\mid\theta) \) (the observation-noise model), ③ build a surrogate standing in for the CAE run, ④ sample the posterior with MCMC or similar. Trouble always originates at one of those 4 places, so this page is organized to trace a symptom back to the stage that produced it.
How is this different from just fitting by least squares? What do I gain by going to the trouble of getting a distribution?
The value of an answer that is a distribution rather than a point is that the uncertainty of the identification propagates straight into the next prediction. A least-squares point estimate stops at "Young's modulus is 205 GPa"; Bayes gets you to "205±3 GPa, strongly correlated with plate thickness." That width and that correlation structure are exactly the raw material for making predictions with confidence intervals from the calibrated model. Put the other way round: if you produce a distribution and then use only the point (the MAP value), you have thrown away most of what the work bought you.
| Check | Remedy |
|---|---|
| Are you reading convergence diagnostics as numbers? | Run 4 or more chains and take \( \hat{R} < 1.01 \) with an effective sample size (ESS) in the hundreds as the bar. Never judge from trace plots by eye alone |
| Parameter scale disparity | Log-transform and standardize every parameter before sampling. Young's modulus (10¹¹) mixed with a friction coefficient (10⁻¹) wrecks the proposal distribution |
| Strong posterior correlation | A strongly correlated pair (stiffness and thickness, say) is probably acting only through its product. Reparameterize (into product and ratio), or move to a sampler that copes with correlation (NUTS/HMC, ensemble methods) |
| Likelihood evaluation far too slow | At 1 evaluation = 1 CAE run, an MCMC of tens of thousands of evaluations is impossible. A surrogate is effectively mandatory |
| Multimodality | If separate chains settle in different modes the solution is non-unique. Use tempering (parallel annealing), or add observations that separate the modes |
This is the sign that the data carries no information about that parameter. Sensitivity analysis comes first in the diagnosis — if the observable has no sensitivity to the parameter (confirm with Morris screening), no amount of additional data will tighten the posterior. The remedies, in order: ① add observables, sensor locations, or load cases that do carry sensitivity (revisit the experiment design); ② drop the insensitive parameters from the calibration set and propagate them at their prior (do not force an identification); ③ if the observation noise was set too generously, tighten it against measured scatter. "Identify every parameter at once" is a recipe for failure — restricting the calibration to the few most sensitive parameters is the standard move.
This is the most serious symptom in Bayesian calibration, and in most cases what is happening is absorption of model discrepancy. Physics the model lacks (contact stiffness, damping, heat loss) is being compensated for by calibration parameters taking nonphysical values. Such a calibrated model matches near the calibration data but misses badly under extrapolation. The remedies are as follows.
Adding a discrepancy term creates non-identifiability between the parameters and the discrepancy (either can explain the data), so set the discrepancy prior (smoothness, amplitude) conservatively, and settle up front whether the aim of the calibration is the true value of the physical parameters or predictive accuracy.
The cause is overfitting to the calibration data, or extrapolation of the discrepancy term. As verification discipline, always measure predictive performance on hold-out conditions that were not used for calibration (a different load, a different temperature range). Predict with a prediction interval obtained by propagating the whole posterior, and score it by coverage — what fraction of the new data lands inside the interval. When coverage departs badly from nominal (about 95% for a 95% interval), suspect ① underestimated observation noise (intervals too narrow), ② a discrepancy term that cannot extrapolate (invalid outside the calibration domain), ③ divergence between the calibration domain and the prediction domain. A discrepancy term is valid only inside the region covered by the calibration data — that is a limit of principle in Bayesian calibration, and something the report must state as its range of applicability.
As long as MCMC runs on the surrogate, surrogate error feeds directly into posterior error. The trap is validating the surrogate across the whole prior while never checking accuracy in the narrow region where the posterior concentrates. The remedy is sequential refinement: ① estimate the posterior coarsely with an initial surrogate, ② add CAE samples in the posterior's high-probability region and update the surrogate, ③ re-estimate the posterior — repeated a few times (with Kriging, the predictive variance tells you where to add points). Before the final report, do not skip running the real CAE at the posterior MAP point and a few representative points to measure the gap against the surrogate. If surrogate error is the same order as the observation noise, that calibration result cannot be trusted.
The noise standard deviation \( \sigma \) in the likelihood directly governs the width of the posterior. Fixing σ at a plausible-looking value is dangerous; practice offers 3 standard moves. ① Estimate σ jointly as an unknown parameter (give it a weakly informative prior). ② If repeat measurements exist, set it from the observed scatter. ③ If observations come from several sensors, consider correlated noise (a covariance) — the independent-noise assumption overstates the information content and narrows the posterior unfairly. The culprit behind a "posterior that is too narrow" is usually an optimistic noise model.
Honestly, doing all of this is a heavy lift… is there a sensible simplified version?
There is — take it in stages. Level 1: least-squares point estimate plus sensitivity analysis (the evidence for which parameters matter). Level 2: add an uncertainty width on top, via bootstrap or a Laplace approximation. Level 3: full Bayes (MCMC plus a discrepancy term). Plenty of real work reaches a sound decision at Level 2, and Level 3 earns its keep on high-risk jobs where identification uncertainty swings the design decision. Whatever the level, the part you never skip is hold-out validation and an explicit range of applicability — that alone turns a calibration result from "curve fitting" into a verifiable engineering deliverable.
Related: Bayesian calibration (consolidated), Kriging surrogates, Morris screening.