Sparse PCE and LAR

Theoretical Foundations of Sparse PCE

What Polynomial Chaos Expansion Is

Polynomial chaos expansion represents a response with random inputs \( \mathbf{X} = (X_1,\dots,X_d) \) as a series in polynomials orthogonal to the input distribution:

$$ f(\mathbf{X}) \approx \sum_{\boldsymbol{\alpha} \in \mathcal{A}} c_{\boldsymbol{\alpha}}\, \Psi_{\boldsymbol{\alpha}}(\mathbf{X}) $$

The basis follows the Wiener-Askey correspondence (Gaussian → Hermite, uniform → Legendre, etc.). PCE's decisive advantage: the moment the coefficients are known, statistics fall out analytically:

$$ \mathbb{E}[f] = c_{\mathbf{0}}, \qquad \mathrm{Var}[f] = \sum_{\boldsymbol{\alpha} \ne \mathbf{0}} c_{\boldsymbol{\alpha}}^2 $$

Sobol' sensitivity indices come as partial sums of coefficients at zero extra cost. Mean, variance, and sensitivities read off the expansion without waiting for Monte Carlo convergence — this is why PCE anchors CAE uncertainty quantification.

The Curse of Dimensionality and the Sparsity Assumption

How LAR Selects the Basis

LAR is a greedy algorithm that admits basis functions in order of correlation with the residual — but "gently": competing candidates advance together along equiangular directions, which stabilizes selection compared with hard stepwise inclusion. In sparse PCE, LAR generates a candidate-model sequence (1 basis → 2 → …), each scored by leave-one-out error, and the best model wins. Which bases, and how many, become data-driven decisions — no manual order-tuning.

Computational Details

Two Routes to the Coefficients

Sparsification in Practice — Truncation and Adaptive Order

Implemented sparse PCE narrows the candidate space in three stages:

1. Hyperbolic truncation (q-norm) — restrict candidates to \( \|\boldsymbol{\alpha}\|_q \le p \) with \( q \approx 0.5\text{–}0.75 \), pruning high-order interactions up front
2. LAR selection — pick the active set minimizing LOO error
3. Adaptive order — repeat while raising the maximum order \( p \), stopping at the LOO-best

With this stack, the sample requirement drops to roughly 2–3× the active basis count, and \( d = 10\text{–}20 \) problems typically reach production accuracy in tens to ~200 runs.

The Validation Metric — LOO Error with Correction

Regression PCE's LOO error is closed-form (via the projection-matrix diagonal — no n retrainings). Report relative LOO error \( \epsilon_{LOO} \) or \( Q^2 = 1 - \epsilon_{LOO} \), targeting \( Q^2 \ge 0.99 \) for moment estimation, ~0.95 for trend studies. Finite-sample LOO is optimistic, so use the corrected LOO (basis/sample-ratio correction, as implemented in UQLab and others).

Applying It in Practice

Standard Workflow

1. Define input distributions — types and parameters from measurement, standards, literature. The distribution rationale is the foundation of the whole analysis (different distributions ⇒ different bases and statistics)
2. Handle correlations — map correlated inputs to independent standard variables (Nataf and friends) before expanding
3. Design of experiments — LHS with 2–3× the expected active bases; from \( 10d \) when in doubt
4. Build adaptive sparse PCE — LAR + LOO select order and bases automatically
5. Validate — corrected LOO, plus a few hold-out CAE points if affordable
6. Post-process — mean, variance, Sobol' indices from coefficients; Monte Carlo on the expansion for distributions and tails

Choosing Between PCE and Kriging

Rule of thumb: UQ and sensitivity → PCE; sequential sampling and optimization → Kriging. PC-Kriging (sparse-PCE trend + GP residual) takes both advantages and is available in UQLab.

CAE-Specific Cautions

Numerical noise (remeshing, convergence truncation) leaks into high-order coefficients and systematically inflates variance and Sobol' indices. If noise is suspected: measure it by re-running identical points, tighten convergence one order, and deliberately cap the order (anti noise-fitting). Excluding failed runs distorts the sample distribution — apply the same discipline (range revision, documentation) described for Morris screening.

Implementation Tools

Tool Support

Minimal OpenTURNS Example

import openturns as ot

dist = ot.ComposedDistribution([ot.Normal(210e9, 6e9),      # E
                                ot.Uniform(1.8e-3, 2.2e-3)]) # thickness
X = ot.LHSExperiment(dist, 60).generate()
Y = run_fem_batch(X)                        # your CAE batch runner

algo = ot.FunctionalChaosAlgorithm(X, Y, dist)  # LARS + model selection by default
algo.run()
result = algo.getResult()
sens = ot.FunctionalChaosSobolIndices(result)
print(sens.getSobolIndex(0), sens.getSobolIndex(1))  # first-order Sobol'

Coefficients, active bases, and LOO error all live on the result object — always attach the active-basis count and LOO error to the report.

Research Frontiers

The Compressive-Sensing Connection

"Recover sparse coefficients from few samples" is compressive sensing, and its recovery algorithms — \( \ell_1 \) minimization (basis pursuit), OMP, Bayesian compressive sensing — have all been ported to PCE, along with recovery-guarantee theory for sample requirements. The empirical bottom line across algorithms: model selection via corrected LOO decides the final accuracy, whichever recovery engine you run.

Arbitrary Distributions and Correlated Inputs (aPC, Copulas)

Real data rarely follows clean normals or uniforms. Arbitrary PC builds orthogonal bases numerically from data moments; copula frameworks with Rosenblatt/Nataf transforms handle dependence explicitly — letting you separate "distribution idealization" as its own modeling error. Second-level analyses that propagate distribution-estimation uncertainty are an active topic.

Discontinuous Responses and Multi-Element PCE

Buckling, contact, phase change — discontinuities degrade global polynomials fundamentally (Gibbs). Multi-element gPC tiles the input space with low-order local expansions, with adaptive splitting at detected discontinuities. The practical tell: LOO stops improving as order rises — at that point switch to ME-PCE or a Kriging-family model.

Troubleshooting

Symptoms, Causes, and Fixes

The Minimum Reporting Set

Related: Morris screening, Kriging surrogates, Bayesian calibration.