CAE Control via Reinforcement Learning

A method that uses Reinforcement Learning (RL) for optimal control of simulation parameters and active flow control. The environment is defined as a CFD simulation, and the policy is optimized based on reward signals.

Expressing this mathematically, it looks like this.

Policy Gradient Method:

CAE control using Reinforcement Learning is an important method aiming for the fusion of data-driven approaches and physics-based modeling. While computational cost is a major bottleneck in conventional CAE analysis, introducing CAE control via Reinforcement Learning can significantly improve the trade-off between computational efficiency and prediction accuracy. The mathematical foundation of this method is based on function approximation theory and statistical learning theory, with theoretical research topics including guarantees of generalization performance and rigorous analysis of convergence. Particularly, dealing with the "curse of dimensionality" in high-dimensional input cases is a key practical challenge, and approaches like dimensionality reduction and leveraging sparsity are important.

It shows the basic mathematical framework for applying machine learning models to CAE.

The loss function in AI×CAE is composed as a weighted sum of a data-driven term and a physics constraint term:

Here, $\mathcal{L}_{\text{data}}$ is the squared error with observed data, $\mathcal{L}_{\text{physics}}$ is the residual of the governing equations, and $\mathcal{L}_{\text{reg}}$ is the regularization term. Adjusting the weight parameters $\lambda$ greatly affects learning stability and accuracy.

The biggest challenge for surrogate models is prediction accuracy outside the range of training data (extrapolation region). Incorporating physical laws can improve extrapolation performance, but complete guarantees are difficult.

When the dimensionality of the input parameter space is high, the required number of samples increases exponentially. Efficient sample placement using Active Learning or Latin Hypercube Sampling (LHS) is extremely important.

• The training data must sufficiently represent the physics of the analysis target.
• The relationship between input parameters and output must be smooth (if discontinuities exist, domain partitioning is necessary).
• Reducing computational cost is the main objective; conventional solvers should be used in conjunction for final verification requiring high accuracy.
• If the quality of training data (mesh-converged, V&V completed) is insufficient, model reliability decreases.

Understanding the dimensionless parameters governing the physical phenomenon being analyzed forms the basis for appropriate model selection and parameter setting.

• Péclet Number Pe: Relative importance of convection vs. diffusion. Pe >> 1 indicates convection-dominated (stabilization techniques required).
• Reynolds Number Re: Ratio of inertial forces to viscous forces. A fundamental parameter for fluid problems.
• Biot Number Bi: Ratio of internal conduction to surface convection. For Bi < 0.1, the lumped capacitance method is applicable.
• Courant Number CFL: Indicator of numerical stability. For explicit methods, CFL ≤ 1 is required.

For order-of-magnitude estimation of analysis results, dimensional analysis based on Buckingham's Π theorem is effective. Using characteristic length $L$, characteristic velocity $U$, and characteristic time $T = L/U$, the order of each physical quantity is estimated in advance to confirm the validity of the analysis results.

Choosing appropriate boundary conditions is directly linked to solution uniqueness and physical validity. Insufficient boundary conditions lead to an ill-posed problem, while excessive ones create contradictions.

Yeah, you're on the right track! Actually getting your hands dirty is the best way to learn. If you have any questions, feel free to ask anytime.