# Convolutional AEs for low-dimensional parameterizations of Navier-Stokes flow

IFAC Seminar – Data-driven Methods in Control – 2021

# Introduction

$\dot x = f(x) + Bu$

Control of an inverted pendulum

• 9 degrees of freedom
• but nonlinear controller.

Stabilization of a laminar flow

• 50’000 degrees of freedom
• but linear regulator.

## Control of Nonlinear & Large-Scale Systems

A general approach would include

• powerful backends (linear algebra / optimization)
• exploitation of general structures
• data-driven surrogate models
• all of it?!

# SDC Representation

$\dot x = [A(x)]\,x + Bu$

• Under mild conditions, the flow $f(x)$ can be factorized $\dot x = [A(x)]\,x + Bu$ – a state dependent coefficient system – with some $A\colon \mathbb R^{n} \to \mathbb R^{n\times n}.$

• Control through a state-dependent state-feedback law $u=-[B^*P(x)]\,x.$

## Nonlinear SDRE Feedback

• Set $u=-[B^TP(x)]\,x.$

• with $P(x)$ as the solution to the state-dependent Riccati equation $A(x)^TP + PA(x) - PBB^TP + C^TC=0$

• the system $\dot x = f(x) + Bu \;=[A(x)- BB^TP(x)]\,x$ can be controlled towards an equilibrium; see, e.g., Banks, Lewis, and Tran (2007).

## Linear Updates as an Alternative

Theorem Benner and Heiland (2018)

• If $P_0$ is the Riccati solution for $x=x_0$

• and if $E$ solves the linear equation $A(x)E + E(A(x_0)-BB^TP_0)=A(x_0)-A(x)$

• with $\|E\| \leq \epsilon < 1$,

• then $u=-B^TP_0(I+E)^{-1}$ stabilizes the system.

$\DeclareMathOperator{\spann}{span} \DeclareMathOperator{\Re}{Re}$

# LPV Representation

$\dot x \approx [A_0+\Sigma \,\rho_k(x)A_k]\, x + Bu$

The linear parameter varying (LPV) representation/approximation $\dot x = f(x) + Bu = [\tilde A(\rho(x))]\,x + Bu \approx [A_0+\Sigma \,\rho_k(x)A_k]\, x + Bu$ with affine parameter dependency can be exploited for designing nonlinear controller through scheduling.

## Scheduling of $H_\infty$ Controllers

• If $\rho(x)\in \mathbb R^{k}$ can be confined to a bounded polygon,

• there is globally stabilizing $H_\infty$ controller

• that can be computed

• through solving $k$ coupled LMI in the size of the state dimension;

see Apkarian, Gahinet, and Becker (1995) .

## Series Expansion of SDRE Solution

For $A(x)=\sum_{k=1}^r\rho_k(x)A_k$, the solution $P$ to the SDRE $A(x)^TP + PA(x) - PBB^TP + C^TC=0$ can be expanded in a series $P(x) = P_0 + \sum_{|\alpha| > 0}\rho(x)^{(\alpha)}P_{\alpha}$ where $P_0$ solves a Riccati equation and $P_\alpha$ solve Lyapunov (linear!) equations;

see Beeler, Tran, and Banks (2000).

## We see

Manifold opportunities if only $k$ was small.

# Low-dimensional LPV

Approximation of Navier-Stokes Equations by Convolutional Neural Networks

The Navier-Stokes equations

$\dot v + (v\cdot \nabla) v- \frac{1}{\Re}\Delta v + \nabla p= f,$

$\nabla \cdot v = 0.$

• Let $v$ be the velocity solution and let $V = \begin{bmatrix} V_1 & V_2 & \dotsm & V_r \end{bmatrix}$ be a, say, POD basis with $v(t) \approx VV^Tv(t)=:\tilde v(t),$

• then $\rho(v(t)) = V^Tv(t)$ is a parametrization.

• And with $\tilde v = VV^Tv = V\rho = \sum_{k=1}^rV_k\rho_k,$

• the NSE has the low-dimensional LPV representation via $(v\cdot \nabla) v \approx (\tilde v \cdot \nabla) v = [\sum_{k=1}^r\rho_k(V_k\cdot \nabla)]\,v.$

## Question

Can we do better than POD?

Lee/Carlberg (2019): MOR of dynamical systems on nonlinear manifolds using deep convolutional autoencoders

Kim/Choi/Widemann/Zodi (2020): Efficient nonlinear manifold reduced order model

## Convolution Autoencoders for NSE

1. Consider solution snapshots $v(t_k)$ as pictures.

2. Learn convolutional kernels to extract relevant features.

3. While extracting the features, we reduce the dimensions.

4. Encode $v(t_k)$ in a low-dimensional $\rho_k$.

## Our Example Architecture Implementation

• A number of convolutional layers for feature extraction and reduction

• A full linear layer with nonlinear activation for the final encoding $\rho\in \mathbb R^{r}$

• A linear layer (w/o activation) that expands $\rho \to \tilde \rho\in \mathbb R^{k}$.

## Input:

• Velocity snapshots $v_i$ of an FEM simulation with $n=50'000$ degrees of freedom

• interpolated to two pictures with 63x95 pixels each

• makes a 2x63x69 tensor.

## Training for minimizing:

$\| v_i - VW\rho(v_i)\|^2_M$ which includes

1. the POD modes $V\in \mathbb R^{n\times k}$,

2. a learned weight matrix $W\in \mathbb R^{k\times r}\colon \rho \mapsto \tilde \rho$,

3. the mass matrix $M$ of the FEM discretization.

## Going PINN

Outlook: the induced low-dimensional affine-linear LPV representation of the convection $\| (v_i\cdot \nabla)v_i - (VW\rho_i \cdot \nabla )v_i\|^2_{M^{-1}}$ as the target of the optimization.

Implementation issues:

• Include FEM operators while
• maintaining the backward mode of the training.

## Results

Simulation parameters:

• Cylinder wake at $\Re=40$, time in $[0, 8]$
• 1000 snapshots/data points
• 2D-CNN with 4 layers
• kernelsize, stride = 5, 2.
• batch_size = 40

# Conclusion

## … and Outlook

• LPV with affine-linear dependencies are attractive if only $k$ is small.

• Proof of concept that CNN can improve POD at very low dimensions.

• Next: Include the parametrized convection in the training.

• Outlook: Use for nonlinear controller design.

Thank You!

Apkarian, Pierre, Pascal Gahinet, and Greg Becker. 1995. “Self-Scheduled $H_\infty$ Control of Linear Parameter-Varying Systems: A Design Example.” Autom. 31 (9): 1251–61. https://doi.org/10.1016/0005-1098(95)00038-X.

Banks, H. T., B. M. Lewis, and H. T. Tran. 2007. “Nonlinear Feedback Controllers and Compensators: A State-Dependent Riccati Equation Approach.” Comput. Optim. Appl. 37 (2): 177–218. https://doi.org/10.1007/s10589-007-9015-2.

Beeler, S. C., H. T. Tran, and H. T. Banks. 2000. “Feedback control methodologies for nonlinear systems.” J. Optim. Theory Appl. 107 (1): 1–33.

Benner, Peter, and Jan Heiland. 2018. “Exponential Stability and Stabilization of Extended Linearizations via Continuous Updates of Riccati Based Feedback.” Internat. J. Robust and Nonlinear Cont. 28 (4): 1218–32. https://doi.org/10.1002/rnc.3949.

// reveal.js plugins