Blacksburg – May 2023

Jan Heiland & Peter Benner & Steffen Werner (MPI Magdeburg)

\[\dot x = f(x) + Bu\]

Control of an inverted pendulum

- 9 degrees of freedom
- but nonlinear controller.

Stabilization of a laminar flow

- 50’000 degrees of freedom
- but linear regulator.

A general approach would include

- powerful backends (linear algebra / optimization)
- exploitation of general structures
- model order reduction
- data-driven surrogate models
- all of it?!

\[\begin{align} \dot x -Bu & = f(x) \\ & \approx [A_0+\rho_1(x)A_1+ \dotsm + \rho_r(x) A_r]\, x \end{align}\]

The *linear parameter varying* (LPV)
representation/approximation \[
\dot x \approx \bigl [\Sigma \,\rho_i(x)A_i \bigr]\, x + Bu
\] for nonlinear controller comes with

- a general structure (
**linear**but parameter-varying)

and extensive theory on

- LPV controller design

Spoiler:

In this talk, we will consider LPV series expansions of control laws.

For linear parameter-varying systems \[ \dot x = A(\rho(x))\,x + Bu \] there exist established methods that provide control laws based one

- robustness against parameter variations (Peaucelle and Arzelier 2001)
- adaption with the parameter, i.e.
*gain scheduling*, (Apkarian, Gahinet, and Becker 1995)

A major issue: require solutions of coupled LMI systems.

Consider the optimal regulator control problem

\[ \int_0^\infty \|y\|^2 + \alpha \|u\|^2\, \mathsf{d}s \to \min_{(y, u)} \] subject to \[ \dot x = A(\rho(x))\,x+Bu, \quad y=Cx. \]

**Theorem** (Beeler, Tran, and Banks 2000)

If there exists \(\Pi\) as a function of \(x\) such that \[ \begin{aligned} & \dot{\Pi}(x)+\bigl[\frac{\partial(A(\rho(x)))}{\partial x}\bigr]^T \Pi(x)\\ & \quad+\Pi(x) A(\rho(x))+A^T(\rho(x)) \Pi(x)-\frac{1}{\alpha} \Pi(x) BB^T \Pi(x)=-C^TC . \end{aligned} \]

Then \[u=-\frac{1}{\alpha}B^T\Pi(x)\,x\] is an optimal feedback for the control problem.

In **Praxis**, parts of the HJB are discarded and we use
\(\Pi(x)\) that solely solves the
state-dependent Riccati equation (SDRE) \[
\Pi(x) A(\rho(x))+A^T(\rho(x)) \Pi(x)-\frac{1}{\alpha} \Pi(x)
BB^T\Pi(x)=-C^TC,
\] and the SDRE feedback \[
u=-\frac{1}{\alpha}B^T\Pi(x)\,x.
\]

- numerous application examples and
- proofs of performance (Banks, Lewis, and Tran 2007)
- also beyond smallness conditions (Benner and Heiland 2018)

Although the SDRE is an approximation already,

the repeated solve of the Riccati equation is not feasible.

However, for affine LPV systems, a series expansion

enables an efficient approximation at runtime.

We note that \(\Pi\) depends on \(x\) through \(A(\rho(x))\).

Thus, we can consider \(\Pi\) as a function in \(\rho\) and its corresponding multivariate Taylor expansion up to order \(K\) \[\begin{equation} \label{eq:taylor-expansion-P} \Pi (\rho) \approx \Pi (0) + \sum_{1\leq |\beta| \leq K} \rho^{(\beta)}P_{\beta}, \end{equation}\] where

- \(\beta=(\beta_1, \dotsc, \beta_r)\in \mathbb N^r\) is a multiindex and the
- \(P_{\beta}\in \mathbb R^{n\times
n}\) are
**constant**matrices.

**Theorem**

If \(A(\rho)\) is affine, i.e. \(A(\rho) = A_0 + \sum_{k=1}^r \rho_k A_k\).

Then the coefficients of the first order Taylor approximation \[ \Pi (\rho) \approx \Pi(0) + \sum_{|\beta| = 1} \rho^{(\beta)}P_{\beta} =: P_0 + \sum_{k=1}^r \rho_k L_k. \] are the solutions to

- \(A_{0}^{T} P_{0} + P_{0} A_{0} - P_{0} B B^{T} P_{0} = -C^{T} C\),

and, for \(k=1,\dotsc,r\),

- \((A_{0} - B B^{T} P_{0})^{T} L_{k} + L_{k} ( A_{0} - B B^{T} P_{0} )= -(A_{k}^{T} P_{0} + P_{0} A_{k})\).

**Proof**

Insert the Taylor expansion of \(\Pi\) and the LPV representation of \(A\) into the SDRE and *match the
coefficients*.

**Corollary**

The corresponding nonlinear feedback is realized as \[ u = -\frac{1}{\alpha}B^T[P_0 + \sum_{k=1}^r \rho_k(x) L_k]\,x. \]

Cp., e.g., (Beeler, Tran, and Banks 2000) and (Alla, Kalise, and Simoncini 2023).

A representation/approximation of the nonlinear system via \[ \dot x = [A_0 + \sum_{k=1}^r \rho_k(x) A_k]\, x + Bu \] enables the nonlinear feedback design through truncated expansions of the SDRE.

A general procedure

If \(f(0)=0\) and under mild conditions, the flow \(f\) can be factorized \[ f( x) = [A(x)]\,x \] with some \(A\colon \mathbb R^{n} \to \mathbb R^{n\times n}\).

- If \(f\) has a strongly continuous Jacobian \(\partial f\), then \[ f(x) = [\int_0^1 \partial f(sx)\mathsf{d} s]\, x \]
- The trivial choice of \[ f(x) = [\frac{1}{x^Tx}f(x)x^T]\,x \] doesn’t work well – neither do the improvements (Lin, Vandewalle, and Liang 2015).

For the factorization \(f(x)=A(x)\,x\), one can say that

- it is not unique
- it can be a design parameter
- often, it is indicated by the structure.

… like in the advective term in the *Navier-Stokes* equations:
\[
(v\cdot \nabla)v = \mathcal A_s(v)\,v
\] with \(s\in[0,1]\) and the
linear operator \(\mathcal A_s(v)\)
defined via \[\mathcal A_s(v)\,w :=
s\,(v\cdot \nabla)w + (1-s)\, (w\cdot \nabla)v.\]

Now, we have an *state-dependent coefficient*
representation

\[ f(x) = A(x)\,x.\]

Trivially, this is an LPV representation \[ \dot x = A(\rho(x))\, x + Bu \] with \(\rho(x) = x\).

Take any model order reduction scheme that compresses (via \(\mathcal P\)) the state and lifts it back (via \(\mathcal L\)) so that \[ \tilde x = \mathcal L(\hat x) = \mathcal L (\mathcal P(x)) \approx x \]

- Then \(\rho = \mathcal P(x)\) gives a low-dimensional LPV approximation by means of \[ A(x)\,x \approx A(\tilde x)\, x = A(\mathcal L \rho (x))\,x. \]

If \(x\mapsto A(x)\) itself is affine linear

and \(\mathcal L\) is linear,

then \[ \dot x \approx A(\mathcal L \rho(x))\,x + Bu = [A_0 + \sum_{i=1}^r \rho_i(x) A_i]\, x + Bu \] is

**affine**with- \(\rho_i(x)\) being the components of \(\rho(x)\in \mathbb R^r\)
- and constant matrices \(A_0\), \(A_1\), …, \(A_r \in \mathbb R^{n\times n}\).

Generally, a nonlinear \(f\) can be factorized as \(f(x) = A(x)\,x\).

Model order reduction provides a low dimensional LPV representation \(A(x)\,x\approx A(\mathcal \rho(x))\,x\).

The needed affine-linearity in \(\rho\) follows from system’s structure (or from another layer of approximation (see, e.g, (Koelewijn and Tóth 2020)).

The *Navier-Stokes* equations

\[ \dot v + (v\cdot \nabla) v- \frac{1}{\mathsf{Re}}\Delta v + \nabla p= f, \]

\[ \nabla \cdot v = 0. \]

Control Problem:

- use two small outlets for fluid at the cylinder boundary
- to stabilize the unstable steady state
- with a few point observations in the wake.

Simulation model:

- we use
*finite elements*to obtain - the dynamical model of type

\(\dot x = Ax + N(x,x) + Bu, \quad y = Cx\)

- with \(N\) being bilinear in \(x\)
- and a state dimension of about \(n=50'000\).

Nonlinear controller design for \[ \dot x = f(x) + Bu \] by LPV approximations and truncated SDRE expansions.

Compute an affine LPV approximative model with \[f(x)\approx A_0x + \sum_{k=1}^r \rho_k(x)A_kx.\]

Solve one

*Riccati*and \(r\)*Lyapunov*equations for \(P_0\) and the \(L_k\)s.Close the loop with \(u = -\frac{1}{\alpha}B^T[P_0x + \sum_{k=1}^r \rho_k(x) L_kx ].\)

We use POD coordinates with the matrix \(V\in \mathbb R^{n\times r}\) of POD modes \(v_k\)

\(\rho(x) = V^T x\),

\(\tilde x = V\rho(x)=\sum_{k=1}^r\rho_i(x)v_k.\)

Then: \[N(x,x)\approx N(\tilde x, x) = N(\sum_{k=1}^r\rho_i(x)v_k, x) = \sum_{k=1}^r\rho_i(x) N(v_k, x) \] which is readily realized as \[ [\sum_{k=1}^r\rho_i(x) A_k]\,x.\]

This requires solutions of large-scale (\(n=50'000\)) matrix equations

- Riccati – nonlinear but fairly standard
- Lyapunovs – linear but indefinite.

We use state-of-the-art low-rank ADI iterations (ask Steffen for details).

- Setup: Start from the steady-state
- Goal: Stabilize the steady-state

Comparison of feedback designs

`LQR`

– plain LQR controller`xSDRE-r`

– truncated (at`r`

) SDRE feedback

We check the performance with respect to two parameters

\(\alpha\) … the regularization parameter that penalizes the control

\(t_{\mathsf c} > 0\) … time before the controller is activated

…

The parameter \(t_c\) describes the domain of attraction.

For

`r=0`

the`xSDRE-r`

feedback recovers the`LQR`

feedback.

Norm plot of the feedback signals.

`LQR`

fails to stabilize- increasing
`r`

means better performance - stability achieved at
`r=10`

Less regularization

- less smooth feedback actions
- again
`LQR`

fails `xSDRE`

can achieve stability- stability achieved for certain
`r`

Measurable and reliable improvements with respect to \(\alpha\)

*more performant feedback action at higher regularization*

no measurable performance gain with respect to \(t_{\mathsf c}\)

*no extension of the domain of attraction*

still much space for improvement

- find better bases for the parametrization?
- increase the
`r`

? - second order truncation of the SDRE?

- General approach to model
**structure**reduction by low-dimensional affine LPV systems.

\[f(x) \quad \to\quad A(x)\,x\quad \to\quad \tilde A(\rho(x))\,x\quad \to\quad [A_0 + \sum_{k=1}^r\rho_k(x)A_k]\,x\]

Proof of concept for nonlinear controller design with POD and truncated SDRE (Heiland and Werner 2023).

General and performant but still heuristic approach.

Detailed roadmap for developing the LPV (systems) theory is available.

PhD student wanted!

Thank You!

Apkarian, Pierre, Pascal Gahinet, and Greg Becker. 1995.
“Self-Scheduled \(H_\infty\)
Control of Linear Parameter-Varying Systems: A Design Example.”
*Autom.* 31 (9): 1251–61. https://doi.org/10.1016/0005-1098(95)00038-X.

Peaucelle, D., and D. Arzelier. 2001. “Robust Performance Analysis
with LMI-Based Methods for Real Parametric Uncertainty via
Parameter-Dependent Lyapunov Functions.” *IEEE
Trans. Automat. Control* 46 (4): 624–30. https://doi.org/10.1109/9.917664.

Koelewijn, Patrick J. W., and Roland Tóth. 2020. “Scheduling
Dimension Reduction of LPV Models - A Deep
Neural Network Approach.” In *2020 American Control
Conference*, 1111–17. IEEE. https://doi.org/10.23919/ACC45564.2020.9147310.

Banks, H. T., B. M. Lewis, and H. T. Tran. 2007. “Nonlinear
Feedback Controllers and Compensators: A State-Dependent
Riccati Equation Approach.” *Comput. Optim.
Appl.* 37 (2): 177–218. https://doi.org/10.1007/s10589-007-9015-2.

Beeler, S. C., H. T. Tran, and H. T. Banks. 2000. “Feedback
Control Methodologies for Nonlinear Systems.” *J. Optim.
Theory Appl.* 107 (1): 1–33. https://doi.org/10.1023/A:1004607114958.

Benner, Peter, and Jan Heiland. 2018. “Exponential Stability and
Stabilization of Extended Linearizations via
Continuous Updates of Riccati Based Feedback.”
*Internat. J. Robust and Nonlinear Cont.* 28 (4): 1218–32. https://doi.org/10.1002/rnc.3949.

Alla, A., D. Kalise, and V. Simoncini. 2023. “State-Dependent
Riccati Equation Feedback Stabilization for Nonlinear
PDEs.” *Adv. Comput. Math.* 49 (1): 9. https://doi.org/10.1007/s10444-022-09998-4.

Lin, Li-Gang, Joos Vandewalle, and Yew-Wen Liang. 2015.
“Analytical Representation of the State-Dependent Coefficients in
the SDRE/SDDRE Scheme for Multivariable Systems.”
*Autom.* 59: 106–11. https://doi.org/10.1016/j.automatica.2015.06.015.

Heiland, Jan, Peter Benner, and Rezvan Bahmani. 2022.
“Convolutional Neural Networks for Very Low-Dimensional LPV
Approximations of Incompressible Navier-Stokes Equations.”
*Frontiers in Applied Mathematics and Statistics* 8. https://doi.org/10.3389/fams.2022.879140.

Heiland, Jan, and Steffen W. R. Werner. 2023. “Low-Complexity
Linear Parameter-Varying Approximations of Incompressible
Navier-Stokes Equations for Truncated
State-Dependent Riccati Feedback.” arxiv.