# Low-dimensional LPV approximations for nonlinear control

Blacksburg – May 2023

# Introduction

$\dot x = f(x) + Bu$

Control of an inverted pendulum

• 9 degrees of freedom
• but nonlinear controller.

Stabilization of a laminar flow

• 50’000 degrees of freedom
• but linear regulator.

## Control of Nonlinear & Large-Scale Systems

A general approach would include

• powerful backends (linear algebra / optimization)
• exploitation of general structures
• model order reduction
• data-driven surrogate models
• all of it?!

# LPV Representation

\begin{align} \dot x -Bu & = f(x) \\ & \approx [A_0+\rho_1(x)A_1+ \dotsm + \rho_r(x) A_r]\, x \end{align}

The linear parameter varying (LPV) representation/approximation $\dot x \approx \bigl [\Sigma \,\rho_i(x)A_i \bigr]\, x + Bu$ for nonlinear controller comes with

• a general structure (linear but parameter-varying)

and extensive theory on

• LPV controller design

Spoiler:

In this talk, we will consider LPV series expansions of control laws.

## LPV system approaches

For linear parameter-varying systems $\dot x = A(\rho(x))\,x + Bu$ there exist established methods that provide control laws based one

• robustness against parameter variations (Peaucelle and Arzelier 2001)
• adaption with the parameter, i.e. gain scheduling, (Apkarian, Gahinet, and Becker 1995)

A major issue: require solutions of coupled LMI systems.

# SDRE series expansion

Consider the optimal regulator control problem

$\int_0^\infty \|y\|^2 + \alpha \|u\|^2\, \mathsf{d}s \to \min_{(y, u)}$ subject to $\dot x = A(\rho(x))\,x+Bu, \quad y=Cx.$

Theorem (Beeler, Tran, and Banks 2000)

If there exists $\Pi$ as a function of $x$ such that \begin{aligned} & \dot{\Pi}(x)+\bigl[\frac{\partial(A(\rho(x)))}{\partial x}\bigr]^T \Pi(x)\\ & \quad+\Pi(x) A(\rho(x))+A^T(\rho(x)) \Pi(x)-\frac{1}{\alpha} \Pi(x) BB^T \Pi(x)=-C^TC . \end{aligned}

Then $u=-\frac{1}{\alpha}B^T\Pi(x)\,x$ is an optimal feedback for the control problem.

In Praxis, parts of the HJB are discarded and we use $\Pi(x)$ that solely solves the state-dependent Riccati equation (SDRE) $\Pi(x) A(\rho(x))+A^T(\rho(x)) \Pi(x)-\frac{1}{\alpha} \Pi(x) BB^T\Pi(x)=-C^TC,$ and the SDRE feedback $u=-\frac{1}{\alpha}B^T\Pi(x)\,x.$

• numerous application examples and
• proofs of performance (Banks, Lewis, and Tran 2007)
• also beyond smallness conditions (Benner and Heiland 2018)
• Although the SDRE is an approximation already,

• the repeated solve of the Riccati equation is not feasible.

• However, for affine LPV systems, a series expansion

• enables an efficient approximation at runtime.

## The series expansion

We note that $\Pi$ depends on $x$ through $A(\rho(x))$.

Thus, we can consider $\Pi$ as a function in $\rho$ and its corresponding multivariate Taylor expansion up to order $K$ $\begin{equation} \label{eq:taylor-expansion-P} \Pi (\rho) \approx \Pi (0) + \sum_{1\leq |\beta| \leq K} \rho^{(\beta)}P_{\beta}, \end{equation}$ where

• $\beta=(\beta_1, \dotsc, \beta_r)\in \mathbb N^r$ is a multiindex and the
• $P_{\beta}\in \mathbb R^{n\times n}$ are constant matrices.

Theorem

If $A(\rho)$ is affine, i.e. $A(\rho) = A_0 + \sum_{k=1}^r \rho_k A_k$.

Then the coefficients of the first order Taylor approximation $\Pi (\rho) \approx \Pi(0) + \sum_{|\beta| = 1} \rho^{(\beta)}P_{\beta} =: P_0 + \sum_{k=1}^r \rho_k L_k.$ are the solutions to

• $A_{0}^{T} P_{0} + P_{0} A_{0} - P_{0} B B^{T} P_{0} = -C^{T} C$,

and, for $k=1,\dotsc,r$,

• $(A_{0} - B B^{T} P_{0})^{T} L_{k} + L_{k} ( A_{0} - B B^{T} P_{0} )= -(A_{k}^{T} P_{0} + P_{0} A_{k})$.

Proof

Insert the Taylor expansion of $\Pi$ and the LPV representation of $A$ into the SDRE and match the coefficients.

Corollary

The corresponding nonlinear feedback is realized as $u = -\frac{1}{\alpha}B^T[P_0 + \sum_{k=1}^r \rho_k(x) L_k]\,x.$

Cp., e.g., (Beeler, Tran, and Banks 2000) and (Alla, Kalise, and Simoncini 2023).

## Intermediate Summary

A representation/approximation of the nonlinear system via $\dot x = [A_0 + \sum_{k=1}^r \rho_k(x) A_k]\, x + Bu$ enables the nonlinear feedback design through truncated expansions of the SDRE.

# How to Design an LPV approximation

A general procedure

If $f(0)=0$ and under mild conditions, the flow $f$ can be factorized $f( x) = [A(x)]\,x$ with some $A\colon \mathbb R^{n} \to \mathbb R^{n\times n}$.

1. If $f$ has a strongly continuous Jacobian $\partial f$, then $f(x) = [\int_0^1 \partial f(sx)\mathsf{d} s]\, x$
2. The trivial choice of $f(x) = [\frac{1}{x^Tx}f(x)x^T]\,x$ doesn’t work well – neither do the improvements (Lin, Vandewalle, and Liang 2015).

For the factorization $f(x)=A(x)\,x$, one can say that

1. it is not unique
2. it can be a design parameter
3. often, it is indicated by the structure.

… like in the advective term in the Navier-Stokes equations: $(v\cdot \nabla)v = \mathcal A_s(v)\,v$ with $s\in[0,1]$ and the linear operator $\mathcal A_s(v)$ defined via $\mathcal A_s(v)\,w := s\,(v\cdot \nabla)w + (1-s)\, (w\cdot \nabla)v.$

Now, we have an state-dependent coefficient representation

$f(x) = A(x)\,x.$

## $\dot x = A(x)\,x + Bu$

• Trivially, this is an LPV representation $\dot x = A(\rho(x))\, x + Bu$ with $\rho(x) = x$.

• Take any model order reduction scheme that compresses (via $\mathcal P$) the state and lifts it back (via $\mathcal L$) so that $\tilde x = \mathcal L(\hat x) = \mathcal L (\mathcal P(x)) \approx x$

• Then $\rho = \mathcal P(x)$ gives a low-dimensional LPV approximation by means of $A(x)\,x \approx A(\tilde x)\, x = A(\mathcal L \rho (x))\,x.$

## Observation

• If $x\mapsto A(x)$ itself is affine linear

• and $\mathcal L$ is linear,

• then $\dot x \approx A(\mathcal L \rho(x))\,x + Bu = [A_0 + \sum_{i=1}^r \rho_i(x) A_i]\, x + Bu$ is affine with

• $\rho_i(x)$ being the components of $\rho(x)\in \mathbb R^r$
• and constant matrices $A_0$, $A_1$, …, $A_r \in \mathbb R^{n\times n}$.

## Intermediate Summary

• Generally, a nonlinear $f$ can be factorized as $f(x) = A(x)\,x$.

• Model order reduction provides a low dimensional LPV representation $A(x)\,x\approx A(\mathcal \rho(x))\,x$.

• The needed affine-linearity in $\rho$ follows from system’s structure (or from another layer of approximation (see, e.g, (Koelewijn and Tóth 2020)).

# Numerical Realization

The Navier-Stokes equations

$\dot v + (v\cdot \nabla) v- \frac{1}{\mathsf{Re}}\Delta v + \nabla p= f,$

$\nabla \cdot v = 0.$

Control Problem:

• use two small outlets for fluid at the cylinder boundary
• to stabilize the unstable steady state
• with a few point observations in the wake.

Simulation model:

• we use finite elements to obtain
• the dynamical model of type

$\dot x = Ax + N(x,x) + Bu, \quad y = Cx$

• with $N$ being bilinear in $x$
• and a state dimension of about $n=50'000$.

## The Algorithm

Nonlinear controller design for $\dot x = f(x) + Bu$ by LPV approximations and truncated SDRE expansions.

1. Compute an affine LPV approximative model with $f(x)\approx A_0x + \sum_{k=1}^r \rho_k(x)A_kx.$

2. Solve one Riccati and $r$ Lyapunov equations for $P_0$ and the $L_k$s.

3. Close the loop with $u = -\frac{1}{\alpha}B^T[P_0x + \sum_{k=1}^r \rho_k(x) L_kx ].$

## Step-1 – Compute the LPV Approximation

We use POD coordinates with the matrix $V\in \mathbb R^{n\times r}$ of POD modes $v_k$

• $\rho(x) = V^T x$,

• $\tilde x = V\rho(x)=\sum_{k=1}^r\rho_i(x)v_k.$

Then: $N(x,x)\approx N(\tilde x, x) = N(\sum_{k=1}^r\rho_i(x)v_k, x) = \sum_{k=1}^r\rho_i(x) N(v_k, x)$ which is readily realized as $[\sum_{k=1}^r\rho_i(x) A_k]\,x.$

## Step-2 – Compute $P_0$ and the $L_k$s

This requires solutions of large-scale ($n=50'000$) matrix equations

1. Riccati – nonlinear but fairly standard
2. Lyapunovs – linear but indefinite.

## Step-3 – Close the Loop

• Setup: Start from the steady-state

Comparison of feedback designs

• LQR – plain LQR controller
• xSDRE-r – truncated (at r) SDRE feedback

## Parameters of the Control Setup

We check the performance with respect to two parameters

• $\alpha$ … the regularization parameter that penalizes the control

• $t_{\mathsf c} > 0$ … time before the controller is activated

• The parameter $t_c$ describes the domain of attraction.

• For r=0 the xSDRE-r feedback recovers the LQR feedback.

Norm plot of the feedback signals.

• LQR fails to stabilize
• increasing r means better performance
• stability achieved at r=10

Less regularization

• less smooth feedback actions
• again LQR fails
• xSDRE can achieve stability
• stability achieved for certain r

## Conclusion for the Numerical Results

• Measurable and reliable improvements with respect to $\alpha$

• more performant feedback action at higher regularization
• no measurable performance gain with respect to $t_{\mathsf c}$

• no extension of the domain of attraction
• still much space for improvement

• find better bases for the parametrization?
• increase the r?
• second order truncation of the SDRE?

# Conclusion

## … and Outlook

• General approach to model structure reduction by low-dimensional affine LPV systems.

$f(x) \quad \to\quad A(x)\,x\quad \to\quad \tilde A(\rho(x))\,x\quad \to\quad [A_0 + \sum_{k=1}^r\rho_k(x)A_k]\,x$

• Proof of concept for nonlinear controller design with POD and truncated SDRE (Heiland and Werner 2023).

• General and performant but still heuristic approach.

• Detailed roadmap for developing the LPV (systems) theory is available.

• PhD student wanted!

Thank You!

## References--SDRE

Banks, H. T., B. M. Lewis, and H. T. Tran. 2007. “Nonlinear Feedback Controllers and Compensators: A State-Dependent Riccati Equation Approach.” Comput. Optim. Appl. 37 (2): 177–218. https://doi.org/10.1007/s10589-007-9015-2.
Beeler, S. C., H. T. Tran, and H. T. Banks. 2000. “Feedback Control Methodologies for Nonlinear Systems.” J. Optim. Theory Appl. 107 (1): 1–33. https://doi.org/10.1023/A:1004607114958.
Benner, Peter, and Jan Heiland. 2018. “Exponential Stability and Stabilization of Extended Linearizations via Continuous Updates of Riccati Based Feedback.” Internat. J. Robust and Nonlinear Cont. 28 (4): 1218–32. https://doi.org/10.1002/rnc.3949.

## References--SDRE ctd

Alla, A., D. Kalise, and V. Simoncini. 2023. “State-Dependent Riccati Equation Feedback Stabilization for Nonlinear PDEs.” Adv. Comput. Math. 49 (1): 9. https://doi.org/10.1007/s10444-022-09998-4.
Lin, Li-Gang, Joos Vandewalle, and Yew-Wen Liang. 2015. “Analytical Representation of the State-Dependent Coefficients in the SDRE/SDDRE Scheme for Multivariable Systems.” Autom. 59: 106–11. https://doi.org/10.1016/j.automatica.2015.06.015.

## References--LPV/SDRE Approximation

Heiland, Jan, Peter Benner, and Rezvan Bahmani. 2022. “Convolutional Neural Networks for Very Low-Dimensional LPV Approximations of Incompressible Navier-Stokes Equations.” Frontiers in Applied Mathematics and Statistics 8. https://doi.org/10.3389/fams.2022.879140.
Heiland, Jan, and Steffen W. R. Werner. 2023. “Low-Complexity Linear Parameter-Varying Approximations of Incompressible Navier-Stokes Equations for Truncated State-Dependent Riccati Feedback.” arxiv.