Homework 11 - Due: 04/30

Reading

For minimum-norm optimal control see FDLS Section 22. For LQR and introduction to optimal control theory, see BMP, Sections 10.1–10.5, 11.1–11.2 and 11.4–11.5. For time-optimal control see Dr. Liberzon’s handout.

Problem 1

Consider the optimal control problem

$$\ddot{x}=u,J(u)={\int }_{t_0}^{t_1}(x^4(t)+u^2(t))dt+x^3(t_1)$$

1. Write down a partial differential equation for the optimal cost $V$, and a boundary condition for it.

2. Simplify the PDE by computing the minimum in it. Using this minimum calculation, write down an expression for the optimal control law in state feedback form. (This expression can contain partial derivatives of the optimal cost, evaluated along the optimal trajectory.)

Problem 2

Consider the minimum-time parking problem discussed in class: bring a car modeled by the system $\ddot{x}=u$, $u\in [-1,1]$ to rest at the origin in shortest possible time. Answer the same questions (a) and (b) as in the previous problem.

Problem 3

Consider the optimal control problem given by the system $$\dot{x}=xu$$ with $x\in \mathbb{R}$ and $u\in [-1,1]$, no running cost ($L=0$), and terminal cost $M(x)=x$. In other words, the cost functional is $J(u)=x(t_1)$. The final time $t_1$ is fixed and finite.

1. Find the optimal cost function (also called the value function) $V(t,x)$ by inspection (without using the HJB equation).

2. Write down the HJB equation for this problem, and simplify it by computing the minimum in it.

3. Does the optimal cost function from part (a) satisfy the HJB equation from part (b) everywhere?

Problem 4

Consider the LQR problem $$\begin{array}{rl}{\dot{x}}_1& =x_2,\\ {\dot{x}}_2& =u\end{array}J(u)={\int }_{t_0}^{t_1}(x_2^2+u^2)dt$$ Write down the Riccati differential equation (with its boundary condition) and the expressions for the optimal cost $V$ and the optimal state feedback control $u^{\ast }$ (these expressions will depend on the solution $P$ to the Riccati equation, but you don’t need to compute this solution).

Problem 5

Consider the infinite-horizon LQR problem $$\begin{array}{rl}{\dot{x}}_1& =x_2,\\ {\dot{x}}_2& =u\end{array}J(u)={\int }_{t_0}^{\mathrm{\infty }}(x_2^2+u^2)dt$$

Find the optimal cost $V$ and the optimal control $u^{\ast }$ in state feedback form. Show that the closed-loop system is stable but not asymptotically stable. Which condition of the theorem that guarantees closed-loop asymptotic stability is violated?

Problem 6

The LQR theory can be extended to problems where the derivative of the control variable appears in the performance index (i.e., we penalize not only large control values but also sudden changes in it). Consider minimization of the cost function

$$J(u):=\underset{0}{\overset{\mathrm{\infty }}{\int }}{x}^{T}Qx+u^T{R}_{1}u+{\dot{u}}^T{R}_2\dot{u}dt$$

subject to $\dot{x}=Ax+Bu$ where $x\in {\mathbb{R}}^n$ and $u\in {\mathbb{R}}^r$. We can solve this problem by introducing the additional state variable $z=u$ and treating $v=\dot{u}$ as the control variable.

1. Assume $Q\ge 0$, $R_1\ge 0$, $R_2>0$ and that $x(0)=x_0$ and $u(0)=u_0$ are specified. Obtain a general solution of the above problem by simply applying the LQR theory developed in class.
2. Apply your solution to the specific problem where the dynamics and cost function are as follows:

$$\dot{x}=\left[\begin{array}{cc}0& 1\\ 0& 0\end{array}\right]x+\left[\begin{array}{c}0\\ 1\end{array}\right]u;x_1(0)=x_2(0)=1,u(0)=1$$

$$J=\underset{0}{\overset{\mathrm{\infty }}{\int }}{x}_1^2+{\dot{u}}^{2}dt$$

Hint: For Part (b) your solution should have a control law of the form $v=k_1{x}_1+k_2{x}_2+k_{0}u$ where the $k_i$ are scalar quantities to be determined as part of the solution process.