\subsection{Homework 11 - Due: 04/30}
\subsection{Reading}
For minimum-norm optimal control see FDLS Section 22. For LQR and
introduction to optimal control theory, see BMP, Sections 10.1--10.5,
11.1--11.2 and 11.4--11.5. For time-optimal control see
\href{http://liberzon.csl.illinois.edu/teaching/time-optimal.pdf}{Dr.~Liberzon's
handout}.
\subsubsection{Problem 1}
Consider the optimal control problem
\[
\ddot x=u,\qquad
J(u)=\int_{t_0}^{t_1}\left(x^4(t)+u^2(t)\right)dt+x^3(t_1)
\]
\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\item
Write down a partial differential equation for the optimal cost \(V\),
and a boundary condition for it.
\item
Simplify the PDE by computing the minimum in it. Using this minimum
calculation, write down an expression for the optimal control law in
state feedback form. (This expression can contain partial derivatives
of the optimal cost, evaluated along the optimal trajectory.)
\end{enumerate}
\subsubsection{Problem 2}
Consider the minimum-time parking problem discussed in class: bring a
car modeled by the system \(\ddot x=u\), \(u\in[-1,1]\) to rest at the
origin in shortest possible time. Answer the same questions (a) and (b)
as in the previous problem.
\subsubsection{Problem 3}
Consider the optimal control problem given by the system \[
\dot x=xu
\] with \(x\in\mathbb{R}\) and \(u\in[-1,1]\), no running cost
(\(L=0\)), and terminal cost \(M(x)=x\). In other words, the cost
functional is \(J(u)=x(t_1)\). The final time \(t_1\) is fixed and
finite.
\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\item
Find the optimal cost function (also called the value function)
\(V(t,x)\) by inspection (without using the HJB equation).
\item
Write down the HJB equation for this problem, and simplify it by
computing the minimum in it.
\item
Does the optimal cost function from part (a) satisfy the HJB equation
from part (b) everywhere?
\end{enumerate}
\subsubsection{Problem 4}
Consider the LQR problem \[
\begin{aligned}
\dot x_1&=x_2,\\
\dot x_2&=u
\end{aligned}
\qquad\qquad
J(u)=\int_{t_0}^{t_1}\!\!(x_2^2+u^2)dt
\] Write down the Riccati differential equation (with its boundary
condition) and the expressions for the optimal cost \(V\) and the
optimal state feedback control \(u^*\) (these expressions will depend on
the solution \(P\) to the Riccati equation, but you don't need to
compute this solution).
\subsubsection{Problem 5}
Consider the infinite-horizon LQR problem \[
\begin{aligned}
\dot x_1&=x_2,\\
\dot x_2&=u
\end{aligned}
\qquad\qquad
J(u)=\int_{t_0}^{\infty}\!\!(x_2^2+u^2)dt
\]
Find the optimal cost \(V\) and the optimal control \(u^*\) in state
feedback form. Show that the closed-loop system is stable but not
asymptotically stable. Which condition of the theorem that guarantees
closed-loop asymptotic stability is violated?
\subsubsection{Problem 6}
The LQR theory can be extended to problems where the derivative of the
control variable appears in the performance index (i.e., we penalize not
only large control values but also sudden changes in it). Consider
minimization of the cost function
\[
J (u) := \int \limits _{0} ^{\infty} x^T Qx + u^TR_1 u + {\dot u}^TR_2 {\dot u} \; dt
\]
subject to \(\dot x = Ax + Bu\) where \(x \in \mathbb{R}^n\) and
\(u \in \mathbb{R}^r\). We can solve this problem by introducing the
additional state variable \(z = u\) and treating \(v = \dot u\) as the
control variable.
\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
Assume \(Q \ge 0\), \(R_1 \ge 0\), \(R_2 >0\) and that \(x(0)=x_0\)
and \(u(0)=u_0\) are specified. Obtain a general solution of the above
problem by simply applying the LQR theory developed in class.
\item
Apply your solution to the specific problem where the dynamics and
cost function are as follows:
\end{enumerate}
\[
\dot x = \begin{bmatrix} 0 &1 \\ 0 & 0 \end{bmatrix} x + \begin{bmatrix} 0 \\ 1\end{bmatrix}u ; \qquad x_1(0)=x_2(0)=1, \; u(0)=1
\]
\[
J = \int \limits _0 ^{\infty} x_1^2 + \dot u^2 dt
\]
\textbf{Hint:} For Part (b) your solution should have a control law of
the form \(v = k_1x_1+k_2x_2+k_0u\) where the \(k_i\) are scalar
quantities to be determined as part of the solution process.