Atendendo a algumas respostas apreciativas e inúmeras ignorativas, aqui está......

25 outubro 2009

Four ways to solve the Bellman equation

The Bellman equation of optimal control and dynamic programming is a first-order partial differential equation of the Hamilton-Jacobi type. It can be studied by finding the Cauchy characteristic vector field in the 1-jet, resulting in the Pontryagin maximum principle. The characteristics can then be used to compute solutions by integration from boundary values. The ordinary differential equation that results cannot be projected down to the 0-jet, because the Bellman equation is nonlinear, and must be solved in the 1-jet.

For the problem of state observation the initial conditions are not known a priori and must be estimated, so the approach is of limited practical use. There are at least 4 ways to continue the study of the exterior differential system that results:
  1. Move up to the 2-jet.
  2. Use separation of variables.
  3. Define the cost geometrically as a Riemmanian geodesic energy integral.
  4. Forget about characteristics and compute symmetries.
  5. All of the above.
In the problem of state observation, the relevant case where the Bellman equation has a solution is the Kalman filter for linear systems with quadratic costs. The solution is computed by method (5) - all 4 approaches coincide. How about more general cases?
Postar um comentário