How to prove a Dynamic programming strategy will work for an algorithm

algorithmsdynamic programming

How can I prove that a dynamic programming (dp) strategy for a problem will work or not? For greedy algorithms, we can prove by showing the sub problems exhibits matroid property.
Is there any such methods for dp algorithms?

Best Answer

In principle, any multi-stage optimization problem is solvable using dynamic programming, if you can come up with the right notion of the state. You do not mention which class of problems you are working on, so it hard to give general guidelines. I'll make the following assumptions here:

  • You are looking at deterministic optimization problems (as opposed to stochastic optimization problems).
  • You have to choose finite number of decisions, and in the end want to maximize rewards that depend on all your decisions (the argument also works if you are interested in minimizing costs).
  • You can decompose the total reward into sum of individual rewards, ideally, in such a way that decision n influences the total reward for the first time only from the n-th term onwards.

In order to solve such a problem by dynamic programming, you need to come up with a state staten that satisfies the following properties:

staten+1 = function(staten, decisionn)

and

rewardn = function(staten, decisionn)

If these conditions are satisfied, then dynamic programming gives an optimal solution. First you define value functions

Vn(staten) = maxdecisionn[ rewardn + Vn+1(staten+1)]

and recursively solve for these value functions in a backward manner.

NOTE: These are the simplest class of problems that can be solved using dynamic programming. For more details, see any standard book on dynamic programming, e.g., Berksekas.

Related Topic