CIS 5511. Programming Techniques

Dynamic Programming


1. The idea

When a problem is solved by "divide-and-conquer", sometimes the same subproblem appear multiple times.

For example, Fibonacci numbers are defined by the following recurrence:

    F(0) = 0
    F(1) = 1
    F(i) = F(i-1) + F(i-2)
It is easy to directly write a recursive algorithm according to this definition:
    1  if i == 0
    2      return 0
    3  if i == 1
    4      return 1
    5  return Fibonacci-R(i-1) + Fibonacci-R(i-2)
The complexity of this algorithm is O(2n). However, it is easy to see that the algorithm is not efficient. For example, to calculate F(4), the algorithm calculates F(3) and F(2) separately though in calculating F(3), F(2) is already calculated.

It is easy to see that the algorithm can be rewrite in a "bottom-up" way, with intermediate results saved in a data structure. In this case, an array is enough. This gives us a "dynamic programming" version of the algorithm:

    1  A[0] = 0        // assume 0 is a valid index
    2  A[1] = 1
    3  for j = 2 to i
    4      A[j] = A[j-1] + A[j-2]
    5  return A[i]
The complexity of this algorithm is O(n).

From this simple example, we see the basic elements of dynamic programming, that is, though the solution is still defined in a "top-down" manner (from solution to subsolution), it is built in a "bottom-up" manner (from subsolution to solution). Since the subsolutions to subproblems are kept in a data structure for possible later use, each of them is only calculated once.

Memoization is variation of dynamic programming that keeps the efficiency, while maintaining a top-down strategy.

    1  A[0] = 0        // assume 0 is a valid index
    2  A[1] = 1
    3  for j = 2 to i
    4      A[j] = -1
    5  Fibonacci-M2(A, i)

    Fibonacci-M2(A, i)
    1  if A[i] < 0
    2      then A[i] = Fibonacci-M2(A, i-1) + Fibonacci-M2(A, i-2)
    3  return A[i]
The complexity of this algorithm is also O(n).

Dynamic programming is often used in optimization. For this kind of problem, the optimal solution corresponds to a special way to divide the problem into subproblems. When different ways of division are compared, it is quite common that some subproblems are involved in multiple times, therefore dynamic programming provides a more efficient algorithm than (direct) recursion.

The complexity of dynamic programming comes from the need of keeping intermediate results, as well as remembering the structure of the optimal solution, i.e., how the problem is divided into subproblems, step by step.


2. Example: matrix-chain multiplication

To calculate the product of a matrix-chain A1A2...An, n − 1 matrix multiplications are needed, though in each step any pair of matrices can be multiplied. Furthermore, different choices have different costs.

The product of a p-by-q matrix and a q-by-r matrix is a p-by-r matrix, which contains p*r elements, each calculated from q scalar multiplications and q − 1 additions. If we count the number of scalar multiplications and use it to represent running time, then for such an operation, it is p*q*r.

For matrix-chain A1A2A3, if the three have sizes 10-by-2, 2-by-20, and 20-by-5, respectively, then the cost of (A1A2)A3 is 10*2*20 + 10*20*5 = 1400, while the cost of A1(A2A3) is 2*20*5 + 10*2*5 = 300.

In general, for matrix-chain Ai...Aj where each Ak has dimensions Pk-1-by-Pk, then the minimum cost of the product m[i,j] can be obtained by looking for the best way to cut it into Ai...Ak and Ak+1...Aj. Therefore we have

    m[i, j] = 0                                                if i = j
              min{ m[i,k] + m[k+1,j] + Pi-1PkPj }               if i < j
Since certain m[i,j] will be used again and again, we use dynamic programming to solve this problem by calculate m for sub-chains with increasing length, and using another matrix s to keep the cutting point k for each m[i,j].

Example: Given the following matrix dimensions:
A1 is 30-by-35
A2 is 35-by-15
A3 is 15-by-5
A4 is 5-by-10
A5 is 10-by-20
A6 is 20-by-25

then the output of the program is

which means that A1A2A3A4A5A6 should be calculated as (A1(A2A3))((A4A5)A6), and the number of scalar multiplication is 15,125.

The matrix-chain multiplication problem can be solved by either a bottom-up dynamic programming algorithm or a top-down, memoized dynamic-programming algorithm. The running time of both algorithms is Θ(n3), and the algorithm needs Θ(n2) space. Please note that this algorithm determines the best order to carry out the matrix multiplications, without doing the multiplications themselves.


3. More examples

(1) Longest Common Subsequence (LCS)

For sequence X and Y, their LCS can be recursively constructed. Let us define c[i, j] to be the length of an LCS of the sequences Xi and Yj, then

Unlike substring, LCS does not need to be continuous.


Time cost: Θ(m*n). Space cost: Θ(m*n).


(2) Optimal Binary Search Tree

If the keys in a (constant) BST have different probabilities to be searched, then a balanced tree may not give the lowest average cost (though it gives the lowest worst cost). Using dynamic programming, BST with optimal average search time can be built by comparing the expected cost of each candidate (i.e., with each node as root), in a way similar to the matrix-chain multiplication algorithm. The input of this algorithm are the probabilities for each key values to be searchered, as well as those of the other values not in the tree.