CIS 5511. Programming Techniques

Greedy Algorithms

 

1. The idea

A greedy algorithm is a problem-solving technique often used in optimization problems. At each step, the algorithm makes the locally optimal choice, without reconsidering previous decisions. The hope is that a sequence of locally optimal choices will lead to a globally optimal solution.

Greedy algorithms are usually simple and efficient because they avoid exploring all possible combinations of choices. However, a locally optimal decision does not always contribute to a globally optimal solution.

For example, in the planning matrix-chain multiplication problem discussed previously, a greedy strategy may repeatedly choose the multiplication that appears to reduce the largest matrix dimension. Although this choice seems beneficial at the current step, it can prevent the algorithm from finding the globally optimal parenthesization. In the second example, this algorithm produces the solution ((A1*A2)*A3)*(A4*(A5*A6)), which is not optimal.

Unlike dynamic programming, a greedy algorithm typically does not compare all relevant alternatives or remember previously computed results. Consequently, many greedy algorithms do not always find an optimal solution, though some of them do.

 

2. Example: Activity Selection

Problem: for a given set of activities A1 ... An, each with its start and finish time for using a resource exclusively, find a schedule that maximizes the number of compatible activities. Please note that this is not the same as maximizing the time for the resource to be used.

A greedy solution: find the activity with the earliest finish time that is consistent with the current schedule, add it into the schedule, then recursively create the rest of the schedule.

In the following algorithm, array s and f represent the start and finish time of the activities, and they are sorted in monotonically increasing order of finish time:

Beside the two arrays, the algorithm also takes indexes k and n as input, to indicate the range of activities to be considered. Initially, it is called as Recursively-Activity-Selector(s, f, 0, n) by algorithm Activity-Selector(s, f).

The recursive algorithm can be converted into a more efficient iterative algorithm:

Both algorithms take Θ(n) time.

The optimality of this algorithm follows from an exchange argument. Any optimal schedule can be transformed into another optimal schedule that contains the activity with the earliest finish time. After selecting this activity, the remaining problem has the same form as the original one.

 

3. Example: Knapsack problem

Given a set of items, each with a weight wi and a value vi, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit W and the total value is as large as possible.

If it is allowed to take a fraction of an item, the problem is called "fractional knapsack problem" and a greedy algorithm can find an optimal solution by repeatedly picking the item with the highest unit value until that item is exhausted or W has been reached.

If every item must be completely taken or left, the problem is called "0-1 knapsack problem" and no greedy algorithm (such as the previous one, or repeatedly picking the item with the highest value) can always find an optimal solution. Instead, dynamic programming should be used for that.

Example:

This example illustrates an important lesson: a greedy strategy may be optimal for one version of a problem (fractional knapsack) but not for a closely related version (0-1 knapsack).

 

4. Example: Huffman coding

Huffman coding is an efficient way to compress data for communication and storage. The idea is to use variable-length code for data units, where those with higher frequencies have shorter codes. There is no ambiguity as long as no codeword is also a prefix of some other codeword.

In the following example, the variable-length codeword has an average of 2.24 bits:

Assuming binary prefix coding, a solution can be represented by a binary tree, with each leaf corresponds to an unit, and its path from root represents the code for the unit (left as 0, right as 1). An optimal code is a binary tree with the minimum expected path length from the root to the leaves.

In the following algorithm, C contains a set of units, and Q is a priority queue containing binary trees prioritized by the frequency of their roots.

Procedure HUFFMAN produces an optimal prefix code. Example:

Huffman coding is one of the most important examples of a greedy algorithm. The repeated selection of the two least frequent symbols can be shown to be part of some optimal prefix code, which guarantees the correctness of the algorithm.

 

5. Related techniques

Additional common greedy algorithms:

 

6. Alternative quality-cost tradeoffs

When an in-time solution is preferred over an optimal solution, several approaches besides greedy algorithms can be employed to balance solution quality against computational cost: