CIS 9615. Analysis of Algorithms

Basic Concepts


1. Solving problem with algorithm

Problem: specification of (1) input domain, (2) output range, and (3) mapping from each input to an output.

Algorithm: a (finite and predetermined) sequence of (directly executable or already defined) computational steps that produces an output for a valid input in finite time.

Desired properties of algorithm, as a solution to a problem:

Compromises are usually needed among these factors and others, depending on the practical application.

Programming: to implement an algorithm in a computer language.


2. Time efficiency analysis

The actual time used by an algorithm depends on: To compare the efficiency of different algorithms, several simplifications are applied:
  1. assuming constant time cost for each type of computational step, then only count the number of steps,
  2. measuring the size of instance, usually by length of description,
  3. for a given size, only considering the worst case (or average case, best case) situation,
  4. measuring the efficiency (time complexity) of an algorithm in a situation as a function of instance size,
  5. only considering how fast the function value increases as size increases,
  6. classifying functions into categories by order of growth.
Remember: in this process there are many assumptions made and many factors ignored!

The asymptotic efficiency of algorithms, represented by a simple function g(n):

The growth order of a function is usually written as f(n) = F(g(n)), which actually mean f(n) ∈ F(g(n)). Intuitively, the five notations correspond to relations =, ≤, ≥, <, and >, respectively, though not all functions are comparable in growth order.

To decide the relation between two functions:

Furthermore, in the first two cases f(n) = O(g(n)), and in the last two, f(n) = Ω(g(n)).

The major growth orders, from low to high, are:

Asymptotic notions can be used in equations and inequalities, as well as in certain calculations. For example, when several functions are added, only the one with the highest growth order matters.


3. Analysis of recurrences

An algorithm is usually analyzed by "Divide-and-Conquer": analyzing the parts recursively, then adding the costs together.

Recursion: part of a problem is a simpler problem of the same type.

Assume the algorithm

  1. directly solves problems instances in constant time if their size is smaller than a constant c,
  2. otherwise uses D(n) time to divide a problem into number a of smaller subproblems of the same type, each with 1/b of the original size,
  3. combines their solutions in C(n) time to get the solution for the original problem,
then the algorithm's cost follows the following recurrence:

To solve a recurrence:

  1. substitution method: use mathematical induction to prove a function that was guessed previously;
  2. recursion tree method: represent the equation as a tree to reveal the function — the height of the tree is logbn, and the number of leaves is alogbn (same as nlogba);
  3. the master method:

    In the recursion tree, the three cases correspond to different relative cost spent on root/leaves.

    As a special case, when a = b, logba = 1, and there are n leaves. The above results are simplified to (1) Θ(n), (2) Θ(nlgn), and (3) Θ(f(n)), respectively.