3223-01

CIS 3223. Data Structures and Algorithms

Introduction

1. Algorithm analysis

Algorithm: a step-by-step solution of a problem that produces an output for a valid input.

Each step must be a basic operation or algorithm that has been defined previously.
The order of the steps must be accurately specified.
The output must be produced in finite steps.

Algorithm and program: relation and difference. Pseudocode.

Desired properties of algorithm:

correctness (with respect to the problem),
efficiency (time and space).

Algorithm analysis on these properties.

2. Time efficiency analysis

How to compare the time efficiency of different algorithms?

Example: sorting.

Why not simply use a stop clock?

To separate the impact of a conceptual method from its concrete implementation, as well as to focus on its performance on hard instances of the problem.

Execution time, in terms of number of major operations, as a function of instance size.

Example: number of comparison and swapping in selection sort.

For a given instance size, time-cost among instances may have best-case, worst-case, and average case.

3. The Big-O notation

Running time vs. its growth rate when problem size increases.

To cluster time-cost functions by their growth order. Using simple function to represent other functions.

The asymptotic efficiency of algorithms, represented by a simple function g(n):

Θ(g(n)) = {f(n) : there exist positive constants c₁, c₂, and n₀ such that 0 ≤ c₁g(n) ≤ f(n) ≤ c₂g(n) for all n ≥ n₀} --- g(n) is an asymptotic bound of f(n).
O(g(n)) = {f(n) : there exist positive constants c and n₀ such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n₀} --- g(n) is an asymptotic upper bound of f(n).
Ω(g(n)) = {f(n) : there exist positive constants c and n₀ such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n₀} --- g(n) is an asymptotic lower bound of f(n).

The growth order of a function is usually written as f(n) = F(g(n)), which actually mean f(n) ∈ F(g(n)). Intuitively, the three notations correspond to relations =, ≤, and ≥, respectively, though not all functions are comparable in growth order.

To decide the relation between two functions:

f(n) = Θ(g(n)) if and only if f(n)/g(n) has a non-zero limit when n goes to infinite.
f(n) = O(g(n)) if and only if f(n)/g(n) has a limit when n goes to infinite.
f(n) = Ω(g(n)) if and only if g(n)/f(n) has a limit when n goes to infinite.

The major growth orders, from low to high, are:

constant: Θ(1)
logarithmic: Θ(log_an), a > 1
polynomial: Θ(n^a), a > 0 --- further divided into linear (Θ(n)), quadratic (Θ(n²)), cubic (Θ(n³)), and so on
exponential: Θ(aⁿ), a > 1

Asymptotic notions can be used in equations and inequalities, as well as in certain calculations. For example, when several functions are added, only the one with the highest growth order matters.

4. Analysis vs. testing

To decide the time efficiency of algorithms for practical purposes, usually both theoretical analysis and actual testing are necessary.