CIS 3223. Data Structures and Algorithms
Introduction
1. Algorithm analysis
Algorithm: a step-by-step solution of a problem that produces an output for a valid input.
- Each step must be a basic operation or algorithm that has been defined previously.
- The order of the steps must be accurately specified.
- The output must be produced in finite steps.
Algorithm and
program: relation and difference. Pseudocode.
Desired properties of algorithm:
- correctness (with respect to the problem),
- efficiency (time and space).
Algorithm analysis on these properties.
2. Time efficiency analysis
How to compare the time efficiency of different algorithms?
Example: sorting.
Why not simply use a stop clock?
To separate the impact of a conceptual method from its concrete implementation, as well as to focus on its performance on hard instances of the problem.
Execution time, in terms of number of major operations, as a function of instance size.
Example: number of comparison and swapping in
selection sort.
For a given instance size, time-cost among instances may have best-case, worst-case, and average case.
3. The Big-O notation
Running time vs. its growth rate when problem size increases.
To cluster time-cost functions by their growth order. Using simple function to represent other functions.
The asymptotic efficiency of algorithms, represented by a simple function g(n):
- Θ(g(n)) = {f(n) : there exist positive constants c1, c2, and n0 such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0} --- g(n) is an asymptotic bound of f(n).
- O(g(n)) = {f(n) : there exist positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0} --- g(n) is an asymptotic upper bound of f(n).
- Ω(g(n)) = {f(n) : there exist positive constants c and n0 such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0} --- g(n) is an asymptotic lower bound of f(n).
The growth order of a function is usually written as f(n) = F(g(n)), which actually mean f(n) ∈ F(g(n)). Intuitively, the three notations correspond to relations =, ≤, and ≥, respectively, though not all functions are comparable in growth order.
To decide the relation between two functions:
- f(n) = Θ(g(n)) if and only if f(n)/g(n) has a non-zero limit when n goes to infinite.
- f(n) = O(g(n)) if and only if f(n)/g(n) has a limit when n goes to infinite.
- f(n) = Ω(g(n)) if and only if g(n)/f(n) has a limit when n goes to infinite.
The major growth orders, from low to high, are:
- constant: Θ(1)
- logarithmic: Θ(logan), a > 1
- polynomial: Θ(na), a > 0 --- further divided into linear (Θ(n)), quadratic (Θ(n2)), cubic (Θ(n3)), and so on
- exponential: Θ(an), a > 1
Asymptotic notions can be used in equations and inequalities, as well as in certain calculations. For example, when several functions are added, only the one with the highest growth order matters.
4. Analysis vs. testing
To decide the time efficiency of algorithms for practical purposes, usually both theoretical analysis and actual testing are necessary.