CIS 9615. Analysis of Algorithms

Summary and Beyond


1. The algorithmic approach to problem solving

The algorithmic approach typically solves a problem in the following steps:
  1. Problem definition: to specify input instances and the desired output for each instance
  2. Language selection: to decide available data structures, machine instructions, and control structures
  3. Algorithm design: to use finite sequence of instructions/structures to go from input to output
  4. Correctness analysis: to prove that the algorithm realizes the specified input-output relation
  5. Efficiency analysis: to estimate the running time and space costs of the algorithm
  6. Coding: to implement the algorithm in a programming language
  7. Testing: to verify the correctness and complexity of the program with representative instances
In this way, an algorithm is taken to be the solution of the problem. When an issue appears in a step, very often it is necessary to redo a previous step. When there are multiple candidate solutions, some steps need to be carried out for each of them, then the results are compared. For (theoretically or practically) unsolvable problems, the common approach is to redefine them by (1) reducing the instance scope, or (2) relaxing the solution scope.

Consequently, a problem has different status in the process, starting at 1:

  1. computability-unknown problem --- to look for an algorithm or to prove there is none [go to 2 or 3]
  2. uncomputable problem --- to redefine the problem [go back to 1]
  3. computable problem [start at i]
    1. complexity-unknown problem --- to look for a tractable algorithm or to prove there is none [go to ii or iii]
    2. intractable problem --- to redefine the problem [go back to 1]
    3. tractable problem --- solved

According to this approach, the problem-solving process eventually stops with a tractable algorithm for a given problem or a variant of it. Then, the computer simply applies the algorithm on the instances of the problem, so the algorithm can be seen as a function that maps a problem instance to its solution.


2. Alternative approaches

How to let computer solve a problem without giving it a tractable algorithm for the problem in advance?

Though the activities of a computer are controlled by algorithms, there is no self-contradiction in the above request, since (domain-)problem-independent algorithms are different from (domain-)problem-specific algorithms.

A "non-algorithmic" approach to problem-solving is explored in project NARS, which solves problems in a fundamentally different way:

This non-algorithmic approach to problem solving does not guarantee to find the best solution for all problems. Instead, it is an attempt to use as much of the available knowledge and resources as possible. This approach is closer to human problem solving process, and cannot be analyzed in terms of computability and complexity in the traditional way.

For further information, read the relevant paper, and try the demonstration.

Related approaches explored in Artificial Intelligence (AI):


3. Comparison

In a broad sense, a "problem" can be "solved" in different ways: There are also different relations between a "problem" and a "solution": In summary, which approach should be followed depends on the nature of the problem to be solved, as well as the working environment of the system.