1. The algorithmic approach to problem solving

1. Problem definition: to specify input instances and the desired output for each instance
2. Language selection: to decide available data structures, machine instructions, and control structures
3. Algorithm design: to use finite sequence of instructions/structures to go from input to output
4. Correctness analysis: to prove that the algorithm realizes the specified input-output relation
5. Efficiency analysis: to estimate the running time and space costs of the algorithm
6. Coding: to implement the algorithm in a programming language
7. Testing: to verify the correctness and complexity of the program with representative instances

Consequently, a problem has different status in the process, starting at 1:

1. computability-unknown problem --- to look for an algorithm or to prove there is none [go to 2 or 3]
2. uncomputable problem --- to redefine the problem [go back to 1]
3. computable problem [start at i]
   1. complexity-unknown problem --- to look for a tractable algorithm or to prove there is none [go to ii or iii]
   2. intractable problem --- to redefine the problem [go back to 1]
   3. tractable problem --- solved

According to this approach, the problem-solving process eventually stops with a tractable algorithm for a given problem or a variant of it. Then, the computer simply applies the algorithm on the instances of the problem, so the algorithm can be seen as a function that maps a problem instance to its solution.

2. Alternative approaches

Though the activities of a computer are controlled by algorithms, there is no self-contradiction in the above request, since (domain-)problem-independent algorithms are different from (domain-)problem-specific algorithms.

A "non-algorithmic" approach to problem-solving is explored in project NARS, which solves problems in a fundamentally different way:

• Do not define a "problem" as a set of instances and use the same method to solve all of them. Instead, treat each "problem instance" as a "problem" on its own, and solve it in a case-by-case manner.
• Do not draw a sharp line between solutions and non-solutions for a given problem, and treat all solutions as equally good. Instead, allow solutions to be partial, and compare candidate solutions to decide which one is better.
• Do not insist on "one problem, one solution". Instead, allow the system to generate zero, one, or a sequence of solutions, each of which is better than the previous ones.
• Do not depend on a predetermined algorithm to solve a whole problem. Instead, cut a problem-solving process into steps. Though each step follows an algorithm, the overall process is formed by linking steps together at run time in a context-sensitive manner.
• Do not attempt to use all relevant knowledge to solve a problem. Instead, in each step only consider a constant amount of knowledge, selected according to their priority values.
• Do not predetermine the method by which a problem is solved in each step. Instead, let the selected problem and available knowledge decide how the problem is processed in that step.
• Do not solve problems one after another. Instead, process problems (and subproblems) in parallel, but at different speed, according to their priority values.
• Do not throw away the intermediate results at the end of a problem-solving process. Instead, keep them as knowledge for future problems, and let all problems interact with the same knowledge base.
• Do not attempt to keep all knowledge forever. Instead, remove items with the lowest priority when the knowledge base is full.
• Do not keep a fixed resources distribution. Instead, adjust the priority distributions according to the experience of the system and the current context, so as to give important and relevant items more resources.

For further information, read the relevant paper, and try the demonstration.

Related approaches explored in Artificial Intelligence (AI):

• heuristic search
• anytime algorithm
• flexible computation
• learning from example
• learning from feedback
• evolutionary algorithm

3. Comparison

• Classical algorithm: a sequence of instructions with determined steps and length. For occurrences of the same problem instance, it provides the same result, path, and cost.
• Randomized algorithm: an algorithm with determined length, but some undetermined steps. For occurrences of the same problem instance, it typically provides the same result, but may have different path and cost. It can be seen as multiple classical algorithms plus a random selection among them. If the same instance is repeated infinitely, the process will be a probabilistic distribution of the algorithms. If the random number is taken as an input argument, it is reduced to the classical case.
• Anytime algorithm: an algorithm with determined steps, but undetermined length. For occurrences of the same problem instance, it follows the same path, but stops at different points, so gives different results and has different costs. If time is introduced as an input argument, it is reduced to the classical case.
• Learning algorithm: in the training stage, the processing of the same problem instance may have variable steps and lengths, though it eventually converges to the classical case.
• Case-by-case: the problem-solving process has undetermined steps and length, as well as result, path, and cost, even for occurrences of the same problem instance. However, at macro level (whole life) and micro level (single step), it is reduced to the classical case.

• absolutely correct/optimal: as defined in a classical algorithm
• probably correct/optimal: correct/optimal within a frequency range
• approximately correct/optimal: correct/optimal within a quality range
• relatively correct/optimal: with respect to available knowledge and resources