CIS 5603. Artificial Intelligence

Learning as Inference

With respect to the relation between learning and reasoning, there are three strategies: (1) integrating a learning module and a reasoning module, (2) using learning to carry our reasoning, and (3) using reasoning to carry our learning. Here the focus is on the last one, which is sometimes called "symbolic" learning approaches.

1. Theoretical issues

Traditional study on reasoning has been focused on deduction, which follows inference rules that are "truth-preserving" in the model-theoretic sense.

Non-deductive reasoning has been studied by many thinkers:

Aristotle considered induction as "argument from the particular to the universal", so is the opposite of deduction.
Bacon's method of induction requires systematic observations and generalizations.
Hume on induction: induction (and causal inference) presupposes that the future will resemble the past, so cannot be logically justified. Therefore, non-deductive inference is often referred to as "ampliative" or "inductive".
Peirce defined induction and abduction as "reversed deduction" in term logic. He also identified their cognitive functions as "generalization" and "explanation", respectively.

2. Learning via reasoning

Some attempts are made to extend and revise classical logic to cover learning:

Inductive logic programming: from concrete cases to general rules. Package in SWI-Prolog.
In Michalski's Inferential Learning Theory, non-deductive inference generates hypothesis to be confirmed.
Non-Axiomatic Logic unifies deduction, induction, abduction, analogy, etc. in a multi-valued logic with experience-grounded semantics. Learning is understood as self-organization, and carried out by reasoning.

Representative projects on analogical reasoning are not logic-based:

Structure mapping engine (SME) maps between existing conceptual structures.
Copycat builds structures while establishing mapping between them.

3. Concept learning

Some AI techniques consider a concept as specified by its (sufficient and necessary) conditions, which can be learned from (positive or negative) samples. Examples include

Version space search where hypotheses are revised by examples.
Decision tree learning acquires multiple concepts together by splitting the data by questions.
Automated Mathematician (AM) discovered interesting mathematical concepts and conjectures.

4. Bayesian learning

Bayes' theorem is widely used as a learning rule, in which observed events are merged into background knowledge while a prior distribution is replaced by a posterior distribution.

Bayesian learning has been used to uniformly represent various types of inference, so as to gradually learn new knowledge of various types on complicated problems. There have been arguments on its potential in artificial general intelligence and human-level concept learning.

On the other hand, some limitations of Bayesian learning have been pointed out. Also, the request on consistency does not allow local updating, which is needed for real-time learning.

Readings

Luger: Sections 10.2-5, 7.3.2
Poole and Mackworth: Sections 10.2
Russell and Norvig: Sections 19.1, 19.3, 21.1-2