CIS 5603. Artificial Intelligence

Learning as Inference

With respect to the relation between learning and reasoning, there are three strategies: This lecture focuses on the last type.

1. Theoretical issues

Traditional study on reasoning has been focused on deduction, which follows inference rules that are "truth-preserving" in the model-theoretic sense.

Non-deductive reasoning has been studied by many thinkers:

2. Learning via reasoning

In AI research, various attempts have been made to extend and revise classical logic to cover learning: Representative projects on analogical reasoning are not logic-based:

3. Concept learning

Some AI techniques consider a concept as specified by its (sufficient and necessary) conditions, which can be learned from (positive or negative) samples. Examples include

Theoritical models of categorization from cognitive science.

4. Bayesian learning

Bayes' theorem is widely used as a learning rule, in which observed events are merged into background knowledge while a prior distribution is replaced by a posterior distribution.

Bayesian learning has been used to uniformly represent various types of inference, so as to gradually learn new knowledge of various types on complicated problems. There have been arguments on its potential in building artificial general intelligence, as well as in modeling the human brain and mind.

On the other hand, some limitations of Bayesian learning have been pointed out. Also, the request on consistency does not allow local updating, which is needed for real-time learning.


Readings