Learning as Optimization
A group of learning techniques take learning as a process to optimize an evaluation function to improve the quality of the results according to certain standard, rather an to approximate specific input-output mapping.
1. Unsupervised learning
Unsupervised learning often refers to classifying or clustering of instances. The objective of the learning algorithm is usually to find a partition of the dataset with minimum within-cluster distances and maximum between-cluster distances, though sometimes the requirements for a partition are relaxed by allowing overlapping or probabilistic classification.
Some techniques come from statistics:
-
K-means clustering repeatedly puts each instance into one of the k clusters that has the nearest mean until a stable partition is formed.
-
Expectation-maximization (EM) extends the idea of K-means from deterministic cluster assignments to probabilistic hidden assignments.
Unsupervised learning is often used to provide labeled data for supervised learning:
Many types of learning in the current machine learning research are distinguished with respect to the implicit assumption of a "label" that correctly categorizes each data item. Compared to concept learning in the human mind, many factors have not been taken into account in this assumption, such as the dependency on motivation and context, and the necessity of multiple inheritance.
Artificial neural networks can organize memory according to the patterns discovered in data.
- Hebbian learning is based on the idea that "Neurons that fire together, wire together". Hopfield network uses symmetric weighted connections (often learned by a Hebbian rule) to create an energy-based associative memory, with Boltzmann Machines as a stochastic extension.
- Self-Organizing Maps (MOP)s let neurons compete to represent inputs, then strengthen the winner and its neighbors. Consequently, similar instances are moved closer to create a low-dimensional topological representation of data.
2. Reinforcement learning
The study of reinforcement started in behavioral psychology. A (human or animal) behavior can often be strengthen or weaken by reward and punishment, respectively, as in classical conditioning.
In AI, reinforcement learning is defined as finding a mapping from states to actions (a "policy") that maximizes the expected total reward in a Markov decision process, where initially only the set of possible states and the set of possible actions are known.
Conceptual issues:
Temporal difference and Q-learning: gradually update the value function according to expected improvement.
Application: AlphaGo and AlphaZero.
Potentials (e.g., The Alberta Plan for AI Research) vs.
limitations.
3. Meta-level learning
Basic ideas:
Major issues:
- The object-level / meta-level distinction depends on architecture.
- The need for meta-meta-level rules/algorithms.
- The system's coherence and integrity across self-modifications
4. Evolutionary learning
Evolutionary computation:
optimization via natural selection in a "species" of solutions for a given problem.
Genetic algorithm:
- Each generation is a population of a constant number of solutions.
- Each solution (as a phenotype) is coded by a sequence of "genes" (genotype) in the form of a binary string.
- There is a fitness function that evaluates candidate solutions.
- Candidates with higher fitness have higher probability to reproduce solutions for the next generation.
- Reproduction happens via crossover between a pair of selected solutions.
- Mutation may occur randomly during reproduction.
- Some high-fitness solutions are retained in the next generation.
- Repeat production until the fitness converges or reach a certain threshold, if it happens within the affordable time.
Examples:
walkers,
eaters
Limitations: not easy to represent a solution as a binary string with meaningful substrings; the learning time is too long.
Genetic Programming:
to approximate a function using automatically generated programs, usually represented as trees.
Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents.
Related field: Artificial Life
Theoretical issue: the relationship among intelligence, evolution, and life.
Readings
- Poole and Mackworth: Sections 4.8, 10.3, 13.1-5
- Russell and Norvig: Sections 20.3, 21.7, Chapter 22
- Luger: Sections 10.6-7, 11.5-6, 12.1-3