1. What Is Unsupervised Learning?

In unsupervised learning, a learning system receives data without explicit target labels and must discover useful patterns, structures, or regularities on its own. Unlike supervised learning, where the correct output is provided for each training example, unsupervised learning attempts to uncover hidden relationships within the data.

Common goals of unsupervised learning include:

• Clustering: grouping similar instances together.
• Feature discovery: identifying hidden pattern or useful relationships.

Unsupervised learning can also support supervised learning. Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data. Generative models, such as Generative Adversarial Network (GAN), learn the statistical structure of data and can generate new instances that resemble the training data.

2. Clustering

Clustering is the task of partitioning a dataset into groups whose members are similar to one another.

A desirable clustering typically has:

• Small within-cluster distance.
• Large between-cluster distance.

Depending on the algorithm, clusters may be:

• Disjoint or overlapping.
• Deterministic or probabilistic.
• Flat or hierarchical.

Since no labels are available, evaluating clustering quality is often more challenging than evaluating supervised learning models.

3. K-Means Clustering

K-means is one of the simplest and most widely used clustering algorithms.

Given a predefined number of clusters, k, the algorithm repeatedly performs two steps.

1. Assign each instance to the cluster whose centroid (mean) is closest.
2. Update the centroid of each cluster using the instances currently assigned to it.

These steps are repeated until the cluster assignments stabilize or the improvement becomes negligible, as in Example 10.9.

4. Expectation-Maximization (EM)

The Expectation-Maximization (EM) algorithm generalizes the idea behind K-means by allowing probabilistic cluster memberships.

Instead of assigning each instance completely to one cluster, EM estimates the probability that an instance belongs to each cluster.

EM alternates between two phases:

1. Expectation (E) Step: Compute the posterior probability that each hidden-variable assignment is correct, based on the current model.
2. Maximization (M) Step: Update the model parameters to maximize the expected likelihood computed in the E-step.

These steps repeat until convergence, as in Example 10.10 and the Old Faithful eruption data example.

K-means can be viewed as a special case of EM in which cluster assignments are deterministic rather than probabilistic.

5. Hebbian Learning

Artificial neural networks can discover and encode regularities in data without external supervision. Instead of memorizing individual examples, the network organizes its internal representations according to recurring patterns.

• Hebbian learning is based on the idea that "Neurons that fire together wire together" that captures statistical regularities in data and provides a biologically inspired mechanism for self-organization. Hopfield network uses symmetric weighted connections (often learned by a Hebbian rule) to create an energy-based associative memory, with Boltzmann Machines as a stochastic extension.
• Self-Organizing Maps (SOMs) let neurons compete to represent inputs, then strengthen the winner and its neighbors. Consequently, similar instances are moved closer to create a low-dimensional topological representation of data.

6. Summary

Unsupervised learning enables AI systems to discover structure in unlabeled data.

A central theme of unsupervised learning is that useful knowledge can emerge directly from the statistical structure of experience without requiring explicit labels or external supervision.