3203. Introduction to Artificial Intelligence

Artificial Neural Networks

Artificial Neural Networks (ANNs) are machine learning models inspired by biological neural systems. They are widely used for classification, prediction, pattern recognition, and many other intelligent tasks.

1. Artificial neuron

An artificial neuron: realizes a function z = f(x) in two steps:
  1. Calculates a weighted sum of its inputs. That is, y = w1x1 + w2x2 + b. Here, w1 and w2 are weights, and b is the bias.
  2. Applies an activation function to generate an output. For instance, if y > 0 then z = 1 else z = 0

Example: If x znd z are Boolean variables and w1 = w2 = 1, then

2. Perceptron

A perceptron is a learning algorithm that creates a binary classifier. It adjusts its weights using training examples so that it can correctly classify new inputs.

A single-layer perceptron contains one layer of computational neurons, where each neuron's output is calculated independently. The learning algorithm repeatedly adjusts the weight values according to the training set until convergence.

Single-layer perceptrons can learn linearly separable patterns like AND and OR, but not XOR.

3. Feedforward Neural Networks

Feedforward neural networks introduce one or more hidden layers between the input and output layers. These hidden layers allow the network to learn more complex relationships.

Unlike a single-layer perceptron, a multilayer network can learn nonlinear functions such as XOR step-by-step, given that

a XOR b = (a AND NOT b) OR (NOT a AND b)

However, the weight values of such a network cannot be directly adjusted as in single-layer networks (a form of Credit Assignment problem).

Backpropagation algorithm solves this problem by adjusting weight values by layer:

  1. Perform a forward pass to compute outputs.
  2. Calculate the output error.
  3. Propagate the error backward through the network.
  4. Update weights using gradient descent.
  5. Repeat until the error is minimized.

Demonstration of learning XOR.