3203. Introduction to Artificial Intelligence

Perception and Action

Some AI systems interact directly with their environments, either physical environments (such as robots and autonomous vehicles) or virtual ones (such as game agents and software assistants). Such systems are called sensorimotor systems because they combine:

1. Perception

Perception is the process by which an intelligent system interprets sensory input and constructs an internal representation of a changing environment. Unlike sensation, which merely records stimuli, perception organizes, filters, abstracts, and relates sensory data to the system's existing knowledge and goals. AI research on perception has focused on vision and sound.

A central challenge in AI perception is selecting suitable features for each level of abstraction. Initially, the features were selected by the designers for each level. In computer vision, an influential approach was proposed by David Marr, who considered vision to be the transformation of a 2D projective image on the retina into a 3D model of objects and events in the world.

Traditional computer vision relied heavily on handcrafted features and explicit models of visual processing. Recent deep-learning approaches shift much of this design effort to learning algorithms that acquire useful representations directly from data. In this "feature learning" process, features are generated and selected by a learning algorithm according to their contribution to the overall task.

For instance, in a Convolutional Neural Network (CNN), convolution kernels are applied to generate feature maps that are further abstracted by the next layers. Trained end-to-end using backpropagation, CNN CNNs work well at recognizing objects in ImageNet data.

In a convolutional neural network (CNN), a kernel (sometimes called a convolution mask, or filter) is a learned linear operator that is applied to local patches. The following first covers one-dimensional kernels, used for sequences such as sound or text, then two-dimensional kernels, as used in images. Higher-dimensional kernels are also used, for example in video.

A one-dimensional kernel is a vector that can convert one sequence into another using dot product, as shown in Figure 8.8(a).

A two-dimensional kernel is a matrix that can convert one two-dimensional array into another using convolution, as shown in Figure 8.9.

A typical CNN contains multiple layers:

Input → [Convolution → Activation → Pooling]* → Fully Connected Layers → Output
That is, the input is fed to multiple convolution/Activation/Pooling layers, followed by Fully Connected Layers before reaching the output.

Though CNNs have been used successfully on many problems, it still has issues, such as recognizing object parts without fully understanding their relationships. These issues have motivated other approaches, such as Capsule neural network.

 

2. Action of Robots/Agents

Action refers to any operation that changes the environment or the agent's relationship to it.

Robots interact directly with their environments using sensors and actuators. Their hardware is primarily studied in electrical and mechanical engineering, while their software often involves AI algorithms with special characteristics:

One of the most important ideas in modern AI is that perception and action should not be treated as separate processes. Intelligent behavior emerges from a continuous feedback cycle in which perception guides action, and action creates new perceptions.

This feedback loop is evident in robotics, autonomous vehicles, and reinforcement learning systems, where successful behavior depends on constant adaptation to environmental changes.