CIS 5603. Artificial Intelligence

Perceiving and Acting

 

Some AI systems can directly interact with the outside world (whether physical or virtual). Such a sensorimotor mechanism is also a necessary front-end for a language interface.

1. Perception

Roughly speaking, sensors convert external stimuli (physical, chemical, biological, etc.) into sensations represented within AI systems, which are usually not identical to human sensations, though they may be similar to various degrees.

Perception is the process of interpreting sensory input and constructing an internal representation of the environment. Unlike sensation, which merely records stimuli, perception organizes, filters, abstracts, and relates sensory data to the system's existing knowledge and goals.

AI research on perception has focused primarily on vision and sound. The major challenge is to choose proper features for each level of abstraction. Initially, the levels and the features at each level are chosen by the designers. For instance, in computer vision, an influential approach was proposed by David Marr, who viewed vision as transforming a 2D projective image on the retina into a 3D model of objects and events in the world.

Traditional computer vision relied heavily on handcrafted features and explicit models of visual processing. Recent deep learning approaches shift much of this design effort to learning algorithms that acquire useful representations directly from data. In this "feature learning" process, features are generated and selected by a learning algorithm according to their contribution to the overall task.

Research in cognitive science suggests alternative approaches for AI perception: Related fields:

 

2. Robots and Agents

Robots interact directly with their environment through sensors and actuators.

Some categorical clarifications:

Robots are often equipped with specialized hardware, a subject primarily studied in electrical and mechanical engineering. Robot software is closer to AI and addresses unique requirements: An especially important idea is that perception and action should not be viewed as independent processes. Intelligent agents often act in order to improve perception, while perceptual information continuously guides action. Intelligent behavior emerges from a continuous feedback loop in which perception guides action and action creates new perceptual experiences. This feedback loop is evident in robotics, autonomous vehicles, and reinforcement learning systems, where successful behavior depends on constant adaptation to environmental changes.

Robots provide a common platform for many AI techniques, with current special emphasis on action and body control. Several control paradigms exist for robots:

Architecture Main Idea Limitation Examples
Deliberative Plan before acting Slow in dynamic environments Shakey, BDI agents
Reactive Respond directly to stimuli Limited long-term planning Cog, RL robots/agents
Hybrid Combine planning and reaction Integration complexity AuRA, Robust-TAMP

Examples of perception-action systems include autonomous vehicles, mobile robots, drone navigation, reinforcement-learning agents, and embodied AI systems. These systems continuously interact with their environments rather than solving isolated problems. Modern AI approaches emphasize learning ability—whether symbolic or subsymbolic—and have expanded into multi-agent and swarm systems.

From AI's perspective, the key issue in robots and agents is not their programmed or controlled behavior, but their learned behavior, especially in changing environments. Various approaches include:

The public image of "intelligent robots" often differs from what AI researchers actually pursue in this field. Examples include:

 

3. Theoretical Issues


Readings