Some AI systems can directly interact with the outside (either physical or virtual) world without human users or other computer systems in between. Such a sensorimotor mechanism is also a necessary front-end of a language interface.
Perception is the process by which an intelligent system interprets sensory input and constructs an internal representation of a changing environment. Unlike sensation, which merely records stimuli, perception organizes, filters, abstracts, and relates sensory data to the system's existing knowledge and goals.
AI research on perception has been focused on vision and sound. The major challenge is to choose proper features for each level of abstraction. Initially, the features are selected by the designers for each level. In computer vision, an influential approach was proposed by David Marr, who considered vision as transforming a 2D projective image on the retina into a 3D model of objects and events in the world.
Traditional computer vision relied heavily on handcrafted features and explicit models of visual processing. Recent deep-learning approaches shift much of this design effort to learning algorithms that acquire useful representations directly from data. In this "feature learning" process, features are generated and selected by a learning algorithm according to their contribution to the overall task.
For instance, in a Convolutional Neural Network (CNN), convolution kernels are applied to generate feature maps that are further abstracted by the next layers. Trained end-to-end using backpropagation, CNN works well on ImageNet data in recognizing objects.
Capsule neural network: Adding structures into CNN to capture certain properties (e.g., the orientation of the recognized features inside an image).
Beyond feature extracting and learning, there are alternative approaches about perception:
Robots provide a common platform for many AI techniques, though the current special focus is on action control and body control. There have been several control paradigms of robot:
| Architecture | Main Idea | Limitation | Examples |
|---|---|---|---|
| Deliberative | Plan before acting | Slow in dynamic environments | Shakey, BDI agents |
| Reactive | Respond directly to stimuli | Limited long-term planning | Cog, RL robots/agents |
| Hybrid | Combine planning and reaction | Integration complexity | AuRA, Robust-TAMP |
Examples of perception-action systems include autonomous vehicles, mobile robots, drone navigation, reinforcement-learning agents, and embodied AI systems. These systems continuously interact with their environments rather than solving isolated problems. Modern AI approaches stress learning ability, either sumbolic or subsymbolic, and developed to multi-agent and swarm systems.
From AI's perspective, the key issue in robots and agents is not their programmed or controlled behavior, but their learned behavior, especially in changing environments. Various attempts:
