Fundamental Limits in Information Networks: Communication, Inference and Learning

Xiugang Wu
Assistant Professor, Electrical and Computer Engineering, and Computer and Information Sciences
University of Delaware
SERC 306
Monday, December 3, 2018 - 14:00
Information networks surround us today in different forms and levels, ranging from neural networks to social networks, to wireless networks and the Internet. The nodes in these networks accomplish tasks such as communication, inference, and learning by exchanging information with each other. What are the fundamental laws that govern information flow in networks and how can the desired task be achieved most efficiently? This question was successfully answered by Shannon in 1948 for the case of a single point-point channel and when the desired task is the reliable communication of data, giving birth to the field of information theory.

In this talk, I will demonstrate how information theory, when enriched with new tools and ideas, can be used to characterize the fundamental limits on information flow in networks more complex than a point-point channel or for tasks other than communication. To this end, I will start by presenting our recent solution to a central problem in network communication that has been open for more than 30 years and named "The Capacity of the Relay Channel." I will then move on to establishing the fundamental limits of inference under rate constraints,and connect it to the information bottleneck method. Finally, I will discuss a general principle for jointly designing the feature extractor and the inference based on a minimax approach to learning..