Information Theory

Set theory and the axioms of probability; the pervasiveness of information.

Uncertainty and Information: Foundations of Generalized Information Theory contains comprehensive and up–to–date coverage of results that have emerged from a research program begun by the author in the early 1990s under the name "generalized information theory" (GIT).

Pervasiveness of Information, Probability Axioms, Spaces and Fields

This monograph offers a new foundation for information theory that is based on the notion of information-as-distinctions, being directly measured by logical entropy, and on the re-quantification as Shannon entropy, which is the fundamental concept for the theory of coding and communications.

"In 1948 Claude Shannon published the paper that single-handedly started the field of information theory, ""The Mathematical Theory of Communication."" This groundbreaking paper laid the foundation for virtually all aspects of modern-day communications.

This book is an evolution from my book A First Course in Information Theory published in 2002 when network coding was still at its infancy. The last few years have witnessed the rapid development of network coding into a research ?eld of its own in information science. With its root in infor- tion theory, network coding has not only brought about a paradigm shift in network communications at large, but also had signi?cant in?uence on such speci?c research ?elds as coding theory, networking, switching, wireless c- munications,distributeddatastorage,cryptography,andoptimizationtheory. While new applications of network coding keep emerging, the fundamental - sults that lay the foundation of the subject are more or less mature. One of the main goals of this book therefore is to present these results in a unifying and coherent manner. While the previous book focused only on information theory for discrete random variables, the current book contains two new chapters on information theory for continuous random variables, namely the chapter on di?erential entropy and the chapter on continuous-valued channels. With these topics included, the book becomes more comprehensive and is more suitable to be used as a textbook for a course in an electrical engineering department.