Topics in Statistical Information Theory

The relevance of information theory to statistical theory and its applications to stochastic processes is a unifying influence in these TOPICS. The integral representation of discrimination information is presented in these TOPICS reviewing various approaches used in the literature…

This text presents a collection of mathematical exercises with the aim of guiding readers to study topics in statistical physics, equilibrium thermodynamics, information theory, and their various connections. It explores essential tools from linear algebra, elementary functional analysis, and probability theory in detail and demonstrates their applications in topics such as entropy, machine learning, error-correcting codes, and quantum channels. The theory of communication and signal theory are also in the background, and many exercises have been chosen from the theory of wavelets and machine learning. Exercises are selected from a number of different domains, both theoretical and more applied. Notes and other remarks provide motivation for the exercises, and hints and full solutions are given for many. For senior undergraduate and beginning graduate students majoring in mathematics, physics, or engineering, this text will serve as a valuable guide as theymove on to more advanced work.

For over a decade, complex networks have steadily grown as an important tool across a broad array of academic disciplines, with applications ranging from physics to social media. A tightly organized collection of carefully-selected papers on the subject, Towards an Information Theory of Complex Networks: Statistical Methods and Applications presents theoretical and practical results about information-theoretic and statistical models of complex networks in the natural sciences and humanities. The book's major goal is to advocate and promote a combination of graph-theoretic, information-theoretic, and statistical methods as a way to better understand and characterize real-world networks.

The subjects of stochastic processes, information theory, and Lie groups are usually treated separately from each other. This unique two-volume set presents these topics in a unified setting, thereby building bridges between fields that are rarely studied by the same people. Unlike the many excellent formal treatments available for each of these subjects individually, the emphasis in both of these volumes is on the use of stochastic, geometric, and group-theoretic concepts in the modeling of physical phenomena.

The Minimum Message Length (MML) Principle is an information-theoretic approach to induction, hypothesis testing, model selection, and statistical inference. MML, which provides a formal specification for the implementation of Occam's Razor, asserts that the best explanation of observed data is the shortest. Further, an explanation is acceptable (i.e. the induction is justified) only if the explanation is shorter than the original data.