Markov Chain

Authors: Banisch, Sven
Introduces and describes a new approach for modelling certain types of complex dynamical systems
Self-contained presentation and introductory level
Useful as advanced text and as self-study guide

Since their popularization in the 1990s, Markov chain Monte Carlo (MCMC) methods have revolutionized statistical computing and …

in failure time distributions for systems modeled by finite chains. This introductory chapter attempts to provide an over­ view of the material and ideas covered. The presentation is loose and fragmentary, and should be read lightly initially. Subsequent perusal from time to time may help tie the mat­ erial together and provide a unity less readily obtainable otherwise. The detailed presentation begins in Chapter 1, and some readers may prefer to begin there directly. §O.l. Time-Reversibility and Spectral Representation. Continuous time chains may be discussed in terms of discrete time chains by a uniformizing procedure (§2.l) that simplifies and unifies the theory and enables results for discrete and continuous time to be discussed simultaneously.

Since their popularization in the 1990s, Markov chain Monte Carlo (MCMC) methods have revolutionized statistical computing and have had an especially profound impact on the practice of Bayesian statistics. Furthermore, MCMC methods have enabled the development and use of intricate models in an astonishing array of disciplines as diverse as fisheries science and economics.

This textbook explains the fundamentals of Markov Chain Monte Carlo (MCMC) without assuming advanced knowledge of mathematics and programming. MCMC is a powerful technique that can be used to integrate complicated functions or to handle complicated probability distributions. MCMC is frequently used in diverse fields where statistical methods are important – e.g. Bayesian statistics, quantum physics, machine learning, computer science, computational biology, and mathematical economics. This book aims to equip readers with a sound understanding of MCMC and enable them to write simulation codes by themselves.