Markov Processes

The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas.

The modem theory of Markov processes has its origins in the studies of A. A. MARKOV (1906-1907) on sequences of experiments "connected in a chain" and in the attempts to describe mathematically the physical phenomenon known as Brownian motion (L. BACHELlER 1900, A. EIN­ STEIN 1905). The first correct mathematical construction of a Markov process with continuous trajectories was given by N. WIENER in 1923. (This process is often called the Wiener process.) The general theory of Markov processes was developed in the 1930's and 1940's by A. N. KOL­ MOGOROV, W. FELLER, W. DOEBLlN, P. LEVY, J. L. DOOB, and others. During the past ten years the theory of Markov processes has entered a new period of intensive development. The methods of the theory of semigroups of linear operators made possible further progress in the classification of Markov processes by their infinitesimal characteristics. The broad classes of Markov processes with continuous trajectories be­ came the main object of study.

This 3rd edition provides an insight into the mathematical crossroads formed by functional analysis (the macroscopic approach), partial differential equations (the mesoscopic approach) and probability (the microscopic approach) via the mathematics needed for the hard parts of Markov processes. It brings these three fields of analysis together, providing a comprehensive study of Markov processes from a broad perspective. The material is carefully and effectively explained, resulting in a surprisingly readable account of the subject.

The volume Stochastic Processes by K. Itö was published as No. 16 of Lecture Notes Series from Mathematics Institute, Aarhus University in August, 1969, based on Lectures given at that Institute during the academie year 1968­ 1969. The volume was as thick as 3.5 cm., mimeographed from typewritten manuscript and has been out of print for many years. Since its appearance, it has served, for those abIe to obtain one of the relatively few copies available, as a highly readable introduetion to basic parts of the theories of additive processes (processes with independent increments) and of Markov processes. It contains, in particular, a clear and detailed exposition of the Lévy-It ö decomposition of additive processes. Encouraged by Professor It ó we have edited the volume in the present book form, amending the text in a number of places and attaching many footnotes. We have also prepared an index. Chapter 0 is for preliminaries. Here centralized sums of independent ran­ dom variables are treated using the dispersion as a main tooI. Lévy's form of characteristic functions of infinitely divisible distributions and basic proper­ ties of martingales are given. Chapter 1 is analysis of additive processes. A fundamental structure the­ orem describes the decomposition of sample functions of additive processes, known today as the Lévy-Itó decomposition. This is thoroughly treated, as­ suming no continuity property in time, in a form close to the original 1942 paper of Itó, which gave rigorous expression to Lévy's intuitive understanding of path behavior.