Discrete Time Processes

More and more digital devices are being used for informa­ tion processing and control purposes in a variety of systems applications, including industrial processes, power networks, biological systems and communication networks. This trend has been helped by the advent of microprocessors and the consequent availability of cheap distributed computing power. For those applications, where digital devices are used, it is reasonable to model the system in discrete-time. In addition there are other application areas, e.g. econometric systems, business systems, certain command and control systems, environmental systems, where the underlying models are in discrete-time and here discrete-time approaches to analysis and control are the most appropriate.

In a unified form, this monograph presents fundamental results on the approximation of centralized and decentralized stochastic control problems, with uncountable state, measurement, and action spaces. It demonstrates how quantization provides a system-independent and constructive method for the reduction of a system with Borel spaces to one with finite state, measurement, and action spaces. In addition to this constructive view, the book considers both the information transmission approach for discretization of actions, and the computational approach for discretization of states and actions. Part I of the text discusses Markov decision processes and their finite-state or finite-action approximations, while Part II builds from there to finite approximations in decentralized stochastic control problems.

In a unified form, this monograph presents fundamental results on the approximation of centralized and decentralized stochastic control problems, with uncountable state, measurement, and action spaces. It demonstrates how quantization provides a system-independent and constructive method for the reduction of a system with Borel spaces to one with finite state, measurement, and action spaces. In addition to this constructive view, the book considers both the information transmission approach for discretization of actions, and the computational approach for discretization of states and actions. Part I of the text discusses Markov decision processes and their finite-state or finite-action approximations, while Part II builds from there to finite approximations in decentralized stochastic control problems.

This book develops the theory of continuous and discrete stochastic processes within the context of cell biology. In the second edition the material has been significantly expanded, particularly within the context of nonequilibrium and self-organizing systems. Given the amount of additional material, the book has been divided into two volumes, with volume I mainly covering molecular processes and volume II focusing on cellular processes.