Elementary Decision Theory

This book presents recent research in decision making under uncertainty, in particular reinforcement learning and learning with expert advice. The core elements of decision theory, Markov decision processes and reinforcement learning have not been previously collected in a concise volume. Our aim with this book was to provide a solid theoretical foundation with elementary proofs of the most important theorems in the field, all collected in one place, and not typically found in
introductory textbooks. This book is addressed to graduate students that are interested in statistical decision making under uncertainty and the foundations of reinforcement learning.

Since its inception 20 years ago the theory of fuzzy sets has advanced in a variety of ways and in many disciplines. Applications of this theory can be found in artificial intelligence, computer science, control engineering, decision theory, expert systems, logic, management science, operations research, pattern recognition, robotics and others. Theoretical advances have been made in many directions. In fact it seems extremely difficult for a newcomer to the field or for somebody who wants to apply fuzzy set theory to his problems to recognize properly the present "state of the art." Therefore, many applications use fuzzy set theory on a much more elementary level than appropriate and necessary. On the other hand, theoretical publications are already so specialized and assume such a background in fuzzy set theory that they are hard to understand.

This monograph is written for advanced graduate students, Ph.D. students, and researchers in mathematical statistics and decision theory. All major topics are introduced on a fairly elementary level and then developed gradually to higher levels. The book is self-contained as it provides full proofs, worked-out examples, and problems. It can be used as a basis for graduate courses, seminars, Ph.D. programs, self-studies, and as a reference book.

Understanding the stochastic enviornment is as much important to the manager as to the economist. From production and marketing to financial management, a manager has to assess various costs imposed by uncertainty. The economist analyzes the role of incomplete and too often imperfect information structures on the optimal decisions made by a firm. The need for understanding the role of uncertainty in quantitative decision models, both in economics and management science provide the basic motivation of this monograph. The stochastic environment is analyzed here in terms of the following specific models of optimization: linear and quadratic models, linear programming, control theory and dynamic programming. Uncertainty is introduced here through the para­ meters, the constraints, and the objective function and its impact evaluated. Specifically recent developments in applied research are emphasized, so that they can help the decision-maker arrive at a solution which has some desirable charac­ teristics like robustness, stability and cautiousness. Mathematical treatment is kept at a fairly elementary level and applied as­ pects are emphasized much more than theory. Moreover, an attempt is made to in­ corporate the economic theory of uncertainty into the stochastic theory of opera­ tions research. Methods of optimal decision rules illustrated he re are applicable in three broad areas: (a) applied economic models in resource allocation and economic planning, (b) operations research models involving portfolio analysis and stochastic linear programming and © systems science models in stochastic control and adaptive behavior.