Nonlinear Optimization

Intended for advanced undergraduate/graduate students as well as scientists and engineers, this textbook presents a multi-disciplinary view of optimization, providing a thorough examination of algorithms, methods, techniques, and tools from diverse areas of optimization.

This book constitutes the refereed proceedings of the 9th International Conference on Machine Learning, Optimization, and Data Science, LOD 2023, which took place in Grasmere, UK, in September 2023. The 72 full papers included in this book were carefully reviewed and selected from 119 submissions. The proceedings also contain 9 papers from and the Third Symposium on Artificial Intelligence and Neuroscience, ACAIN 2023. The contributions focus on the state of the art and the latest advances in the integration of machine learning, deep learning, nonlinear optimization and data science to provide and support the scientific and technological foundations for interpretable, explainable and trustworthy AI.

Optimization is the process by which the optimal solution to a problem, or optimum, is produced. The word optimum has come from the Latin word optimus, meaning best. And since the beginning of his existence Man has strived for that which is best. There has been a host of contributions, from Archimedes to the present day, scattered across many disciplines. Many of the earlier ideas, although interesting from a theoretical point of view, were originally of little practical use, as they involved a daunting amount of com­ putational effort. Now modern computers perform calculations, whose time was once estimated in man-years, in the figurative blink of an eye. Thus it has been worthwhile to resurrect many of these earlier methods.

This book attempts to present the concepts which underlie the various optimization procedures which are commonly used. It is written primarily for those scientists such as economists, operations researchers, and en­ gineers whose main tools of analysis involve optimization techniques and who possess a (not very sharp) knowledge of one or one-and-a-half year's calculus through partial differentiation and Taylor's theorem and some acquaintance with elementary vector and matrix terminology. Such a scientist is frequently confronted with expressions such as Lagrange multi­ pliers, first-and second-order conditions, linear programming and activity analysis, duality, the Kuhn-Tucker conditions, and, more recently, dy­ namic programming and optimal control. He or she uses or needs to use these optimization techniques, and would like to feel more comfortable with them through better understanding of their underlying mathematical concepts, but has no immediate use for a formal theorem-proof treatment which quickly abstracts to a general case of n variables and uses a style and terminology that are discouraging to people who are not mathematics majors.

Practical Optimization: Algorithms and Engineering Applications provides a hands-on treatment of the subject of optimization. A comprehensive set of problems and exercises makes the book suitable for use in one or two semesters of a first-year graduate course or an advanced undergraduate course. Each half of the book contains a full semester’s worth of complementary yet stand-alone material. The practical orientation of the topics chosen and a wealth of useful examples also make the book suitable as a reference work for practitioners in the field.

The main subject of this book is perturbation analysis of continuous optimization problems. In the last two decades considerable progress has been made in that area, and it seems that it is time now to present a synthetic view of many important results that apply to various classes of problems. The model problem that is considered throughout the book is of the form (P) Min/(x) subjectto G(x) E K. xeX Here X and Y are Banach spaces, K is a closed convex subset of Y, and / : X -+ IR and G : X -+ Y are called the objective function and the constraint mapping, respectively.

Bayesian decision theory is known to provide an effective framework for the practical solution of discrete and nonconvex optimization problems. This book is the first to demonstrate that this framework is also well suited for the exploitation of heuristic methods in the solution of such problems, especially those of large scale for which exact optimization approaches can be prohibitively costly. The book covers all aspects ranging from the formal presentation of the Bayesian Approach, to its extension to the Bayesian Heuristic Strategy, and its utilization within the informal, interactive Dynamic Visualization strategy.

This book presents the theoretical details and computational performances of algorithms used for solving continuous nonlinear optimization applications imbedded in GAMS. Aimed toward scientists and graduate students who utilize optimization methods to model and solve problems in mathematical programming, operations research, business, engineering, and industry, this book enables readers with a background in nonlinear optimization and linear algebra to use GAMS technology to understand and utilize its important capabilities to optimize algorithms for modeling and solving complex, large-scale, continuous nonlinear optimization problems or applications.