Department of Mechanics: Seminar: Stingl 2024
Design Optimization via the Continuous Stochastic Gradient Method
==== Michael Stingl, Lukas Pflug, and A. Uihlein, Department of Mathematics, Friedrich-Alexander-Universität Erlangen-Nürnberg ====
day month 20204, ab:cd-ef:gh CET, Room B-168 @ Thákurova 7, 166 29 Prague 6
Abstract:
We consider large-scale optimization problems with objective functions or constraints depending on random or distributed parameters. Important examples are problems in which expected properties are minimized or problems subject to chance constraints. Here, standard deterministic optimization approaches rely on a discretization of the appearing integrals. However, the underlying quadrature rule can introduce artificial local minima, resulting in an overall poor performance of the optimizer. Since circumventing this effect severely increases the computational cost, stochastic gradient-type methods have gained popularity over the years, most recently also for the solution of topology optimization problems. However, standard schemes from the literature are typically limited to expected loss functions and still require many iterations. This implies that a lot of expensive state problems have to be solved. Thus, we present the continuous stochastic gradient method (CSG), which provides an efficient hybrid approach without these limitations. In CSG, samples calculated in previous iterations are collected in an optimal linear combination to obtain an approximation to the full gradient and objective function value. It can be shown that the approximation error for both quantities vanishes during the optimization process. Therefore, CSG inherits many convergence properties known from full gradient methods, like convergence for constant step sizes. Moreover, the approximation idea in CSG is not limited to simple gradient schemes but can be combined with more elaborated sequential programming techniques. As an example, the combination with the well-known MMA scheme is shown. After presenting the main theoretical properties of CSG, its efficiency is demonstrated for applications from elasticity and/or nano-particle optics.
Michael Stingl, Lukas Pflug, and
A. Uihlein, Department of Mathematics, Friedrich-Alexander-Universität Erlangen-Nürnberg ====
day month 20204, ab:cd-ef:gh CET, Room B-168 @ Thákurova 7, 166 29 Prague 6
Abstract:
We consider large-scale optimization problems with objective functions or constraints depending on random or distributed parameters. Important examples are problems in which expected properties are minimized or problems subject to chance constraints. Here, standard deterministic optimization approaches rely on a discretization of the appearing integrals. However, the underlying quadrature rule can introduce artificial local minima, resulting in an overall poor performance of the optimizer. Since circumventing this effect severely increases the computational cost, stochastic gradient-type methods have gained popularity over the years, most recently also for the solution of topology optimization problems. However, standard schemes from the literature are typically limited to expected loss functions and still require many iterations. This implies that a lot of expensive state problems have to be solved. Thus, we present the continuous stochastic gradient method (CSG), which provides an efficient hybrid approach without these limitations. In CSG, samples calculated in previous iterations are collected in an optimal linear combination to obtain an approximation to the full gradient and objective function value. It can be shown that the approximation error for both quantities vanishes during the optimization process. Therefore, CSG inherits many convergence properties known from full gradient methods, like convergence for constant step sizes. Moreover, the approximation idea in CSG is not limited to simple gradient schemes but can be combined with more elaborated sequential programming techniques. As an example, the combination with the well-known MMA scheme is shown. After presenting the main theoretical properties of CSG, its efficiency is demonstrated for applications from elasticity and/or nano-particle optics.