Abstract: In 1961, Minsky, one of the founders of AI, perceived a fundamental flaw within the burgeoning field of artificial neural networks. He doubted that such a nonlinear system could be effectively trained using gradient methods, because unless the “structure of the search space is special, the optimization may do more harm than good.” Fast forward to today, and we observe deep neural networks — far more complex than those envisioned at the field's inception — being successfully trained with methods akin to gradient descent. It has, indeed, become evident that the objective function displays a highly benign structure that we are only starting to comprehend. In this lecture, I aim to summarize our current understanding of this enigmatic optimization process. I will discuss several themes, including intrinsic dimensionality, the optimization landscape, and implicit regularization, all within the context of deep networks and large language models. Speaker Bio: Jorge Nocedal is the Walter P. Murphy Professor of Industrial Engineering and Management Sciences and (by courtesy) Engineering Sciences and Applied Mathematics at Northwestern University. Nocedal is also the Director of the Center for Optimization and Statistical Learning. Nocedal's main area of research is optimization, with applications in machine learning, engineering design, and the physical sciences. Research activities range from the design of new algorithms, to their software implementation and mathematical analysis. Areas of emphasis include large scale problems (with millions of variables), optimization under uncertainty, and parallel computing.